[ANN] outorg.el -- reverse Org-Babel

[ANN] outorg.el -- reverse Org-Babel

[Thorsten Jolitz]

[-- Attachment #1: Type: text/plain, Size: 1177 bytes --]


Hi List, 

I would like to announce 'outorg.el', a kind of "reverse Org-Babel".

Literate Programming is great when writing text with some code. It's not so
great when writing code with some (comment) text, since then the direct
approach is lost and the focus seems to be on the 'wrong side'. 

'outorg' is based on the idea that it would be nice to be able to 

1. structure your source-code files like Org-mode files and use all the
   structure editing and navigation commands available. 
2. have the full power of Org-mode available when writing comment parts
   that are longer than the usual one- or two-liners. 

With 'outorg', you can stay in you favorite language's major-mode while
programming, but with a real Org-mode 'look-and-feel', and rapidly
switch to a temporary buffer in Org-mode for some comment editing.
Exiting the temporary buffer then stores the edited comment text back to
the original source-code buffer (out-commented with the language's
'comment-start' character). 

It's still version 0.9, but it works already quite good with Emacs Lisp
(;; *) and PicoLisp (## *), so I thought I present it to the public.
Feedback and Bug-reports are welcome.


[-- Attachment #2: outorg - reverse org-babel --]
[-- Type: text/plain, Size: 5376 bytes --]

                    ________________________________

                     OUTORG.EL -- REVERSE ORG-BABEL

                            Thorsten Jolitz
                    ________________________________


                             2013-02-11 Mo


Table of Contents
_________________

Org-style outline navigation and comment editing
.. Copyright
.. Credits
.. Commentary
..... What is 'outorg'?
..... Features
.. Emacs Version
.. Github repo
.. Installation
.. ChangeLog
.. Bugs





Org-style outline navigation and comment editing
================================================

Copyright
~~~~~~~~~

  Copyright (C) 2013 Thorsten Jolitz

  This file is not (yet) part of GNU Emacs

  Author: Thorsten Jolitz (format "tjolitz%sgmail%s" "@" ".com")


Credits
~~~~~~~

  This library is based on, or rather an extension of, Per Abrahamsen's
  'out-xtra.el' ([http://tinyurl.com/aql9p97]), and may replace it many
  cases. Some new ideas were taken from Fabrice Niessen's '.emacs'
  ([http://www.mygooglest.com/fni/dot-emacs.html#sec-2]), and some
  inspiration from Eric Schulte's and Dan Davidson's 'Org-babel'
  ([http://orgmode.org/worg/org-contrib/babel/]).


Commentary
~~~~~~~~~~

What is 'outorg'?
-----------------

  It's a modified extension of 'out-xtra', that is itself an extension
  to Emacs 'outline-minor-mode'. Its first goal is to enable full
  outline-navigation and structure editing in source code buffers that
  are structured by an outline-hierarchy with Org-style headlines like

  ,----
  | ;; * 1st-level Entry
  | ;; ** 2nd-level Entry
  `----

  Its second goal, and new idea, is to take advantage of the fact that
  these kind of headlines look just like out-commented Org-mode
  headlines (e.g. in Emacs Lisp source code buffers). Thus,
  functionality has been added to enable editing of such 'outlined'
  source-code buffers in a temporary Org-mode buffer and insert the
  modified content back into the original source code buffer.

  This is actually the same idea employed in Org-mode (Org-Babel) for
  comfortable editing of source code blocks in Org-mode buffers: a
  temporary buffer is opened and put into the programming language
  major-mode, the source-code block is copied into the temporary buffer,
  edited (using the programming-language major-mode functionality), and
  the modified content saved back to the original Org-mode file. Here,
  the focus is on the text - an Org-mode file can be viewed as text with
  source-code.

  With 'outorg', the focus is on the source code, not on the text. It
  should be used when its about editing (extended) comments in
  source-code files, i.e. when the main editing activity is actually
  programming, with an occasional need to write sections of (comment)
  text. Just like it is very convenient to edit the source code buffers
  in Org-mode with the full editing power of the associated major-mode,
  its quite an advantage to have the full power of Org-mode available
  when writing the comment sections of your source code files.


Features
--------

  This file provides (almost) the same nice extra features for outline
  minor mode like Per Abrahamsen's 'out-xtra':

  - Change default minor mode key prefix to `C-c'.
  - Complete keybindings and menu support.
  - Add command to show top level headers.
  - Add command to hide all other entries than the one containing point.

  `outorg' follows a different idea than `out-xtra': it consists of
  generic functionality that calculates the adequate outline-regexp and
  outline-level for the active major-mode, rather than defining several
  blocks of major-mode specific functionality.

  New features of `outorg' are:

  1. Generic functionality that should work whereever `comment-region'
     and `uncomment-region' work.
  2. Fontification of headlines (copied from Fabrice Niessen's '.emacs')
  3. Toggling between editing in (e.g.) Lisp mode and in Org mode,
     similar to the editing of source-code blocks in Org-mode.

  It is highly recommended to use `outorg' together with `outline-magic'
  for the Org-style `outline-cycle' command.


Emacs Version
~~~~~~~~~~~~~

  `outorg.el' works with [GNU Emacs 24.2.1 (x86_64-unknown-linux-gnu,
  GTK+ Version 3.6.4) of 2013-01-20 on eric]. No attempts of testing
  with older versions or other types of Emacs have be made (yet).


Github repo
~~~~~~~~~~~

  You can clone the repo:

  ,----
  | $ git clone git@github.com:tj64/outorg.git
  `----

  or just download the source from

  [https://github.com/tj64/outorg]


Installation
~~~~~~~~~~~~

  Insert

  ,----
  | (require 'outorg)
  `----

  in your .emacs file to install. If you want a different prefix key,
  insert first

  ,----
  | (defvar outline-minor-mode-prefix "\C-c")
  `----

  or whatever. The prefix can only be changed before outline (minor)
  mode is loaded.


ChangeLog
~~~~~~~~~

         date  author(s)        version 
  --------------------------------------
   2013-02-11  Thorsten Jolitz      0.9 


Bugs
~~~~

  `outorg' is line-based, it only works with 'one-line' comments, i.e.
  with comment-sections like those produced by `comment-region' (a
  command that comments or uncomments each line in the region). Those
  special multi-line comments found in many programming languages are
  not recognized and lead to undefined behaviour.

  And it's still version 0.9 - work in progress.

[-- Attachment #3: Type: text/plain, Size: 23 bytes --]


-- 
cheers,
Thorsten

Re: [ANN] outorg.el -- reverse Org-Babel

[François Pinard]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> With 'outorg', you can stay in you favorite language's major-mode while
> programming, but with a real Org-mode 'look-and-feel', and rapidly
> switch to a temporary buffer in Org-mode for some comment editing.
> Exiting the temporary buffer then stores the edited comment text back to
> the original source-code buffer (out-commented with the language's
> 'comment-start' character).

Hi, Thorsten.

Just from reading your description, "outorg" seems strangely similar to
"poporg" (https://github.com/pinard/poporg), which I announced on this
list maybe two weeks ago.  I would presume you missed it? :-)

I'll save myself a pointer to "outorg", for later perusing and study.

François

Re: [ANN] outorg.el -- reverse Org-Babel

[Thorsten Jolitz]

François Pinard <pinard@iro.umontreal.ca> writes:

Hi, Francois,

> Just from reading your description, "outorg" seems strangely similar to
> "poporg" (https://github.com/pinard/poporg), which I announced on this
> list maybe two weeks ago.  I would presume you missed it? :-)

I missed that completely, you can check the code - there should be no
similarities whatsoever. I was entirely inspired by my recent discovery
of the possibilities of outline-minor-mode and Org-Babel. 

What a bad luck ... ;(

But anyway, probably the best we can do is to bring both version to a
usable state and then let the users decide if they want to use it - and
which one.

At first glance, the projects seem so different that a merge appears
impossible or just to much work - and not worth the pain. 

> I'll save myself a pointer to "outorg", for later perusing and study.

I'll do the same thing with 'poporg' ;)

-- 
cheers,
Thorsten

Re: [ANN] outorg.el -- reverse Org-Babel

[François Pinard]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> [about the nearly coincident publication of *outorg* and *poporg*]
> What a bad luck ... ;(

Oh, I'm not much into authorship wars, you know, as long as the need
gets covered.  Free time being a scarce resource (for me at least!), I
prefer when we can all make the best use of it.

Keep happy!

François

Re: [ANN] outorg.el -- reverse Org-Babel

[Thorsten Jolitz]

François Pinard <pinard@iro.umontreal.ca> writes:

> Thorsten Jolitz <tjolitz@gmail.com> writes:
>
>> [about the nearly coincident publication of *outorg* and *poporg*]
>> What a bad luck ... ;(
>
> Oh, I'm not much into authorship wars, you know, as long as the need
> gets covered.  Free time being a scarce resource (for me at least!), I
> prefer when we can all make the best use of it.
>
> Keep happy!

Thanks, I really did miss your anouncement, and it was not the feedback
I was looking for to hear that another (after looking at your code:
better and more refined) version of the idea already exists. A bit
discouraging, I must admit. 

So I consider 'outorg' now as a private project for my private use. Keep
going with 'poporg' and make it part of emacs/org-mode - and look at
'outorg' as a proof that there is definitely a need out there that
should get covered.

-- 
cheers,
Thorsten

Re: [ANN] outorg.el -- reverse Org-Babel

[François Pinard]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> François Pinard <pinard@iro.umontreal.ca> writes:
>> Keep happy!

> A bit discouraging, I must admit.

Don't be!  We need all the courage we can get!

> Keep going with 'poporg' and make it part of emacs/org-mode - and look
> at 'outorg' as a proof that there is definitely a need out there that
> should get covered.

poporg has many flaws, and is surely not up to the quality of Org.
Nevertheless, I much use it even if imperfect, while knowing it should
be improved in many ways.

If we could make something better out of two imperfections, it could be
worth trying.  If only I had more time!  Sigh!

François

Re: [ANN] outorg.el -- reverse Org-Babel

[Bastien]

Hi François and Thorsten,

When great minds meet... ;)  It's sometimes good to have two 
different implementations, ideas can flow from one to another.

If you think one library is mature and useful enough for the
contrib/ directory, please feel free to submit it!

Note that both your libraries (supporting some Org syntax in
comments) are a perfect match for Christopher recent chance
in master, which allows a more powerful orgstruct-mode in
those files -- with folding etc.  If you didn't, have a look:

  C-h v orgstruct-heading-prefix-regexp RET

is a starting point.

Thanks!

-- 
 Bastien

Re: [ANN] outorg.el -- reverse Org-Babel

[Thorsten Jolitz]

Bastien <bzg@altern.org> writes:

Hi Bastien,

> It's sometimes good to have two different implementations, ideas can
> flow from one to another.

I agree ... besides that I did not have any intention to duplicate or
compete, the Org-mode mailing list is just so busy that its hard to keep
up. 

> If you think one library is mature and useful enough for the
> contrib/ directory, please feel free to submit it!

OK

> Note that both your libraries (supporting some Org syntax in
> comments) are a perfect match for Christopher recent chance
> in master, which allows a more powerful orgstruct-mode in
> those files -- with folding etc.  If you didn't, have a look:

I will try to refactor the whole thing so that it becomes independent
from outline-minor-mode. 

>   C-h v orgstruct-heading-prefix-regexp RET

I will have a look, thanks. 

-- 
cheers,
Thorsten

Re: [ANN] outorg.el -- reverse Org-Babel

[Thorsten Jolitz]

Bastien <bzg@altern.org> writes:

Hi Bastien,

> Note that both your libraries (supporting some Org syntax in
> comments) are a perfect match for Christopher recent chance
> in master, which allows a more powerful orgstruct-mode in
> those files -- with folding etc.  If you didn't, have a look:
>
>   C-h v orgstruct-heading-prefix-regexp RET

for equivalence with outline-minor-mode, one would need 

,--------------------------------
| orgstruct-heading-prefix-regexp
| orgstruct-heading-prefix-level
`--------------------------------

just like: 

,-------------------------------------------------------------------------------
| outline-regexp is a variable defined in `outline.el'.
| 
| Regular expression to match the beginning of a heading.
| Any line whose beginning matches this regexp is considered to start a heading.
| Note that Outline mode only checks this regexp at the start of a line,
| so the regexp need not (and usually does not) start with `^'.
| The recommended way to set this is with a Local Variables: list
| in the file it applies to.  See also `outline-heading-end-regexp'.
`-------------------------------------------------------------------------------

,----------------------------------------------------------------------------
| outline-level is a variable defined in `outline.el'.
| 
| *Function of no args to compute a header's nesting level in an outline.
| It can assume point is at the beginning of a header line and that the match
| data reflects the `outline-regexp'.
`----------------------------------------------------------------------------

on the other hand:

,---------------------------------------------------------
| (define-derived-mode org-mode outline-mode "Org"
| [...]
|   (org-set-local 'outline-regexp org-outline-regexp)
|   (org-set-local 'outline-level 'org-outline-level) ...)
`---------------------------------------------------------

I'm not sure if I really understand orgstruct-mode, but it seems to me
that when orgstruct-mode is active in a (e.g. elisp) buffer, all outline
functions and vars are available, and so are org-outline-regexp and
org-outline-level. 

Then a library like outorg could use outline functionality in its
functions and it would work no matter if the buffer is in
outline-minor-mode or in orgstruct-mode. The only change necesary would
be to set outline-regexp and outline-level conditionally on the
minor-mode active. 

Or why not use outline-regexp and outline-level directly, like org.el
does in some places: 

,---------------------------------------------------------------------------
| 6624:            (if (re-search-forward (concat "^" outline-regexp) nil t)
`---------------------------------------------------------------------------

,-----------------------------------------------------
| 36 matches for "[^-]outline-level" in buffer: org.el
`-----------------------------------------------------

I'm not really sure how much of Org-mode and Outline is available when
orgstruct is loaded, and if and why there is a need for
'orgstruct-heading-prefix-regexp' - couldn't orgstruct just set a
buffer-local 'outline-regexp' instead? Some enlightment is welcome. 

-- 
cheers,
Thorsten
 

Re: [ANN] outorg.el -- reverse Org-Babel

[Bastien]

Hi Thorsten,

Thorsten Jolitz <tjolitz@gmail.com> writes:

> Some enlightment is welcome. 

Can't really help here... as I lost track of the exact problem 
you are trying to solve :)

-- 
 Bastien

Re: [ANN] outorg.el -- reverse Org-Babel

[Thorsten Jolitz]

François Pinard <pinard@iro.umontreal.ca> writes:

> poporg has many flaws, and is surely not up to the quality of Org.
> Nevertheless, I much use it even if imperfect, while knowing it should
> be improved in many ways.

same here on the outorg site, although I'm pretty sure its more
alpha-stage. 

but since it is based on such a simple idea (-> design outline-regexp
and outline-level in a way that applying 'comment-region' and
'uncomment-region' to each line does the conversion job between Org and
whatever programming-language syntax) I hope a few bug-fixes and
improvements will move it closer towards version 1.0.

> If we could make something better out of two imperfections, it could be
> worth trying.  If only I had more time!  Sigh!

after a quick look at your code I would say that outorg and poporg are
very different beasts - the effort to merge them would probably be much
higher than to bring them both (independently) to a stable usable state. 

I will do some major changes motivated by this threat, maybe afterwards
it easier to see where the similarities and where the differences are. 

-- 
cheers,
Thorsten

Re: [ANN] outorg.el -- reverse Org-Babel

[Samuel Wales]

This looks great.

One suggestion: what about optionally allowing you to keep your Org
notes in Org?  That would allow all Org features.

You'd do it by putting a specially-formatted Org ID in the comment,
and then you have a key that bounces back and forth.

  ;;; $[id 123451243512345]

Then in Org you have that ID as a property.

That way you can choose whether to have your documentation in the
source file using Org syntax or in Org.

Samuel

-- 
The Kafka Pandemic: http://thekafkapandemic.blogspot.com

The disease DOES progress.  MANY people have died from it.  ANYBODY
can get it.  There is no hope without action.

Re: [ANN] outorg.el -- reverse Org-Babel

[Thorsten Jolitz]

Samuel Wales <samologist@gmail.com> writes:

> One suggestion: what about optionally allowing you to keep your Org
> notes in Org?  That would allow all Org features.
>
> You'd do it by putting a specially-formatted Org ID in the comment,
> and then you have a key that bounces back and forth.
>
>   ;;; $[id 123451243512345]
>
> Then in Org you have that ID as a property.
>
> That way you can choose whether to have your documentation in the
> source file using Org syntax or in Org.

I'm not sure I fully understand this, but working with 'org-id' would
definitely have been another option. Maybe even in combination with
dynamic blocks?

-- 
cheers,
Thorsten

Re: [ANN] outorg.el -- reverse Org-Babel

[Bastien]

Hi Thorsten,

Thorsten Jolitz <tjolitz@gmail.com> writes:

> I'm not sure I fully understand this, but working with 'org-id' would
> definitely have been another option. Maybe even in combination with
> dynamic blocks?

Instead of just supporting new special links in non-org buffers,
maybe we can try to supports links in general.

But at the same time, we don't want everybody on Earth to end up
with strings like [[http://orgmode.org][links]] in the source code
they write/read.  I'd really think twice before implementing such
features.

2 cts of course,

-- 
 Bastien

Re: [ANN] outorg.el -- reverse Org-Babel

[Samuel Wales]

Hi Thorsten,

The idea is to allow the option of keeping your notes in your agenda
files, so that you can do everything Org does.  If you only keep them
in comments, you can't do searches, for example.

To do this, you put an ID in the comments.  Then you have a command
that will take you to the entry with that ID in your agenda files.
The same command -- e.g. C-c ' -- will take you back to the correct
place in the source file.  If you are in the source file, it will
search for the closest ID.

So you can say this:

  (let ((foo bar))
    ;; $[id ...]
    (something-to-be-documented

and then in your Org files you have

  * my elisp notes
  ** something
  :PROPERTIES:
  :ID: ...
  :END:

  my notes

I don't know how dynamic blocks would fit into that.

This puts elisp in elisp and Org in Org.

I like your idea too and consider this to be just an optional feature.

Samuel

On 2/12/13, Thorsten Jolitz <tjolitz@gmail.com> wrote:
> Samuel Wales <samologist@gmail.com> writes:
>
>> One suggestion: what about optionally allowing you to keep your Org
>> notes in Org?  That would allow all Org features.
>>
>> You'd do it by putting a specially-formatted Org ID in the comment,
>> and then you have a key that bounces back and forth.
>>
>>   ;;; $[id 123451243512345]
>>
>> Then in Org you have that ID as a property.
>>
>> That way you can choose whether to have your documentation in the
>> source file using Org syntax or in Org.
>
> I'm not sure I fully understand this, but working with 'org-id' would
> definitely have been another option. Maybe even in combination with
> dynamic blocks?
>
> --
> cheers,
> Thorsten
>
>
>


-- 
The Kafka Pandemic: http://thekafkapandemic.blogspot.com

The disease DOES progress.  MANY people have died from it.  ANYBODY
can get it.  There is no hope without action.

Re: [ANN] outorg.el -- reverse Org-Babel

[Alan Schmitt]

Thorsten Jolitz writes:

> 'outorg' is based on the idea that it would be nice to be able to 
>
> 1. structure your source-code files like Org-mode files and use all the
>    structure editing and navigation commands available. 

Does it mean it can superseed orgstruct-mode, or should one use both?

In any case, I find this and poporg great ideas!

Alan

Re: [ANN] outorg.el -- reverse Org-Babel

[Thorsten Jolitz]

Alan Schmitt <alan.schmitt@polytechnique.org> writes:

>> 'outorg' is based on the idea that it would be nice to be able to 
>>
>> 1. structure your source-code files like Org-mode files and use all the
>>    structure editing and navigation commands available. 
>
> Does it mean it can superseed orgstruct-mode, or should one use both?

'outorg' is actually based on outline-minor-mode. Since all I wanted in
the source-buffers was outline functionality (combined with full
Org-mode functionality in the edit-buffers), I thought all I need is
outline-minor-mode.

But I will try to refactor the whole thing so that it becomes immaterial
if orgstruct-mode or outline-minor-mode is used (I assume there are
equivalents for 'outline-regexp' and 'outline-level' in orgstruct-mode).

-- 
cheers,
Thorsten

Re: [ANN] outorg.el -- reverse Org-Babel

[Bastien]

Hi Thorsten,

Thorsten Jolitz <tjolitz@gmail.com> writes:

> But I will try to refactor the whole thing so that it becomes immaterial
> if orgstruct-mode or outline-minor-mode is used (I assume there are
> equivalents for 'outline-regexp' and 'outline-level' in
> orgstruct-mode).

See `org-outline-regexp' and `org-outline-level'.

HTH,

-- 
 Bastien

Re: [ANN] outorg.el -- reverse Org-Babel

[Thorsten Jolitz]

Bastien <bzg@altern.org> writes:

Hi Bastien,

>> But I will try to refactor the whole thing so that it becomes immaterial
>> if orgstruct-mode or outline-minor-mode is used (I assume there are
>> equivalents for 'outline-regexp' and 'outline-level' in
>> orgstruct-mode).
>
> See `org-outline-regexp' and `org-outline-level'.

just posted a question with regards to this ...

-- 
cheers,
Thorsten