[Babel] Deep interest in LP feature of org-mode

[Babel] Deep interest in LP feature of org-mode

[Olaf.Hamann]

Hello all,

this starts with kind of an introduction to the list as I will post more 
in future
and leads to a few concept questions concerning org-babel-detangle at 
the end.


First of all many thanks to the authors of the babel package!
Integrating babel into core orgmode made me discovering it.

This comes to the right time because I'm very interested in "Literate 
Programming" and have to leave my beloved LP tool.

I use LP techniques when putting scripts coded in different languages 
together to solve a certain task,
as it is normal when creating media output with a collection of 
different command line tools.

LP helps me to present code sniplets of different languages together in 
one place according to the concept for solving the task,
totally ignoring needed position in procedural code files.
So later adding a feature is easy as participated code parts are still 
in one place, no need to search through the diverse script files.

I worked with funnelweb (http://www.ross.net/funnelweb/) which is really 
nice because of its very precise syntax
  (one can put together binary data with tangeling), simplicity of 
concept, expressive error messages,  stability in performance and 
scalability.

But I have to leave that behind because UTF-8 is not supported and I'm 
no C programmer, so I can't help myself to adapt that tool.
(for rewriting the engine in emacs-lisp I'm unfortunately not smart enough)

So this weekend I started to explore org-babel for my purposes.
I played a little bit around with changing a .fw-Script so that it works 
as an .org-Script with babel-blocks.
After hacking in the macro-expansion I was able to tangle the files 
identical to former .fw-output.

Changing the org-file to add features or change some code was really fun!
Needed org-syntax for that is simple enough, possibility of inherited 
tree properties to store the header arguments is great.
Performance looks good to me, although the file I played with was not 
that big.

So I look forward to work on with org-babel and especially have interest 
to benefit from org-babel-detangle.
At the moment that is not working for me because of it's lack of 
supporting nested src_blocks (using noweb syntax).


I read the posted suggestion for a kind of spidering version of 
org-babel-detangle (thanks to Seth Burleigh ).
There occur some questions to me as I'm not that intimate with the 
concepts behind org-babel at the moment.

* How would one handle conflicts?
   (e.g. different changes in tangled code which all reference to one 
begin_src block)

* Is it possible to provide functionality like the org-mode macro 
expansion mechanism?
    (How to distinguish macro-expanded values from changed code)

* Would it be possible to take over more org-mode export features into 
src-blocks (calculating, calling blocks inside blocks)

And, more in general:
* When thinking of LP (preprocessing text) with org-mode
   it's hard for me to compare the pros and cons of
    exporting the org-file to ascii
    vs. tangling it to code-file.

* Is it possible, that I use the wrong approach with org-babel when I 
only need text processing features
    !! Of course, using src blocks to call extern interpreter over 
several code blocks
        and keeping the same session is really a great thing !!
        The document provided on the website is very impressive.
         ( 
http://repo.or.cz/w/Worg.git/blob_plain/HEAD:/org-contrib/babel/examples/foo.pdf 
)

* Where is the right place to discover more about org-babel:
    http://orgmode.org or http://eschulte.github.com/babel-dev/  ?
    or both ?

* Where are the latest code changes offered?



Greetings,
Olaf

Re: [Babel] Deep interest in LP feature of org-mode

[Eric Schulte]

Hi Olaf,

It is great to hear that you are enjoying LP with Org-mode.

I have some comments/answers inline below.

Best -- Eric

"Olaf.Hamann" <O.Hamann@gmx.net> writes:

> Hello all,
>
> this starts with kind of an introduction to the list as I will post
> more in future
> and leads to a few concept questions concerning org-babel-detangle at
> the end.
>
>
> First of all many thanks to the authors of the babel package!
> Integrating babel into core orgmode made me discovering it.
>
> This comes to the right time because I'm very interested in "Literate
> Programming" and have to leave my beloved LP tool.
>
> I use LP techniques when putting scripts coded in different languages
> together to solve a certain task,
> as it is normal when creating media output with a collection of
> different command line tools.
>
> LP helps me to present code sniplets of different languages together
> in one place according to the concept for solving the task,
> totally ignoring needed position in procedural code files.
> So later adding a feature is easy as participated code parts are still
> in one place, no need to search through the diverse script files.
>
> I worked with funnelweb (http://www.ross.net/funnelweb/) which is
> really nice because of its very precise syntax
>  (one can put together binary data with tangeling), simplicity of
> concept, expressive error messages,  stability in performance and
> scalability.
>
> But I have to leave that behind because UTF-8 is not supported and I'm
> no C programmer, so I can't help myself to adapt that tool.
> (for rewriting the engine in emacs-lisp I'm unfortunately not smart enough)
>
> So this weekend I started to explore org-babel for my purposes.
> I played a little bit around with changing a .fw-Script so that it
> works as an .org-Script with babel-blocks.
> After hacking in the macro-expansion I was able to tangle the files
> identical to former .fw-output.
>
> Changing the org-file to add features or change some code was really fun!
> Needed org-syntax for that is simple enough, possibility of inherited
> tree properties to store the header arguments is great.
> Performance looks good to me, although the file I played with was not
> that big.
>
> So I look forward to work on with org-babel and especially have
> interest to benefit from org-babel-detangle.
> At the moment that is not working for me because of it's lack of
> supporting nested src_blocks (using noweb syntax).
>
>
> I read the posted suggestion for a kind of spidering version of
> org-babel-detangle (thanks to Seth Burleigh ).
> There occur some questions to me as I'm not that intimate with the
> concepts behind org-babel at the moment.
>
> * How would one handle conflicts?
>   (e.g. different changes in tangled code which all reference to one
> begin_src block)
>

There is currently no mechanism in place for handling conflicts, rather
the contents from the tangled source file simply overwrites the contents
of the code block.  This would severely impede the utility of
de-tangling for code blocks making use of variable noweb or macro
expansion.

While it should be possible to implement a sane handling of such things
(as has been discussed previously WRT noweb) the implementation will not
be a trivial task.

>
> * Is it possible to provide functionality like the org-mode macro
> expansion mechanism?
>    (How to distinguish macro-expanded values from changed code)
>

See my recent email for an implementation of macro expansion.  The
resolution of expanded macros when detangling code would not be easy...

>
> * Would it be possible to take over more org-mode export features into
> src-blocks (calculating, calling blocks inside blocks)
>

Not sure exactly what you mean here, could you provide a concrete
example?

>
> And, more in general:
> * When thinking of LP (preprocessing text) with org-mode
>   it's hard for me to compare the pros and cons of
>    exporting the org-file to ascii
>    vs. tangling it to code-file.
>

These should be different use cases.  Exporting (weaving in LP syntax)
is generally for creating a document view for reading by humans, while
tangling to source code is for organizing a document for consumption by
machine.

>
> * Is it possible, that I use the wrong approach with org-babel when I
> only need text processing features

That is possible, if you only need text-processing, then you probably
shouldn't be calling code blocks, rather you should simply type into an
Org-mode buffer and then export to your desired back-end, and let
Org-mode handle all formatting.  See the relevant part of the Org-mode
manual http://orgmode.org/manual/Exporting.html

> 
>    !! Of course, using src blocks to call extern interpreter over
> several code blocks
>        and keeping the same session is really a great thing !!
>        The document provided on the website is very impressive.
>         (http://repo.or.cz/w/Worg.git/blob_plain/HEAD:/org-contrib/babel/examples/foo.pdf)
>
> * Where is the right place to discover more about org-babel:
>    http://orgmode.org or http://eschulte.github.com/babel-dev/  ?
>    or both ?
>

The best Babel resource is the manual (readable through Emacs or Online).
http://orgmode.org/manual/Working-With-Source-Code.html

>
> * Where are the latest code changes offered?
>

Your best bet is to track the Org-mode git repository, this is where all
new code is released.  See http://orgmode.org/index.html#sec-3_2

Cheers -- Eric

>
>
>
> Greetings,
> Olaf

Re: [Babel] Deep interest in LP feature of org-mode

[Olaf.Hamann]

Thank you for your quick reply and the precise answers, Eric.

Comparing org-export vs. org-tangle I've mixed up how I used text 
processing:
* for myself I often use it simply for creating codefiles (tangling)  - 
org-export would be sufficient
* for tasks in office there is need for creating a nice doc file, too 
(weaving) - org-babel for code creation
Thank you for making that clear to me.

I will revisit my nebulous feature request of calling blocks inside 
blocks - there is no actual need, as you identified.

Greetings,
Olaf