[RFC] org-noweb: org-tangle + org-weave for literate programming using Org

[RFC] org-noweb: org-tangle + org-weave for literate programming using Org

[Nicolas Girard]

Hi folks,

I've released today a couple of tools named "org-tangle" and
"org-weave" in a public GitHub repository:
https://github.com/ngirard/org-noweb

I thought that Org's literate programming abilities deserved to be
made accessible non-interactively from the command-line via "official"
commands, and I hope this repository will help establish them.

The benefits of such an approach I can think of are:
- to provide people with reference tools they can refer to;
- to isolate common behavior from people's customization that can be
repeated consistently and reliably;
- to lower the entry barrier for newcomers;
- to attract the people who might just be interested in Org as a
literate programming tool first, before grasping other areas of the
Org "platform".

Now, I'd be very glad if we could use this code as a starting point
and discuss about it.

For instance:

- What should these tools do, and how ? It seemed to me that they
should only glue functionality from external tools and libraries,
rather than embedding functionality in their cores.

  For instance, if org-weave ever had to gain cross-indexing and
referencing features similar to its oldest siblings (weave, noweave),
it should be achieved by leveraging some dedicated elisp package,
LaTeX style file, or whatever, rather than implementing them itself.

  What do you think ?

- Into which language should these tools be written ? I've chosen the
Bash shell because it's available almost everywhere. Does it seem ok
for you ? Would it be useful to write them in POSIX-compliant code ?

- About =org-tangle=: should it take only one org file, or several
files, as an argument ?

  I gave =org-tangle= the same behaviour than the code snippet that
can be found in Org manual
([[info:org#Batch%20execution][info:org#Batch execution]]), so for now
you can type

  #+begin_src sh
    org-tangle file1.org file2.org
  #+end_src

  But it makes more sense to me, that org-tangle takes only one file,
and optionally the name of a source code block to be extracted, like

  #+begin_src sh
    org-tangle file.org [chunkname]
  #+end_src

  What do you think ?

- About =org-babel-use-quick-and-dirty-noweb-expansion=: should it be
set to 't' by default ? I'd be tempted to say yes, given the dramatic
performance gain

- etc, etc...

Also, while not necessary, I thought it would be nice if org-noweb
tools ate their own dogfoot and extracted themselves ; so I've written
them in literate programming style and I have to say I really enjoyed
the process.

The tools are a mix of elisp and shell code, and it seems to me like
Org + literate style really shine here at making the code readable and
understandable.

That said, there are a few quirks into the code I'd be glad to see disappear:

1. It made sense for me to write the command line options as an Org
table (look at =#tblname: options= in
https://raw.githubusercontent.com/ngirard/org-noweb/master/org-tangle.org)
; but if you see the docstring for the =-E= option, I wanted to write
"Default value is <<emacs>>" but couldn't find a way to get this
reference to expand ; so I had to manually write "Default value is
xxxxx". Any hints ?
2. The boilerplate fonctions =escape-quotes= and =format-options= :
could it be done in a better way ?

Cheers,

Nicolas

Re: [RFC] org-noweb: org-tangle + org-weave for literate programming using Org

[Grant Rettke]

Use Emacs to run Emacs Lisp and set
org-babel-use-quick-and-dirty-noweb-expansion to nil.
Grant Rettke | AAAS, ACM, ASA, FSF, IEEE, SIAM, Sigma Xi
gcr@wisdomandwonder.com | http://www.wisdomandwonder.com/
“Wisdom begins in wonder.” --Socrates
((λ (x) (x x)) (λ (x) (x x)))
“Life has become immeasurably better since I have been forced to stop
taking it seriously.” --Thompson


On Tue, Jun 10, 2014 at 9:46 AM, Nicolas Girard
<girard.nicolas@gmail.com> wrote:
> Hi folks,
>
> I've released today a couple of tools named "org-tangle" and
> "org-weave" in a public GitHub repository:
> https://github.com/ngirard/org-noweb
>
> I thought that Org's literate programming abilities deserved to be
> made accessible non-interactively from the command-line via "official"
> commands, and I hope this repository will help establish them.
>
> The benefits of such an approach I can think of are:
> - to provide people with reference tools they can refer to;
> - to isolate common behavior from people's customization that can be
> repeated consistently and reliably;
> - to lower the entry barrier for newcomers;
> - to attract the people who might just be interested in Org as a
> literate programming tool first, before grasping other areas of the
> Org "platform".
>
> Now, I'd be very glad if we could use this code as a starting point
> and discuss about it.
>
> For instance:
>
> - What should these tools do, and how ? It seemed to me that they
> should only glue functionality from external tools and libraries,
> rather than embedding functionality in their cores.
>
>   For instance, if org-weave ever had to gain cross-indexing and
> referencing features similar to its oldest siblings (weave, noweave),
> it should be achieved by leveraging some dedicated elisp package,
> LaTeX style file, or whatever, rather than implementing them itself.
>
>   What do you think ?
>
> - Into which language should these tools be written ? I've chosen the
> Bash shell because it's available almost everywhere. Does it seem ok
> for you ? Would it be useful to write them in POSIX-compliant code ?
>
> - About =org-tangle=: should it take only one org file, or several
> files, as an argument ?
>
>   I gave =org-tangle= the same behaviour than the code snippet that
> can be found in Org manual
> ([[info:org#Batch%20execution][info:org#Batch execution]]), so for now
> you can type
>
>   #+begin_src sh
>     org-tangle file1.org file2.org
>   #+end_src
>
>   But it makes more sense to me, that org-tangle takes only one file,
> and optionally the name of a source code block to be extracted, like
>
>   #+begin_src sh
>     org-tangle file.org [chunkname]
>   #+end_src
>
>   What do you think ?
>
> - About =org-babel-use-quick-and-dirty-noweb-expansion=: should it be
> set to 't' by default ? I'd be tempted to say yes, given the dramatic
> performance gain
>
> - etc, etc...
>
> Also, while not necessary, I thought it would be nice if org-noweb
> tools ate their own dogfoot and extracted themselves ; so I've written
> them in literate programming style and I have to say I really enjoyed
> the process.
>
> The tools are a mix of elisp and shell code, and it seems to me like
> Org + literate style really shine here at making the code readable and
> understandable.
>
> That said, there are a few quirks into the code I'd be glad to see disappear:
>
> 1. It made sense for me to write the command line options as an Org
> table (look at =#tblname: options= in
> https://raw.githubusercontent.com/ngirard/org-noweb/master/org-tangle.org)
> ; but if you see the docstring for the =-E= option, I wanted to write
> "Default value is <<emacs>>" but couldn't find a way to get this
> reference to expand ; so I had to manually write "Default value is
> xxxxx". Any hints ?
> 2. The boilerplate fonctions =escape-quotes= and =format-options= :
> could it be done in a better way ?
>
> Cheers,
>
> Nicolas
>

Re: [RFC] org-noweb: org-tangle + org-weave for literate programming using Org

[Waldemar Quevedo]

[-- Attachment #1: Type: text/plain, Size: 3842 bytes --]

Hi Nicolas,

This is the right direction I think. I added a similar command to a project
that I've been working but I use the Ruby parser so the functionality is
less: https://github.com/wallyqs/org-converge/blob/master/bin/org-tangle

So... I have a added a link to your project.

Thanks for sharing!

- Wally


On Tue, Jun 10, 2014 at 11:46 PM, Nicolas Girard <girard.nicolas@gmail.com>
wrote:

> Hi folks,
>
> I've released today a couple of tools named "org-tangle" and
> "org-weave" in a public GitHub repository:
> https://github.com/ngirard/org-noweb
>
> I thought that Org's literate programming abilities deserved to be
> made accessible non-interactively from the command-line via "official"
> commands, and I hope this repository will help establish them.
>
> The benefits of such an approach I can think of are:
> - to provide people with reference tools they can refer to;
> - to isolate common behavior from people's customization that can be
> repeated consistently and reliably;
> - to lower the entry barrier for newcomers;
> - to attract the people who might just be interested in Org as a
> literate programming tool first, before grasping other areas of the
> Org "platform".
>
> Now, I'd be very glad if we could use this code as a starting point
> and discuss about it.
>
> For instance:
>
> - What should these tools do, and how ? It seemed to me that they
> should only glue functionality from external tools and libraries,
> rather than embedding functionality in their cores.
>
>   For instance, if org-weave ever had to gain cross-indexing and
> referencing features similar to its oldest siblings (weave, noweave),
> it should be achieved by leveraging some dedicated elisp package,
> LaTeX style file, or whatever, rather than implementing them itself.
>
>   What do you think ?
>
> - Into which language should these tools be written ? I've chosen the
> Bash shell because it's available almost everywhere. Does it seem ok
> for you ? Would it be useful to write them in POSIX-compliant code ?
>
> - About =org-tangle=: should it take only one org file, or several
> files, as an argument ?
>
>   I gave =org-tangle= the same behaviour than the code snippet that
> can be found in Org manual
> ([[info:org#Batch%20execution][info:org#Batch execution]]), so for now
> you can type
>
>   #+begin_src sh
>     org-tangle file1.org file2.org
>   #+end_src
>
>   But it makes more sense to me, that org-tangle takes only one file,
> and optionally the name of a source code block to be extracted, like
>
>   #+begin_src sh
>     org-tangle file.org [chunkname]
>   #+end_src
>
>   What do you think ?
>
> - About =org-babel-use-quick-and-dirty-noweb-expansion=: should it be
> set to 't' by default ? I'd be tempted to say yes, given the dramatic
> performance gain
>
> - etc, etc...
>
> Also, while not necessary, I thought it would be nice if org-noweb
> tools ate their own dogfoot and extracted themselves ; so I've written
> them in literate programming style and I have to say I really enjoyed
> the process.
>
> The tools are a mix of elisp and shell code, and it seems to me like
> Org + literate style really shine here at making the code readable and
> understandable.
>
> That said, there are a few quirks into the code I'd be glad to see
> disappear:
>
> 1. It made sense for me to write the command line options as an Org
> table (look at =#tblname: options= in
> https://raw.githubusercontent.com/ngirard/org-noweb/master/org-tangle.org)
> ; but if you see the docstring for the =-E= option, I wanted to write
> "Default value is <<emacs>>" but couldn't find a way to get this
> reference to expand ; so I had to manually write "Default value is
> xxxxx". Any hints ?
> 2. The boilerplate fonctions =escape-quotes= and =format-options= :
> could it be done in a better way ?
>
> Cheers,
>
> Nicolas
>
>

[-- Attachment #2: Type: text/html, Size: 5071 bytes --]

Re: [RFC] org-noweb: org-tangle + org-weave for literate programming using Org

[Nicolas Girard]

2014-06-10 19:54 GMT+02:00 Grant Rettke <gcr@wisdomandwonder.com>:
> On Tue, Jun 10, 2014 at 9:46 AM, Nicolas Girard
> <girard.nicolas@gmail.com> wrote:
>>
>> - About =org-babel-use-quick-and-dirty-noweb-expansion=:
>>   should it be set to 't' by default ? I'd be tempted to say yes,
>>   given the dramatic performance gain
> Use Emacs to run Emacs Lisp and set
> org-babel-use-quick-and-dirty-noweb-expansion to nil.

Hi Grant,

thanks for your reply. There might be a slight misunderstanding though.
My question was: should org-tangle and org-weave enable
"org-babel-use-quick-and-dirty-noweb-expansion" before doing their
jobs ? For now I let the default value to be =nil=, and I was
wondering if it wouldn't be bette to do the opposite instead, that is,
enable "quick-and-dirty-noweb-expansion" by default and provide a
-noquick option.
What do you think ?

Cheers,
Nicolas

Re: [RFC] org-noweb: org-tangle + org-weave for literate programming using Org

[Grant Rettke]

Understood.

The quick and dirty approach removes the ability for headings to
inherit the noweb properties of, and override, the properties of its
parent header.

That feature enables the true literate programming to remove it would
be a great loss.
Grant Rettke | AAAS, ACM, ASA, FSF, IEEE, SIAM, Sigma Xi
gcr@wisdomandwonder.com | http://www.wisdomandwonder.com/
“Wisdom begins in wonder.” --Socrates
((λ (x) (x x)) (λ (x) (x x)))
“Life has become immeasurably better since I have been forced to stop
taking it seriously.” --Thompson


On Wed, Jun 11, 2014 at 3:18 AM, Nicolas Girard
<girard.nicolas@gmail.com> wrote:
> 2014-06-10 19:54 GMT+02:00 Grant Rettke <gcr@wisdomandwonder.com>:
>> On Tue, Jun 10, 2014 at 9:46 AM, Nicolas Girard
>> <girard.nicolas@gmail.com> wrote:
>>>
>>> - About =org-babel-use-quick-and-dirty-noweb-expansion=:
>>>   should it be set to 't' by default ? I'd be tempted to say yes,
>>>   given the dramatic performance gain
>> Use Emacs to run Emacs Lisp and set
>> org-babel-use-quick-and-dirty-noweb-expansion to nil.
>
> Hi Grant,
>
> thanks for your reply. There might be a slight misunderstanding though.
> My question was: should org-tangle and org-weave enable
> "org-babel-use-quick-and-dirty-noweb-expansion" before doing their
> jobs ? For now I let the default value to be =nil=, and I was
> wondering if it wouldn't be bette to do the opposite instead, that is,
> enable "quick-and-dirty-noweb-expansion" by default and provide a
> -noquick option.
> What do you think ?
>
> Cheers,
> Nicolas