Literate Programming - Continue a Source Block?

Literate Programming - Continue a Source Block?

[Neeum Zawan]

Hi,

With noweb, one can continue a source block that one started
earlier. Can this not be done with Babel?

If not, I'm struggling a little with how to do LP using Babel...

Thanks.

Re: Literate Programming - Continue a Source Block?

[Sebastien Vauban]

Hi Neeum,

Neeum Zawan wrote:
> With noweb, one can continue a source block that one started
> earlier. Can this not be done with Babel?
>
> If not, I'm struggling a little with how to do LP using Babel...

Of course, this can be done here as well: simply reuse the same "tangle"
target (file), and blocks are concatenated in the order they appear.

Second solution: create one sole block that will be tangled, and which
contains your other blocks (using the <<ref>> syntax), in the order you want.

Best regards,
  Seb

-- 
Sebastien Vauban

Re: Literate Programming - Continue a Source Block?

[Neeum Zawan]

"Sebastien Vauban"
<wxhgmqzgwmuf@spammotel.com> writes:

> Hi Neeum,
>
> Neeum Zawan wrote:
>> With noweb, one can continue a source block that one started
>> earlier. Can this not be done with Babel?
>>
>> If not, I'm struggling a little with how to do LP using Babel...
>
> Of course, this can be done here as well: simply reuse the same "tangle"
> target (file), and blocks are concatenated in the order they appear.

But this will only allow me to append to the end of the file (up to that
point), correct? I want to append to a source block that may be in the
middle. Consider the following example:

(Begin-example)

#+BABEL: :noweb yes

* Outline

The structure of the configuration file will be:


#+srcname: outline
#+begin_src emacs-lisp :tangle .emacs :exports code 
;; Store configurations that impact appearance
<<visual-config>>

;; Languages configurations
<<lang-config>>

;; GNUS configurations
<<gnus-config>>

#+end_src

* General Appearance

This section controls the general appearance (color of cursor, fonts,
etc).

#+srcname: visual-config
#+begin_src emacs-lisp :exports code 
;; Config here.

#+end_src

* GNUS configuration

#+srcname: gnus-config
#+begin_src emacs-lisp :exports code 
;; Config here.

#+end_src

* Languages

#+srcname: lang-config
#+begin_src emacs-lisp :exports code 

<<python-config>>

<<perl-config>>
#+end_src

** Python

Now here's where the problem begins. I want to add some configurations
that impact some Python settings. I can dump those in
python-config. However, I also want to add some visual settings for
python-mode (change font colors, etc). For whatever reason, I think that
part should go into visual-config.

How do I now append the relevant portion to visual-config while I'm in
_this_ portion of the org document?

(End-example)

The above is somewhat artificial, but in a proper programming project
something like this will occur frequently: A new feature will be added
at some later point and I'll want to update various blocks of code.

> Second solution: create one sole block that will be tangled, and which
> contains your other blocks (using the <<ref>> syntax), in the order you want.

I had thought of this, but I find it somewhat lacking. Consider my
example above. I could have created a <<visual-python>> in my
<<visual-config>> block. However:

1. That requires me to know I'm going to need it later when I write
visual-config. 

2. If I didn't know I'd need it, I'd have to continually modify various
parts of the org document every time I add a new feature to my code. I
find this suboptimal and error prone.

3. For me, one of the main motivations to do LP is to document the
evolution of the project, such that someone else can read the document
and understand. Part of this is that I want all aspects of a single
feature to appear under a single heading. When the reader reads the
document, he shouldn't have to see too many aspects of the code early on
that will be explained much later in the document.

Now the original noweb allows what I'm asking for. If you begin a source
block with a name of an existing block but append an "=" symbol, it
knows to append to that source block.

It would be great if org-mode could add that capability. Another
approach is that if multiple source blocks with the same name are found,
the default behavior is to concatenate all those source blocks together
(and then add a header option for overwrite if the user wanted to
overwrite instead of append). 

Thoughts?

Re: Literate Programming - Continue a Source Block?

[Eric Schulte]

>
> The above is somewhat artificial, but in a proper programming project
> something like this will occur frequently: A new feature will be added
> at some later point and I'll want to update various blocks of code.
>

Currently the best method is that suggested previously/below of using
named references in the target code block, as suggested below.  While it
shouldn't be overly difficult to add the behavior your described with
something like a ":noweb-append" header argument, e.g.,

  #+begin_src emacs-lisp :noweb-append visual-config
   ...
  #+end_src

This behavior is not currently implemented.

>
>> Second solution: create one sole block that will be tangled, and which
>> contains your other blocks (using the <<ref>> syntax), in the order you want.
>
> I had thought of this, but I find it somewhat lacking. Consider my
> example above. I could have created a <<visual-python>> in my
> <<visual-config>> block. However:
>
> 1. That requires me to know I'm going to need it later when I write
>    visual-config. 
>

nit picking here, but while this does require a small edit to
<<visual-config>>, there is no need for prior knowledge of the need for
<<visual-python>>.

>
> 2. If I didn't know I'd need it, I'd have to continually modify various
> parts of the org document every time I add a new feature to my code. I
> find this suboptimal and error prone.
>

Technically only one edit per new block introduced, which does not seem
overly onerous.

>
> 3. For me, one of the main motivations to do LP is to document the
> evolution of the project, such that someone else can read the document
> and understand. Part of this is that I want all aspects of a single
> feature to appear under a single heading. When the reader reads the
> document, he shouldn't have to see too many aspects of the code early on
> that will be explained much later in the document.
>

I agree, this is a motivating example.

>
> Now the original noweb allows what I'm asking for. If you begin a
> source block with a name of an existing block but append an "="
> symbol, it knows to append to that source block.
>
> It would be great if org-mode could add that capability.

I agree, and the functionality you describe shouldn't be overly
difficult to implement.

I like the concision of the "=original-name" syntax used by noweb, but I
would lean towards the use of a ":noweb-append" type header argument as
suggested above because currently the names of blocks in Babel carry no
semantic content and I'd prefer to leave it this way.

> Another approach is that if multiple source blocks with the same name
> are found, the default behavior is to concatenate all those source
> blocks together (and then add a header option for overwrite if the
> user wanted to overwrite instead of append).
>
> Thoughts?
>

Thanks for the motivating example and the thorough explanation of
behavior.

I'll certainly put this on my long-term development queue, however, that
does not guarantee an implementation in the near future.  If anyone is
interested in this functionality and is up for writing some elisp I am
happy to offer advice and code pointers immediately.

Best -- Eric

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Literate Programming - Continue a Source Block?

[Neeum Zawan]

Eric Schulte <schulte.eric@gmail.com> writes:

>>> Second solution: create one sole block that will be tangled, and which
>>> contains your other blocks (using the <<ref>> syntax), in the order you want.
>>
>> I had thought of this, but I find it somewhat lacking. Consider my
>> example above. I could have created a <<visual-python>> in my
>> <<visual-config>> block. However:
>>
>> 1. That requires me to know I'm going to need it later when I write
>>    visual-config. 
>>
>
> nit picking here, but while this does require a small edit to
> <<visual-config>>, there is no need for prior knowledge of the need for
> <<visual-python>>.
>
>>
>> 2. If I didn't know I'd need it, I'd have to continually modify various
>> parts of the org document every time I add a new feature to my code. I
>> find this suboptimal and error prone.
>>
>
> Technically only one edit per new block introduced, which does not seem
> overly onerous.

In this case, yes. In a real programming project, it could be a number
of them. For example, I may have a code block dedicated to
imports/includes which I want to be on the top of the file - and I may
have to append to that when adding a new feature. And then the actual
code for the new feature may require edits to other parts of one's
program. Ideally, everything should be decoupled, but reality rarely
follows the ideal ;-)

>> Now the original noweb allows what I'm asking for. If you begin a
>> source block with a name of an existing block but append an "="
>> symbol, it knows to append to that source block.
>>
>> It would be great if org-mode could add that capability.
>
> I agree, and the functionality you describe shouldn't be overly
> difficult to implement.
>
> I like the concision of the "=original-name" syntax used by noweb, but I
> would lean towards the use of a ":noweb-append" type header argument as
> suggested above because currently the names of blocks in Babel carry no
> semantic content and I'd prefer to leave it this way.

I suppose it may also break compatibility in case someone out there uses
the =symbol.

Had it been thought of earlier, I would have preferred the default
behavior being append if you have multiple blocks of the same name, and
an explicit option *not* to append but to overwrite, but your idea makes
the most sense with respect to preserving backward compatibility. 

In addition to append, there probably should be another option for
overwriting instead of appending (neither is possible right now).

Also, just on the side, I'm not sure it's documented anywhere what
happens if you have multiple source code blocks of the same name. At the
moment, it seems only the first is used (I would have expected the
last). 

> Thanks for the motivating example and the thorough explanation of
> behavior.
>
> I'll certainly put this on my long-term development queue, however, that
> does not guarantee an implementation in the near future.  If anyone is
> interested in this functionality and is up for writing some elisp I am
> happy to offer advice and code pointers immediately.

Wish I knew elisp. Anyway, hopefully someone will get it done one day.

Thanks.

Re: Literate Programming - Continue a Source Block?

[Avdi Grimm]

On Wed, Jun 8, 2011 at 5:20 PM, Neeum Zawan <mailinglists@nawaz.org> wrote:
> n this case, yes. In a real programming project, it could be a number
> of them. For example, I may have a code block dedicated to
> imports/includes which I want to be on the top of the file - and I may
> have to append to that when adding a new feature.

Ooh! Ooh! Me too!

I was just wanting this the other day. I wanted to be able to do:

    #+srcname: requires
    #+begin_src ruby
      require 'library_a'
    #+end_src

     ...

    #+srcname: requires
    #+begin_src ruby
      require 'library_b'
    #+end_src

And then tie it all together with:

    #+begin_src ruby :tangle foo.rb
      <<requires>>
    #+end

But no such luck. <<requires>> just picked up the most recent block
with that srcname, IIRC.

-- 
Avdi Grimm
http://avdi.org

Re: Literate Programming - Continue a Source Block?

[Eric Schulte]

>>
>> I like the concision of the "=original-name" syntax used by noweb, but I
>> would lean towards the use of a ":noweb-append" type header argument as
>> suggested above because currently the names of blocks in Babel carry no
>> semantic content and I'd prefer to leave it this way.
>
> I suppose it may also break compatibility in case someone out there uses
> the =symbol.
>
> Had it been thought of earlier, I would have preferred the default
> behavior being append if you have multiple blocks of the same name, and
> an explicit option *not* to append but to overwrite, but your idea makes
> the most sense with respect to preserving backward compatibility. 
>
> In addition to append, there probably should be another option for
> overwriting instead of appending (neither is possible right now).
>

I've just pushed up a patch which implements optional block combination
during tangling.  Specifically a new customization variable named
`org-babel-tangle-named-block-combination' is introduced which can take
the following values

nil    the default, no behavior is changed

append the bodies of all blocks of the same name are appended
       during tangling

first  only the body of the first block of any given name is kept
       during tangling

last   only the body of the last block of any given name is kept during
       tangling

>
> Also, just on the side, I'm not sure it's documented anywhere what
> happens if you have multiple source code blocks of the same name. At the
> moment, it seems only the first is used (I would have expected the
> last). 
>

Yes, currently block names are intended to be unique, and some of the
Babel functionality (e.g., named block evaluation) make this assumption.

The behavior of multiple blocks with the same name is undefined
behavior.  I've expanded the relevant documentation.

>
>> Thanks for the motivating example and the thorough explanation of
>> behavior.
>>
>> I'll certainly put this on my long-term development queue, however, that
>> does not guarantee an implementation in the near future.  If anyone is
>> interested in this functionality and is up for writing some elisp I am
>> happy to offer advice and code pointers immediately.
>
> Wish I knew elisp. Anyway, hopefully someone will get it done one day.
>

Hopefully this gets at the behavior you're after.  I'd be interested to
hear any thought you have on this new functionality.

Cheers -- Eric

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Literate Programming - Continue a Source Block?

[Achim Gratz]

Eric Schulte <schulte.eric@gmail.com> writes:

> append the bodies of all blocks of the same name are appended
>        during tangling

Shouldn't this be called "concat(enate)" or "collate"?  Or, since
several blocks with the same name seem a bit dubious, would it not be
cleaner to have an index part to the block name and a range expression
for the concatenation during tangling?  I might want to tangle them in
different order than their appearance in the source, for instance.


Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Literate Programming - Continue a Source Block?

[Eric Schulte]

Achim Gratz <Stromeko@nexgo.de> writes:

> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> append the bodies of all blocks of the same name are appended
>>        during tangling
>
> Shouldn't this be called "concat(enate)" or "collate"?

I think "append" is just as clear as concatenate, and collate implies
shuffling which is not happening.

> Or, since several blocks with the same name seem a bit dubious, would
> it not be cleaner to have an index part to the block name and a range
> expression for the concatenation during tangling?  I might want to
> tangle them in different order than their appearance in the source,
> for instance.
>

I'm wary of adding too much duplicate functionality.  It is already
possible to organize the tangling of many named code blocks using noweb
reference expansion (a feature which I've used myself on real projects
in the past).  This existing method allows for unique block names and
for arbitrary tangling order.

Simplicity is the only reason that the new name-based appending behavior
was implemented, simplicity which (in my opinion) is lost when the user
defines a naming and sorting schema.

Cheers -- Eric

>
>
> Achim.

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Literate Programming - Continue a Source Block?

[Achim Gratz]

Eric Schulte <schulte.eric@gmail.com> writes:
> I think "append" is just as clear as concatenate,

Fair enough, in my mind "append" needs an existing entity, but I guess
it could be an empty one.

> and collate implies shuffling which is not happening.

Well, I was getting ahead of myself...  I hope you don't mind me harping
on about this.

> I'm wary of adding too much duplicate functionality.  It is already
> possible to organize the tangling of many named code blocks using noweb
> reference expansion (a feature which I've used myself on real projects
> in the past).  This existing method allows for unique block names and
> for arbitrary tangling order.

The examples presented in this thread pointed out one difficulty with
how noweb references currently work, namely that you need to know in
advance all block names for finally tangling them and that when adding
or removing things you need to remember to keep these references current
in potentially many places.

> Simplicity is the only reason that the new name-based appending behavior
> was implemented, simplicity which (in my opinion) is lost when the user
> defines a naming and sorting schema.

My general objection is that since you now require that the names be all
the same, there is no way to distinguish between those blocks anymore.
This will come back to bite you when you then later need to make a
distinction since you'll then have to rework everything to unique names
again.  I consider that sort of simplicity a trap for the unwary.

Splitting between a basename and an index extension would instead allow
for appending with the basename and still getting at individual blocks
using the full name (with the index part).  I've had one specific
use-case in mind where this would be needed.  You'd normally just
concatenate all source blocks (let's assume for the moment that the
index part is separated from the basename by "::")

  <<source::*>>

Let's say you have three source blocks and need to point-patch the
second one for this run:

  <<source::1>>
  <<source-patch::2>>
  <<source::3>>

If the three source blocks are all just named "source", there's no way
to do this short of changing their names and all the references that use
them.

The index part doesn't have to be numeric, it just needs to be
orderable.  Anyone not using index extensions gets an implicit one by
order of appearance, so that wouldn't require any code duplication and
you can in this case still say

  <<source>>

Being able to provide custom naming and sorting schemes could be
deferred to hook functions the user has to provide, as they will indeed
be rarely needed.  So an implementation that keeps current behaviour
would provide an implicit indexer (by order of appearance), an empty
sorter and three selectors (all=append, first last).  Anyone who needs
more than that has to customize those functions, but I suppose the more
common application scenarios will sneak into the baseline sooner or
later.  The move from implicit to explicit indexing should be supported
by providing another hook in the implicit indexer.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: Literate Programming - Continue a Source Block?

[Eric Schulte]

Achim Gratz <Stromeko@nexgo.de> writes:

> Eric Schulte <schulte.eric@gmail.com> writes:
>> I think "append" is just as clear as concatenate,
>
> Fair enough, in my mind "append" needs an existing entity, but I guess
> it could be an empty one.
>
>> and collate implies shuffling which is not happening.
>
> Well, I was getting ahead of myself...  I hope you don't mind me harping
> on about this.
>
>> I'm wary of adding too much duplicate functionality.  It is already
>> possible to organize the tangling of many named code blocks using noweb
>> reference expansion (a feature which I've used myself on real projects
>> in the past).  This existing method allows for unique block names and
>> for arbitrary tangling order.
>
> The examples presented in this thread pointed out one difficulty with
> how noweb references currently work, namely that you need to know in
> advance all block names for finally tangling them and that when adding
> or removing things you need to remember to keep these references current
> in potentially many places.
>
>> Simplicity is the only reason that the new name-based appending behavior
>> was implemented, simplicity which (in my opinion) is lost when the user
>> defines a naming and sorting schema.
>
> My general objection is that since you now require that the names be all
> the same, there is no way to distinguish between those blocks anymore.
> This will come back to bite you when you then later need to make a
> distinction since you'll then have to rework everything to unique names
> again.  I consider that sort of simplicity a trap for the unwary.
>
> Splitting between a basename and an index extension would instead allow
> for appending with the basename and still getting at individual blocks
> using the full name (with the index part).  I've had one specific
> use-case in mind where this would be needed.  You'd normally just
> concatenate all source blocks (let's assume for the moment that the
> index part is separated from the basename by "::")
>
>   <<source::*>>
>
> Let's say you have three source blocks and need to point-patch the
> second one for this run:
>
>   <<source::1>>
>   <<source-patch::2>>
>   <<source::3>>
>
> If the three source blocks are all just named "source", there's no way
> to do this short of changing their names and all the references that use
> them.
>
> The index part doesn't have to be numeric, it just needs to be
> orderable.  Anyone not using index extensions gets an implicit one by
> order of appearance, so that wouldn't require any code duplication and
> you can in this case still say
>
>   <<source>>
>
> Being able to provide custom naming and sorting schemes could be
> deferred to hook functions the user has to provide, as they will indeed
> be rarely needed.  So an implementation that keeps current behaviour
> would provide an implicit indexer (by order of appearance), an empty
> sorter and three selectors (all=append, first last).  Anyone who needs
> more than that has to customize those functions, but I suppose the more
> common application scenarios will sneak into the baseline sooner or
> later.  The move from implicit to explicit indexing should be supported
> by providing another hook in the implicit indexer.
>
>
> Regards,
> Achim.

Hi Achim,

Thanks for sharing your thoughts.  How would you feel about moving away
from special source block names and moving towards implementing this
behavior with a header argument?  Specifically two header arguments
which would both take the name of a named code block, these header
arguments would be as follows

- append :: will append the current code block's body to the named code
            block during tangling

- overwrite :: will overwrite the named code block's body with the body
               of the current code block

for simple concatenate the value of the append header argument could be
easily set on the file or subtree level.

I feel that for many reasons (most of which have been discussed in
relation to a header argument solution earlier in this thread) such a
solution would be simpler and more consistent with the rest of Babel
than the current name-based solution.

Thoughts? -- Eric

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Literate Programming - Continue a Source Block?

[Achim Gratz]

Eric Schulte <schulte.eric@gmail.com> writes:
> Thanks for sharing your thoughts.  How would you feel about moving away
> from special source block names and moving towards implementing this
> behavior with a header argument?

I'm not feeling strongly either way... I'm wanting to use Babel for some
stuff at work, but haven't got around to learn it well enough to set
things in motion.  So my current investment is still small.  :-)

>  Specifically two header arguments
> which would both take the name of a named code block, these header
> arguments would be as follows
>
> - append :: will append the current code block's body to the named code
>             block during tangling
>
> - overwrite :: will overwrite the named code block's body with the body
>                of the current code block
>
> for simple concatenate the value of the append header argument could be
> easily set on the file or subtree level.

This looks good, but it sort of turns the tables (I hope I get this
right, it was a longer day than I hoped).  With noweb references, you
need to resolve the name of the source block at the target.  With the
header arguments you propose you need to resolve the target block name
at the source.  Both ways are useful, but it seems that it would be
difficult to track all targets from a single source even if supposedly
one could have a list of targets (that would only be useful for
"overwrite" unless you are really lucky).  This asymmetry is
inescapeable when you try to use source blocks like "include files".
Another problem arises when you need to (again) reorder source blocks at
the target.  You can't do that from the source, since not only would the
source then need to know all targets, it would also need to know all
other sources and their ordering in all targets.

> I feel that for many reasons (most of which have been discussed in
> relation to a header argument solution earlier in this thread) such a
> solution would be simpler and more consistent with the rest of Babel
> than the current name-based solution.

I'd think if there was an API to get at the contents of the header
arguments (perhaps using or modeled after the properties API), both the
forward and backward resolution could use the same internal mechanism
and noweb references could be changed (or extended) to (optionally) use
a match with some header argument (e.g. :id or :srcname).  So let's say
that'd then allow:

  <<:id "bla">>

to build a list of all source blocks with that header argument and
tangle them in list order.  And since noweb references now really expect
a list, you can just as well feed them a custom function that cobbles
that list from multiple header arguments or properties.

Thinking along those lines I'm beginning to wonder if not the whole
header should be put into a drawer (I've often wished this was the case
for #+TBLFM).



Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Literate Programming - Continue a Source Block?

[Eric Schulte]

Rather than feeling our way forward step by step it seems that simply
following the behavior of noweb would both
1. allow for easy transition between noweb and babel
2. benefit from the years of experience and design accumulated in the
   noweb project

Does anyone on this list know the noweb system well enough to specify
its behavior in this regard, and to describe what functional changes
would be required to bring Babel into line with noweb behavior?

Thanks -- Eric

Achim Gratz <Stromeko@nexgo.de> writes:

> Eric Schulte <schulte.eric@gmail.com> writes:
>> Thanks for sharing your thoughts.  How would you feel about moving away
>> from special source block names and moving towards implementing this
>> behavior with a header argument?
>
> I'm not feeling strongly either way... I'm wanting to use Babel for some
> stuff at work, but haven't got around to learn it well enough to set
> things in motion.  So my current investment is still small.  :-)
>
>>  Specifically two header arguments
>> which would both take the name of a named code block, these header
>> arguments would be as follows
>>
>> - append :: will append the current code block's body to the named code
>>             block during tangling
>>
>> - overwrite :: will overwrite the named code block's body with the body
>>                of the current code block
>>
>> for simple concatenate the value of the append header argument could be
>> easily set on the file or subtree level.
>
> This looks good, but it sort of turns the tables (I hope I get this
> right, it was a longer day than I hoped).  With noweb references, you
> need to resolve the name of the source block at the target.  With the
> header arguments you propose you need to resolve the target block name
> at the source.  Both ways are useful, but it seems that it would be
> difficult to track all targets from a single source even if supposedly
> one could have a list of targets (that would only be useful for
> "overwrite" unless you are really lucky).  This asymmetry is
> inescapeable when you try to use source blocks like "include files".
> Another problem arises when you need to (again) reorder source blocks at
> the target.  You can't do that from the source, since not only would the
> source then need to know all targets, it would also need to know all
> other sources and their ordering in all targets.
>
>> I feel that for many reasons (most of which have been discussed in
>> relation to a header argument solution earlier in this thread) such a
>> solution would be simpler and more consistent with the rest of Babel
>> than the current name-based solution.
>
> I'd think if there was an API to get at the contents of the header
> arguments (perhaps using or modeled after the properties API), both the
> forward and backward resolution could use the same internal mechanism
> and noweb references could be changed (or extended) to (optionally) use
> a match with some header argument (e.g. :id or :srcname).  So let's say
> that'd then allow:
>
>   <<:id "bla">>
>
> to build a list of all source blocks with that header argument and
> tangle them in list order.  And since noweb references now really expect
> a list, you can just as well feed them a custom function that cobbles
> that list from multiple header arguments or properties.
>
> Thinking along those lines I'm beginning to wonder if not the whole
> header should be put into a drawer (I've often wished this was the case
> for #+TBLFM).
>
>
>
> Regards,
> Achim.

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Literate Programming - Continue a Source Block?

[Achim Gratz]

Eric Schulte <schulte.eric@gmail.com> writes:
> Does anyone on this list know the noweb system well enough to specify
> its behavior in this regard, and to describe what functional changes
> would be required to bring Babel into line with noweb behavior?

Far from knowing it well, but the basics are, well basic:

Documentation text is started via '@', code chunks are defined via
'<<chunk_name>>=' and a given chunk can be extended by '<<chunk_name>>='
with the same name again, which allows to intermingle documentation and
definition in a linear fashion.  Code chunk references are just
'<<chunk_name>>' without the equal sign and can be used non-linearly.
For comparison with Babel the noweb Hackers Guide is probably a good
source since it describes the pipeline representation which is used by
the tools (and should maybe be emulated by the Babel backend).

http://www.cs.tufts.edu/~nr/noweb/guide.html

Notably, noweb assumes a tree (or forest) of code chunks of which the
root nodes are to be tangled.

Babel probably needs to take care of another two layers of complexity
since it goes at least one step further than LP: after the equivalent of
tangling all root chunks (producing the runnable programs) it needs to
produce and integrate back into the documentation their results by
running them and optionally keeping track of session information if they
are not independent.

That still assumes that the final documentation can be arrived at after
running all programs and collecting their results, but the real fun
starts when one of those programs produces another program that also
needs to be run.  :-)


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

Re: Literate Programming - Continue a Source Block?

[Eric Schulte]

[-- Attachment #1: Type: text/plain, Size: 2336 bytes --]

Achim Gratz <Stromeko@nexgo.de> writes:

> Eric Schulte <schulte.eric@gmail.com> writes:
>> Does anyone on this list know the noweb system well enough to specify
>> its behavior in this regard, and to describe what functional changes
>> would be required to bring Babel into line with noweb behavior?
>
> Far from knowing it well, but the basics are, well basic:
>
> Documentation text is started via '@', code chunks are defined via
> '<<chunk_name>>=' and a given chunk can be extended by '<<chunk_name>>='
> with the same name again, which allows to intermingle documentation and
> definition in a linear fashion.  Code chunk references are just
> '<<chunk_name>>' without the equal sign and can be used non-linearly.
> For comparison with Babel the noweb Hackers Guide is probably a good
> source since it describes the pipeline representation which is used by
> the tools (and should maybe be emulated by the Babel backend).
>
> http://www.cs.tufts.edu/~nr/noweb/guide.html
>

Hi Achim,

Thanks for sending along this reference.  That combined with the related
"Literate Programming Simplified" [1] serve as a very interesting
introduction to Noweb.  I have not studied noweb previously and I am
impressed by the simplicity of the pipelined filter-based design.

Noweb and Babel are certainly very different beasts.  Noweb achieves
sparse simplicity allowing use with a variety of tools while Babel lives
inside the very rich environment of Org-mode documents.  Noweb enables
literate programming LP while babel is a reproducible research tool
which also provides support for LP.

It seems that lifting a solution whole-piece from noweb will not make
sense, as noweb does not have a need for unique code block names in the
same way as babel.

How about the following solution, which is based on a new :noweb-ref
header argument.

When expanding ``noweb'' style references the bodies of all code block
with /either/ a block name matching the reference name /or/ a :noweb-ref
header argument matching the reference name will be concatenated
together to form the replacement text.

By setting this header argument at the sub-tree or file level, simple
code block concatenation may be achieved.  For example, when tangling
the following Org-mode file, the bodies of code blocks will be
concatenated into the resulting pure code file.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: append.org --]
[-- Type: text/x-org, Size: 472 bytes --]

#+begin_src sh :tangle yes :noweb yes :shebang #!/bin/sh
  <<fullest-disk>>
#+end_src

* the mount point of the fullest disk
  :PROPERTIES:
  :noweb-ref: fullest-disk
  :END:

** query all mounted disks
#+begin_src sh
  df \
#+end_src

** strip the header row
#+begin_src sh
  |sed '1d' \
#+end_src

** sort by the percent full
#+begin_src sh
  |awk '{print $5 " " $6}'|sort -n |tail -1 \
#+end_src

** extract the mount point
#+begin_src sh
  |awk '{print $2}'
#+end_src

[-- Attachment #3: Type: text/plain, Size: 191 bytes --]


This should provide feature-parity with noweb, and satisfy most LP needs.

Best -- Eric

Footnotes: 
[1]  www.cs.tufts.edu/~nr/pubs/lpsimp.pdf

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Literate Programming - Continue a Source Block?

[Neeum Zawan]

Eric Schulte <schulte.eric@gmail.com> writes:

> Rather than feeling our way forward step by step it seems that simply
> following the behavior of noweb would both
> 1. allow for easy transition between noweb and babel
> 2. benefit from the years of experience and design accumulated in the
>    noweb project
>
> Does anyone on this list know the noweb system well enough to specify
> its behavior in this regard, and to describe what functional changes
> would be required to bring Babel into line with noweb behavior?

Looking at the basic functionality:

<<...>>= starts a code block. Further uses of it will just append. No
delimiter for end (it ends when you start another code/doc block). 

It can be referenced using <<...>> (without the = symbol). The location
of the reference is not relevant. 

So by default, no overwriting or ordering. 

However, noweb is extensible, and Achim linked to the syntax one can use
to extend it. I haven't read those details. 

Hope that helps.

Re: Literate Programming - Continue a Source Block?

[Neeum Zawan]

Achim Gratz <Stromeko@nexgo.de> writes:

> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> append the bodies of all blocks of the same name are appended
>>        during tangling
>
> several blocks with the same name seem a bit dubious, would it not be
> cleaner to have an index part to the block name and a range expression
> for the concatenation during tangling?  I might want to tangle them in
> different order than their appearance in the source, for instance.

For my purposes, an index would be too much to maintain, and while I
wouldn't mind it as an option, I wouldn't want to *have* to use them if
all I want is to append and/or overwrite. 

The solution I proposed in my response to Eric may meet you halfway
there in some ways.

When I initially wrote about this problem, my goal was simply to enable
what could easily be done with noweb (and of course, I actually had a
need for it). I actually have very little experience with LP, so I don't
know in practice what will work and what won't.

The initial benefit that I can see with indexing is that one may want to
add some lines in the middle of a block. However, this can only really
work if you just happen to end/begin your source blocks at that
point. If you didn't know in advance that you'd add some lines in the
middle, then you likely didn't end/begin your blocks at that point, and
you'll have to go and artificially begin/end your blocks there. This may
seem confusing to a reader, so it makes sense to put an explicit noweb
style reference. 

If, on the other hand, you knew it in advance, you probably should put
the noweb reference any way.

My comments are from the perspective of my use case for LP, and I can
see people may use LP for other reasons, so if you have a better
scenario, let us know.

Re: Literate Programming - Continue a Source Block?

[Neeum Zawan]

Eric Schulte <schulte.eric@gmail.com> writes:
>>>
>>> I like the concision of the "=original-name" syntax used by noweb, but I
>>> would lean towards the use of a ":noweb-append" type header argument as
>>> suggested above because currently the names of blocks in Babel carry no
>>> semantic content and I'd prefer to leave it this way.
>>
>> I suppose it may also break compatibility in case someone out there uses
>> the =symbol.
>>
>> Had it been thought of earlier, I would have preferred the default
>> behavior being append if you have multiple blocks of the same name, and
>> an explicit option *not* to append but to overwrite, but your idea makes
>> the most sense with respect to preserving backward compatibility. 
>>
>> In addition to append, there probably should be another option for
>> overwriting instead of appending (neither is possible right now).
>>
>
> I've just pushed up a patch which implements optional block combination
> during tangling.  Specifically a new customization variable named
> `org-babel-tangle-named-block-combination' is introduced which can take
> the following values

This does solve the problems I have, but I think the noweb-append header
option discussed earlier is much more flexible. The potential problems
with a custom variable is that it can't be set at a per-buffer or
per-document level. I was thinking along the lines of:

:noweb-append option

where option is one of:

- nil (the default)

- append (appends to the block of the same name as it is up to that
  point in the document - acts as nil if this is the first block of that
  name).

- overwrite (overwrites the block as it is up to that point - acts as
  nil if this is the first block).

I think these three provide all the abilities of what you proposed, but
allows for much more. Some additional benefits:

1. Can be set at a per-buffer level or a per-block level. 

2. Can selectively append/overwrite. One scenario where this would be
   useful is where you may have had some source blocks that had been
   appended, but then later on as the project evolves, you decide to
   rewrite much of that code. You can then just do an overwrite
   (i.e. erases all that you had up to that point), and then again allow
   for the new code to be evolved with possible future appends (so
   multiple appends/overwrites in one document). You may have reason to
   keep the old code in the document for some reason or other. If that
   didn't make sense I can explain in more detail.

> Hopefully this gets at the behavior you're after.  I'd be interested to
> hear any thought you have on this new functionality.

I don't want to make it sound like I'm complaining above. What you've
proposed probably takes care of my current needs (and I imagine is a bit
easier to code than what I've proposed) - but I just think having a new
header for the source block would make it much more flexible. 

I haven't yet tried the new patch - I'll have to figure out how to do a
custom babel install (at the moment I get it via orgmode which is
installed system-wide). Is it possible for me to just install babel in
my custom emacs directory and not have it impact other aspects of
org-mode? 

Thanks for the quick commit!

-- 
My neighbor has a circular driveway. He never leaves home.


 

Re: Literate Programming - Continue a Source Block?

[Eric Schulte]

Hi Neeum,

Thanks for your feedback.  Your point is well taken about the
flexibility of header arguments, and the ability of a header argument
based solution to overwrite blocks.

I would mention that variables such as the newly introduced
`org-babel-tangle-named-block-combination' may be easily set on a
per-file bases using file local variables---basically adding a line like
the following to the top of your Org-mode file.

# -*- org-babel-tangle-named-block-combination:append -*-

While there is no way to specify this on the block level, I would tend
to think that specifications such as ":tangle no" or simply renaming the
block would suffice in most cases.

This still does not address the subtree level.  I think that a general
solution for setting variable values at the subtree level might be
widely useful and could be something worth pursuing Org-wide.

Moving forward from here, my proposal would be that we all try out the
current support using noweb references and the new named-block behavior
and discover experientially if and what new features would be desirable.
On a related note I hereby declare that the newly implemented
named-block behavior should be considered experimental, and support for
it may be removed if a better solution presents itself.

Cheers -- Eric

Neeum Zawan <mailinglists@nawaz.org> writes:

> Eric Schulte <schulte.eric@gmail.com> writes:
>>>>
>>>> I like the concision of the "=original-name" syntax used by noweb,
>>>> but I would lean towards the use of a ":noweb-append" type header
>>>> argument as suggested above because currently the names of blocks
>>>> in Babel carry no semantic content and I'd prefer to leave it this
>>>> way.
>>>
>>> I suppose it may also break compatibility in case someone out there
>>> uses
>>> the =symbol.
>>>
>>> Had it been thought of earlier, I would have preferred the default
>>> behavior being append if you have multiple blocks of the same name,
>>> and an explicit option *not* to append but to overwrite, but your
>>> idea makes the most sense with respect to preserving backward
>>> compatibility.
>>>
>>> In addition to append, there probably should be another option for
>>> overwriting instead of appending (neither is possible right now).
>>>
>>
>> I've just pushed up a patch which implements optional block
>> combination during tangling.  Specifically a new customization
>> variable named `org-babel-tangle-named-block-combination' is
>> introduced which can take the following values
>
> This does solve the problems I have, but I think the noweb-append header
> option discussed earlier is much more flexible. The potential problems
> with a custom variable is that it can't be set at a per-buffer or
> per-document level. I was thinking along the lines of:
>
> :noweb-append option
>
> where option is one of:
>
> - nil (the default)
>
> - append (appends to the block of the same name as it is up to that
>   point in the document - acts as nil if this is the first block of that
>   name).
>
> - overwrite (overwrites the block as it is up to that point - acts as
>   nil if this is the first block).
>
> I think these three provide all the abilities of what you proposed, but
> allows for much more. Some additional benefits:
>
> 1. Can be set at a per-buffer level or a per-block level. 
>
> 2. Can selectively append/overwrite. One scenario where this would be
>    useful is where you may have had some source blocks that had been
>    appended, but then later on as the project evolves, you decide to
>    rewrite much of that code. You can then just do an overwrite
>    (i.e. erases all that you had up to that point), and then again allow
>    for the new code to be evolved with possible future appends (so
>    multiple appends/overwrites in one document). You may have reason to
>    keep the old code in the document for some reason or other. If that
>    didn't make sense I can explain in more detail.
>
>> Hopefully this gets at the behavior you're after.  I'd be interested
>> to hear any thought you have on this new functionality.
>
> I don't want to make it sound like I'm complaining above. What you've
> proposed probably takes care of my current needs (and I imagine is a
> bit easier to code than what I've proposed) - but I just think having
> a new header for the source block would make it much more flexible.
>
> I haven't yet tried the new patch - I'll have to figure out how to do a
> custom babel install (at the moment I get it via orgmode which is
> installed system-wide). Is it possible for me to just install babel in
> my custom emacs directory and not have it impact other aspects of
> org-mode? 
>
> Thanks for the quick commit!

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Literate Programming - Continue a Source Block?

[Neeum Zawan]

Eric Schulte <schulte.eric@gmail.com> writes:

> Hi Neeum,
>
> Thanks for your feedback.  Your point is well taken about the
> flexibility of header arguments, and the ability of a header argument
> based solution to overwrite blocks.
>
> I would mention that variables such as the newly introduced
> `org-babel-tangle-named-block-combination' may be easily set on a
> per-file bases using file local variables---basically adding a line like
> the following to the top of your Org-mode file.

Somehow I couldn't get your new function to work. The variable is set to
append, and I removed all org-modes from my system except the latest
from git. I even ensured that this code gets executed:

(append (mapconcat #'identity
    named "\n"))

However, the output I get is the same as before - it just uses the first
source block.

Incidentally, why do we need "\n" as a separator? What if I wanted to
do (in Python)

def func(a, b, <<func_args>>):
  stuff

In other words, I want to add more arguments later on. Wouldn't a \n
mess things up here?


Thanks.

Re: Literate Programming - Continue a Source Block?

[Eric Schulte]

[-- Attachment #1: Type: text/plain, Size: 1242 bytes --]

Hi Neeum,

Neeum Zawan <mailinglists@nawaz.org> writes:

> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> Hi Neeum,
>>
>> Thanks for your feedback.  Your point is well taken about the
>> flexibility of header arguments, and the ability of a header argument
>> based solution to overwrite blocks.
>>
>> I would mention that variables such as the newly introduced
>> `org-babel-tangle-named-block-combination' may be easily set on a
>> per-file bases using file local variables---basically adding a line like
>> the following to the top of your Org-mode file.
>
> Somehow I couldn't get your new function to work. The variable is set to
> append, and I removed all org-modes from my system except the latest
> from git. I even ensured that this code gets executed:
>
> (append (mapconcat #'identity
>     named "\n"))
>
> However, the output I get is the same as before - it just uses the first
> source block.
>

Could you try the attached example file?  I first evaluated the
following elisp code to set the combination variable's value to append.

#+begin_src emacs-lisp
  (setq org-babel-tangle-named-block-combination 'append)
#+end_src

I then call org-babel-tangle in the attached Org-mode file to generate
the attached elisp file.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: combined-tangle-example.org --]
[-- Type: text/x-org, Size: 564 bytes --]

* continued code blocks
   :PROPERTIES:
   :tangle:   yes
   :comments: yes
   :END:
#+source: foo
#+begin_src emacs-lisp
  (message "foo:%S" 1)
#+end_src

#+begin_src emacs-lisp
  (message "un-named")
#+end_src

#+source: bar
#+begin_src emacs-lisp
  (message "bar:%S" 1)
#+end_src

#+source: foo
#+begin_src emacs-lisp
  (message "foo:%S" 2)
#+end_src

#+source: bar
#+begin_src emacs-lisp
  (message "bar:%S" 2)
#+end_src

#+begin_src emacs-lisp :tangle no :results silent
  (with-temp-buffer
    (insert-file-contents "scraps.el")
    (eval-buffer))
#+end_src

[-- Attachment #3: combined-tangle-example.el --]
[-- Type: application/emacs-lisp, Size: 486 bytes --]

[-- Attachment #4: Type: text/plain, Size: 422 bytes --]


>
> Incidentally, why do we need "\n" as a separator? What if I wanted to
> do (in Python)
>
> def func(a, b, <<func_args>>):
>   stuff
>
> In other words, I want to add more arguments later on. Wouldn't a \n
> mess things up here?
>

Ah, this is a good point.  I've just changed the code to avoid inserting
superfluous newlines.  Thanks!

Cheers -- Eric

>
>
> Thanks.
>
>

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Literate Programming - Continue a Source Block?

[Neeum Zawan]

[-- Attachment #1: Type: text/plain, Size: 384 bytes --]

Eric Schulte <schulte.eric@gmail.com> writes:

> Could you try the attached example file?  I first evaluated the
> following elisp code to set the combination variable's value to append.

Your example works if there are no noweb references. 

See the  modified one where I have noweb references. Note that when
expanding the reference, it inserts only the first block it finds. 



 

[-- Attachment #2: combined --]
[-- Type: text/plain, Size: 657 bytes --]

#+begin_src emacs-lisp :tangle yes :noweb yes
<<foo>>

Random text

<<bar>>
#+end_src



* continued code blocks
   :PROPERTIES:
   :tangle:   yes
   :comments: yes
   :END:
#+srcname: foo
#+begin_src emacs-lisp
  (message "foo:%S" 1)
#+end_src

#+begin_src emacs-lisp
  (message "un-named")
#+end_src

#+srcname: bar
#+begin_src emacs-lisp
  (message "bar:%S" 1)
#+end_src

#+srcname: foo
#+begin_src emacs-lisp
  (message "foo:%S" 2)
#+end_src

#+srcname: bar
#+begin_src emacs-lisp
  (message "bar:%S" 2)
#+end_src

#+begin_src emacs-lisp :tangle no :results silent
  (with-temp-buffer
    (insert-file-contents "scraps.el")
    (eval-buffer))
#+end_src

Re: Literate Programming - Continue a Source Block?

[Eric Schulte]

Hi Neeum,

You are correct, the current implementation only specially concatenates
blocks during tangling, *not* during noweb resolution.

It would be possible to also implement the concatenation behavior during
noweb expansion, however I'd prefer to first wait for a response to my
recent other email to this thread asking for a more clear explication of
existing noweb behavior.

The only remaining times when such concatenation behavior could be
implemented would be during block reference expansion, and during block
evaluation, but I think expanding at those times would be unnecessary
and confusing.

Thanks -- Eric

Neeum Zawan <mailinglists@nawaz.org> writes:

> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> Could you try the attached example file?  I first evaluated the
>> following elisp code to set the combination variable's value to append.
>
> Your example works if there are no noweb references. 
>
> See the  modified one where I have noweb references. Note that when
> expanding the reference, it inserts only the first block it finds. 
>
>
>
>  
> #+begin_src emacs-lisp :tangle yes :noweb yes
> <<foo>>
>
> Random text
>
> <<bar>>
> #+end_src
>
>
>
> * continued code blocks
>    :PROPERTIES:
>    :tangle:   yes
>    :comments: yes
>    :END:
> #+srcname: foo
> #+begin_src emacs-lisp
>   (message "foo:%S" 1)
> #+end_src
>
> #+begin_src emacs-lisp
>   (message "un-named")
> #+end_src
> #+srcname: bar
> #+begin_src emacs-lisp
>   (message "bar:%S" 1)
> #+end_src
>
> #+srcname: foo
> #+begin_src emacs-lisp
>   (message "foo:%S" 2)
> #+end_src
>
> #+srcname: bar
> #+begin_src emacs-lisp
>   (message "bar:%S" 2)
> #+end_src
>
> #+begin_src emacs-lisp :tangle no :results silent
>   (with-temp-buffer
>     (insert-file-contents "scraps.el")
>     (eval-buffer))
> #+end_src

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Literate Programming - Continue a Source Block?

[Neeum Zawan]

Eric Schulte <schulte.eric@gmail.com> writes:
> It would be possible to also implement the concatenation behavior during
> noweb expansion, however I'd prefer to first wait for a response to my
> recent other email to this thread asking for a more clear explication of
> existing noweb behavior.
>
> The only remaining times when such concatenation behavior could be
> implemented would be during block reference expansion, and during block
> evaluation, but I think expanding at those times would be unnecessary
> and confusing.

I'm having trouble understanding the difference between what you discuss
in the first paragraph and in the second. What's the difference between
noweb expansion and block reference expansion?

As for actual noweb behavior, I don't really know - will have to play
with it and let you know...

Re: Literate Programming - Continue a Source Block?

[Eric Schulte]

Neeum Zawan <mailinglists@nawaz.org> writes:

> Eric Schulte <schulte.eric@gmail.com> writes:
>> It would be possible to also implement the concatenation behavior during
>> noweb expansion, however I'd prefer to first wait for a response to my
>> recent other email to this thread asking for a more clear explication of
>> existing noweb behavior.
>>
>> The only remaining times when such concatenation behavior could be
>> implemented would be during block reference expansion, and during block
>> evaluation, but I think expanding at those times would be unnecessary
>> and confusing.
>
> I'm having trouble understanding the difference between what you discuss
> in the first paragraph and in the second. What's the difference between
> noweb expansion and block reference expansion?
>

By block reference I meant when variables are assigned to the values of
blocks through reference.  So in the following example the "number"
block has its body returned through "reference expansion".

#+source: number
#+begin_src emacs-lisp
  4
#+end_src

#+begin_src emacs-lisp :var num=number
  (* num 2)
#+end_src

Best -- Eric

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Literate Programming - Continue a Source Block?

[Neeum Zawan]

Eric Schulte <schulte.eric@gmail.com> writes:

> How about the following solution, which is based on a new :noweb-ref
> header argument.
>
> When expanding ``noweb'' style references the bodies of all code block
> with /either/ a block name matching the reference name /or/ a :noweb-ref
> header argument matching the reference name will be concatenated
> together to form the replacement text.
>
> By setting this header argument at the sub-tree or file level, simple
> code block concatenation may be achieved.  For example, when tangling
> the following Org-mode file, the bodies of code blocks will be
> concatenated into the resulting pure code file.

Hi,

Your example is not completely clear. I noticed you didn't put any names
for the source blocks that use the noweb-ref. Is it necessary not to
name them, or can one name them but their names will still have to be
unique and are orthogonal to concatenation (i.e. they have names, but
the concatenation is due to whatever the argument of noweb-ref is)?

I think either way, this solution serves my purpose. My original
suggestion was to have a header whose value would be "append" to allow
for concatenation if the name matches a previous source block. It seems
your solution above is the same except instead of looking at the name of
the source block, it looks at the argument of noweb-ref.

Overwriting is still not supported, but I don't know if that's all that
important (I don't have an immediate need for it). And noweb by default
did not have it either, so perhaps it's not needed for most tasks (OTOH,
you may want to think about what the best solution is if later on you
decide to add overwriting capability).

Thanks. Hope to see this or something like it soon.

Re: Literate Programming - Continue a Source Block?

[Eric Schulte]

Neeum Zawan <mailinglists@nawaz.org> writes:

> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> How about the following solution, which is based on a new :noweb-ref
>> header argument.
>>
>> When expanding ``noweb'' style references the bodies of all code block
>> with /either/ a block name matching the reference name /or/ a :noweb-ref
>> header argument matching the reference name will be concatenated
>> together to form the replacement text.
>>
>> By setting this header argument at the sub-tree or file level, simple
>> code block concatenation may be achieved.  For example, when tangling
>> the following Org-mode file, the bodies of code blocks will be
>> concatenated into the resulting pure code file.
>
> Hi,
>
> Your example is not completely clear. I noticed you didn't put any names
> for the source blocks that use the noweb-ref. Is it necessary not to
> name them,

The source code blocks may have names, as before all code block names
should be unique or the behavior may be undefined.

> or can one name them but their names will still have to be unique and
> are orthogonal to concatenation (i.e. they have names, but the
> concatenation is due to whatever the argument of noweb-ref is)?
>

The value of the :noweb-ref header argument supersedes the name of the
code block for noweb expansion.  To preserve existing semantics and to
avoid requiring the use of the :noweb-ref header argument in simple
cases, the code block name will be used to resolve noweb references in
blocks which do not have a :noweb-ref header argument.

>
> I think either way, this solution serves my purpose. My original
> suggestion was to have a header whose value would be "append" to allow
> for concatenation if the name matches a previous source block. It seems
> your solution above is the same except instead of looking at the name of
> the source block, it looks at the argument of noweb-ref.
>

Yup, happy this sounds like it should work for your original need.

>
> Overwriting is still not supported, but I don't know if that's all that
> important (I don't have an immediate need for it). And noweb by default
> did not have it either, so perhaps it's not needed for most tasks

This was my thinking.

> (OTOH, you may want to think about what the best solution is if later
> on you decide to add overwriting capability).
>

If someone finds a real need for overwriting code blocks, hopefully the
specifics of their need will point towards an implementation.

>
> Thanks. Hope to see this or something like it soon.
>

You welcome, hope this turns out to be helpful -- Eric

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Literate Programming - Continue a Source Block?

[Sebastien Vauban]

Hi Eric and Neeum,

Eric Schulte wrote:
>> Overwriting is still not supported, but I don't know if that's all that
>> important (I don't have an immediate need for it). And noweb by default
>> did not have it either, so perhaps it's not needed for most tasks
>
> This was my thinking.
>
>> (OTOH, you may want to think about what the best solution is if later
>> on you decide to add overwriting capability).
>
> If someone finds a real need for overwriting code blocks, hopefully the
> specifics of their need will point towards an implementation.

The only case that pops up to my mind now, of such a use case where
overwriting could be "needed" (well, let's say useful) is for some pedagogical
document that one would write, where code is constructed from a simplistic
(and buggy) approach to a correct one.

One could say: the code to do that is this one, and show the block contents.

Then, discover problems to it, explain them in the document, and make a new
version of the block with the same name (for tangling reasons).

Then, highlight some limitations of the new code, fix them in a new block with
the same name, etc.

Does that mean it needs to be implemented?  Up to you...

Best regards,
  Seb

-- 
Sebastien Vauban

Re: Literate Programming - Continue a Source Block?

[Neeum Zawan]

"Sebastien Vauban"
<wxhgmqzgwmuf@spammotel.com> writes:

> The only case that pops up to my mind now, of such a use case where
> overwriting could be "needed" (well, let's say useful) is for some pedagogical
> document that one would write, where code is constructed from a simplistic
> (and buggy) approach to a correct one.
>
> One could say: the code to do that is this one, and show the block contents.
>
> Then, discover problems to it, explain them in the document, and make a new
> version of the block with the same name (for tangling reasons).
>
> Then, highlight some limitations of the new code, fix them in a new block with
> the same name, etc.
>
> Does that mean it needs to be implemented?  Up to you...

I gave a similar example earlier, and I think Eric said that can be
handled by just changing the tangle header for the code you
overwrite. Not as convenient, though.

Re: Literate Programming - Continue a Source Block?

[Olaf.Hamann]

Am 08.06.2011 17:20, schrieb Neeum Zawan:
> [...]
> Now the original noweb allows what I'm asking for. If you begin a source
> block with a name of an existing block but append an "=" symbol, it
> knows to append to that source block.
>
> It would be great if org-mode could add that capability. Another
> approach is that if multiple source blocks with the same name are found,
> the default behavior is to concatenate all those source blocks together
> (and then add a header option for overwrite if the user wanted to
> overwrite instead of append).
>
> Thoughts?
>
I missed this feature too - I used it   (so called additive macros) very 
often with funnelweb.