Re: A requires/provides approach to linking source code blocks

[Tom Gillespie <tgbugs@gmail.com>]

We have been receiving many new feature suggestions and requests
coming in for org babel. I think that Tim's suggestion is the right
one. Nearly all of these need to be implemented as an extension first
and tested independently. Further, even if this is done, it should be
clear that there is zero expectation that such extensions will be
incorporated.

Once I wrap up the formal grammar for org, one of the next things I
plan to work on is a clear specification for org babel. This is
critical because so many of the suggestions that come in deal with
individuals' specific problems and thus fail to account for how such
features interact with existing features and how the newly proposed
feature would block some other features in the future, confuse users,
etc. Such suggestions also often fail to account for increased
complexity, nor have they been exposed to a sufficient number of
examples to reveal fundamental ambiguities in how they could be
interpreted. The issues with variable behavior between ob langs for
:pre :post :prologue :epilogue etc. are already enough to keep us busy
for quite some time.

With regard to this thread in particular, it is of some interest, but
there are fundamental issues, including the fact that certain
languages (e.g. racket) expect module code to exist somewhere on the
file system. There are ways around many of these issues, in fact there
are likely many ways around any individual issue, so org babel needs
to systematically consider the issues and provide a clear
specification, or at least a guide for how such cases should be
handled.

To give an example from one of my continual pain points: I start
writing python or racket in an org src block and then I want it to be
a library so that it can be reused by other code both inside and
outside the org file without having to resort to noweb.

What is the best way to handle this? I don't know. Right now I tangle
things and resort to awful hacks for the reuse-in-this-org-file case, but
I'm guessing there is a better generic solution which would allow _any_
org block to be exported as a library instead of nowebbed in.

Before jumping for any particular suggestion for how to handle this
we need to explore the diversity of cases that various ob langs
present, so that we can find a solution that will work for all of
them. After all, packaging code to a library for reuse is an
extremely common pattern that org babel should be able to
abstract, but it is a major undertaking, not just the addition of a
keyword here and there.

In short I suggest that we issue a general moratorium on new org babel
feature suggestions until we can stabilize what we already have and
provide a clear specification for correct behavior. Until we have that spec
we could encourage users to create extensions that implement those
features.

Best,
Tom


PS The other next thing that I am working on might be another way out
for this particular feature request. Namely, it is simplifying and
extending org keyword syntax so that new keywords (with options) and
associated keywords can be specified using keyword syntax within a
single org file. This would make it possible to get useful high level
keyword behavior in a single file without burdening the core
implementation with more special cases for associated keywords, and it
would allow users to write small elisp functions that could do some of
what is suggested here, all without need to add anything to the core
org implementation.