Re: problems with export and :cache

[Aaron Ecay <aaronecay@gmail.com>]

Hi Nicolas,

2015ko urriak 29an, Nicolas Goaziou-ek idatzi zuen:
> 
> Andreas Leha <andreas.leha@med.uni-goettingen.de> writes:
> 
>> Generally, I think that caching is a sensitive area.  And if we think
>> that it is unpredictable and advise people to stay off of it, we are
>> better off removing it than offering it in this state.  At least until
>> it behaves (more) predictable.
> 
> Is it necessarily broken?

If this means “can it ever work?” then I think the answer is “yes it
can”.  But I think the current implementation is broken and likely to
remain so for the foreseeable future.  The issues are:

1. :cache only works for code which is a pure function of its header args
2. When combined with :session, the environment that the code is evaluated
   in is not created anew each time it is run.  This makes it much easier
   to leak references to (e.g.) variables defined in other blocks
3. The proper notion of purity is not easily defined when the code does
   things like modifying the emacs environment, touching the filesystem,
   or accessing the language’s RNG.
4. We (org devs) don’t actually understand how the semantics of cache
   interacts with other babel features.  See:
   <http://mid.gmane.org/86fvqqc8jb.fsf@somewhere.org>.

1-3 are likely to be extremely confusing for users, especially less
technically sophisticated ones (what’s a “pure function” anyway)?  The
inability to give a clear introductory explanation of the feature in
combination with 4 indicating we don’t actually understand it ourselves
makes me feel like we should not be advertising, let alone recommending,
it.

The only other literate programming environment that I know of that
implements such a feature is knitr (for R).  They address these issues
by providing (optional) free-variable analysis to construct a dependency
graph between code blocks.  There is also some handling of RNG seed
values.  The documentation <http://yihui.name/knitr/demo/cache/> is much
more comprehensive, including a prominent statement about the dangers of
side effect-ful code and a nuanced discussion of several issues,
including the RNG.

Aaron

PS I’ve looked through my old notes on the interaction of cache with
babel export.  I can’t reproduce any of my old test cases, so maybe that
bug has been fixed as a side effect of some other change.
 
-- 
Aaron Ecay