Re: problems with export and :cache

[Andreas Leha <andreas.leha@med.uni-goettingen.de>]

Hi Thomas,

Thomas S. Dye <tsd@tsdye.com> writes:
> Aloha all,
>
> Aaron Ecay <aaronecay@gmail.com> writes:
>
>> Hi Nicolas,
>>
>> 2015ko urriak 29an, Nicolas Goaziou-ek idatzi zuen:
>>> 
>>> Andreas Leha <andreas.leha@med.uni-goettingen.de> writes:
>>> 
>>>> Generally, I think that caching is a sensitive area.  And if we think
>>>> that it is unpredictable and advise people to stay off of it, we are
>>>> better off removing it than offering it in this state.  At least until
>>>> it behaves (more) predictable.
>>> 
>>> Is it necessarily broken?
>>
>> If this means “can it ever work?” then I think the answer is “yes it
>> can”.  But I think the current implementation is broken and likely to
>> remain so for the foreseeable future.  The issues are:
>>
>> 1. :cache only works for code which is a pure function of its header args
>> 2. When combined with :session, the environment that the code is evaluated
>>    in is not created anew each time it is run.  This makes it much easier
>>    to leak references to (e.g.) variables defined in other blocks
>> 3. The proper notion of purity is not easily defined when the code does
>>    things like modifying the emacs environment, touching the filesystem,
>>    or accessing the language’s RNG.
>> 4. We (org devs) don’t actually understand how the semantics of cache
>>    interacts with other babel features.  See:
>>    <http://mid.gmane.org/86fvqqc8jb.fsf@somewhere.org>.
>>
>> 1-3 are likely to be extremely confusing for users, especially less
>> technically sophisticated ones (what’s a “pure function” anyway)?  The
>> inability to give a clear introductory explanation of the feature in
>> combination with 4 indicating we don’t actually understand it ourselves
>> makes me feel like we should not be advertising, let alone recommending,
>> it.
>>
>> The only other literate programming environment that I know of that
>> implements such a feature is knitr (for R).  They address these issues
>> by providing (optional) free-variable analysis to construct a dependency
>> graph between code blocks.  There is also some handling of RNG seed
>> values.  The documentation <http://yihui.name/knitr/demo/cache/> is much
>> more comprehensive, including a prominent statement about the dangers of
>> side effect-ful code and a nuanced discussion of several issues,
>> including the RNG.
>>
>> Aaron
>>
>> PS I’ve looked through my old notes on the interaction of cache with
>> babel export.  I can’t reproduce any of my old test cases, so maybe that
>> bug has been fixed as a side effect of some other change.
>
> FWIW, I think relying on cache to speed up export is the wrong
> approach.  Having all code run during each export is, to me, a feature
> that makes a document "live."  I'm willing to be patient during export
> to get this feature.
>
> If speed is important and a live document isn't desired, then one
> solution is to rename the results and refer to this name in the
> document, rather than to the name of the code block that produced the
> results.  I do this manually, which is OK, but I've often wanted
> something like :persist-as my-result so I can be certain to have a good
> link from the results back to the code that produced them.

There we go again.  Another advice not to use the :cache.

>
> That said, points 2 and 4 seem to me serious issues that users must
> understand if they choose to use :cache.  So, at the least, the
> documentation needs revision.
>

Just to add (maybe that belongs to point 4 anyway): And the caching
should work predictably -- even during export.

Regards,
Andreas