Re: Please document the caching and its user options

[Eli Zaretskii <eliz@gnu.org>]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: emacs-orgmode@gnu.org
> Date: Fri, 14 Jun 2024 13:12:42 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I needed to visit org.org, the Org manual, today, and to my surprise
> > saw Emacs writing some data files into the ~/.cache/org-persist/
> > directory.  What's more, Emacs popped a buffer out of the blue telling
> > me that it could not safely encode the data written to (I presume)
> > some of those files, and asked me to select a safe coding-system.
> >
> > By randomly poking here and there, I've succeeded to figure out that
> > this is due to org-element's caching of data from parsing Org files.
> > It seems this caching is turned on by default, but is not documented
> > in the Org manual, and in particular there's nothing in the manual
> > about turning off the caching.
> >
> > Please document the caching features of Org in the manual, including
> > how to turn that off.  (I also question the wisdom of turning this on
> > by default without as much as a single request for confirmation from
> > the user.)
> 
> Hmm. What aspect of caching do you want us to document?

First and foremost, that it exists, and is turned on by default.  The
manual is currently completely silent about it.

Next, please document the user options that control this caching, and
especially those options which can be used to turn this caching off or
direct it to a different place.

> FYI, Org mode has been doing various forms of caching since
> forever. Recently, we just employed a bit more regular API and
> introduced one more kind of caching - parser cache. In addition to the
> previously existing image cache, publishing cache, ID cache, clock
> cache, etc.

I'm not a heavy user of Org, but I do have several Org files that I
visit from time to time.  This was the first time I got prompted about
anything related to this caching.  Moreover, I think this was the
first time the Org file I visited was parsed by Org and the results
cached: I have a feature on my system that prominently indicates when
the machine is heavily loaded, and I was surprised to see it in action
when I visited org.org.  I never had this activated before just by
visiting an Org file.  I presumed the high load was due to the
parsing.  So either this is very new, or maybe my Org files are much
simpler than doc/misc/org.org, and so the parsing I triggered before
was much less expensive.

I hope you now understand why I wrote this report now and not before,
and why I was surprised: this caching was never so explicitly and
prominently into my face, so I could have completely missed its
existence.

> > Please also make sure that the code which actually writes the data to
> > the cache files makes a point of binding coding-system-for-write to a
> > proper value (probably utf-8-unix), or forces
> > buffer-file-coding-system of the buffer from which it writes to have
> > such a safe value, to avoid annoying and unexpected prompting of the
> > user to select a proper encoding.  Lisp programs that write files in
> > the background cannot fail to set a proper encoding, because the call
> > to select-safe-coding-system is not supposed to be triggered by Lisp
> > programs unless they run as a direct result of a user-invoked command.
> 
> I believe that this particular problem has been solved in
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=c8f88589c
> It is a part of Org 9.7.

Maybe.  Try visiting org.org on a system whose locale is set to, say,
Latin-1, and see if you get the warnings about a safe coding-system.

But why do you use utf-8 there and not utf-8-unix?  Come to think
about it, why not emacs-internal?  Those files are used internally by
Org, so they should be able to encode any characters supported by
Emacs, not just those which have UTF-8 encoding.  And using native EOL
convention is not needed, and will get in the way if the user shares
these files between systems.