Re: [BUG] org-capture autoload bug? [9.5.2 (9.5.2-gfbff08 @ /home/ignacio/.emacs.d/elpa/org-9.5.2/)]

[Ignacio Casso <ignaciocasso@hotmail.com>]

> While I don't know if this is a bug, it certainly doesn't seem to be
> doing the right thing from an 'intuitive' point of view. I would expect
> when a variable is bound to a value inside a let and a function is then
> called which uses that variable, the initial let bound value should be
> used and the result be the same regardless of whether org-capture has or
> has not been loaded. It means there is a hidden side-effect here, which
> isn't good and probably needs more analysis. If you had set the value
> using setq rather than as a let form, it wouldn't be overridden when
> org-capture is loaded, so why does it when it is a let binding? 
>

I've investigated this a little bit further and this is not specific to
org-capture. For example if you evaluate (let ((org-agenda-files
'("/tmp/foo.org"))) (org-agenda-list)) before org-agenda is laoded, the
let-binding of org-agenda-files is ignored too.

I think the reason is the following:

1) Loading the file defining those autoload functions runs defcustom for
the variable we are using.

2) defcustom calls the function F passed as argument with the :set
keyword (set-default by default) to initialize the variable. That
function receives as argument the variable and a value.

2.1) If the variable was already initialized (e.g., with setq) with a
value V, the value passed to F is V. It it was not, the value passed to
F is the value passed as STANDARD argument to defcustom. So far, so
good.

2.2) If the variable is let-bound with lexical-binding non-nil, there is
an error in Emacs 29, since at the time the variable is let-bound Emacs
does not know it is special and thus it uses lexical binding, but when
defcustom runs it finds out that it is special and therefore it should
have used dynamic binding. In Emacs 27 this check is not done, and the
let-binding is considered lexical, so it has no effect for uses of the
variable while the let body is being executed, unless they appear
textually within that body. In particular, the value passed to F is
again the STANDARD argument of defcustom.

2.2.1) An exception to this is when the variable is autoloaded, since
Emacs can know that it is special by the time is evaluates let.

2.2.2) But in general this means that the programmer should know whether
an autoload function is going to be loaded at that execution
point if he is going to do this.

2.3) If the variable is let-bound with lexical-binding nil, things then
break completely. First, the value passed to F is again the STANDARD
argument of defcustom. This makes sense since F would initialize the
variable globally and the let binding was supposed to be local. But it
means that again the let binding gets shadowed. But it's worse than
that. The initialization of the variable does not work and only has the
same scope as the let-binding, so after the let body is finished, the
variable is void. Furthermore, since the feature in question is already
provided, there is no easy way to leave that state and initialize the
variable again.

This makes me conclude that let-bindings of variables used by autoloaded
functions should be avoided unless we know for sure that the function
will be loaded at that point in execution.

> Might be worth asking a general question on emacs-devel? Stephan or Eli
> can probably provide some clarification here (maybe this is somehow
> related to the changes associated with the lexical binding stuff?)

Yes, I will email them explaining this, since I think the point 2.3 is a
bug. Maybe there is no better way to do this, but I think at least there
should be a warning if a misuse of let + autoload can leave Emacs in
what I think is a broken state.