[BUG] ob-tangle overrides user customization of org-id-link-to-org-use-id [9.5.2 (9.5.2-gfbff08 @ /home/ionasal/.emacs.d/elpa/org-9.5.2/)]

[BUG] ob-tangle overrides user customization of org-id-link-to-org-use-id [9.5.2 (9.5.2-gfbff08 @ /home/ionasal/.emacs.d/elpa/org-9.5.2/)]

[Allen Li]

In ob-tangle.el, the line

(defvar org-id-link-to-org-use-id nil) ; Dynamically scoped

appears to override the user's customization of
org-id-link-to-org-use-id.

Specifically, this happens if the user has python in
org-babel-do-load-languages, which leads to the require chain:

require ob-python
require ob
require ob-tangle

I believe the line should be changed to

(defvar org-id-link-to-org-use-id) ; Dynamically scoped

which makes the variable dynamically scoped without setting its value.

I don't know if ob-tangle.el refers to
org-id-link-to-org-use-id before it has a value set.  If so, the
change I suggest would cause an error and ob-tangle.el should be fixed
to require org-id before using said variable.

Emacs  : GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.27, cairo version 1.17.4)
 of 2021-03-26
Package: Org mode version 9.5.2 (9.5.2-gfbff08 @ /home/ionasal/.emacs.d/elpa/org-9.5.2/)

Re: [BUG] ob-tangle overrides user customization of org-id-link-to-org-use-id [9.5.2 (9.5.2-gfbff08 @ /home/ionasal/.emacs.d/elpa/org-9.5.2/)]

[Ihor Radchenko]

Allen Li <darkfeline@felesatra.moe> writes:

> In ob-tangle.el, the line
>
> (defvar org-id-link-to-org-use-id nil) ; Dynamically scoped
>
> appears to override the user's customization of
> org-id-link-to-org-use-id.

Have you seen this happening?

FYI, defvar does not overwrite variable value that is already set.
Try the following:
(setq foo 'my-value) ;; foo = 'my-value
(defvar foo 'value) ;; foo = 'my-value !!
foo ;; foo still = 'my-value

defvar only matters if the variable is not yet defined:
bar ;; => error void-variable
(defvar bar 'value) ;; bar = 'value

Best,
Ihor

Re: [BUG] ob-tangle overrides user customization of org-id-link-to-org-use-id [9.5.2 (9.5.2-gfbff08 @ /home/ionasal/.emacs.d/elpa/org-9.5.2/)]

[Allen Li]

Ihor Radchenko <yantar92@gmail.com> writes:

> Allen Li <darkfeline@felesatra.moe> writes:
>
>> In ob-tangle.el, the line
>>
>> (defvar org-id-link-to-org-use-id nil) ; Dynamically scoped
>>
>> appears to override the user's customization of
>> org-id-link-to-org-use-id.
>
> Have you seen this happening?

Yes, that's why I filed the bug.

> FYI, defvar does not overwrite variable value that is already set.
> Try the following:
> (setq foo 'my-value) ;; foo = 'my-value
> (defvar foo 'value) ;; foo = 'my-value !!
> foo ;; foo still = 'my-value
>
> defvar only matters if the variable is not yet defined:
> bar ;; => error void-variable
> (defvar bar 'value) ;; bar = 'value

Yes, that is so.

Even if org-id-link-to-org-use-id were always set at this point, there
would be no reason to use

(defvar org-id-link-to-org-use-id nil)

instead of

(defvar org-id-link-to-org-use-id)

>
> Best,
> Ihor

Re: [BUG] ob-tangle overrides user customization of org-id-link-to-org-use-id [9.5.2 (9.5.2-gfbff08 @ /home/ionasal/.emacs.d/elpa/org-9.5.2/)]

[Ihor Radchenko]

Allen Li <darkfeline@felesatra.moe> writes:

>> Allen Li <darkfeline@felesatra.moe> writes:
>>
>>> In ob-tangle.el, the line
>>>
>>> (defvar org-id-link-to-org-use-id nil) ; Dynamically scoped
>>>
>>> appears to override the user's customization of
>>> org-id-link-to-org-use-id.
>>
>> Have you seen this happening?
>
> Yes, that's why I filed the bug.

If you saw this, then the root cause is probably not in defvar.
Are you able to provide a reproducer?

>> defvar only matters if the variable is not yet defined:
>> bar ;; => error void-variable
>> (defvar bar 'value) ;; bar = 'value
>
> Yes, that is so.
>
> Even if org-id-link-to-org-use-id were always set at this point, there
> would be no reason to use
>
> (defvar org-id-link-to-org-use-id nil)
>
> instead of
>
> (defvar org-id-link-to-org-use-id)

I agree. The question is whether changing to
(defvar org-id-link-to-org-use-id)
solves your problem.

Best,
Ihor

Re: [BUG] ob-tangle overrides user customization of org-id-link-to-org-use-id [9.5.2 (9.5.2-gfbff08 @ /home/ionasal/.emacs.d/elpa/org-9.5.2/)]

[Allen Li]

Ihor Radchenko <yantar92@gmail.com> writes:

> Allen Li <darkfeline@felesatra.moe> writes:
>> Even if org-id-link-to-org-use-id were always set at this point, there
>> would be no reason to use
>>
>> (defvar org-id-link-to-org-use-id nil)
>>
>> instead of
>>
>> (defvar org-id-link-to-org-use-id)
>
> I agree. The question is whether changing to
> (defvar org-id-link-to-org-use-id)
> solves your problem.

Thanks for your concern.  By editing ob-tangle.el, I can confirm that
changing this does solve my problem.

If I must, I could provide a reproducible example, but I feel like
that costs unnecessary effort on the part of everyone involved if we
agree on the above point.

(Similarly, I could provide a patch, but I feel like it would be less
effort for everyone if a maintainer who agrees with the above point just
edits that one line.)

>
> Best,
> Ihor

Re: [BUG] ob-tangle overrides user customization of org-id-link-to-org-use-id [9.5.2 (9.5.2-gfbff08 @ /home/ionasal/.emacs.d/elpa/org-9.5.2/)]

[Ihor Radchenko]

Allen Li <darkfeline@felesatra.moe> writes:

>> I agree. The question is whether changing to
>> (defvar org-id-link-to-org-use-id)
>> solves your problem.
>
> Thanks for your concern.  By editing ob-tangle.el, I can confirm that
> changing this does solve my problem.
>
> If I must, I could provide a reproducible example, but I feel like
> that costs unnecessary effort on the part of everyone involved if we
> agree on the above point.

Let me clarify what I am worrying about.
We have 132 occurrences of (defvar foo nil) in the code.
I am genuinely surprised that changing defvar fixed the problem for you
and at the same time nobody reported similar issues with the other 131
defvars. That's why I really want to get a reproducer and understand
what is going on there. So far, I don't understand how defvar can break
anything except in case of some strange compilation/mixed installation
problems.

Best,
Ihor

Re: [BUG] ob-tangle overrides user customization of org-id-link-to-org-use-id [9.5.2 (9.5.2-gfbff08 @ /home/ionasal/.emacs.d/elpa/org-9.5.2/)]

[Allen Li]

Ihor Radchenko <yantar92@gmail.com> writes:

> Allen Li <darkfeline@felesatra.moe> writes:
>
>>> I agree. The question is whether changing to
>>> (defvar org-id-link-to-org-use-id)
>>> solves your problem.
>>
>> Thanks for your concern.  By editing ob-tangle.el, I can confirm that
>> changing this does solve my problem.
>>
>> If I must, I could provide a reproducible example, but I feel like
>> that costs unnecessary effort on the part of everyone involved if we
>> agree on the above point.
>
> Let me clarify what I am worrying about.
> We have 132 occurrences of (defvar foo nil) in the code.
> I am genuinely surprised that changing defvar fixed the problem for you
> and at the same time nobody reported similar issues with the other 131
> defvars. That's why I really want to get a reproducer and understand
> what is going on there. So far, I don't understand how defvar can break
> anything except in case of some strange compilation/mixed installation
> problems.

I see.  In my opinion those occurrences should be fixed even if no one
is reporting issues because it is bad/improper code, and it is not
especially surprising that no one has reported it yet; there is always a
first person who reports a bug, and there are always more old bugs to be
found in a big project like Emacs/Org mode.

This is annoying to reproduce because it relies on the normal Emacs
startup process, which loads packages.  Anyway, I have found a reproduction.

This is using the Emacs install on Arch Linux.

GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.27, cairo version 1.17.4)
 of 2021-03-26

Have the packages org-9.5.2 and org-contrib-0.3 installed in
~/.emacs.d/elpa.  I have these installed from
https://elpa.gnu.org/packages/ and https://elpa.nongnu.org/nongnu/
respectively.

Start emacs by loading a tmp.el file with the contents provided below.

1. emacs -Q --load tmp.el
2. C-h v org-id-link-to-org-use-id

Note that the value is nil instead of the expected
'create-if-interactive

3. Press RET on the customize link.

Note that the Customize UI says "CHANGED outside Customize."

Expected behavior:

User sees org-id-link-to-org-use-id set to the value they saved in
Customize.

Personal aside:

It is easy to get subtle bugs in between autoloading, package activation
and Customize when some bit of code is not following all of the right
conventions.  I suspect most Emacs users are adding setqs to their init
file until things work.  Meanwhile I foolishly read and follow all of
the conventions and thus I make the above claim ("It is easy to get
subtle bugs...") from my experience running into other bugs like this one.
Hence my stance on fixing the aforementioned 132 occurrences, lest they
cause yet another subtle bug later on.

;; tmp.el starts here.

;; This is usually called by Emacs before loading init.el.
;; It is skipped when using -Q so we call it manually.
;; https://www.gnu.org/software/emacs/manual/html_node/elisp/Startup-Summary.html
(package-activate-all)

;; This is added to init.el and is the standard way user
;; customizations are saved/loaded.
(custom-set-variables
 ;; custom-set-variables was added by Custom.
 ;; If you edit it by hand, you could mess it up, so be careful.
 ;; Your init file should contain only one such instance.
 ;; If there is more than one, they won't work right.
 '(org-id-link-to-org-use-id 'create-if-interactive)
 '(org-babel-load-languages '((emacs-lisp . t) (python . t) (shell . t))))

;; tmp.el ends here.

Re: [BUG] ob-tangle overrides user customization of org-id-link-to-org-use-id [9.5.2 (9.5.2-gfbff08 @ /home/ionasal/.emacs.d/elpa/org-9.5.2/)]

[Ihor Radchenko]

Allen Li <darkfeline@felesatra.moe> writes:

> I see.  In my opinion those occurrences should be fixed even if no one
> is reporting issues because it is bad/improper code, and it is not
> especially surprising that no one has reported it yet; there is always a
> first person who reports a bug, and there are always more old bugs to be
> found in a big project like Emacs/Org mode.
>
> This is annoying to reproduce because it relies on the normal Emacs
> startup process, which loads packages.  Anyway, I have found a reproduction.
>
> ...
> 3. Press RET on the customize link.
>
> Note that the Customize UI says "CHANGED outside Customize."

I was also able to reproduce the problem following your steps. Thank you!

Unfortunately, fixing the occurrences of (defvar foo nil) is not
completely straightforward. Some of them are real defvars.

If someone is willing to check all the occurrences of
(defvar +[^ ]+ +nil) and remove nil values where we merely put a defvar
to silence byte-compiler, please do it.

Also, I feel that my Elisp-foo is not good enough in this specific
scenario. I invite others to reply if they feel that changing
(defvar foo nil) to (defvar foo) for externally defined variables is
_not_ universally safe.

Best,
Ihor

Re: [BUG] ob-tangle overrides user customization of org-id-link-to-org-use-id [9.5.2 (9.5.2-gfbff08 @ /home/ionasal/.emacs.d/elpa/org-9.5.2/)]

[Allen Li]

Ihor Radchenko <yantar92@gmail.com> writes:
> Unfortunately, fixing the occurrences of (defvar foo nil) is not
> completely straightforward. Some of them are real defvars.

I see, I thought you meant those defvars were also used for dynamic
scoping.

>
> If someone is willing to check all the occurrences of
> (defvar +[^ ]+ +nil) and remove nil values where we merely put a defvar
> to silence byte-compiler, please do it.

My initial request was just to fix this one instance that is actively
troubling me.  If there are any other known cases of defvar, they should
also be fixed.  If not, then let's wait until someone identifies them.

If any maintainers reading this could fix this one defvar, that'd be
great.

>
> Also, I feel that my Elisp-foo is not good enough in this specific
> scenario. I invite others to reply if they feel that changing
> (defvar foo nil) to (defvar foo) for externally defined variables is
> _not_ universally safe.
>
> Best,
> Ihor

Re: [BUG] ob-tangle overrides user customization of org-id-link-to-org-use-id [9.5.2 (9.5.2-gfbff08 @ /home/ionasal/.emacs.d/elpa/org-9.5.2/)]

[Ihor Radchenko]

Allen Li <darkfeline@felesatra.moe> writes:

>> If someone is willing to check all the occurrences of
>> (defvar +[^ ]+ +nil) and remove nil values where we merely put a defvar
>> to silence byte-compiler, please do it.
>
> My initial request was just to fix this one instance that is actively
> troubling me.  If there are any other known cases of defvar, they should
> also be fixed.  If not, then let's wait until someone identifies them.
>
> If any maintainers reading this could fix this one defvar, that'd be
> great.

I just pushed the fix onto bugfix as dd6486a07.
Most of the occurrences of (defvar foo nil) in Org are just actual
variable definitions without docstring (which is not good, but not an
error).

Also, I only skimmed through org-agenda.el. I am afraid to touch that
monster even though most of such defvars are in org-agenda.el

Best,
Ihor