Re: [PATCH] Autoload `org-assert-version' and remove org-loaddefs.el

[Max Nikulin <manikulin@gmail.com>]

On 10/04/2023 23:58, Ihor Radchenko wrote:
> Max Nikulin writes:
>> Currently I do not understand:
>> - Why presence of .el files in the same directory (old Org version) with
>> .elc files affects result of compilation (at least for Emacs-28)?
> 
> I may be missing your point, but `load-prefer-newer' maybe?

I mean installing ELPA Org package to Emacs < 29 when another Org 
version is loaded. The result is different when Emacs is running from 
source tree (raw org .el sources are available) or from install tree 
(lisp files are compressed to .el.gz). In the former case Org is 
compiled correctly. In the latter case compressed sources causes 
mixed-compilation issue with following failures to load Org. I think, it 
is a reason why you were unable to reproduce mixed-compilation issue on 
Gentoo.

The following commit

https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=9dfd945a2c2
Fix byte compilation of package built-ins
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=49708
(no discussion in the bug report)

introduced `package--reload-previously-loaded` instead of 
`package--reload-previously-loaded` and `package--list-loaded-files' (A 
side note. I missed a public function that reloads subset of files from 
a given directory that are present in `load-path' to use it in 
`org-reload').

`package--list-of-conflicts' had a couple of bugs:

1. Looking up for loaded earlier files, composition of 
`file-name-sans-extension` and `find-library-name' strips just `.gz', 
not `.el.gz' thus comparing e.g. '/<path>/org.el' (.gz stripped) and 
'/<path>/org' (.elc stripped). As a result new .el Org files are not 
reloaded over Org .elc files from an older version.

2. A debian-specific issue when elpa-org deb package is installed:

ls -l /usr/share/emacs/site-lisp/elpa/org-9.3.1/org.*
lrwxrwxrwx 1 root root     53 Jan 29  2021 
/usr/share/emacs/site-lisp/elpa/org-9.3.1/org.el -> 
/usr/share/emacs/site-lisp/elpa-src/org-9.3.1//org.el
-rw-r--r-- 1 root root 710150 Jan 29  2021 
/usr/share/emacs/site-lisp/elpa/org-9.3.1/org.elc

Notice that .el files are symlinks to another directory. Due to 
`file-truename' in `package--list-of-conflicts' .el files are not 
reloaded because full paths are compared.

So prior to version 29, Emacs had code that is intended to reload 
earlier loaded files when ELPA package is installed, but it was buggy. 
It seems both issues are fixed in new `package--reload-previously-loaded'.

Now I understand what happens, but I have not figure out what is the 
best strategy to prevent failures. Of course, it should be clearly 
stated in the manual and in the README that Org should not be loaded yet 
when `package-install' is called.

>> - Why even when the `org-assert-version' macro is defined, an error is
>> signaled on attempt to load a compiled file with unexpanded
>> (org-assert-version) call (a file compiled with warning "the function
>> ‘org-assert-version’ is not known")?
> 
> This is because `org-assert-version' was not defined in Emacs during
> compile time. During compilation, Emacs produces byte code calling a
> function. AFAIU, the byte code is equivalent to
> (funcall #'org-assert-version), which fails with error (try it with M-:).

You are right. I did not realized that actually 2 kinds of errors happens:
- Invalid function: org-assert-version
- Symbol’s function definition is void: org-assert-version
   (and compile Warning: the function ‘org-assert-version’ is not known 
to be defined.

"void"/"not known" happens when old org-macs.el is loaded. "Invalid 
function" is the sign of mixed compilation. `org-assert-version' is 
defined, but it is a macro, so can not be called during loading of an 
.elc file since it was not expanded during compilation.

So `fboundp' is not the correct way to generate meaningful message 
during loading result of mixed compilation. `macrop' is more selective 
(and `functionp' as well).

Relevant section of the elisp manual
- (info "(elisp) Function Cells")
- (info "(elisp) Calling Functions")
- (info "(elisp) What Is a Function")