On 30/04/2023 17:39, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
>>       ./epm.el -Q --epm-dir $(emacs_pkgdir)/emacs-%e install compat
> 
> Good idea. Although, we should not overdo this package management thing.
> If we really need complex functionality here, we should better just use
> cask/eldev instead of re-inventing the wheel.

I have not tried cask or eldev, so I can not reason on supposed workflow.

> Are you willing to improve the draft to be ready for upstream?

See the attachment. Interface is subject to change to better fit 
particular use cases.

> I think that we need to zoom out a bit and discuss the contexts where
> Org build system is used:
> 
> 1. During Org development, by developers who know what they are doing

...and who prefer failure to files installed to unexpected directory 
when configuration contains a mistake or it is not activated.

> 2. By ordinary users, not necessarily familiar with GNU make and all the
>     associated build process conventions.

For those who are not familiar with convention any target may be 
specified in docs, keeping usual meaning of the default "all" target.

> For example, my `package-user-dir' contains a
> number of forks with additional patches applied - it is not the
> environment I want to develop (and test) Org in.

I think, for ordinary user it is better to keep build time and runtime 
`package-user-dir' the same. Developers, especially when multiple 
versions are involved, should have isolated sandboxes similar to 
"python3 -m venv" and activating scripts setting proper environment 
variables.

> Org users will likely use make autoloads, make, make docs, and make
> repro.

make autoloads should be necessary only to run org uncompiled. My 
impression is that some bugs may exist, so make clean and make autoloads 
are necessary during updates.

> However, make repro and
> optionally make docs should avoid re-using user packages as it may cause
> inconsistent results if the `package-user-dir' is messed up.

I agree concerning "make repro", but unsure for docs.

> One way to handle the above scenarios might be your idea with AUTODEP.
> By default, it will be "auto":
>   - make compile, docs  :: Re-use default `package-user-dir'
>   - make repro    :: Auto-download and ignore `package-user-dir'
>   - other targets :: Prompt the user
> Alternatives will be meant to be used as
> 
>      AUTODEP=download/user/no make target.
> 
> triggering unconditional downloading, using `package-user-dir', and not
> using any guess, correspondingly.

In general I agree that strategy may depend on specified target. The 
only issue that make allows to specify several targets. An I am unsure 
concerning user prompt.

I have realized that as soon as build dependencies are involved, 
Makefile should use emacs -q, not emacs -Q since -Q excludes site-lisp 
directories created by e.g. elpa-compat debian package.

> I think that it is stretching a bit beyond the complexity we should
> allow within Org build system. In your scenario, I can simply do
> make cleanpkg and re-download the latest dependencies.

I would prefer clear error message that package version is not 
satisfactory. However such feature may be added later.

>> #!/bin/sh
>> ":"; # -*- mode: emacs-lisp; lexical-binding: t; -*-
>> ":"; exec emacs --script "$0" "$@"
> 
> Let's not lock to bash. AFAIK, our makefiles can currently work on
> Windows. Using /bin/sh will lead to regression.

It is POSIX shell, not BASH. I am unsure if make can be used on windows 
when e.g. cygwin is not available. Makefiles are full of POSIX tool 
invocations.

Anyway, since emacs binary is customizable in Makefile, the correct way 
to use this script from Makefile is like (perhaps with more flags)

$(EMACS_Q) --script mk/epm.el --epm-dir $(EPMDIR) missing compat

A convenience way

mk/epm.el -q report compat htmlize

is for default emacs from PATH when a user works with shell prompt.


>> 	   (dir (directory-file-name (expand-file-name fmt-expanded))))
>>        ;; `package-user-dir' ~/.emacs.d/elpa by default
>>        ;; `package-directory-list' does not include it
> 
> What does this comment refer to?

To lack of my experience with package.el and site-lisp infrastructure.