Suggestion: convert dispatchers to use transient

Suggestion: convert dispatchers to use transient

[Hugo Heagren]

Org uses various dispatchers, where invoking a command gives the user a
choice of different sub-commands, chosen by pressing a relevant key,
from a list displayed on the screen. Some of these dispatchers include
options which can affect the command chosen. Examples include org-
capture, org-beamer-select-environment and org-export-dispatch.

These desptachers are idiosyncratic, written for purpose, and each
behave differently. They have varying levels of customisability, and
this is reached in different ways for each. Overall, I think the user-
experience could be more consistent and more easily customisable.

Luckily, recent versions of emacs ship with transient.el, a powerful
way of building such interfaces in a consistent and easily extensible
way.

So, I propose to rewrite the current dispatchers as transients. What
does the community think? I would be happy to work on this unless
others strongly object, but I don't  know everything about org, so if
others could help me with a list of other dispatchers which could also
be converted that would be helpful.

Blue skies, Hugo

Re: Suggestion: convert dispatchers to use transient

[Tom Gillespie]

The backward compatibility requirements for org mean that it won't be
possible to replace the existing implementation
for quite a while. That said, I imagine that having optional transient
dispatchers for users on newer versions of emacs would be appreciated.
Best,
Tom

Re: Suggestion: convert dispatchers to use transient

[Tim Cross]

Hugo Heagren <hugo@heagren.com> writes:

> Org uses various dispatchers, where invoking a command gives the user a
> choice of different sub-commands, chosen by pressing a relevant key,
> from a list displayed on the screen. Some of these dispatchers include
> options which can affect the command chosen. Examples include org-
> capture, org-beamer-select-environment and org-export-dispatch.
>
> These desptachers are idiosyncratic, written for purpose, and each
> behave differently. They have varying levels of customisability, and
> this is reached in different ways for each. Overall, I think the user-
> experience could be more consistent and more easily customisable.
>
> Luckily, recent versions of emacs ship with transient.el, a powerful
> way of building such interfaces in a consistent and easily extensible
> way.
>
> So, I propose to rewrite the current dispatchers as transients. What
> does the community think? I would be happy to work on this unless
> others strongly object, but I don't  know everything about org, so if
> others could help me with a list of other dispatchers which could also
> be converted that would be helpful.
>

This has been discussed previously and I believe the overall consensus
was that moving to transient is the right direction to take.

However, the stumbling point at this time is that org needs to retain
backwards compatibility with at least 2 previous Emacs releases. The
current release is 27.2, so we need to maintain compatibility with 26.x
and 25.x. Emacs 28 is at 'release candidate' status, so we can expect it
will be released within the next 6 months or so. I expect the next org
release will also occur about then. We would then only need
compatibility with 26.x, 27.2 in addition to 28.0

After 28.0 is released, if transient is a GNU ELPA package, we can
probably just make it an org dependency and Emacs 26.x and 27.x should
be able to install and run it (would need to be verified). 

Re: Suggestion: convert dispatchers to use transient

[Jim Porter]

On 2/3/2022 2:07 AM, Tim Cross wrote:
> After 28.0 is released, if transient is a GNU ELPA package, we can
> probably just make it an org dependency and Emacs 26.x and 27.x should
> be able to install and run it (would need to be verified).

The transient package is on GNU ELPA already[1], and based on its 
package metadata[2], supports Emacs 25.1+, so I don't think there should 
be any compatibility concerns (at least not with the availability of 
transient; maybe there are other reasons that dispatchers shouldn't 
change without an option to go back).

- Jim

[1] https://elpa.gnu.org/packages/transient.html
[2] 
https://github.com/magit/transient/blob/440a341831398b825dc2288a10821cf7be1999ca/lisp/transient.el#L9

Re: Suggestion: convert dispatchers to use transient

[Tim Cross]

Jim Porter <jporterbugs@gmail.com> writes:

> On 2/3/2022 2:07 AM, Tim Cross wrote:
>> After 28.0 is released, if transient is a GNU ELPA package, we can
>> probably just make it an org dependency and Emacs 26.x and 27.x should
>> be able to install and run it (would need to be verified).
>
> The transient package is on GNU ELPA already[1], and based on its package
> metadata[2], supports Emacs 25.1+, so I don't think there should be any
> compatibility concerns (at least not with the availability of transient; maybe
> there are other reasons that dispatchers shouldn't change without an option to
> go back).
>
> - Jim
>
> [1] https://elpa.gnu.org/packages/transient.html
> [2]
> https://github.com/magit/transient/blob/440a341831398b825dc2288a10821cf7be1999ca/lisp/transient.el#L9

That is good to know. I knew it was getting added, but wasn't sure how
compatible it would be with older versions.

Not 100% sure about best migration strategy. On one level, providing the
ability for users to choose which interface sounds nice, but on the
other, a main goal is to simplify code and reduce maintenance. Providing
a choice will actually complicate matters and increase maintenance.

Is the transient.el included in Emacs 28 the same as the one on GNU
ELPA? I'm assuming it is, but I have to admit I'm still not 100% clear
on how Emacs handles the situation where you use a library that is both
built-in and available in ELPA. Does Emacs use the latest version
available or does it use the built-in version until you explicitly
select the ELPA versions?

Re: Suggestion: convert dispatchers to use transient

[Jim Porter]

On 2/3/2022 1:30 PM, Tim Cross wrote:
> Is the transient.el included in Emacs 28 the same as the one on GNU
> ELPA?

Currently, yes. They're both 0.3.7. However, that could obviously change 
in the future (e.g. a user on 28.1 would likely have transient 0.3.7, 
though GNU ELPA might have transient 0.3.8 if it were released one day).

> I'm assuming it is, but I have to admit I'm still not 100% clear
> on how Emacs handles the situation where you use a library that is both
> built-in and available in ELPA. Does Emacs use the latest version
> available or does it use the built-in version until you explicitly
> select the ELPA versions?
Emacs will use the built-in version of a package provided that package 
meets the version requirements. For the version of Org Mode that comes 
with Emacs, it would always use the built-in transient (largely because 
maintainers wouldn't do it any other way). If you download a newer Org 
version from GNU ELPA, I believe it checks the package metadata and then 
will fetch transient from GNU ELPA as well if the built-in version is 
too old.

(I haven't looked at the code to verify this, but it's how Eglot worked 
when I installed it. Eglot wants newer versions of eldoc, xref, project, 
etc than Emacs 27.2 provides, so package.el automatically installs them 
when installing Eglot.)

- Jim

Re: Suggestion: convert dispatchers to use transient

[Christopher M. Miles]

[-- Attachment #1: Type: text/plain, Size: 866 bytes --]


Tim Cross <theophilusx@gmail.com> writes:

> Is the transient.el included in Emacs 28 the same as the one on GNU
> ELPA? I'm assuming it is, but I have to admit I'm still not 100% clear
> on how Emacs handles the situation where you use a library that is both
> built-in and available in ELPA. Does Emacs use the latest version
> available or does it use the built-in version until you explicitly
> select the ELPA versions?

I'm curious about this question too. What if user explicitly load a package (not built-in but also
have Emacs built-in version like Dired etc) before org-mode requires it?

-- 
[ stardiviner ]
       I try to make every word tell the meaning that I want to express.

       Blog: https://stardiviner.github.io/
       IRC(freenode): stardiviner, Matrix: stardiviner
       GPG: F09F650D7D674819892591401B5DF1C95AE89AC3

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Suggestion: convert dispatchers to use transient

[João Pedro de Amorim Paula]

On 04 February 2022 08:30, Tim Cross <theophilusx@gmail.com> wrote:

> I'm assuming it is, but I have to admit I'm still not 100% clear on
> how Emacs handles the situation where you use a library that is both
> built-in and available in ELPA. Does Emacs use the latest version
> available or does it use the built-in version until you explicitly
> select the ELPA versions?

Welp, I happened to ponder the same question after trying to implement a
function to install packages that are not installed already (I'm not
using any helper configuration such as use-package, which would already
handle this). From what I gathered empirically, it appears that if
something is built-in, Emacs' package.el won't try to install -- it
checks with package-installed-p on any call to package-install, which
checks if the package is built-in with package-built-in-p as a fallback
on cond --, but you can force installation from the archives by passing
a package description object (defined as package-desc on package.el)
instead of a symbol

    (let ((pkg-desc (assq 'org package-archive-contents)))
      (package-install pkg-desc))

and Emacs seems to be loading the newest version when it is a dependency
of something else. Though I'm not really sure, as most of the packages I
have from ELPA have the same version as the ones built-in on Emacs
28.0.91. I'll try and install Emacs 27 to check this out.

In the mean time, I guess this would be a good opportunity to share a
couple of functions I have with the purpose of installing packages.

    (defun pkg-description (package)
      "Return the description for PACKAGE.
    If PACKAGE is installed, the will be present on `package-alist',
    otherwise look for it in `package-archive-contents'."
      (or (cadr (assoc package package-alist))
          (cadr (assoc package package-archive-contents))))

    (defun pkg-ensure-archive (package)
      "Install PACKAGE from the archives, if not already installed."
      (when-let ((pkg-desc (pkg-description package)))
        (unless (package-installed-p pkg-desc)
          (package-install-from-archive pkg-desc))))

    (defun require-package (package &optional force)
      "Ensure that PACKAGE is installed.
    If FORCE is non-nil, force installation regardless if PACKAGE is
    built-in or not.

    First, use `package-installed-p' to check if PACKAGE was
    installed via the Emacs package manager, otherwise, try to
    `require' PACKAGE; this ensures that we don't require PACKAGE if
    it was installed using the package manager. If both of those
    fail, run `package-refresh-contents' and install PACKAGE."
      (unless (and (not force)
                   (or (package-installed-p package)
                       (require package nil 'no-error)))
        (unless (assoc package package-archive-contents)
          (package-refresh-contents))
        (if force
            (pkg-ensure-archive package)
          (package-install package))))

On the last one, the main function that I use, if FORCE is non-nil it
will download and install the package from the archives even if it is
built-in.

Best regards,

-- 
João Pedro de Amorim Paula
IT undergraduate at Universidade Federal do Rio Grande do Norte (UFRN)

Re: Suggestion: convert dispatchers to use transient

[Samuel Wales]

jus want to state the obvious here in case it is useful [but everybody
probbly lready knows] --- not everybody uses packages or maybe trusts
them has internet etc.

On 2/5/22, João Pedro de Amorim Paula <jpedrodeamorim@gmail.com> wrote:
> On 04 February 2022 08:30, Tim Cross <theophilusx@gmail.com> wrote:
>
>> I'm assuming it is, but I have to admit I'm still not 100% clear on
>> how Emacs handles the situation where you use a library that is both
>> built-in and available in ELPA. Does Emacs use the latest version
>> available or does it use the built-in version until you explicitly
>> select the ELPA versions?
>
> Welp, I happened to ponder the same question after trying to implement a
> function to install packages that are not installed already (I'm not
> using any helper configuration such as use-package, which would already
> handle this). From what I gathered empirically, it appears that if
> something is built-in, Emacs' package.el won't try to install -- it
> checks with package-installed-p on any call to package-install, which
> checks if the package is built-in with package-built-in-p as a fallback
> on cond --, but you can force installation from the archives by passing
> a package description object (defined as package-desc on package.el)
> instead of a symbol
>
>     (let ((pkg-desc (assq 'org package-archive-contents)))
>       (package-install pkg-desc))
>
> and Emacs seems to be loading the newest version when it is a dependency
> of something else. Though I'm not really sure, as most of the packages I
> have from ELPA have the same version as the ones built-in on Emacs
> 28.0.91. I'll try and install Emacs 27 to check this out.
>
> In the mean time, I guess this would be a good opportunity to share a
> couple of functions I have with the purpose of installing packages.
>
>     (defun pkg-description (package)
>       "Return the description for PACKAGE.
>     If PACKAGE is installed, the will be present on `package-alist',
>     otherwise look for it in `package-archive-contents'."
>       (or (cadr (assoc package package-alist))
>           (cadr (assoc package package-archive-contents))))
>
>     (defun pkg-ensure-archive (package)
>       "Install PACKAGE from the archives, if not already installed."
>       (when-let ((pkg-desc (pkg-description package)))
>         (unless (package-installed-p pkg-desc)
>           (package-install-from-archive pkg-desc))))
>
>     (defun require-package (package &optional force)
>       "Ensure that PACKAGE is installed.
>     If FORCE is non-nil, force installation regardless if PACKAGE is
>     built-in or not.
>
>     First, use `package-installed-p' to check if PACKAGE was
>     installed via the Emacs package manager, otherwise, try to
>     `require' PACKAGE; this ensures that we don't require PACKAGE if
>     it was installed using the package manager. If both of those
>     fail, run `package-refresh-contents' and install PACKAGE."
>       (unless (and (not force)
>                    (or (package-installed-p package)
>                        (require package nil 'no-error)))
>         (unless (assoc package package-archive-contents)
>           (package-refresh-contents))
>         (if force
>             (pkg-ensure-archive package)
>           (package-install package))))
>
> On the last one, the main function that I use, if FORCE is non-nil it
> will download and install the package from the archives even if it is
> built-in.
>
> Best regards,
>
> --
> João Pedro de Amorim Paula
> IT undergraduate at Universidade Federal do Rio Grande do Norte (UFRN)
>


-- 
The Kafka Pandemic

A blog about science, health, human rights, and misopathy:
https://thekafkapandemic.blogspot.com

Re: Suggestion: convert dispatchers to use transient

[João Pedro de Amorim Paula]

On 05 February 2022 18:49, Samuel Wales <samologist@gmail.com> wrote:

> jus want to state the obvious here in case it is useful [but everybody
> probbly lready knows] --- not everybody uses packages or maybe trusts
> them has internet etc.

Indeed! What I posted assumes using packages, but they also work with
any package-desc object. So, if you download it locally and build the
package-desc for it it should also work, just have to put it in place of
the assq:

>>     (let ((pkg-desc (assq 'org package-archive-contents)))
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>       (package-install pkg-desc))

Although Tim was referring to packages on ELPA that are also built-in.

-- 
João Pedro de A. Paula
IT undergraduate at Universidade Federal do Rio Grande do Norte (UFRN)

Re: Suggestion: convert dispatchers to use transient

[Anders Johansson]

I agree that transient dispatchers would be great and consistent. What is not so great with transient currently is it’s lacking support for lisp-variable/option workflows. Transient is good for command-line arguments, but setting options for emacs functions is not as streamlined (this would be relevant for the export dispatcher for example). 

Some new classes for these types of options would probably need to be defined, since the default infix-classes assume all options are strings.

I recently did try to implement a transient dispatcher this for my orgqda package:
https://gitlab.com/andersjohansson/orgqda/-/blob/main/orgqda-transient.el

Partly inspired by org-ql:
https://github.com/alphapapa/org-ql/blob/048de35b07d8d29fc6fa49c34d7ba1fd1c912011/org-ql-view.el#L501

I now store the options as plists in buffer-local-variables, which I am not sure is optimal and perhaps goes against the idea of saving options with transient. So still some work to do there.

By the way. Getting started with transient is way easier with this tutorial than with the rather abstract info pages:
https://github.com/magit/transient/wiki/Developer-Quick-Start-Guide

Best,
Anders Johansson