[BUG] Setting org-map-continue from to avoid skipping headings while archiving no longer works [9.5.2 (9.5.2-gd01235 @ /home/john/.emacs.d/straight/build/org/)]

[BUG] Setting org-map-continue from to avoid skipping headings while archiving no longer works [9.5.2 (9.5.2-gd01235 @ /home/john/.emacs.d/straight/build/org/)]

[John Mathena]

[-- Attachment #1: Type: text/plain, Size: 2187 bytes --]

It's a common workflow to iterate over all Org headings in a given file,
archiving those marked with the "DONE" keyword. The only problem is that
archiving changes the structure of the file you're operating on, so if you
have test.org:

    * DONE A
    * DONE B
    * DONE C
    * DONE D

A will be archived, moving [B, C, D] up, with point on B. Then, point is
advanced to the next heading, so B gets skipped in favor of C getting
archived. The end result is B and D remaining in the file instead of being
archived.

In Org 9.4.4, you can fix this issue with a snippet like this:

(defun my-org-archive-done-tasks ()
  "Archive all top-level tasks with the DONE keyword in the current file."
  (interactive)
  (org-map-entries
   (lambda ()
     (org-archive-subtree)
     (setq org-map-continue-from (org-element-property :begin
(org-element-at-point))))
   "LEVEL=1/DONE"
   'file))

...where you set org-map-continue-from in order to avoid skipping every
other heading. However, upon updating to Org 9.5.2-gd01235, this function
no longer works - if you have a file like test.org above, eval
"my-org-archive-done-tasks", and try to run it on the file, every other
heading is still skipped. I verified this by trying this function on
test.org twice, first after running "emacs -Q", and then after doing "emacs
-Q -l ~/.emacs.d/straight/repos/straight.el/bootstrap.el" and "M-x
straight-use-package org"

I believe this is a result of recent additions in org-scan-tags - I hit a
dead end there while trying to determine the exact cause of the change (the
caching stuff that was added is spooky for a new user like me).

Is the snippet above no longer expected to work in Org 9.5.2 due to the
logic for mapping over entries having changed, or is this a bug? If it's
the former, what's the intended way of archiving all DONE tasks in a file
now?

Emacs   : GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version
3.24.27, cairo version 1.17.4) of 2021-03-26
Package: Org mode version 9.5.2 (9.5.2-gd01235 @
/home/john/.emacs.d/straight/build/org/)

(And the function above *does* work in Package: Org mode version 9.4.4
(release_9.4.4 @ /usr/share/emacs/27.2/lisp/org/))

Best,
John

[-- Attachment #2: Type: text/html, Size: 2574 bytes --]

Re: [BUG] Setting org-map-continue from to avoid skipping headings while archiving no longer works [9.5.2 (9.5.2-gd01235 @ /home/john/.emacs.d/straight/build/org/)]

[Ihor Radchenko]

John Mathena <jmmathena@gmail.com> writes:

> In Org 9.4.4, you can fix this issue with a snippet like this:
> ...
>      (setq org-map-continue-from (org-element-property :begin
>
> I believe this is a result of recent additions in org-scan-tags - I hit a
> dead end there while trying to determine the exact cause of the change (the
> caching stuff that was added is spooky for a new user like me).
>
> Is the snippet above no longer expected to work in Org 9.5.2 due to the
> logic for mapping over entries having changed, or is this a bug? If it's
> the former, what's the intended way of archiving all DONE tasks in a file
> now?

Hi. Sorry for the inconvenience. The logic has indeed been changed to
speed things up.

I tried to make sure that org-map-continue-from is not broken, but your
scenario is apparently not covered by our tests.

Let me investigate and fix this.

Best,
Ihor

Re: [BUG] Setting org-map-continue from to avoid skipping headings while archiving no longer works [9.5.2 (9.5.2-gd01235 @ /home/john/.emacs.d/straight/build/org/)]

[Ihor Radchenko]

Ihor Radchenko <yantar92@gmail.com> writes:

> I tried to make sure that org-map-continue-from is not broken, but your
> scenario is apparently not covered by our tests.
>
> Let me investigate and fix this.

It turned out to be more tricky than I thought.

Fixed by 06f58e475

Let me know if you are still seeing the problem.

Best,
Ihor

Re: [BUG] Setting org-map-continue from to avoid skipping headings while archiving no longer works [9.5.2 (9.5.2-gd01235 @ /home/john/.emacs.d/straight/build/org/)]

[John Mathena]

> Let me know if you are still seeing the problem.

This may be a deficiency in my function (I basically copied it
wholesale after realizing the source of the "skipping" issue so there
may be context I'm missing), but I'm noticing a problem when the
heading-to-be-archived comes last in the file. For instance, if you
call the function defined in my first email while in test.org:

* TODO A
* TODO B
* DONE C

...Emacs will hang, locking up until you "C-g". This minimal example
was confirmed on Emacs 29.0.50 and Org 9.5.2-gb8656a (and I'd
previously experienced the same problem in Emacs 27.2 before I
upgraded, although I didn't run any "emacs -Q" tests on that
particular build).

Again, I'd totally understand if this is just an end-user problem,
where I need to write a proper end condition for my archiving function
- if you think that's the case, let me know! I just wanted to bring
your attention to Emacs locking up in case it was unforeseen on your
end.

Best,
John

Re: [BUG] Setting org-map-continue from to avoid skipping headings while archiving no longer works [9.5.2 (9.5.2-gd01235 @ /home/john/.emacs.d/straight/build/org/)]

[Ihor Radchenko]

John Mathena <jmmathena@gmail.com> writes:

> * TODO A
> * TODO B
> * DONE C
>
> ...Emacs will hang, locking up until you "C-g". This minimal example
> was confirmed on Emacs 29.0.50 and Org 9.5.2-gb8656a (and I'd

Thanks for reporting! This was a problem on Org side. Should be fixed now.

> Again, I'd totally understand if this is just an end-user problem,
> where I need to write a proper end condition for my archiving function
> - if you think that's the case, let me know! I just wanted to bring
> your attention to Emacs locking up in case it was unforeseen on your
> end.

Your function indeed looks fishy in sense that it can sometimes force
Org to call your function twice on the same element. I think your
specific combination of matcher and the function should do just fine
(after I fixed the bug), but you may need to be careful about moving to
element at point at the end of buffer if you reuse this code for more
complex tasks.

Best,
Ihor