[BUG] Infinite loop in org-agenda-show-new-time

[BUG] Infinite loop in org-agenda-show-new-time

[Matt Lundin]

When the agenda buffer is filtered by tag, I find that rescheduling an
item (via org-agenda-do-date-later, org-agenda-schedule, etc.) often
results in an infinite loop.

I believe this loop occurs in the function org-agenda-show-new-time.

Here are the steps I took to debug this:

1. I add a counter to the function:

--8<---------------cut here---------------start------------->8---
(defun org-agenda-show-new-time (marker stamp &optional prefix)
  "Show new date stamp via text properties."
  ;; We use text properties to make this undoable
  (let ((inhibit-read-only t))
    (setq stamp (concat prefix " => " stamp " "))
    (setq my-counter 0) ;; <-- my addition
    (save-excursion
      (goto-char (point-max))
      (while (not (bobp))
	(when (equal marker (org-get-at-bol 'org-marker))
	  (org-move-to-column (- (window-width) (length stamp)) t)
	  (org-agenda-fix-tags-filter-overlays-at (point))
	  (setq my-counter (1+ my-counter)) ;; <-- also my addition
          (if (featurep 'xemacs)
	      ;; Use `duplicable' property to trigger undo recording
              (let ((ex (make-extent nil nil))
                    (gl (make-glyph stamp)))
                (set-glyph-face gl 'secondary-selection)
                (set-extent-properties
                 ex (list 'invisible t 'end-glyph gl 'duplicable t))
                (insert-extent ex (1- (point)) (point-at-eol)))
            (add-text-properties
             (1- (point)) (point-at-eol)
	     (list 'display (org-add-props stamp nil
			      'face 'secondary-selection))))
	  (beginning-of-line 1))
	(beginning-of-line 0)))))
--8<---------------cut here---------------end--------------->8---

2. I narrow an agenda diary buffer by tag (e.g., "home").

3. I reschedule an item.

4. Emacs hangs.

5. I hit C-g to stop the process.

6. I check the value of my-counter. The longer I let the process run,
the higher the value of my-counter. E.g., letting it run for
approximately ten seconds results in the following:

--8<---------------cut here---------------start------------->8---
my-counter's value is 32193

Documentation:
Not documented as a variable.
--8<---------------cut here---------------end--------------->8---

Without C-g the number will keep increasing indefinitely.

Thanks,
Matt

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Nick Dokos]

Matt Lundin <mdl@imapmail.org> writes:

> When the agenda buffer is filtered by tag, I find that rescheduling an
> item (via org-agenda-do-date-later, org-agenda-schedule, etc.) often
> results in an infinite loop.
>
> I believe this loop occurs in the function org-agenda-show-new-time.
>
> Here are the steps I took to debug this:
>
> 1. I add a counter to the function:
>
>
> 2. I narrow an agenda diary buffer by tag (e.g., "home").
>
> 3. I reschedule an item.
>
> 4. Emacs hangs.
>
> 5. I hit C-g to stop the process.
>
> 6. I check the value of my-counter. The longer I let the process run,
> the higher the value of my-counter. E.g., letting it run for
> approximately ten seconds results in the following:
>
> my-counter's value is 32193
>

My one feeble attempt to reproduce this failed. Looking at the code

--8<---------------cut here---------------start------------->8---
      (while (not (bobp))
	(when (equal marker (org-get-at-bol 'org-marker))
	  (org-move-to-column (- (window-width) (length stamp)) t)
	  (org-agenda-fix-tags-filter-overlays-at (point))
          ...
	  (beginning-of-line 1))
	(beginning-of-line 0)))))
--8<---------------cut here---------------end--------------->8---

let's assume we are not at the beginning of the buffer, so we don't exit
the loop that way.  If the when succeeds, we do a couple of things and
then do (beginning-of-line 1). This just takes us to the beginning of
the current line. But after the when is done, we do (beginning-of-line
0) which should take us to the previous line.  So we should be making
steady progress towards the beginning of the buffer and the loop should
terminate.

Since you can reproduce it (and you've already done the hard work of
figuring out where the inf loop is), maybe you can edebug the function
and step through it a couple of times to see what's happening.
-- 
Nick

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Matt Lundin]

Nick Dokos <ndokos@gmail.com> writes:

> Matt Lundin <mdl@imapmail.org> writes:
>
> My one feeble attempt to reproduce this failed. Looking at the code

Here are the steps to reproduce the problem:

1. Create file test.org with the following content:

--8<---------------cut here---------------start------------->8---
* TODO A							       :home:
  SCHEDULED: <2013-08-05 Mon>
* TODO B							       :work:
  SCHEDULED: <2013-08-05 Mon>
* TODO C							       :play:
  SCHEDULED: <2013-08-05 Mon>
* TODO D							       :home:
  SCHEDULED: <2013-08-05 Mon>
* TODO E							       :work:
  SCHEDULED: <2013-08-05 Mon>
* TODO F							       :play:
  SCHEDULED: <2013-08-05 Mon>
* TODO G							       :home:
  SCHEDULED: <2013-08-05 Mon>
* TODO H							       :work:
  SCHEDULED: <2013-08-05 Mon>
* TODO I							       :play:
  SCHEDULED: <2013-08-05 Mon>
--8<---------------cut here---------------end--------------->8---

2. /usr/bin/emacs -Q 

3. find file test.org

4. M-x org-agenda -> hit "<" to restrict to buffer and then "a" for
diary

5. / home

6. Attempt to reschedule one of the visible items.

>       (while (not (bobp))
> 	(when (equal marker (org-get-at-bol 'org-marker))
> 	  (org-move-to-column (- (window-width) (length stamp)) t)
> 	  (org-agenda-fix-tags-filter-overlays-at (point))
>           ...
> 	  (beginning-of-line 1))
> 	(beginning-of-line 0)))))
>
> let's assume we are not at the beginning of the buffer, so we don't exit
> the loop that way.  If the when succeeds, we do a couple of things and
> then do (beginning-of-line 1). This just takes us to the beginning of
> the current line. But after the when is done, we do (beginning-of-line
> 0) which should take us to the previous line.  So we should be making
> steady progress towards the beginning of the buffer and the loop should
> terminate.
>
> Since you can reproduce it (and you've already done the hard work of
> figuring out where the inf loop is), maybe you can edebug the function
> and step through it a couple of times to see what's happening.

Thanks for the pointers. Running edebug with the file above reveals that
org-move-to-column is not working with the invisible sections of the
buffer.

With the sample file above, I filter the agenda to display only items
tagged :home:.

--8<---------------cut here---------------start------------->8---
Day-agenda (W32):
Monday      5 August 2013 W32
  test:       Scheduled:  TODO A                                          :home:
  test:       Scheduled:  TODO D                                          :home:
  test:       Scheduled:  TODO G                                          :home:
--8<---------------cut here---------------end--------------->8---

When stepping through org-agenda-do-date-later, edebug reveals that the
point goes to the end of the buffer, as expected and then works its way
backward. When it arrives at the beginning of the line with task "G", it
finds and match and executes the following functions:

--8<---------------cut here---------------start------------->8---
	  (org-move-to-column (- (window-width) (length stamp)) t)
	  (org-agenda-fix-tags-filter-overlays-at (point))
--8<---------------cut here---------------end--------------->8---

The problem is that org-move-to-column shifts the point several lines
forward. In fact, if I make all contents of the agenda buffer visible
after edebug executes org-move-to-column, I find that the point is now
all the way at the end of line "I," which, of course, will trigger an
endless loop. In other words, org-move-to-column moves the point to the
end of the entire invisible section.

--8<---------------cut here---------------start------------->8---
Day-agenda (W32):
Monday      5 August 2013 W32
  test:       Scheduled:  TODO A                                          :home:
  test:       Scheduled:  TODO B                                          :work:
  test:       Scheduled:  TODO C                                          :play:
  test:       Scheduled:  TODO D                                          :home:
  test:       Scheduled:  TODO E                                          :work:
  test:       Scheduled:  TODO F                                          :play:
  test:       Scheduled:  TODO G                                          :home:
  test:       Scheduled:  TODO H                                          :work:
  test:       Scheduled:  TODO I                                          :play:
--8<---------------cut here---------------end--------------->8---
                                                                                ^
                                                                                |
                                                                             here

This bug was introduced with the following commit:

--8<---------------cut here---------------start------------->8---
commit fafb5f3429c41cba1eddb9fc78d9f9e0980acbe2
Author: Bastien Guerry <bzg@altern.org>
Date:   Mon Feb 11 14:56:38 2013 +0100

    org-agenda.el: Fix bug when displaying a temporary overlay
    
    * org-agenda.el (org-agenda-schedule, org-agenda-deadline):
    Cosmetic changes.
    (org-agenda-show-new-time): Fix bug when displaying a
    temporary overlay with the scheduled/deadline information.
    
    Thanks to Thomas Morgan for reporting this bug and testing the patch.
--8<---------------cut here---------------end--------------->8---

This commit removed the local binding of buffer-invisibility-spec to
nil in org-agenda-show-new-time.

Here is the bug this change was meant to fix:

http://permalink.gmane.org/gmane.emacs.orgmode/52667

Might we revert this change? The original bug was cosmetic. This bug,
however, interferes in an essential way with the functioning of agenda
buffers.

Best,
Matt

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Nick Dokos]

Matt Lundin <mdl@imapmail.org> writes:

> Nick Dokos <ndokos@gmail.com> writes:
>
>> Matt Lundin <mdl@imapmail.org> writes:
>>
>> My one feeble attempt to reproduce this failed. Looking at the code
>
> Here are the steps to reproduce the problem:
>

Thanks for the recipe: I still can't reproduce it - it just works for
me.

Org-mode version 8.0.7 (release_8.0.7-367-gd1d918 @
/home/nick/elisp/org-mode/lisp/)

GNU Emacs 24.3.50.2 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.10) of
2013-07-14 on pierrot

>
> Thanks for the pointers. Running edebug with the file above reveals that
> org-move-to-column is not working with the invisible sections of the
> buffer.
>
> With the sample file above, I filter the agenda to display only items
> tagged :home:.
>
> Day-agenda (W32):
> Monday      5 August 2013 W32
>   test:       Scheduled:  TODO A                                          :home:
>   test:       Scheduled:  TODO D                                          :home:
>   test:       Scheduled:  TODO G                                          :home:
>
> When stepping through org-agenda-do-date-later, edebug reveals that the
> point goes to the end of the buffer, as expected and then works its way
> backward. When it arrives at the beginning of the line with task "G", it
> finds and match and executes the following functions:
>
> 	  (org-move-to-column (- (window-width) (length stamp)) t)
> 	  (org-agenda-fix-tags-filter-overlays-at (point))
>
> The problem is that org-move-to-column shifts the point several lines
> forward. In fact, if I make all contents of the agenda buffer visible
> after edebug executes org-move-to-column, I find that the point is now
> all the way at the end of line "I," which, of course, will trigger an
> endless loop. In other words, org-move-to-column moves the point to the
> end of the entire invisible section.
>

org-move-to-column is just a compatibility function that calls
move-to-column whose doc says:

  Move point to column COLUMN in the current line.

So I'm not sure why it ends up on a different line.

> This bug was introduced with the following commit:
>
> commit fafb5f3429c41cba1eddb9fc78d9f9e0980acbe2
> Author: Bastien Guerry <bzg@altern.org>
> Date:   Mon Feb 11 14:56:38 2013 +0100
>
>     org-agenda.el: Fix bug when displaying a temporary overlay
>     
>     * org-agenda.el (org-agenda-schedule, org-agenda-deadline):
>     Cosmetic changes.
>     (org-agenda-show-new-time): Fix bug when displaying a
>     temporary overlay with the scheduled/deadline information.
>     
>     Thanks to Thomas Morgan for reporting this bug and testing the patch.
>
> This commit removed the local binding of buffer-invisibility-spec to
> nil in org-agenda-show-new-time.
>
> Here is the bug this change was meant to fix:
>
> http://permalink.gmane.org/gmane.emacs.orgmode/52667
>
> Might we revert this change? The original bug was cosmetic. This bug,
> however, interferes in an essential way with the functioning of agenda
> buffers.
>

So you revert that commit and the infloop goes away? I looked at the
functions briefly and I don't understand how this commit could affect
what move-to-column does.

Can you provide emacs and org versions? Maybe there is something weird
in what you use.
-- 
Nick

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Matt Lundin]

Nick Dokos <ndokos@gmail.com> writes:

> So you revert that commit and the infloop goes away? I looked at the
> functions briefly and I don't understand how this commit could affect
> what move-to-column does.

Just to confirm: Did you try rescheduling with the agenda buffer
filtered only to the home tag using the sample file I provided? To
trigger the bug one must attempt to reschedule an item when 1) the
agenda is filtered by tags and 2) the item being changed contains
invisible lines immediately beneath it (b/c of the filtering). Both
these conditions are necessary to cause the bug on my end.

When the agenda buffer is filtered to a tag (i.e., when it contains
invisible lines), move-to-column treats the entire invisible region as a
single line. Moving to the end of the line moves the point to the end of
the entire invisible region.

An easy way to see this behavior in action is to do the following:

Take the following sample file (test.org):

--8<---------------cut here---------------start------------->8---
* TODO A							       :home:
  SCHEDULED: <2013-08-06 Tue>
* TODO B							       :work:
  SCHEDULED: <2013-08-06 Tue>
* TODO C							       :play:
  SCHEDULED: <2013-08-06 Tue>
--8<---------------cut here---------------end--------------->8---

1. Call the agenda restricted to that file.

--8<---------------cut here---------------start------------->8---
Day-agenda (W32):
Tuesday     6 August 2013
  test:       Scheduled:  TODO A                                          :home:
  test:       Scheduled:  TODO B                                          :work:
  test:       Scheduled:  TODO C                                          :play:
--8<---------------cut here---------------end--------------->8---

2. Narrow an agenda buffer to the tag home.

--8<---------------cut here---------------start------------->8---
Day-agenda (W32):
Tuesday     6 August 2013
  test:       Scheduled:  TODO A                                          :home:
--8<---------------cut here---------------end--------------->8---

3. Move to the beginning of the task "A" line.

4. Call either M-x move-end-of-line or M-x move-to-column 100 (or
another large number).

5. M-: (point)

On my machine the point is at 287, which is at the at the end of the
task "C" line (i.e., the end of the entire invisible region).

If I call move-end-of-line on the task "A" line when the buffer is not
filtered, the point moves to 125 - i.e., the end of the task A line.

In other words, within the agenda buffer, move-to-column and
move-end-of-line will move to the point to the end of the entire
invisible region. That is why removing the local binding of
buffer-invisibility-spec to nil triggers this bug, because when that
variable is nil, the function org-agenda-show-new-time temporarily
treats the agenda buffer as if it were visible (i.e., it ignores the
invisible overlay).

I reproduced the problem with emacs -Q, so I can confirm that it is not
a result of my configuration.

> org-move-to-column is just a compatibility function that calls
> move-to-column whose doc says:
>
>   Move point to column COLUMN in the current line.
>
> So I'm not sure why it ends up on a different line.

I think the following explains why this is happening:
(info "(elisp) Invisible Text")

,----
|    Ordinarily, functions that operate on text or move point do not care
| whether the text is invisible.  The user-level line motion commands
| ignore invisible newlines if `line-move-ignore-invisible' is non-`nil'
| (the default), but only because they are explicitly programmed to do so.
| 
|    However, if a command ends with point inside or at the boundary of
| invisible text, the main editing loop relocates point to one of the two
| ends of the invisible text.  Emacs chooses the direction of relocation
| so that it is the same as the overall movement direction of the
| command; if in doubt, it prefers a position where an inserted char
| would not inherit the `invisible' property.  Additionally, if the text
| is not replaced by an ellipsis and the command only moved within the
| invisible text, then point is moved one extra character so as to try
| and reflect the command's movement by a visible movement of the cursor.
| 
|    Thus, if the command moved point back to an invisible range (with
| the usual stickiness), Emacs moves point back to the beginning of that
| range.  If the command moved point forward into an invisible range,
| Emacs moves point forward to the first visible character that follows
| the invisible text and then forward one more character.
`----

> Can you provide emacs and org versions? Maybe there is something weird
> in what you use.

(org-version)
"Org-mode version 8.0.7 (release_8.0.7-367-gd1d918 @ /home/matt/org-mode/lisp/)"

(emacs-version)
GNU Emacs 24.3.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.8.2) of 2013-07-30 on -var-lib-archbuild-staging-x86_64-jgc

(shell-command "uname -a")
Linux archdesk 3.10.3-1-ARCH #1 SMP PREEMPT Fri Jul 26 11:26:59 CEST 2013 x86_64 GNU/Linux

However, this has been a persistent problem for the last several months
(i.e., it has survived org, emacs, and system updates).

Matt

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Nick Dokos]

[-- Attachment #1: Type: text/plain, Size: 1299 bytes --]

Matt Lundin <mdl@imapmail.org> writes:

> Nick Dokos <ndokos@gmail.com> writes:
>
>> So you revert that commit and the infloop goes away? I looked at the
>> functions briefly and I don't understand how this commit could affect
>> what move-to-column does.
>
> Just to confirm: Did you try rescheduling with the agenda buffer
> filtered only to the home tag using the sample file I provided? To
> trigger the bug one must attempt to reschedule an item when 1) the
> agenda is filtered by tags and 2) the item being changed contains
> invisible lines immediately beneath it (b/c of the filtering). Both
> these conditions are necessary to cause the bug on my end.
>

Just a quick follow-up to this question: I believe I followed your
recipe exactly except for loading a minimal startup to set load-path and
to do initial setup (C-c a etc). The org file I used was just as in your
mail (just now, I changed the dates to today's date before trying the
recipe again). I do

     M-x org-agenda
     <
     a
     / <TAB> home <RET>

(I have to hit <TAB> before typing "home" to get the tag - not sure
that's different or whether it matters), then arrow down to the D TODO and

     M-x org-agenda-date-later

The result is shown in the attached screenshot: success afaict -
certainly no infinite loops.


[-- Attachment #2: Screenshot after rescheduling item D --]
[-- Type: image/png, Size: 74124 bytes --]

[-- Attachment #3: Type: text/plain, Size: 1690 bytes --]


Here's my minimal.org.el file:

--8<---------------cut here---------------start------------->8---
;;; -*- mode: emacs-lisp -*-
(add-to-list 'load-path (expand-file-name "~/src/emacs/org/org-mode/lisp"))
(add-to-list 'load-path (expand-file-name "~/src/emacs/org/org-mode/contrib/lisp"))

(add-to-list 'auto-mode-alist '("\\.\\(org\\|org_archive\\|txt\\)$" . org-mode))

(require 'org)

(setq debug-on-error t)
(setq debug-on-quit t)
(setq eval-expression-print-length nil)
(setq eval-expression-print-level nil)

(global-set-key "\C-cl" 'org-store-link)
(global-set-key "\C-ca" 'org-agenda)
--8<---------------cut here---------------end--------------->8---

here's the foo.org:

--8<---------------cut here---------------start------------->8---
* TODO A							       :home:
  SCHEDULED: <2013-08-06 Tue>
* TODO B							       :work:
  SCHEDULED: <2013-08-06 Mon>
* TODO C							       :play:
  SCHEDULED: <2013-08-06 Mon>
* TODO D							       :home:
  SCHEDULED: <2013-08-06 Tue>
* TODO E							       :work:
  SCHEDULED: <2013-08-06 Mon>
* TODO F							       :play:
  SCHEDULED: <2013-08-06 Mon>
* TODO G							       :home:
  SCHEDULED: <2013-08-06 Mon>
* TODO H							       :work:
  SCHEDULED: <2013-08-06 Mon>
* TODO I							       :play:
  SCHEDULED: <2013-08-06 Mon>
--8<---------------cut here---------------end--------------->8---

and here's the command line:

    emacs -Q -l minimal.org.el foo.org

Can we get some more votes perhaps? If people try the above recipe
(subject to any corrections Matt might have), does emacs go into an
infinite loop?

I haven't read through the rest of your mail yet (let alone tried
anything) but I'll try to do that later tonight.

-- 
Nick

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Nick Dokos]

Oh, oh: I just tried following the steps in your last email and I *did*
get the infinite loop, with the same backtrace that you did. Then I
tried it again just now and it did not happen again. Is this thing not
100% reproducible?

In any case, I'm working through the rest of your email. I should have
more later on tonight.

-- 
Nick

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Matt Lundin]

Nick Dokos <ndokos@gmail.com> writes:

> Oh, oh: I just tried following the steps in your last email and I *did*
> get the infinite loop, with the same backtrace that you did. Then I
> tried it again just now and it did not happen again. Is this thing not
> 100% reproducible?

FWIW, it is 100% reproducible here when there are invisible lines
immediate following the item I am trying to reschedule.

Matt

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Nick Dokos]

Matt Lundin <mdl@imapmail.org> writes:

> Nick Dokos <ndokos@gmail.com> writes:
>
>> Oh, oh: I just tried following the steps in your last email and I *did*
>> get the infinite loop, with the same backtrace that you did. Then I
>> tried it again just now and it did not happen again. Is this thing not
>> 100% reproducible?
>
> FWIW, it is 100% reproducible here when there are invisible lines
> immediate following the item I am trying to reschedule.
>

OK - I've tried a few time since then and I have not been able to hit it
again. I'll keep trying.
-- 
Nick

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Nick Dokos]

Matt Lundin <mdl@imapmail.org> writes:

> ...
> In other words, within the agenda buffer, move-to-column and
> move-end-of-line will move to the point to the end of the entire
> invisible region. That is why removing the local binding of
> buffer-invisibility-spec to nil triggers this bug, because when that
> variable is nil, the function org-agenda-show-new-time temporarily
> treats the agenda buffer as if it were visible (i.e., it ignores the
> invisible overlay).
>

I haven't been able to work on the problem, but assuming that your
diagnosis above is correct, perhaps the thing to do is to bind
buffeer-invisibility-spec to nil inside org-move-to-column:

--8<---------------cut here---------------start------------->8---
(defun org-move-to-column (column &optional force buffer)
  (let ((buffer-invisibility-spec nil))
	(if (featurep 'xemacs)
	    (org-xemacs-without-invisibility (move-to-column column force buffer))
	  (move-to-column column force))))
--8<---------------cut here---------------end--------------->8---

What do you think?
-- 
Nick

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Matt Lundin]

Nick Dokos <ndokos@gmail.com> writes:

> Matt Lundin <mdl@imapmail.org> writes:
>
>> ...
>> In other words, within the agenda buffer, move-to-column and
>> move-end-of-line will move to the point to the end of the entire
>> invisible region. That is why removing the local binding of
>> buffer-invisibility-spec to nil triggers this bug, because when that
>> variable is nil, the function org-agenda-show-new-time temporarily
>> treats the agenda buffer as if it were visible (i.e., it ignores the
>> invisible overlay).
>>
>
> I haven't been able to work on the problem, but assuming that your
> diagnosis above is correct, perhaps the thing to do is to bind
> buffeer-invisibility-spec to nil inside org-move-to-column:
>
> (defun org-move-to-column (column &optional force buffer)
>   (let ((buffer-invisibility-spec nil))
> 	(if (featurep 'xemacs)
> 	    (org-xemacs-without-invisibility (move-to-column column force buffer))
> 	  (move-to-column column force))))
>
> What do you think?

That solves the problem beautifully. Would it have any negative
side-effects for other org functions, especially those that invoke
org-move-to-column in normal org buffers?

Here's a list of such invocations:

--8<---------------cut here---------------start------------->8---
-*- mode: grep; default-directory: "~/org-mode/lisp/" -*-
Grep started at Wed Aug 14 10:14:17

grep -nH -e org-move-to-column *.el
org-agenda.el:8248:    (org-move-to-column col))
org-agenda.el:8260:      (org-move-to-column col)))
org-agenda.el:8766:      (org-move-to-column col))))
org-agenda.el:9130:	  (org-move-to-column (- (window-width) (length stamp)) t)
org-agenda.el:9232:      (org-move-to-column col))))
org-agenda.el:9252:    (org-move-to-column col)
org-clock.el:1851:    (org-move-to-column c)
org-colview.el:503:	(org-move-to-column col)
org-colview.el:632:      (org-move-to-column col)
org-compat.el:338:(defun org-move-to-column (column &optional force buffer)
org.el:9440:	   (if (< (current-column) gc) (org-move-to-column gc t) (insert " "))
org.el:14377:	  (org-move-to-column (min ncol col) t))
org.el:14532:	(org-move-to-column col)
org.el:14616:      (org-move-to-column (- (window-width) 19) t)
org.el:22037:      (org-move-to-column column))))
org.el:22465:		  (org-move-to-column min-indent t))
org.el:22588:	      (org-move-to-column cc t)
org.el:22593:	     ((not off) (org-move-to-column cc t) (insert ": ")))
org.el:23360:      (org-move-to-column c))))
org-list.el:2170:    (org-move-to-column col)))
org-list.el:2189:    (org-move-to-column col)))
org-macs.el:110:	 (org-move-to-column ,col)))))
org-src.el:358:	  (org-move-to-column
org-src.el:364:	(org-move-to-column
org-src.el:537:	(org-move-to-column (max 0 (- col block-nindent 2)))
org-src.el:771:	(org-move-to-column (if preserve-indentation col (+ col total-nindent delta)))))
org-table.el:1121:	  (org-move-to-column col)
org-table.el:1126:	  (org-move-to-column col))
org-table.el:1353:      (org-move-to-column col)
org-table.el:1358:	(org-move-to-column col))
org-table.el:1507:    (org-move-to-column col)
org-table.el:1559:    (org-move-to-column col)
org-table.el:1599:    (org-move-to-column col)

Grep finished (matches found) at Wed Aug 14 10:14:17
--8<---------------cut here---------------end--------------->8---

Matt

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Nick Dokos]

Matt Lundin <mdl@imapmail.org> writes:

> Nick Dokos <ndokos@gmail.com> writes:
>
>>
>> I haven't been able to work on the problem, but assuming that your
>> diagnosis above is correct, perhaps the thing to do is to bind
>> buffeer-invisibility-spec to nil inside org-move-to-column:
>>
>> (defun org-move-to-column (column &optional force buffer)
>>   (let ((buffer-invisibility-spec nil))
>> 	(if (featurep 'xemacs)
>> 	    (org-xemacs-without-invisibility (move-to-column column force buffer))
>> 	  (move-to-column column force))))
>>
>> What do you think?
>
> That solves the problem beautifully. Would it have any negative
> side-effects for other org functions, especially those that invoke
> org-move-to-column in normal org buffers?
>

I hope not, but I don't know for sure. OTOH, we can try it and, if there
are complaints, we can revert it and apply a more localized version of
the same fix: just wrap the relevant *call* of org-move-to-column in a

     (let ((buffer-invisibility-spec nil))
        (org-move-to-column ....))


But (without any solid evidence) it seems to me that having the behavior
of move-to-column change with the buffer-invisibility-spec setting is a
bug: how can anybody expect reproducible behavior from it if you don't
know where point is going to end up? So I would be surprised if the fix
does have negative side effects on anything: on the contrary, it might
resolve some mysteries. OTOH, putting the let in the compat function
would be a workaround for org, but the real fix should probably be in
move-to-column itself. Perhaps a question to emacs devel is warranted.

-- 
Nick

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Carsten Dominik]

On 14.8.2013, at 17:45, Nick Dokos <ndokos@gmail.com> wrote:

> Matt Lundin <mdl@imapmail.org> writes:
> 
>> Nick Dokos <ndokos@gmail.com> writes:
>> 
>>> 
>>> I haven't been able to work on the problem, but assuming that your
>>> diagnosis above is correct, perhaps the thing to do is to bind
>>> buffeer-invisibility-spec to nil inside org-move-to-column:
>>> 
>>> (defun org-move-to-column (column &optional force buffer)
>>>  (let ((buffer-invisibility-spec nil))
>>> 	(if (featurep 'xemacs)
>>> 	    (org-xemacs-without-invisibility (move-to-column column force buffer))
>>> 	  (move-to-column column force))))
>>> 
>>> What do you think?
>> 
>> That solves the problem beautifully. Would it have any negative
>> side-effects for other org functions, especially those that invoke
>> org-move-to-column in normal org buffers?
>> 
> 
> I hope not, but I don't know for sure. OTOH, we can try it and, if there
> are complaints, we can revert it and apply a more localized version of
> the same fix: just wrap the relevant *call* of org-move-to-column in a
> 
>     (let ((buffer-invisibility-spec nil))
>        (org-move-to-column ....))
> 
> 
> But (without any solid evidence) it seems to me that having the behavior
> of move-to-column change with the buffer-invisibility-spec setting is a
> bug: how can anybody expect reproducible behavior from it if you don't
> know where point is going to end up? So I would be surprised if the fix
> does have negative side effects on anything: on the contrary, it might
> resolve some mysteries. OTOH, putting the let in the compat function
> would be a workaround for org, but the real fix should probably be in
> move-to-column itself. Perhaps a question to emacs devel is warranted.

Hi Nick,

I also do not expect negative consequences.  Please apply the patch, will you?

Thanks!

- Carsten

> 
> -- 
> Nick
> 
> 

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Nick Dokos]

[-- Attachment #1: Type: text/plain, Size: 437 bytes --]

Carsten Dominik <carsten.dominik@gmail.com> writes:

> I also do not expect negative consequences.  Please apply the patch, will you?
>

OK - patch attached. NB: there is at least one place where the "wrap the
call to org-move-to-column" has been applied, in
org.el:`org-comment-or-uncomment-region', presumably for exactly this
reason. If this patch does not cause unexpected problem, then the
wrapped setting above can be cleaned up.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Patch for Matt Lundin's infloop --]
[-- Type: text/x-diff, Size: 1532 bytes --]

From b0aebe182a72df8e7d5428a02e0eca7a6922fefe Mon Sep 17 00:00:00 2001
From: Nick Dokos <ndokos@gmail.com>
Date: Fri, 23 Aug 2013 07:42:44 -0400
Subject: [PATCH] Fix infinite loop in org-agenda-show-new-time

* lisp/org-agenda.el (org-move-to-column): Make sure that
  `buffer-invisibility-spec' is nil when calling move-to-column.

This was caused by a previous commit (fafb5f34) which eliminated the
setting of `buffer-invisibility-spec' in `org-agenda-show-new-time'.

Matt Lundin ran into this and there is a discussion on the ML:

   http://thread.gmane.org/gmane.emacs.orgmode/75288
---
 lisp/org-compat.el | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/lisp/org-compat.el b/lisp/org-compat.el
index d10aeea..c4d15d8 100644
--- a/lisp/org-compat.el
+++ b/lisp/org-compat.el
@@ -336,9 +336,12 @@ Works on both Emacs and XEmacs."
     (indent-line-to column)))
 
 (defun org-move-to-column (column &optional force buffer)
-  (if (featurep 'xemacs)
-      (org-xemacs-without-invisibility (move-to-column column force buffer))
-    (move-to-column column force)))
+  ;; set buffer-invisibility-spec to nil so that move-to-column
+  ;; does the right thing despite the presence of invisible text.
+  (let ((buffer-invisibility-spec nil))
+    (if (featurep 'xemacs)
+	(org-xemacs-without-invisibility (move-to-column column force buffer))
+      (move-to-column column force))))
 
 (defun org-get-x-clipboard-compat (value)
   "Get the clipboard value on XEmacs or Emacs 21."
-- 
1.8.3.101.g727a46b


[-- Attachment #3: Type: text/plain, Size: 77 bytes --]


I hope it conforms to conventions but let me know of any problems.
-- 
Nick

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Andrew Hyatt]

[-- Attachment #1: Type: text/plain, Size: 1138 bytes --]

I've been having this same issue - the issue is quite reproducible for me,
and it has been for years.  I just finally grew tired of the issue and
decided to investigate it, and yes, the issue is org-agenda-show-new-time.

I also have invisible entries in the org buffer, and the call to
org-move-to-column apparently will move several lines forward, which causes
us to process the same lines over and over.

Wrapping the call to org-move-to-column with a let setting the
buffer-invisibility-spec to nil does fix the issue.


On Fri, Aug 23, 2013 at 8:08 AM Nick Dokos <ndokos@gmail.com> wrote:

> Carsten Dominik <carsten.dominik@gmail.com> writes:
>
> > I also do not expect negative consequences.  Please apply the patch,
> will you?
> >
>
> OK - patch attached. NB: there is at least one place where the "wrap the
> call to org-move-to-column" has been applied, in
> org.el:`org-comment-or-uncomment-region', presumably for exactly this
> reason. If this patch does not cause unexpected problem, then the
> wrapped setting above can be cleaned up.
>
>
> I hope it conforms to conventions but let me know of any problems.
> --
> Nick
>

[-- Attachment #2: Type: text/html, Size: 1631 bytes --]

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Bastien]

Hi Andrew,

Andrew Hyatt <ahyatt@gmail.com> writes:

> I've been having this same issue - the issue is quite reproducible
> for me, and it has been for years.  I just finally grew tired of the
> issue and decided to investigate it, and yes, the issue is
> org-agenda-show-new-time.
>
> I also have invisible entries in the org buffer, and the call to
> org-move-to-column apparently will move several lines forward, which
> causes us to process the same lines over and over.
>
> Wrapping the call to org-move-to-column with a let setting the
> buffer-invisibility-spec to nil does fix the issue. 

this problem is from... 2013!
https://lists.gnu.org/archive/html/emacs-orgmode/2013-08/msg00218.html

Is there anything we still need to fix in this area?  If so, can you
send it as a patch against current maint branch?

Thanks,

-- 
 Bastien

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Andrew Hyatt]

[-- Attachment #1: Type: text/plain, Size: 1360 bytes --]

On Sat, Feb 1, 2020 at 4:33 AM Bastien <bzg@gnu.org> wrote:

> Hi Andrew,
>
> Andrew Hyatt <ahyatt@gmail.com> writes:
>
> > I've been having this same issue - the issue is quite reproducible
> > for me, and it has been for years.  I just finally grew tired of the
> > issue and decided to investigate it, and yes, the issue is
> > org-agenda-show-new-time.
> >
> > I also have invisible entries in the org buffer, and the call to
> > org-move-to-column apparently will move several lines forward, which
> > causes us to process the same lines over and over.
> >
> > Wrapping the call to org-move-to-column with a let setting the
> > buffer-invisibility-spec to nil does fix the issue.
>
> this problem is from... 2013!
> https://lists.gnu.org/archive/html/emacs-orgmode/2013-08/msg00218.html
>
> Is there anything we still need to fix in this area?  If so, can you
> send it as a patch against current maint branch?
>

Yes, there's definitely still a problem similar to the one reported -
although I have yet to pare it down to a reproducible case.
I need to look at this more to understand why the fix that was done doesn't
seem to work.  Once I understand that, I'll be close to creating a patch.
The issue is that I don't know this code very well, so any fix I make might
be wrong for some other reason I don't understand.


>
> Thanks,
>
> --
>  Bastien
>

[-- Attachment #2: Type: text/html, Size: 2112 bytes --]

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Bastien]

Hi Andrew,

I have pushed some fixes in this area, if you have a chance to test
Org from the latest maint or master branch, please do so and report
if the problem persists.

Thanks,

-- 
 Bastien

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Andrew Hyatt]

[-- Attachment #1: Type: text/plain, Size: 1059 bytes --]

I've tried the latest version from Feb 2nd, and it still has the same issue.

After some more time with the issue, the issue is when the agenda has items
A, and next line, invisibily (due to org-agenda-dim-blocked-tasks), B.
Trying to set A gets you into an invisible loop because moving to the start
of the line after displaying the time in org-agenda-show-new-time doesn't
take you far enough back to proceed backwards.  The next iteration through
the loop, the time will be re-displayed.

Removing the (beginning-of-line 1) at the end of the time display code in
that function, and substituting (org-agenda-previous-line) seems to fix
it.  I'm not sure if that's the right approach - the previous code didn't
use that function for a reason, but I don't know what that reason was.

On Mon, Feb 3, 2020 at 2:04 PM Bastien <bzg@gnu.org> wrote:

> Hi Andrew,
>
> I have pushed some fixes in this area, if you have a chance to test
> Org from the latest maint or master branch, please do so and report
> if the problem persists.
>
> Thanks,
>
> --
>  Bastien
>

[-- Attachment #2: Type: text/html, Size: 1443 bytes --]

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Bastien]

Hi Andrew,

thanks again.

Andrew Hyatt <ahyatt@gmail.com> writes:

> Removing the (beginning-of-line 1) at the end of the time display
> code in that function, and substituting (org-agenda-previous-line)
> seems to fix it.  I'm not sure if that's the right approach - the
> previous code didn't use that function for a reason, but I don't know
> what that reason was.

I think this approach is correct is it will move over visible lines.

I've pushed a patch, please let me know if it is fixed.

Best,

-- 
 Bastien

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Andrew Hyatt]

[-- Attachment #1: Type: text/plain, Size: 806 bytes --]

It is fixed, but now the new time that's supposed to be displayed via
text-properties does not show up.

Let me spend some time and get a small reproducible case, which will help
us test this.

On Tue, Feb 4, 2020 at 6:38 PM Bastien <bzg@gnu.org> wrote:

> Hi Andrew,
>
> thanks again.
>
> Andrew Hyatt <ahyatt@gmail.com> writes:
>
> > Removing the (beginning-of-line 1) at the end of the time display
> > code in that function, and substituting (org-agenda-previous-line)
> > seems to fix it.  I'm not sure if that's the right approach - the
> > previous code didn't use that function for a reason, but I don't know
> > what that reason was.
>
> I think this approach is correct is it will move over visible lines.
>
> I've pushed a patch, please let me know if it is fixed.
>
> Best,
>
> --
>  Bastien
>

[-- Attachment #2: Type: text/html, Size: 1268 bytes --]

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Matthew Lundin]

Hi Bastien,

Bastien <bzg@gnu.org> writes:
>
> Andrew Hyatt <ahyatt@gmail.com> writes:
>
>> Removing the (beginning-of-line 1) at the end of the time display
>> code in that function, and substituting (org-agenda-previous-line)
>> seems to fix it.  I'm not sure if that's the right approach - the
>> previous code didn't use that function for a reason, but I don't know
>> what that reason was.
>
> I think this approach is correct is it will move over visible lines.
>
> I've pushed a patch, please let me know if it is fixed.

I'm finding that this patch (19676dce758038749887a057208ea33d9a1fad57)
has the by-product of causing multiple paths to flash in the mini-buffer
if org-agenda-show-outline-path is set to t. I believe that is because
it calls org-agenda-previous-line, which in turn calls
org-agenda-do-context-action.

The effect is even more pronounced if org-agenda-follow-mode is on,
causing a significant slowdown in rescheduling items.

Thanks,

Matt

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Bastien]

Hi Matthew,

not sure I replied to this one but in case I didn't, yes, my initial
fix was wrong, I reverted it.

Thanks!

-- 
 Bastien

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Andrew Hyatt]

[-- Attachment #1.1: Type: text/plain, Size: 545 bytes --]

I whittled this down to the smallest reproducible case, which I'm
attaching.  Hopefully should be clear upon viewing the file how to
reproduce.

I'm still unsure of the best solution, I'll have to think about this some
more - but at least the reproducible case will make it easier to debug into
this for anyone who wants to try.



On Tue, Feb 11, 2020 at 3:56 AM Bastien <bzg@gnu.org> wrote:

> Hi Matthew,
>
> not sure I replied to this one but in case I didn't, yes, my initial
> fix was wrong, I reverted it.
>
> Thanks!
>
> --
>  Bastien
>

[-- Attachment #1.2: Type: text/html, Size: 919 bytes --]

[-- Attachment #2: repro.org --]
[-- Type: application/octet-stream, Size: 401 bytes --]

* To reproduce
#+begin_src emacs-lisp
  (setq org-agenda-dim-blocked-tasks 'invisible
	org-enforce-todo-dependencies t)

  ;; Needed to reload & change how org-mode behaves.
  (org-agenda-file-to-front)
  (org-mode)
#+end_src

#+RESULTS:

* TODO A
* TODO Parent
:PROPERTIES:
:ORDERED:  t
:END:
** TODO B - To schedule
Get agenda w/ TODO items, try to schedule
** TODO C (invisible, necessary for bug)

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Bastien]

Hi Andrew,

thanks a lot for the minimal recipe!  Very helpful.
I confirm the bug and I have now (finally) fixed it.

We can close this bug from... 2013 :)

Best,

-- 
 Bastien

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Andrew Hyatt]

[-- Attachment #1: Type: text/plain, Size: 357 bytes --]

Thanks Bastien!  I've tested this out and confirmed it works, and didn't
notice any side effects.

On Fri, Feb 14, 2020 at 6:02 AM Bastien <bzg@gnu.org> wrote:

> Hi Andrew,
>
> thanks a lot for the minimal recipe!  Very helpful.
> I confirm the bug and I have now (finally) fixed it.
>
> We can close this bug from... 2013 :)
>
> Best,
>
> --
>  Bastien
>

[-- Attachment #2: Type: text/html, Size: 677 bytes --]

Re: [BUG] Infinite loop in org-agenda-show-new-time

[Bastien]

Hi Andrew,

Andrew Hyatt <ahyatt@gmail.com> writes:

> Thanks Bastien!  I've tested this out and confirmed it works, and
> didn't notice any side effects. 

Great, thanks for testing and confirming!

-- 
 Bastien