bug#16265: 24.3.50; re-search-forward (error "Invalid search bound (wrong side of point)")

bug#16265: 24.3.50; re-search-forward (error "Invalid search bound (wrong side of point)")

[Eli Zaretskii]

> Date:  Wed, 01 Jan 2014 17:30:12 -0800
> From:  Keith David Bershatsky <esq@lawlist.com>
> Cc:  theonewiththeevillook@yahoo.fr, 16265@debbugs.gnu.org
> 
> Yes, the issue has indeed been resolved in the latest version of Emacs Trunk that I built today.  I used my example to conduct the test, and it works now with `cache-long-scans` enabled.

Thanks, I'm closing the bug, then.

bug#16265: 24.3.50; re-search-forward (error "Invalid search bound (wrong side of point)")

[Keith David Bershatsky]

Nicolas:

This example demonstrates the problem caused when `org-capture` damages the line numbers in the `org-agenda-files`, making it impossible to go to the bottom of the buffer with (goto-char (point-max))  --  consequently, re-search-backward fails -- other functions fail also, e.g., `org-sort-entries`.

(defun example ()
(interactive)
   (let* (
      (org-todo-keywords '((sequence "Active(a)" "Next Action(n)" "Reference(r)"  "Someday(s)" "Delegated(d)" "|" "None(N)") ))
      (sample-todo (concat
        "* TASKS\n\n"
        "** Active [#A] smith @ drawer-one (fishing) | drawer-two (tennis). :lawlist:\n"
        "   DEADLINE: <2013-12-21 Sat 17:00>  SCHEDULED: <2013-12-21 Sat>\n"
        "   :PROPERTIES:\n"
        "   :DRAWER-ONE:  fishing\n"
        "   :DRAWER-TWO:  tennis\n"
        "   :END:\n\n"
        "** Next-Action [#B] doe @ drawer-one (football) | drawer-two (bowling). :fred:\n"
        "   DEADLINE: <2013-12-22 Sun 08:30>  SCHEDULED: <2013-12-22 Sun>\n"
        "   :PROPERTIES:\n"
        "   :DRAWER-ONE:  football\n"
        "   :DRAWER-TWO:  bowling\n"
        "   :END:\n\n"
        "* EVENTS\n\n"
        "** Reference [#C] john @ drawer-one (fishing) | drawer-two (sky-diving). :george:\n"
        "   DEADLINE: <2013-12-23 Mon 10:15>  SCHEDULED: <2013-12-23 Mon>\n"
        "   :PROPERTIES:\n"
        "   :DRAWER-ONE:  fishing\n"
        "   :DRAWER-TWO:  sky-diving\n"
        "   :END:\n\n"
        "* UNDATED\n\n"
        "** Someday [#D] jane @ drawer-one (basket-ball) | drawer-two (bowling). :sam:\n"
        "   DEADLINE: <2013-12-24 Tues 12:00>  SCHEDULED: <2013-12-24 Tues>\n"
        "   :PROPERTIES:\n"
        "   :DRAWER-ONE:  basket-ball\n"
        "   :DRAWER-TWO:  bowling\n"
        "   :END:")))
      (if (get-buffer "foo.org")
        (progn
          (switch-to-buffer "foo.org")
          (erase-buffer)
          (delete-other-windows))
        (switch-to-buffer (get-buffer-create "foo.org")))
      (org-mode)
      (linum-mode 1)
      (insert sample-todo)
      (goto-char (point-min))
      (or (y-or-n-p (format "For this example work, you must save this buffer as a file.  Proceed with example?"))
          (error "Canceled."))
      (write-file "~/Desktop/foo.org" t)
      (let* (
          (filename (buffer-file-name))
          (org-agenda-files (list filename))
          (org-capture-templates '(("n" "Next Action" entry (file+headline filename "TASKS")
      "** Next Action [#A] %?\n   DEADLINE: <%<%Y-%m-%d %a>>\n   :PROPERTIES:\n   :ToodledoID:\n   :ToodledoFolder: TASKS\n   :Hash:\n   :END:"
      :empty-lines 1))))
(search-backward-example)
(org-capture nil "n")
(message "-----------------------------")
(message "Here we can see that the line numbers in 'foo.org' got messed up.")
(sit-for 5)
(insert "Hello World! :lawlist:")
(org-capture-finalize)
(search-backward-example)
(message "-----------------------------")
(message "Here is where things really went wrong.  It's searching the WRONG todo.")
(message "-----------------------------")
(switch-to-buffer "*Messages*"))))


(defun search-backward-example ()
(require 'org-element)
  (let* (element todo-state title deadline scheduled day month year
        (org-todo-keywords '((sequence "Active(a)" "Next Action(n)" "Reference(r)"  "Someday(s)" "Delegated(d)" "|" "None(N)") )))
    (goto-char (point-max))
    (while (re-search-backward "^\*\* \\(Reference\\)" nil t)
      (setq element (org-element-at-point))
      (setq todo-state (org-element-property :todo-keyword element))
      (setq title (org-element-property :title element))
      (setq deadline (ignore-errors (org-element-property :deadline element) ))
      (setq scheduled (ignore-errors (org-element-property :scheduled element) ))
      (setq day (ignore-errors (org-element-property :day-start scheduled)))
      (setq month (ignore-errors (org-element-property :month-start scheduled)))
      (setq year (ignore-errors (org-element-property :year-start scheduled)))
      (message "-----------------------------")
      (message "todo-state:  %s" todo-state)
      (message "deadline:  %s" deadline)
      (message "scheduled:  %s" scheduled)
      (message "title:  %s" title)
      (message "day:  %s" day)
      (message "month:  %s" month)
      (message "year:  %s" year)
      (message "-----------------------------") )))

bug#16265: 24.3.50; re-search-forward (error "Invalid search bound (wrong side of point)")

[Nicolas Richard]

Le 28/12/2013 21:44, Keith David Bershatsky a écrit :
> This example demonstrates the problem caused when `org-capture`
> damages the line numbers in the `org-agenda-files`, making it
> impossible to go to the bottom of the buffer with (goto-char
> (point-max))  --  consequently, re-search-backward fails -- other
> functions fail also, e.g., `org-sort-entries`.

Thanks for the recipe, I can indeed reproduce problems, but they are
different from what you describe. I suspect they are tied, but I'm not
sure because I don't understand them.

Here is a reduced version of your recipe which shows what I see. Run the
following snippet using emacs -Q --batch -l filenamehere :
,---- this is filenamehere.el
| (setq test-file "/tmp/foo.org")
| (setq capture-todo (mapconcat
|                     #'identity
|                     (list "** whatever"
|                           ":PROPERTIES:"
|                           ":ToodledoFolder: TASKS"
|                           ":END:")
|                     "\n"))
| (when (file-exists-p test-file) (delete-file test-file) (message "Deleted file %s" test-file))
| (find-file test-file)
| (org-mode)
| (setq org-agenda-files (list test-file))
| (setq org-capture-templates `(("n" "NextAction" entry (file+headline test-file "TASKS")
| 			       ,capture-todo)))
| (org-capture nil "n")
| (org-capture-finalize)
| (goto-char (point-min))
| (insert "It's important to insert something ; I guess it triggers something\n")
| (search-forward "Toodledo")
| (end-of-line) ; this should land us before "\n:END:" not after it
| (if (looking-at "\n:END:")
|     (message "Great!")
|   (error "Problem here."))
`----

fun facts : if you remove the (insert ...) form, it works. Also, if you
do it interactively, then the problem appears too, but not if you clone
the buffer.

I'll bisect this (I tested 24.3, it works there) and post the result
unless someone understands what's going on before the bisect finishes
(it does take quite some time to rebuild each version of emacs on my
machine).

-- 
Nicolas.

bug#16265: 24.3.50; re-search-forward (error "Invalid search bound (wrong side of point)")

[Nicolas Richard]

Nicolas Richard <theonewiththeevillook@yahoo.fr> writes:
> I'll bisect this (I tested 24.3, it works there) and post the result

bisection is done.
# first bad commit: [f56f1e3993fd79240e03666cf8390f489b4a2435] Switch cache-long-scans to t by default.

and indeed, running
emacs -Q --batch --eval '(setq-default cache-long-scans nil)' -l filenamehere
worked for me.

Keith, can you test if it fixes the problems your are seeing as well ?
(If it doesn't then I guess we'll need another bug report.) If it does,
the next step is to understand why things break when the caching
mechanism is enabled -- I give up there.

-- 
Nico.

bug#16265: 24.3.50; re-search-forward (error "Invalid search bound (wrong side of point)")

[Keith David Bershatsky]

Nicolas:

Yes, `(setq-default cache-long-scans nil)` does indeed fix the problem.  In a version of Emacs Trunk built on October 5, 2013, the default value for `cache-long-scans` is `nil`.  In the recent version of Emacs Trunk built on December 23, 2013, the default value for `cache-long-scans` is `t`.

I would suspect that anyone who uses `org-capture` will have significant problems performing various operations on the `org-agenda-files` subsequent to the initial call.  So my recommendation would be that the Emacs team return the default value of `cache-long-scans` to `nil`; or, fix it somehow so that it doesn't interfere with popular functions like `re-search-forward` and `re-search-backward` after calling `org-capture`.

Thank you very much for your hard work tracking down the issue.

Keith

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

At Tue, 31 Dec 2013 15:16:40 +0100,
Nicolas Richard wrote:
> 
> Nicolas Richard <theonewiththeevillook@yahoo.fr> writes:
> > I'll bisect this (I tested 24.3, it works there) and post the result
> 
> bisection is done.
> # first bad commit: [f56f1e3993fd79240e03666cf8390f489b4a2435] Switch cache-long-scans to t by default.
> 
> and indeed, running
> emacs -Q --batch --eval '(setq-default cache-long-scans nil)' -l filenamehere
> worked for me.
> 
> Keith, can you test if it fixes the problems your are seeing as well ?
> (If it doesn't then I guess we'll need another bug report.) If it does,
> the next step is to understand why things break when the caching
> mechanism is enabled -- I give up there.
> 
> -- 
> Nico.

bug#16265: 24.3.50; re-search-forward (error "Invalid search bound (wrong side of point)")

[Nicolas Richard]

Keith David Bershatsky <esq@lawlist.com> writes:
> Yes, `(setq-default cache-long-scans nil)` does indeed fix the
> problem.

Great. As I said I must give up here, because I don't know what to do at
this point ; but I can dig further if someone tells me what I'm looking
for, or what is to be done.

-- 
Nico.

bug#16265: 24.3.50; re-search-forward (error "Invalid search bound (wrong side of point)")

[Eli Zaretskii]

> Date: Tue, 31 Dec 2013 09:37:15 -0800
> From: Keith David Bershatsky <esq@lawlist.com>
> Cc: 16265@debbugs.gnu.org
> 
> So my recommendation would be that the Emacs team return the default value of `cache-long-scans` to `nil`

Unlikely to happen.

> or, fix it somehow so that it doesn't interfere with popular functions like `re-search-forward` and `re-search-backward` after calling `org-capture`.

I think I fixed this in trunk revision 115826, please try the latest
version of the code.  (I only verified that the simplified test case
works after the fix.)

Thanks.

bug#16265: 24.3.50; re-search-forward (error "Invalid search bound (wrong side of point)")

[Keith David Bershatsky]

Yes, the issue has indeed been resolved in the latest version of Emacs Trunk that I built today.  I used my example to conduct the test, and it works now with `cache-long-scans` enabled.

Great job -- thank you Eli, and thanks again Nicolas!

Keith

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

At Wed, 01 Jan 2014 19:50:12 +0200,
Eli Zaretskii wrote:
> 
> > Date: Tue, 31 Dec 2013 09:37:15 -0800
> > From: Keith David Bershatsky <esq@lawlist.com>
> > Cc: 16265@debbugs.gnu.org
> > 
> > So my recommendation would be that the Emacs team return the default value of `cache-long-scans` to `nil`
> 
> Unlikely to happen.
> 
> > or, fix it somehow so that it doesn't interfere with popular functions like `re-search-forward` and `re-search-backward` after calling `org-capture`.
> 
> I think I fixed this in trunk revision 115826, please try the latest
> version of the code.  (I only verified that the simplified test case
> works after the fix.)
> 
> Thanks.