[BUG] Unmatched #+end-src

[BUG] Unmatched #+end-src

[Martyn Jago]

Hi 

--8<---------------cut here---------------start------------->8---
* Unmatched #+end-src bug

#+end_src
--8<---------------cut here---------------end--------------->8---

With the above simple org file, placing the cursor at the end of
#+end_src and hitting return causes emacs to hang.

The bug can be replicated with the following simple test which also
causes emacs to hang...

--8<---------------cut here---------------start------------->8---
(ert-deftest test-headless-end-src ()
  (with-temp-buffer
    (insert "\n\n\n\n\n\n#+end_")
    (should-not (org-in-item-p))))
--8<---------------cut here---------------end--------------->8---

It appears to be related to the following in 'org-in-item-p
(org-list.el)...

--8<---------------cut here---------------start------------->8---
	       ((looking-at "^[ \t]*#\\+end_")
		(re-search-backward "^[ \t]*#\\+begin_" nil t))
--8<---------------cut here---------------end--------------->8---

I've tried to pin down the bug but its left me perplexed, so I'm going
to defer to more experienced org lispers! 

Regards

Martyn

---
Org-mode version 7.5 (release_7.5.32.gdf26.dirty)
GNU Emacs 24.0.50.1 (i686-pc-linux-gnu, GTK+ Version 2.24.0)
 of 2011-02-25 

Re: [BUG] Unmatched #+end-src

[Aankhen]

Hi,

On Sat, Mar 12, 2011 at 22:26, Martyn Jago <martyn.jago@btinternet.com> wrote:
> --8<---------------cut here---------------start------------->8---
> * Unmatched #+end-src bug
>
> #+end_src
> --8<---------------cut here---------------end--------------->8---
>
> With the above simple org file, placing the cursor at the end of
> #+end_src and hitting return causes emacs to hang.
>
> The bug can be replicated with the following simple test which also
> causes emacs to hang...
>
> [snip]
>
> It appears to be related to the following in 'org-in-item-p
> (org-list.el)...
>
> --8<---------------cut here---------------start------------->8---
>               ((looking-at "^[ \t]*#\\+end_")
>                (re-search-backward "^[ \t]*#\\+begin_" nil t))
> --8<---------------cut here---------------end--------------->8---
>
> I've tried to pin down the bug but its left me perplexed, so I'm going
> to defer to more experienced org lispers!

The =cond= is part of a =while= loop; it just keeps looping, entering
that branch and doing nothing (rather than moving point and picking up
again from there).  Going by the other branches, I think the correct
thing to do is just exit the loop:

--8<---------------cut here---------------start------------->8---
diff --git a/lisp/org-list.el b/lisp/org-list.el
--- a/lisp/org-list.el
+++ b/lisp/org-list.el
@@ -450,17 +450,19 @@ This checks `org-list-ending-method'."
               ;; At upper bound of search or looking at the end of a
               ;; previous list: search is over.
               ((<= (point) lim-up) (throw 'exit nil))
               ((and (not (eq org-list-ending-method 'indent))
                     (looking-at org-list-end-re))
                (throw 'exit nil))
               ;; Skip blocks, drawers, inline-tasks, blank lines
               ((looking-at "^[ \t]*#\\+end_")
-               (re-search-backward "^[ \t]*#\\+begin_" nil t))
+               (condition-case nil
+                   (re-search-backward "^[ \t]*#\\+begin_" nil)
+                 (search-failed (throw 'exit nil))))
               ((looking-at "^[ \t]*:END:")
                (re-search-backward org-drawer-regexp nil t)
                (beginning-of-line))
               ((and inlinetask-re (looking-at inlinetask-re))
                (org-inlinetask-goto-beginning)
                (forward-line -1))
               ((looking-at "^[ \t]*$") (forward-line -1))
               ;; Text at column 0 cannot belong to a list: stop.
--8<---------------cut here---------------end--------------->8---

Hope this helps,
Aankhen

Re: [BUG] Unmatched #+end-src

[Nicolas]

Hello,

Aankhen <aankhen@gmail.com> writes:

>> --8<---------------cut here---------------start------------->8---
>> * Unmatched #+end-src bug
>>
>> #+end_src
>> --8<---------------cut here---------------end--------------->8---
>>
>> With the above simple org file, placing the cursor at the end of
>> #+end_src and hitting return causes emacs to hang.
>>
> The =cond= is part of a =while= loop; it just keeps looping, entering
> that branch and doing nothing (rather than moving point and picking up
> again from there).  Going by the other branches, I think the correct
> thing to do is just exit the loop:

I don't think exiting the loop that way is the right thing to do, as it
always return nil, even though the #+end_ might be in the list.
Moreover, there are other parts of the file that should also be fixed.

I think there are 3 options here:

1. Effectively stop everything, but at least throw an informative error.
2. Consider the #+end_ as normal text after all, and resume the loop.
3. Consider #+end_ as normal text and message the user about his syntax
   problem.

I'm not sure yet what's the best way to go, but I think option 2 is
sufficient.


Regards,

-- 
Nicolas

Re: [BUG] Unmatched #+end-src

[Aankhen]

Hi,

On Sun, Mar 13, 2011 at 01:37, Nicolas <n.goaziou@gmail.com> wrote:
> Aankhen <aankhen@gmail.com> writes:
>
>>> --8<---------------cut here---------------start------------->8---
>>> * Unmatched #+end-src bug
>>>
>>> #+end_src
>>> --8<---------------cut here---------------end--------------->8---
>>>
>>> With the above simple org file, placing the cursor at the end of
>>> #+end_src and hitting return causes emacs to hang.
>>>
>> The =cond= is part of a =while= loop; it just keeps looping, entering
>> that branch and doing nothing (rather than moving point and picking up
>> again from there).  Going by the other branches, I think the correct
>> thing to do is just exit the loop:
>
> I don't think exiting the loop that way is the right thing to do, as it
> always return nil, even though the #+end_ might be in the list.
>
> [snip]

You’re right, I lost the context.  My apologies for the ill-conceived patch.

Aankhen

Re: Re: [BUG] Unmatched #+end-src

[Martyn Jago]

Nicolas <n.goaziou@gmail.com> writes:

Hi

> Hello,
>
> Aankhen <aankhen@gmail.com> writes:
>
>>> --8<---------------cut here---------------start------------->8---
>>> * Unmatched #+end-src bug
>>>
>>> #+end_src
>>> --8<---------------cut here---------------end--------------->8---
>>>
>>> With the above simple org file, placing the cursor at the end of
>>> #+end_src and hitting return causes emacs to hang.
>>>
>> The =cond= is part of a =while= loop; it just keeps looping, entering
>> that branch and doing nothing (rather than moving point and picking up
>> again from there).  Going by the other branches, I think the correct
>> thing to do is just exit the loop:
>
> I don't think exiting the loop that way is the right thing to do, as it
> always return nil, even though the #+end_ might be in the list.
> Moreover, there are other parts of the file that should also be fixed.

I've supplied a patch which passes all of my tests, but I will look at
providing additional tests looking at other cases within this loop since
I'm currently in the habit of writing tests anyway. 

>
> I think there are 3 options here:
>
> 1. Effectively stop everything, but at least throw an informative error.
> 2. Consider the #+end_ as normal text after all, and resume the loop.
> 3. Consider #+end_ as normal text and message the user about his syntax
>    problem.
>
> I'm not sure yet what's the best way to go, but I think option 2 is
> sufficient.
>
>
> Regards,

Regards

Martyn

Re: [BUG] Unmatched #+end-src

[Nicolas]

Hello,

Martyn Jago <martyn.jago@btinternet.com> writes:

> I've supplied a patch which passes all of my tests, but I will look at
> providing additional tests looking at other cases within this loop since
> I'm currently in the habit of writing tests anyway.

Your patch has the same weakness as the previous one and I explained
why. For example, in the following example, calling (org-in-item-p) with
point anywhere on line "some text" will return nil, which is obviously
wrong.

- item 1
  #+end_
  some text
- item 2

Actually, it is not a matter of patch, which is just changing

#+begin_src emacs-lisp
((looking-at "^[ \t]*#\\+end_")
 (re-search-backward "^[ \t]*#\\+begin_" nil t))
#+end_src

into

#+begin_src emacs-lisp
((and (looking-at "^[ \t]*#\\+end_")
      (re-search-backward "^[ \t]*#\\+begin_" nil t)))
#+end_src

once for blocks and once for drawers, in both org-in-item-p and
org-list-struct.


The real problem is: how should Org react when parsing syntactically
erroneous buffers? I concede that freezing Emacs isn't nice, but otoh,
code can't deal with every possible user error.

So, what is the expected behavior here? Consider orphan #+end_ as
normal text, throw an error, or both? An answer to this question would
be more useful than code, honestly.

I hope I am clearer now.

Regards,

-- 
Nicolas

Re: Re: [BUG] Unmatched #+end-src

[Eric Schulte]

>
> The real problem is: how should Org react when parsing syntactically
> erroneous buffers? I concede that freezing Emacs isn't nice, but otoh,
> code can't deal with every possible user error.
>
> So, what is the expected behavior here? Consider orphan #+end_ as
> normal text, throw an error, or both? An answer to this question would
> be more useful than code, honestly.
>

This is just opinion and gut reaction, but my first instinct is to say
that Org just treat an orphan #+end_ as normal text (or technically as
an Org-mode comment).

The same is true for a floating #+begin_src.  Until the block is closed,
it is just a comment.

Just opinions, Best -- Eric

Re: [BUG] Unmatched #+end-src

[Nicolas Goaziou]

[-- Attachment #1: Type: text/plain, Size: 976 bytes --]

Hello,

"Eric Schulte" <schulte.eric@gmail.com> writes:

>> The real problem is: how should Org react when parsing syntactically
>> erroneous buffers? I concede that freezing Emacs isn't nice, but otoh,
>> code can't deal with every possible user error.
>>
>> So, what is the expected behavior here? Consider orphan #+end_ as
>> normal text, throw an error, or both? An answer to this question would
>> be more useful than code, honestly.
>>
>
> This is just opinion and gut reaction, but my first instinct is to say
> that Org just treat an orphan #+end_ as normal text (or technically as
> an Org-mode comment).
>
> The same is true for a floating #+begin_src.  Until the block is closed,
> it is just a comment.

As it's the only answer so far, I guess it was more trivial than
I thought.

Here is a patch that should fix the original problem.

Martyn, as you are writing tests, would you mind running it against
them, before I apply it?

Many thanks in advance.

Regards,

[-- Attachment #2: patch infloop with erroneous constructs --]
[-- Type: text/plain, Size: 2714 bytes --]

From 805e11e419c58a9f3fdb470987056c5ac970817f Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou <n.goaziou@gmail.com>
Date: Thu, 17 Mar 2011 13:01:06 +0100
Subject: [PATCH] org-list: fix infinite loop on erroneous block and drawer constructs

* lisp/org-list.el (org-list-struct,org-in-item-p): don't assume end
  of blocks or drawers necessarily start somewhere. It it isn't the
  case, treat them as normal text.
---
 lisp/org-list.el |   24 ++++++++++++------------
 1 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/lisp/org-list.el b/lisp/org-list.el
index aea8634..8bfd359 100644
--- a/lisp/org-list.el
+++ b/lisp/org-list.el
@@ -465,10 +465,10 @@ This checks `org-list-ending-method'."
 		     (looking-at org-list-end-re))
 		(throw 'exit nil))
 	       ;; Skip blocks, drawers, inline-tasks, blank lines
-	       ((looking-at "^[ \t]*#\\+end_")
-		(re-search-backward "^[ \t]*#\\+begin_" nil t))
-	       ((looking-at "^[ \t]*:END:")
-		(re-search-backward org-drawer-regexp nil t)
+	       ((and (looking-at "^[ \t]*#\\+end_")
+		     (re-search-backward "^[ \t]*#\\+begin_" lim-up t)))
+	       ((and (looking-at "^[ \t]*:END:")
+		     (re-search-backward org-drawer-regexp lim-up t))
 		(beginning-of-line))
 	       ((and inlinetask-re (looking-at inlinetask-re))
 		(org-inlinetask-goto-beginning)
@@ -686,10 +686,10 @@ Assume point is at an item."
 			     (memq (assq (car beg-cell) itm-lst) itm-lst))))
 	       ;; Skip blocks, drawers, inline tasks, blank lines
 	       ;; along the way.
-	       ((looking-at "^[ \t]*#\\+end_")
-		(re-search-backward "^[ \t]*#\\+begin_" nil t))
-	       ((looking-at "^[ \t]*:END:")
-		(re-search-backward drawers-re nil t)
+	       ((and (looking-at "^[ \t]*#\\+end_")
+		     (re-search-backward "^[ \t]*#\\+begin_" lim-up t)))
+	       ((and (looking-at "^[ \t]*:END:")
+		     (re-search-backward drawers-re lim-up t))
 		(beginning-of-line))
 	       ((and inlinetask-re (looking-at inlinetask-re))
 		(org-inlinetask-goto-beginning)
@@ -753,11 +753,11 @@ Assume point is at an item."
 	      (throw 'exit (push (cons 0 (point)) end-lst-2)))
 	     ;; Skip blocks, drawers, inline tasks and blank lines
 	     ;; along the way
-	     ((looking-at "^[ \t]*#\\+begin_")
-	      (re-search-forward "^[ \t]*#\\+end_")
+	     ((and (looking-at "^[ \t]*#\\+begin_")
+		   (re-search-forward "^[ \t]*#\\+end_" lim-down t))
 	      (forward-line 1))
-	     ((looking-at drawers-re)
-	      (re-search-forward "^[ \t]*:END:" nil t)
+	     ((and (looking-at drawers-re)
+		   (re-search-forward "^[ \t]*:END:" lim-down t))
 	      (forward-line 1))
 	     ((and inlinetask-re (looking-at inlinetask-re))
 	      (org-inlinetask-goto-end))
-- 
1.7.4.1


[-- Attachment #3: Type: text/plain, Size: 21 bytes --]


-- 
Nicolas Goaziou

Re: [BUG] Unmatched #+end-src

[Martyn Jago]

[-- Attachment #1: Type: text/plain, Size: 204 bytes --]

Aankhen <aankhen@gmail.com> writes:

Thanks for the heads-up Aankhen, I had indeed missed the enclosing
infinite loop.

I modified your patch slightly since my test still failed, and made
further tests.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Fix-endless-loop-in-org-in-item-p --]
[-- Type: text/x-diff, Size: 995 bytes --]

From b55d846b57fc2ebf3c282cb1fbb27becfdd7d4fd Mon Sep 17 00:00:00 2001
From: Martyn Jago <martyn.jago@btinternet.com>
Date: Sat, 12 Mar 2011 20:38:14 +0000
Subject: [PATCH] Fix endless loop in 'org-in-item-p where #+end_ block has no matching beginning block

---
 lisp/org-list.el |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/lisp/org-list.el b/lisp/org-list.el
index 4b50910..7d8692b 100644
--- a/lisp/org-list.el
+++ b/lisp/org-list.el
@@ -465,8 +465,9 @@ This checks `org-list-ending-method'."
 		     (looking-at org-list-end-re))
 		(throw 'exit nil))
 	       ;; Skip blocks, drawers, inline-tasks, blank lines
-	       ((looking-at "^[ \t]*#\\+end_")
-		(re-search-backward "^[ \t]*#\\+begin_" nil t))
+	       ((and (looking-at "^[ \t]*#\\+end_")
+		     (not (re-search-backward "^[ \t]*#\\+begin_" nil t)))
+		(throw 'exit nil))
 	       ((looking-at "^[ \t]*:END:")
 		(re-search-backward org-drawer-regexp nil t)
 		(beginning-of-line))
-- 
1.7.4.1


[-- Attachment #3: Type: text/plain, Size: 2633 bytes --]


Regards

Martyn

> Hi,
>
> On Sat, Mar 12, 2011 at 22:26, Martyn Jago <martyn.jago@btinternet.com> wrote:
>> --8<---------------cut here---------------start------------->8---
>> * Unmatched #+end-src bug
>>
>> #+end_src
>> --8<---------------cut here---------------end--------------->8---
>>
>> With the above simple org file, placing the cursor at the end of
>> #+end_src and hitting return causes emacs to hang.
>>
>> The bug can be replicated with the following simple test which also
>> causes emacs to hang...
>>
>> [snip]
>>
>> It appears to be related to the following in 'org-in-item-p
>> (org-list.el)...
>>
>> --8<---------------cut here---------------start------------->8---
>>               ((looking-at "^[ \t]*#\\+end_")
>>                (re-search-backward "^[ \t]*#\\+begin_" nil t))
>> --8<---------------cut here---------------end--------------->8---
>>
>> I've tried to pin down the bug but its left me perplexed, so I'm going
>> to defer to more experienced org lispers!
>
> The =cond= is part of a =while= loop; it just keeps looping, entering
> that branch and doing nothing (rather than moving point and picking up
> again from there).  Going by the other branches, I think the correct
> thing to do is just exit the loop:
>
> diff --git a/lisp/org-list.el b/lisp/org-list.el
> --- a/lisp/org-list.el
> +++ b/lisp/org-list.el
> @@ -450,17 +450,19 @@ This checks `org-list-ending-method'."
>                ;; At upper bound of search or looking at the end of a
>                ;; previous list: search is over.
>                ((<= (point) lim-up) (throw 'exit nil))
>                ((and (not (eq org-list-ending-method 'indent))
>                      (looking-at org-list-end-re))
>                 (throw 'exit nil))
>                ;; Skip blocks, drawers, inline-tasks, blank lines
>                ((looking-at "^[ \t]*#\\+end_")
> -               (re-search-backward "^[ \t]*#\\+begin_" nil t))
> +               (condition-case nil
> +                   (re-search-backward "^[ \t]*#\\+begin_" nil)
> +                 (search-failed (throw 'exit nil))))
>                ((looking-at "^[ \t]*:END:")
>                 (re-search-backward org-drawer-regexp nil t)
>                 (beginning-of-line))
>                ((and inlinetask-re (looking-at inlinetask-re))
>                 (org-inlinetask-goto-beginning)
>                 (forward-line -1))
>                ((looking-at "^[ \t]*$") (forward-line -1))
>                ;; Text at column 0 cannot belong to a list: stop.
>
> Hope this helps,
> Aankhen

Re: [BUG] Unmatched #+end-src

[Nicolas]

Hello,

Martyn Jago <martyn.jago@btinternet.com> writes:

> Hi
>
> * Unmatched #+end-src bug
>
> #+end_src
>
> With the above simple org file, placing the cursor at the end of
> #+end_src and hitting return causes emacs to hang.

I've pushed a fix that should solve the problem. Please tell if it
doesn't.

Thank you for your report.

Regards,

-- 
Nicolas