[PATCH] Resolve regexp ambiguity for item headers

[PATCH] Resolve regexp ambiguity for item headers

[Mats Kindahl]

[-- Attachment #1.1: Type: text/plain, Size: 1256 bytes --]

Hi!

I use org-mode quite a lot of most things and also have a number of
"workflows" in my setup, use columnview quite a lot, and also have the
habit of entering whatever text I need to write to resolve an issue
under the item in org-mode format (quite convenient, then I can just
export the subtree and send it as mail).

I have frequently run up against the issue that when trying to generate
a column view for a file, it aborts with a strange error. Since this has
become more frequent lately, I decided to resolve it.

It turns out that if you have something that looks like this:

    * Top level

    ** TODO something

    *** Proposal

    Blabla

    *** Submitted

    Some more blaha.

and you have "SUBMITTED" as one of your todo keywords, this line will be
interpreted as an item with a keyword but no headline. Since a lot of
code assumes that there is always a headline, many functions can get a
"nil" as a headline, failing miserably.

I solved this issue by requiring that there should always be a headline
text (this is after all an assumption made in the code) and rewrote the
org-complex-heading-regexp accordingly.

Patch is attached.

Just my few cents,
Mats Kindahl

-- 
Senior Principal Software Developer
Oracle, MySQL Department


[-- Attachment #1.2: Type: text/html, Size: 1807 bytes --]

[-- Attachment #2: 0001-Resolved-regexp-ambiguity-for-item-headers.patch --]
[-- Type: text/x-patch, Size: 1623 bytes --]

From bc3c021d06bb6f0b5971959f769216defae200bd Mon Sep 17 00:00:00 2001
From: Mats Kindahl <mats.kindahl@oracle.com>
To: emacs-orgmode@gnu.org
Date: Sun, 9 Sep 2012 21:53:45 +0200
Subject: [PATCH] Resolved regexp ambiguity for item headers
X-IMAPbase: 1347255850 2
Status: O
X-UID: 1

* lisp/org.el (org-complex-heading-regexp): Resolve ambiguity in heading regexp
in favor of keywordless headline

When a simple headline is provided that just contain a todo keyword, the
regular expression will match so that the heading is treated as containing just
a keyword and no headline. Since org-column-compact-links only accept a
headline text (and not nil), this causes problems with called from
org-columns-cleanup-item. This occurs if a 'columnview' dynamic block is used
to capture a column view.

This patch solves the problem by re-writing the org-complex-heading-regexp to
require a headline text instead of allowing it to be optional. This means that
items containing only a single word that also happens to be a todo keyword is
interpreted as a headline with no todo keyword and just a headline text.
---
 lisp/org.el |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org.el b/lisp/org.el
index fef7597..e990f49 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -4824,7 +4824,7 @@ but the stars and the body are.")
 	    (concat "^\\(\\*+\\)"
 		    "\\(?: +" org-todo-regexp "\\)?"
 		    "\\(?: +\\(\\[#.\\]\\)\\)?"
-		    "\\(?: +\\(.*?\\)\\)?"
+		    " +\\(.+?\\)"
 		    (org-re "\\(?:[ \t]+\\(:[[:alnum:]_@#%:]+:\\)\\)?")
 		    "[ \t]*$")
 	    org-complex-heading-regexp-format
-- 
1.7.9.5

Re: [PATCH] Resolve regexp ambiguity for item headers

[Nicolas Goaziou]

Hello,

Mats Kindahl <mats.kindahl@oracle.com> writes:

> I solved this issue by requiring that there should always be a headline
> text (this is after all an assumption made in the code) and rewrote the
> org-complex-heading-regexp accordingly.

Actually, I have recently committed a change going exactly the opposite
way. I'm not certain which solution is better, but I tend to think empty
headlines should be valid and parts of Org choking on empty headlines
should be fixed.


Regards,

-- 
Nicolas Goaziou

Re: [PATCH] Resolve regexp ambiguity for item headers

[Mats Kindahl]

[-- Attachment #1: Type: text/plain, Size: 1292 bytes --]


On 09/10/2012 08:24 PM, Nicolas Goaziou wrote:
> Hello,
>
> Mats Kindahl <mats.kindahl@oracle.com> writes:
>
>> I solved this issue by requiring that there should always be a headline
>> text (this is after all an assumption made in the code) and rewrote the
>> org-complex-heading-regexp accordingly.
> Actually, I have recently committed a change going exactly the opposite
> way. I'm not certain which solution is better, but I tend to think empty
> headlines should be valid and parts of Org choking on empty headlines
> should be fixed.

Hi Nicolas,

Interesting. :)

Well... the most important point for me is that it shouldn't choke on
these lines, but otherwise I'm open to suggestions.

My rationale for doing it this way was:

  * The code I looked at assumes that the headline text is there, so
    it's likely that it's the common assumption.
  * It is clearly the case that todo keywords are optional.
  * It is not so clear that the headline text is optional

However:

  * The regular expression matches completely empty headlines, so maybe
    the intention is to allow matching items with just a todo keyword?

I would be very interested if anybody could shed some light on this.

Best wishes,
Mats Kindahl

-- 
Senior Principal Software Developer
Oracle, MySQL Department


[-- Attachment #2: Type: text/html, Size: 2089 bytes --]

Re: [PATCH] Resolve regexp ambiguity for item headers

[Nicolas Goaziou]

Hello,

> Well... the most important point for me is that it shouldn't choke on
> these lines, but otherwise I'm open to suggestions.

In your case, I think that the problem really comes from a bad case
matching: if SUBMITTED is a keyword, "** Submitted" shouldn't be
matched. IOW, todo keywords are/ought to be case sensitive.

> My rationale for doing it this way was:
>
>   * The code I looked at assumes that the headline text is there, so
>     it's likely that it's the common assumption.
>   * It is clearly the case that todo keywords are optional.
>   * It is not so clear that the headline text is optional

From my point of view everything is optional but todo keywords and tags
bind stronger than headline's text. I.e.

** TODO :work:

is an empty headline with a TODO keyword and a "work" tag.

> However:
>
>   * The regular expression matches completely empty headlines, so maybe
>     the intention is to allow matching items with just a todo keyword?

Yes, it is.

Note that I'm not convinced by empty headlines nor do I use them, but as
an outliner, Org should accept them nonetheless. The question is: how
much of the code base shares this opinion?


Regards,

-- 
Nicolas Goaziou

Re: [PATCH] Resolve regexp ambiguity for item headers

[Bastien]

Hi,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

>>   * The regular expression matches completely empty headlines, so maybe
>>     the intention is to allow matching items with just a todo keyword?
>
> Yes, it is.

FWIW I confirm it is.

> Note that I'm not convinced by empty headlines nor do I use them, but as
> an outliner, Org should accept them nonetheless. 

Indeed.  I once removed the support for empty headlines but Carsten
reminded me that we always supported them, so we should continue to
support them.

> The question is: how much of the code base shares this opinion?

Well, hopefully stating the above with clear things up -- and make 
it easier to find out of some parts of the code don't support empty
headlines.

-- 
 Bastien

Re: [PATCH] Resolve regexp ambiguity for item headers

[Mats Kindahl]

Thanks for clearing it up Bastien,

Nicolas, do you have the patch somewhere so that I can take a look at it?

Best wishes,
Mats Kindahl

On 09/19/2012 11:14 AM, Bastien wrote:
> Hi,
>
> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>
>>>   * The regular expression matches completely empty headlines, so maybe
>>>     the intention is to allow matching items with just a todo keyword?
>> Yes, it is.
> FWIW I confirm it is.
>
>> Note that I'm not convinced by empty headlines nor do I use them, but as
>> an outliner, Org should accept them nonetheless. 
> Indeed.  I once removed the support for empty headlines but Carsten
> reminded me that we always supported them, so we should continue to
> support them.
>
>> The question is: how much of the code base shares this opinion?
> Well, hopefully stating the above with clear things up -- and make 
> it easier to find out of some parts of the code don't support empty
> headlines.
>

-- 
Senior Principal Software Developer
Oracle, MySQL Department

Re: [PATCH] Resolve regexp ambiguity for item headers

[Nicolas Goaziou]

Hello,

Mats Kindahl <mats.kindahl@oracle.com> writes:

> Nicolas, do you have the patch somewhere so that I can take a look at
> it?

The patch has been applied to master branch a few weeks ago. It's very
simple: headline's text has been made optional.

There's room from improvement, though: I find strange that "**** " is
a valid empty headline while "****" isn't.


Regards,

-- 
Nicolas Goaziou

Re: [PATCH] Resolve regexp ambiguity for item headers

[Bastien]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> There's room from improvement, though: I find strange that "**** " is
> a valid empty headline while "****" isn't.

This I find natural.  

The prefix for headlines includes the whitespace, so the whitespace
is needed for empty headlines too.  Also, people might want to use 
lines matching "^*+$" (to separate paragraphs or whatever.)

2 cts of course,

-- 
 Bastien

Re: [PATCH] Resolve regexp ambiguity for item headers

[Nick Dokos]

Bastien <bzg@altern.org> wrote:

> Hi Nicolas,
> 
> Nicolas Goaziou <n.goaziou@gmail.com> writes:
> 
> > There's room from improvement, though: I find strange that "**** " is
> > a valid empty headline while "****" isn't.
> 
> This I find natural.  
> 
> The prefix for headlines includes the whitespace, so the whitespace
> is needed for empty headlines too.  Also, people might want to use 
> lines matching "^*+$" (to separate paragraphs or whatever.)
> 
> 2 cts of course,
> 

The trouble of course is that the space is almost invisible, so it leads
to hair-pulling: "why the (&^&^$%&bleep*(^()*^* doesn't this work?"

OTOH, if it happens to you once, you tend to remember it for ever after :-)

I see your 2¢ and lower you to 1.5¢,
Nick

Re: [PATCH] Resolve regexp ambiguity for item headers

[Bastien]

Nick Dokos <nicholas.dokos@hp.com> writes:

> OTOH, if it happens to you once, you tend to remember it for ever
> after :-)

Exactly -- and I think allowing "****" to mean "I'm a headline!" 
would like to even more head-scratching/keyboard-throwing.

-- 
 Bastien