TODO type problem on speedbar and imenu.

TODO type problem on speedbar and imenu.

[Osamu OKANO]

[-- Attachment #1: Type: text/plain, Size: 527 bytes --]

Hi.
I found a problem on speedbar and imenu.
Reproducing date is here
-8<---------------cut here---------------start------------->8---
* someday/maybe
* SOMEDAY/maybe
* SOMEDAY maybe
* someday maybe
* read/review
* READ/review
* conf
#+TYP_TODO: SOMEDAY
--8<---------------cut here---------------end--------------->8---

When a head line includes TODO state name,
I am afraid that displayed head line names on speedbar are wrong.
Because SOMEDAY is TODO type but "someday/maybe" and "SOMEDAY/maybe"
is not TODO task.

Regards.

[-- Attachment #2: org-speedbar-bug2.jpg --]
[-- Type: image/jpeg, Size: 79192 bytes --]

[-- Attachment #3: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: TODO type problem on speedbar and imenu.

[Noorul Islam K M]

Osamu OKANO <okano.osamu@gmail.com> writes:

> Hi.
> I found a problem on speedbar and imenu.
> Reproducing date is here
> * someday/maybe
> * SOMEDAY/maybe
> * SOMEDAY maybe
> * someday maybe
> * read/review
> * READ/review
> * conf
> #+TYP_TODO: SOMEDAY
>
> When a head line includes TODO state name,
> I am afraid that displayed head line names on speedbar are wrong.
> Because SOMEDAY is TODO type but "someday/maybe" and "SOMEDAY/maybe"
> is not TODO task.
>

I am not able to reproduce this on 

GNU Emacs 23.1.1 (i686-pc-linux-gnu, GTK+ Version 2.12.12)
 of 2010-01-30 on noorul

Org-mode version 7.01trans (release_6.36.723.gaf9d)


Thanks and Regards
Noorul

Re: TODO type problem on speedbar and imenu.

[Sébastien Vauban]

Hi Osamu,

> I found a problem on speedbar and imenu.
> Reproducing date is here
> * someday/maybe
> * SOMEDAY/maybe
> * SOMEDAY maybe
> * someday maybe
> * read/review
> * READ/review
> * conf
> #+TYP_TODO: SOMEDAY
>
> When a head line includes TODO state name,
> I am afraid that displayed head line names on speedbar are wrong.
> Because SOMEDAY is TODO type but "someday/maybe" and "SOMEDAY/maybe"
> is not TODO task.

I won't answer your mail. Sorry. Just having a question: how do you have the
red "V" after the file names?  Is it some sort of versioning marker?

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: TODO type problem on speedbar and imenu.

[Osamu OKANO]

Hi.
> I won't answer your mail. Sorry. Just having a question: how do you have the
> red "V" after the file names?  Is it some sort of versioning marker?
I don't set by my effort, but I manage org files with git(vc-git).


I can reproduce this on
GNU Emacs 23.2.1 (i386-mingw-nt6.0.6002) of 2010-05-08 on PC
Org-mode version 7.01trans (release_7.01g.22.gec034.dirty)

Re: TODO type problem on speedbar and imenu.

[David Maus]

[-- Attachment #1.1: Type: text/plain, Size: 879 bytes --]

Osamu OKANO wrote:
>[1  <text/plain; ISO-8859-1 (7bit)>]
>Hi.
>I found a problem on speedbar and imenu.
>Reproducing date is here
>-8<---------------cut here---------------start------------->8---
>* someday/maybe
>* SOMEDAY/maybe
>* SOMEDAY maybe
>* someday maybe
>* read/review
>* READ/review
>* conf
>#+TYP_TODO: SOMEDAY
>--8<---------------cut here---------------end--------------->8---

>When a head line includes TODO state name,
>I am afraid that displayed head line names on speedbar are wrong.
>Because SOMEDAY is TODO type but "someday/maybe" and "SOMEDAY/maybe"
>is not TODO task.

I can confirm this with Org-mode version 7.01trans
(release_7.01g.26.g24bb.dirty) on GNU Emacs 23.2.1 (i486-pc-linux-gnu,
GTK+ Version 2.20.0) of 2010-05-16 on raven, modified by Debian.

 -- David
--
OpenPGP... 0x99ADB83B5A4478E6
Jabber.... dmjena@jabber.org
Email..... dmaus@ictsoc.de

[-- Attachment #1.2: Type: application/pgp-signature, Size: 230 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: TODO type problem on speedbar and imenu.

[Nicolas Goaziou]

Hello,

Osamu OKANO <okano.osamu@gmail.com> writes:

> I found a problem on speedbar and imenu.
> Reproducing date is here
> -8<---------------cut here---------------start------------->8---
> * someday/maybe
> * SOMEDAY/maybe
> * SOMEDAY maybe
> * someday maybe
> * read/review
> * READ/review
> * conf
> #+TYP_TODO: SOMEDAY
> --8<---------------cut here---------------end--------------->8---
>
> When a head line includes TODO state name,
> I am afraid that displayed head line names on speedbar are wrong.
> Because SOMEDAY is TODO type but "someday/maybe" and "SOMEDAY/maybe"
> is not TODO task.

This is a regexp problem: `org-complex-heading-regexp' recognizes
"someday" and a TODO keyword in both "someday/maybe" and
"SOMEDAY/maybe".

It is defined as the following (I let the concat form for better
readability):

#+begin_src emacs-lisp
(concat "^\\(\\*+\\)[ \t]+\\(?:\\("
	(mapconcat 'regexp-quote org-todo-keywords-1 "\\|")
	"\\)\\>\\)?\\(?:[ \t]*\\(\\[#.\\]\\)\\)?[ \t]*\\(.*?\\)"
	"\\(?:[ \t]+\\(:[[:alnum:]_@#%:]+:\\)\\)?[ \t]*$")
#+end_src

The problem in at the "\\>" part. The following change may solve the
problem.

#+begin_src emacs-lisp
(concat "^\\(\\*+\\)[ \t]+\\(?:\\("
	(mapconcat 'regexp-quote org-todo-keywords-1 "\\|")
	"\\)[ \t]+\\)?\\(?:[ \t]*\\(\\[#.\\]\\)\\)?[ \t]*\\(.*?\\)"
	"\\(?:[ \t]+\\(:[[:alnum:]_@#%:]+:\\)\\)?[ \t]*$")
#+end_src

I think it is better to enforce some whitespace after a TODO keyword
than a word boundary. There are a few places where this could also be
changed (i.e. `org-complex-heading-regexp-format').

Any opinion on this?

Regards,

-- 
Nicolas Goaziou

Re: TODO type problem on speedbar and imenu.

[Sebastien Vauban]

Hi Nicolas,

Nicolas Goaziou wrote:
> I think it is better to enforce some whitespace after a TODO keyword than a
> word boundary. There are a few places where this could also be changed (i.e.
> `org-complex-heading-regexp-format').
>
> Any opinion on this?

I share your opinion.

Best regards,
  Seb

-- 
Sebastien Vauban

Re: TODO type problem on speedbar and imenu.

[Bastien]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> I think it is better to enforce some whitespace after a TODO keyword
> than a word boundary. There are a few places where this could also be
> changed (i.e. `org-complex-heading-regexp-format').
>
> Any opinion on this?

Yep.  When you have some free time (!), please suggest a patch and a
small org test file so that we can test.

Thanks!

-- 
 Bastien

Re: TODO type problem on speedbar and imenu.

[Nicolas Goaziou]

[-- Attachment #1: Type: text/plain, Size: 527 bytes --]

Hello,

Bastien <bzg@altern.org> writes:

> Yep.  When you have some free time (!), please suggest a patch and a
> small org test file so that we can test.

Here is an attempt to convert word boundaries into white spaces where it
matters.

I paste again the test file from Osamu Okano for reference.

#+begin_src org
#+TYP_TODO: SOMEDAY 
* someday/maybe     
* SOMEDAY/maybe     
* SOMEDAY maybe     
* someday maybe     
* read/review       
* READ/review       
* conf              
#+end_src


Regards,

-- 
Nicolas Goaziou

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Enforce-white-space-after-todo-keywords.patch --]
[-- Type: text/x-patch, Size: 2819 bytes --]

From 95709480975155387121fdd18cea144145b47e5c Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou <n.goaziou@gmail.com>
Date: Mon, 25 Jul 2011 17:50:28 +0200
Subject: [PATCH] Enforce white space after todo keywords

* lisp/org.el (org-set-regexps-and-options): enforce white space after
  todo keyword, as word boundary isn't sufficient (i.e. in matches * TODO/this)
---
 lisp/org.el |   28 +++++++++++++++-------------
 1 files changed, 15 insertions(+), 13 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index a13730b..9fef2a8 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -4537,38 +4537,40 @@ means to push this value onto the list in the variable.")
 	    org-not-done-heading-regexp
 	    (concat "^\\(\\*+\\)[ \t]+\\("
 		    (mapconcat 'regexp-quote org-not-done-keywords "\\|")
-		    "\\)\\>")
+		    "\\)[ \t]+")
 	    org-todo-line-regexp
 	    (concat "^\\(\\*+\\)[ \t]+\\(?:\\("
 		    (mapconcat 'regexp-quote org-todo-keywords-1 "\\|")
-		    "\\)\\>\\)?[ \t]*\\(.*\\)")
+		    "\\)[ \t]+\\)?\\(.*\\)")
 	    org-complex-heading-regexp
 	    (concat "^\\(\\*+\\)[ \t]+\\(?:\\("
 		    (mapconcat 'regexp-quote org-todo-keywords-1 "\\|")
-		    "\\)\\>\\)?\\(?:[ \t]*\\(\\[#.\\]\\)\\)?[ \t]*\\(.*?\\)"
+		    "\\)[ \t]+\\)?"
+		    "\\(?:\\(\\[#.\\]\\)[ \t]+\\)?"
+		    "\\(.*?\\)"
 		    "\\(?:[ \t]+\\(:[[:alnum:]_@#%:]+:\\)\\)?[ \t]*$")
 	    org-complex-heading-regexp-format
 	    (concat "^\\(\\*+\\)[ \t]+\\(?:\\("
 		    (mapconcat 'regexp-quote org-todo-keywords-1 "\\|")
-		    "\\)\\>\\)?"
-		    "\\(?:[ \t]*\\(\\[#.\\]\\)\\)?"
-		    "\\(?:[ \t]*\\(?:\\[[0-9%%/]+\\]\\)\\)?" ;; stats cookie
-		    "[ \t]*\\(%s\\)"
-		    "\\(?:[ \t]*\\(?:\\[[0-9%%/]+\\]\\)\\)?" ;; stats cookie
-		    "\\(?:[ \t]+\\(:[[:alnum:]_@#%%:]+:\\)\\)?[ \t]*$")
+		    "\\)[ \t]+\\)?"
+		    "\\(?:\\(\\[#.\\]\\)[ \t]+\\)?"
+		    "\\(?:\\(?:\\[[0-9%%/]+\\]\\)[ \t]+\\)?" ; stats cookie
+		    "\\(%s\\)[ \t]*"
+		    "\\(?:\\[[0-9%%/]+\\]\\)?" ; stats cookie
+		    (org-re "\\(?:[ \t]+\\(:[[:alnum:]_@#%%:]+:\\)\\)?[ \t]*$"))
 	    org-nl-done-regexp
 	    (concat "\n\\*+[ \t]+"
 		    "\\(?:" (mapconcat 'regexp-quote org-done-keywords "\\|")
-		    "\\)" "\\>")
+		    "\\)" "[ \t]+")
 	    org-todo-line-tags-regexp
 	    (concat "^\\(\\*+\\)[ \t]+\\(?:\\("
 		    (mapconcat 'regexp-quote org-todo-keywords-1 "\\|")
-		    (org-re
-		     "\\)\\>\\)? *\\(.*?\\([ \t]:[[:alnum:]:_@#%]+:[ \t]*\\)?$\\)"))
+		    "\\)[ \t]+\\)"
+		    (org-re "\\(.*?\\([ \t]:[[:alnum:]:_@#%]+:[ \t]*\\)?$\\)"))
 	    org-looking-at-done-regexp
 	    (concat "^" "\\(?:"
 		    (mapconcat 'regexp-quote org-done-keywords "\\|") "\\)"
-		    "\\>")
+		    "[ \t]+")
 	    org-deadline-regexp (concat "\\<" org-deadline-string)
 	    org-deadline-time-regexp
 	    (concat "\\<" org-deadline-string " *<\\([^>]+\\)>")
-- 
1.7.6

Re: TODO type problem on speedbar and imenu.

[Bastien]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Bastien <bzg@altern.org> writes:
>
>> Yep.  When you have some free time (!), please suggest a patch and a
>> small org test file so that we can test.
>
> Here is an attempt to convert word boundaries into white spaces where it
> matters.

Thanks a lot -- I have tested the patch during the last two hours and
didn't find any problem.  Feel free to apply it.

What about going one step further and removing the possibility of 
using tabs after TODO keywords?

Best,

-- 
 Bastien

Re: TODO type problem on speedbar and imenu.

[Nicolas Goaziou]

Hello,

Bastien <bzg@altern.org> writes:

> Thanks a lot -- I have tested the patch during the last two hours and
> didn't find any problem.  Feel free to apply it.

Done.

> What about going one step further and removing the possibility of 
> using tabs after TODO keywords?

There are quite a few places indeed using "[ \t]" which may use " "
instead. Before TODO keywords, before priority cookies, before
check-boxes, etc.

We might remove any use of [ \t] but for indentation. Though, I don't
use tabs, and there could well be corner cases that I can't see[1].

Regards,

[1] Like "* TODO          Headline          :tag:"

-- 
Nicolas Goaziou

Re: TODO type problem on speedbar and imenu.

[Carsten Dominik]

On 13.8.2011, at 12:16, Nicolas Goaziou wrote:

> Hello,
> 
> Bastien <bzg@altern.org> writes:
> 
>> Yep.  When you have some free time (!), please suggest a patch and a
>> small org test file so that we can test.
> 
> Here is an attempt to convert word boundaries into white spaces where it
> matters.

I am not sure this is a good idea.  With this patch, you always
require the space to be there.  So "* TODO" would not be a task.
I am not sure, this might have side effects.

- Carsten

> 
> I paste again the test file from Osamu Okano for reference.
> 
> #+begin_src org
> #+TYP_TODO: SOMEDAY 
> * someday/maybe     
> * SOMEDAY/maybe     
> * SOMEDAY maybe     
> * someday maybe     
> * read/review       
> * READ/review       
> * conf              
> #+end_src
> 
> 
> Regards,
> 
> -- 
> Nicolas Goaziou
> <0001-Enforce-white-space-after-todo-keywords.patch>

Re: TODO type problem on speedbar and imenu.

[Nicolas Goaziou]

Hello,

Carsten Dominik <carsten.dominik@gmail.com> writes:

> I am not sure this is a good idea.  With this patch, you always
> require the space to be there.  So "* TODO" would not be a task.
> I am not sure, this might have side effects.

Notwithstanding the fontification problem, isn't "* TODO" considered as
a valid task, whose text is "TODO" and without a keyword?

Regards,


-- 
Nicolas Goaziou

Re: TODO type problem on speedbar and imenu.

[Carsten Dominik]

On Aug 16, 2011, at 11:44 PM, Nicolas Goaziou wrote:

> Hello,
> 
> Carsten Dominik <carsten.dominik@gmail.com> writes:
> 
>> I am not sure this is a good idea.  With this patch, you always
>> require the space to be there.  So "* TODO" would not be a task.
>> I am not sure, this might have side effects.
> 
> Notwithstanding the fontification problem, isn't "* TODO" considered as
> a valid task, whose text is "TODO" and without a keyword?

Well, the behavior is really undefined on these border cases.
My worries also stem from the possibility that the match
of these regexps now extends an additional character, and
there may be places in the code which rely on (match-end 0)
being right after (e.g.) the TODO keyword.
I do not know if this is the case, but it
is a definite possibility.

- Carsten

Re: TODO type problem on speedbar and imenu.

[Bastien]

Hi nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Notwithstanding the fontification problem, isn't "* TODO" considered
> as a valid task, whose text is "TODO" and without a keyword?

FWIW, I'm not in favor of this -- I'd rather consider it a task with a
todo keyword but with no "true headline".

-- 
 Bastien

Re: TODO type problem on speedbar and imenu.

[Bastien]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> There are quite a few places indeed using "[ \t]" which may use " "
> instead. Before TODO keywords, before priority cookies, before
> check-boxes, etc.

Enforcing " " _before_ the places you mention would be good change.
Feel free to commit such a change if you have some time.

Thanks,

-- 
 Bastien

Re: TODO type problem on speedbar and imenu.

[Bastien]

Hi Carsten and Nicolas,

Carsten Dominik <carsten.dominik@gmail.com> writes:

>> Notwithstanding the fontification problem, isn't "* TODO" considered as
>> a valid task, whose text is "TODO" and without a keyword?
>
> Well, the behavior is really undefined on these border cases.

There are side-effects to the current behavior.

Basically, in headlines like "* TODO", the TODO keyword will not be
matched correctly.  So in column view, "TODO" will be displayed as the
ITEM instead of the TODO keyword, which will certainly be confusing for
users (see my other email).

I suggest enforcing [ \t\n] instead of [ \t] to make sure those
headlines are handled correctly.

> My worries also stem from the possibility that the match
> of these regexps now extends an additional character, and
> there may be places in the code which rely on (match-end 0)
> being right after (e.g.) the TODO keyword.
> I do not know if this is the case, but it
> is a definite possibility.

Whether this is the case or not, I think the correct fix is to 
enforce [ \t\n] _outside_ the TODO keyword submatch.

Nicolas, can you make the two suggested changes, i.e. using 
[ \t\n] and make sure this string is required outside the TODO 
keyword submatch? 

If you don't have time just let me know and I'll do it.

Thanks!

-- 
 Bastien

Re: TODO type problem on speedbar and imenu.

[Sebastien Vauban]

Hi Bastien and Nicolas,

Bastien wrote:
> Nicolas Goaziou <n.goaziou-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>> Notwithstanding the fontification problem, isn't "* TODO" considered
>> as a valid task, whose text is "TODO" and without a keyword?
>
> FWIW, I'm not in favor of this -- I'd rather consider it a task with a
> todo keyword but with no "true headline".

FWIW bis, I would make the exact same interpretation as you do, Bastien.

Best regards,
  Seb

-- 
Sebastien Vauban

Re: TODO type problem on speedbar and imenu.

[Nicolas Goaziou]

Hello,

Bastien <bzg@altern.org> writes:

> Hi Nicolas,
>
> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>
>> There are quite a few places indeed using "[ \t]" which may use " "
>> instead. Before TODO keywords, before priority cookies, before
>> check-boxes, etc.
>
> Enforcing " " _before_ the places you mention would be good change.
> Feel free to commit such a change if you have some time.

I don't mind providing a commit for this, but the list wasn't
exhaustive. I'd rather have a set of rules which would be part of the
Org format specification.

What about : allow mixing tabs and spaces only when indenting or
filling. One or more spaces everywhere[1] else.

An heading regexp would then be:

"^\\*\\+\\( +TODO\\)?\\( +\\[#.\\]\\)?\\( +.*?\\)?\\([ \t]+\\(:[[:alnum:]]_@#%:\\)\\)?[ \t]*$"

Note the use of [ \t]+ to fill the tags to the right. Also note that
regexp means "^***" is a valid regexp (which isn't the case actually).

Regards,

[1] As for every rule, some exceptions: check-boxes cookies and
counters, which can be sticked to respectively the headline text and the
check-box.

-- 
Nicolas Goaziou

Re: TODO type problem on speedbar and imenu.

[Bastien]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> I don't mind providing a commit for this, but the list wasn't
> exhaustive. I'd rather have a set of rules which would be part of the
> Org format specification.

Agreed.

> What about : allow mixing tabs and spaces only when indenting or
> filling. One or more spaces everywhere[1] else.

If you feel confident this is flexible enough, please go ahead.
We only need to make sure that a task like

* TODO Task       
           ^^^^^^ <= unintentional mixed tabs/spaces

is okay.  In other words: enforce a set of rules, but in a way
that will not surprise users if they accidently hit spaces or tabs
in position like the end of a line.

> An heading regexp would then be:
>
> "^\\*\\+\\( +TODO\\)?\\( +\\[#.\\]\\)?\\( +.*?\\)?\\([ \t]+\\(:[[:alnum:]]_@#%:\\)\\)?[ \t]*$"
>
> Note the use of [ \t]+ to fill the tags to the right. Also note that
> regexp means "^***" is a valid regexp (which isn't the case actually).

Yes, I think keeping "^***" as a valid regexp is a good idea.

> [1] As for every rule, some exceptions: check-boxes cookies and
> counters, which can be sticked to respectively the headline text and the
> check-box.

Okay.

Thanks for looking again into this when you have some time!

-- 
 Bastien

Re: TODO type problem on speedbar and imenu.

[Carsten Dominik]

On Aug 24, 2011, at 4:18 PM, Bastien wrote:

> Hi Nicolas,
> 
> Nicolas Goaziou <n.goaziou@gmail.com> writes:
> 
>> I don't mind providing a commit for this, but the list wasn't
>> exhaustive. I'd rather have a set of rules which would be part of the
>> Org format specification.
> 
> Agreed.
> 
>> What about : allow mixing tabs and spaces only when indenting or
>> filling. One or more spaces everywhere[1] else.

Hi Nicolas

Org currently also uses \t before the tags, when it aligns them.

- Carsten

> 
> If you feel confident this is flexible enough, please go ahead.
> We only need to make sure that a task like
> 
> * TODO Task       
>           ^^^^^^ <= unintentional mixed tabs/spaces
> 
> is okay.  In other words: enforce a set of rules, but in a way
> that will not surprise users if they accidently hit spaces or tabs
> in position like the end of a line.
> 
>> An heading regexp would then be:
>> 
>> "^\\*\\+\\( +TODO\\)?\\( +\\[#.\\]\\)?\\( +.*?\\)?\\([ \t]+\\(:[[:alnum:]]_@#%:\\)\\)?[ \t]*$"
>> 
>> Note the use of [ \t]+ to fill the tags to the right. Also note that
>> regexp means "^***" is a valid regexp (which isn't the case actually).
> 
> Yes, I think keeping "^***" as a valid regexp is a good idea.
> 
>> [1] As for every rule, some exceptions: check-boxes cookies and
>> counters, which can be sticked to respectively the headline text and the
>> check-box.
> 
> Okay.
> 
> Thanks for looking again into this when you have some time!
> 
> -- 
> Bastien
> 

- Carsten

Re: TODO type problem on speedbar and imenu.

[Nicolas Goaziou]

[-- Attachment #1: Type: text/plain, Size: 487 bytes --]

Hello,

Bastien <bzg@altern.org> writes:

> Yes, I think keeping "^***" as a valid regexp is a good idea.

In fact, it isn't. "^***" shouldn't match an headline. Not yet
actually. If we want such headlines, org-outline-regexp and al. have to
be set to something like "\\*+". That would break many things, and
outline functions would match unexpected things, like *bold* at
column 0.

Anyway, here's another attempt to improve headline regexps consistency.

Regards,

-- 
Nicolas Goaziou

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch for headlines regexps --]
[-- Type: text/x-patch, Size: 23395 bytes --]

From 758f1342b01a0dd2b36873a1fa948c6cbf7b1ab0 Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou <n.goaziou@gmail.com>
Date: Thu, 25 Aug 2011 01:58:29 +0200
Subject: [PATCH] Provide more consistent regexps for headlines

* lisp/org-agenda.el (org-search-view): simplify regexp.
(org-agenda-get-todos): use new format string.
* lisp/org-archive.el (org-archive-all-done): simplify regexp.
* lisp/org-ascii.el (org-export-as-ascii): more accurate regexp.
* lisp/org-colview-xemacs.el (org-columns-capture-view): use new
  format string and new string.
* lisp/org-colview.el (org-columns-capture-view): use new format
  string and new string.
* lisp/org-docbook.el (org-export-as-docbook): more accurate
  regexp. Also use new regexp to match generic headlines.
* lisp/org-exp.el (org-export-protect-quoted-subtrees): more accurate
  regexp. Also use new regexp to match generic headlines.
* lisp/org-html.el (org-export-as-html): more accurate regexp. Also
  use new regexp to match generic headlines.
* lisp/org-mouse.el (org-mouse-match-todo-keyword): removed unused
  and now erroneous function.
* lisp/org.el (org-heading-regexp, org-heading-keyword-regexp-format):
  new variables.
(org-set-regexps-and-options): create regexps according to the
following rule: use spaces only to separate elements from an headline,
while allowing mixed tabs and spaces for any indentation job.
(org-nl-done-regexp, org-looking-at-done-regexp): removed variables.
(org-set-font-lock-defaults): fontify again headlines with a keyword
and no other text. Use new format strings.
(org-get-heading, org-toggle-comment, org-prepare-agenda-buffers,
org-toggle-fixed-width-section): use new format string.
(org-todo): more accurate regexps.
(org-point-at-end-of-empty-headline): simplify regexp.

This patch attempts to reduce the number of hard-coded headlines, by
providing two format strings and one generic string to cover most of
the cases of headline construction.
---
 lisp/org-agenda.el         |   30 ++++---
 lisp/org-archive.el        |    2 +-
 lisp/org-ascii.el          |    4 +-
 lisp/org-colview-xemacs.el |    5 +-
 lisp/org-colview.el        |    5 +-
 lisp/org-docbook.el        |    7 +-
 lisp/org-exp.el            |    6 +-
 lisp/org-html.el           |    7 +-
 lisp/org-mouse.el          |    7 --
 lisp/org.el                |  179 +++++++++++++++++++++++++-------------------
 10 files changed, 139 insertions(+), 113 deletions(-)

diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index b157d39..22d3ab7 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -3868,7 +3868,7 @@ in `org-agenda-text-search-extra-files'."
     (if (not regexps+)
 	(setq regexp org-outline-regexp-bol)
       (setq regexp (pop regexps+))
-      (if hdl-only (setq regexp (concat "^" org-outline-regexp ".*?"
+      (if hdl-only (setq regexp (concat org-outline-regexp-bol " .*?"
 					regexp))))
     (setq files (org-agenda-files nil 'ifmode))
     (when (eq (car org-agenda-text-search-extra-files) 'agenda-archives)
@@ -4574,15 +4574,19 @@ the documentation of `org-diary'."
 		      'help-echo
 		      (format "mouse-2 or RET jump to org file %s"
 			      (abbreviate-file-name buffer-file-name))))
-	 (regexp (concat "^\\*+[ \t]+\\("
-			 (if org-select-this-todo-keyword
-			     (if (equal org-select-this-todo-keyword "*")
-				 org-todo-regexp
-			       (concat "\\<\\("
-				       (mapconcat 'identity (org-split-string org-select-this-todo-keyword "|") "\\|")
-				     "\\)\\>"))
-			   org-not-done-regexp)
-			 "[^\n\r]*\\)"))
+	 (regexp (format org-heading-keyword-regexp-format
+			 (cond
+			  ((and org-select-this-todo-keyword
+				(equal org-select-this-todo-keyword "*"))
+			   org-todo-regexp)
+			  (org-select-this-todo-keyword
+			   (concat "\\("
+				   (mapconcat 'identity
+					      (org-split-string
+					       org-select-this-todo-keyword
+					       "|")
+					      "\\|") "\\)"))
+			  (t org-not-done-regexp))))
 	 marker priority category tags todo-state
 	 ee txt beg end)
     (goto-char (point-min))
@@ -4596,11 +4600,11 @@ the documentation of `org-diary'."
 	    (goto-char (1+ beg))
 	    (or org-agenda-todo-list-sublevels (org-end-of-subtree 'invisible))
 	    (throw :skip nil)))
-	(goto-char (match-beginning 1))
+	(goto-char (match-beginning 2))
 	(setq marker (org-agenda-new-marker (match-beginning 0))
 	      category (org-get-category)
 	      category-pos (get-text-property (point) 'org-category-position)
-	      txt (match-string 1)
+	      txt (match-string 2)
 	      tags (org-get-tags-at (point))
 	      txt (org-format-agenda-item "" txt category tags)
 	      priority (1+ (org-get-priority txt))
@@ -4612,7 +4616,7 @@ the documentation of `org-diary'."
 	  'type "todo" 'todo-state todo-state)
 	(push txt ee)
 	(if org-agenda-todo-list-sublevels
-	    (goto-char (match-end 1))
+	    (goto-char (match-end 2))
 	  (org-end-of-subtree 'invisible))))
     (nreverse ee)))
 
diff --git a/lisp/org-archive.el b/lisp/org-archive.el
index 16c35cf..c7a4049 100644
--- a/lisp/org-archive.el
+++ b/lisp/org-archive.el
@@ -404,7 +404,7 @@ sibling does not exist, it will be created at the end of the subtree."
 If the cursor is not on a headline, try all level 1 trees.  If
 it is on a headline, try all direct children.
 When TAG is non-nil, don't move trees, but mark them with the ARCHIVE tag."
-  (let ((re (concat org-outline-regexp-bol "+" org-not-done-regexp)) re1
+  (let ((re org-not-done-heading-regexp) re1
 	(rea (concat ".*:" org-archive-tag ":"))
 	(begm (make-marker))
 	(endm (make-marker))
diff --git a/lisp/org-ascii.el b/lisp/org-ascii.el
index 056f44b..2663886 100644
--- a/lisp/org-ascii.el
+++ b/lisp/org-ascii.el
@@ -283,7 +283,7 @@ publishing directory."
 		    "UNTITLED"))
 	 (email (plist-get opt-plist :email))
 	 (language (plist-get opt-plist :language))
-	 (quote-re0 (concat "^[ \t]*" org-quote-string "\\>"))
+	 (quote-re0 (concat "^\\(" org-quote-string "\\)\\( +\\|[ \t]*$\\)"))
 	 (todo nil)
 	 (lang-words nil)
 	 (region
@@ -406,7 +406,7 @@ publishing directory."
 				   txt))
 			     (setq txt (replace-match "" t t txt)))
 			 (if (string-match quote-re0 txt)
-			     (setq txt (replace-match "" t t txt)))
+			     (setq txt (replace-match "" t t txt 1)))
 
 			 (if org-export-with-section-numbers
 			     (setq txt (concat (org-section-number level)
diff --git a/lisp/org-colview-xemacs.el b/lisp/org-colview-xemacs.el
index b32e741..a85762f 100644
--- a/lisp/org-colview-xemacs.el
+++ b/lisp/org-colview-xemacs.el
@@ -1317,12 +1317,13 @@ of fields."
   (if (featurep 'xemacs)
       (save-excursion
         (let* ((title (mapcar 'cadr org-columns-current-fmt-compiled))
-	       (re-comment (concat "\\*+[ \t]+" org-comment-string "\\>"))
+	       (re-comment (format org-heading-keyword-regexp-format
+				   org-comment-string))
 	       (re-archive (concat ".*:" org-archive-tag ":"))
                (n (length title)) row tbl)
           (goto-char (point-min))
 
-	  (while (re-search-forward "^\\(\\*+\\) " nil t)
+	  (while (re-search-forward org-heading-regexp nil t)
 	    (catch 'next
 	      (when (and (or (null maxlevel)
 			     (>= maxlevel
diff --git a/lisp/org-colview.el b/lisp/org-colview.el
index 829a134..0635419 100644
--- a/lisp/org-colview.el
+++ b/lisp/org-colview.el
@@ -1152,11 +1152,12 @@ containing the title row and all other rows.  Each row is a list
 of fields."
   (save-excursion
     (let* ((title (mapcar 'cadr org-columns-current-fmt-compiled))
-	   (re-comment (concat "\\*+[ \t]+" org-comment-string "\\>"))
+	   (re-comment (format org-heading-keyword-regexp-format
+			       org-comment-string))
 	   (re-archive (concat ".*:" org-archive-tag ":"))
 	   (n (length title)) row tbl)
       (goto-char (point-min))
-      (while (re-search-forward "^\\(\\*+\\) " nil t)
+      (while (re-search-forward org-heading-regexp nil t)
 	(catch 'next
 	  (when (and (or (null maxlevel)
 			 (>= maxlevel
diff --git a/lisp/org-docbook.el b/lisp/org-docbook.el
index f09740f..df8db64 100644
--- a/lisp/org-docbook.el
+++ b/lisp/org-docbook.el
@@ -498,8 +498,9 @@ publishing directory."
 	 ;; We will use HTML table formatter to export tables to DocBook
 	 ;; format, so need to set html-table-tag here.
 	 (html-table-tag (plist-get opt-plist :html-table-tag))
-	 (quote-re0   (concat "^[ \t]*" org-quote-string "\\>"))
-	 (quote-re    (concat "^\\(\\*+\\)\\([ \t]+" org-quote-string "\\>\\)"))
+	 (quote-re0   (concat "^ *" org-quote-string "\\( +\\|[ \t]*$\\)"))
+	 (quote-re    (format org-heading-keyword-regexp-format
+			      org-quote-string))
 	 (inquote     nil)
 	 (infixed     nil)
 	 (inverse     nil)
@@ -969,7 +970,7 @@ publishing directory."
 		    (push (cons num 1) footref-seen))))))
 
 	  (cond
-	   ((string-match "^\\(\\*+\\)[ \t]+\\(.*\\)" line)
+	   ((string-match org-heading-regexp line)
 	    ;; This is a headline
 	    (setq level (org-tr-level (- (match-end 1) (match-beginning 1)
 					 level-offset))
diff --git a/lisp/org-exp.el b/lisp/org-exp.el
index f795fbd..e6185d1 100644
--- a/lisp/org-exp.el
+++ b/lisp/org-exp.el
@@ -1637,7 +1637,8 @@ from the buffer."
 
 (defun org-export-protect-quoted-subtrees ()
   "Mark quoted subtrees with the protection property."
-  (let ((org-re-quote (concat "^\\*+[ \t]+" org-quote-string "\\>")))
+  (let ((org-re-quote (format org-heading-keyword-regexp-format
+			      org-quote-string)))
     (goto-char (point-min))
     (while (re-search-forward org-re-quote nil t)
       (goto-char (match-beginning 0))
@@ -1925,7 +1926,8 @@ table line.  If it is a link, add it to the line containing the link."
 
 (defun org-export-remove-comment-blocks-and-subtrees ()
   "Remove the comment environment, and also commented subtrees."
-  (let ((re-commented (concat "^\\*+[ \t]+" org-comment-string "\\>"))
+  (let ((re-commented (format org-heading-keyword-regexp-format
+			      org-comment-string))
 	case-fold-search)
     ;; Remove comment environment
     (goto-char (point-min))
diff --git a/lisp/org-html.el b/lisp/org-html.el
index bf2d012..8c627fa 100644
--- a/lisp/org-html.el
+++ b/lisp/org-html.el
@@ -1172,8 +1172,9 @@ PUB-DIR is set, use this as the publishing directory."
 			 (plist-get opt-plist :link-home)))
 	 (dummy (setq opt-plist (plist-put opt-plist :title title)))
 	 (html-table-tag (plist-get opt-plist :html-table-tag))
-	 (quote-re0   (concat "^[ \t]*" org-quote-string "\\>"))
-	 (quote-re    (concat "^\\(\\*+\\)\\([ \t]+" org-quote-string "\\>\\)"))
+	 (quote-re0   (concat "^ *" org-quote-string "\\( +\\|[ \t]*$\\)"))
+	 (quote-re    (format org-heading-keyword-regexp-format
+			      org-quote-string))
 	 (inquote     nil)
 	 (infixed     nil)
 	 (inverse     nil)
@@ -1643,7 +1644,7 @@ lang=\"%s\" xml:lang=\"%s\">
 			 t t line))))))
 
 	  (cond
-	   ((string-match "^\\(\\*+\\)[ \t]+\\(.*\\)" line)
+	   ((string-match org-heading-regexp line)
 	    ;; This is a headline
 	    (setq level (org-tr-level (- (match-end 1) (match-beginning 1)
 					 level-offset))
diff --git a/lisp/org-mouse.el b/lisp/org-mouse.el
index 346ba6d..ba625d3 100644
--- a/lisp/org-mouse.el
+++ b/lisp/org-mouse.el
@@ -631,13 +631,6 @@ This means, between the beginning of line and the point."
 	(set-match-data ',match)
 	(apply ',function rest)))))
 
-(defun org-mouse-match-todo-keyword ()
-  (save-excursion
-    (org-back-to-heading)
-    (if (looking-at org-outline-regexp) (goto-char (match-end 0)))
-    (or (looking-at (concat " +" org-todo-regexp " *"))
-	(looking-at " \\( *\\)"))))
-
 (defun org-mouse-yank-link (click)
   (interactive "e")
   ;; Give temporary modes such as isearch a chance to turn off.
diff --git a/lisp/org.el b/lisp/org.el
index 6ee3b4e..2fcc6cf 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -4177,6 +4177,10 @@ collapsed state."
 
 ;;; Variables for pre-computed regular expressions, all buffer local
 
+(defvar org-heading-regexp nil
+  "Matches an headline.
+Stars are put in group 1 and the trimmed body in group 2.")
+(make-variable-buffer-local 'org-heading-regexp)
 (defvar org-drawer-regexp nil
   "Matches first line of a hidden block.")
 (make-variable-buffer-local 'org-drawer-regexp)
@@ -4200,22 +4204,28 @@ group 3: Priority cookie
 group 4: True headline
 group 5: Tags")
 (make-variable-buffer-local 'org-complex-heading-regexp)
+(defvar org-heading-keyword-regexp-format nil
+  "Printf format to make regexp to match an headline with some keyword.
+This regexp will match the headline of any node which has the
+exact keyword that is put into the format.  The keyword isn't in
+any group by default, but the stars and the body are.")
+(make-variable-buffer-local 'org-heading-keyword-regexp-format)
+(defvar org-heading-keyword-maybe-regexp-format nil
+  "Printf format to make regexp to match an headline with some keyword.
+This regexp can match any headline with the specified keyword, or
+a without a keyword.  The keyword isn't in any group by default,
+but the stars and the body are.")
+(make-variable-buffer-local 'org-heading-keyword-maybe-regexp-format)
 (defvar org-complex-heading-regexp-format nil
   "Printf format to make regexp to match an exact headline.
-This regexp will match the headline of any node which hase the exact
-headline text that is put into the format, but may have any TODO state,
-priority and tags.")
+This regexp will match the headline of any node which has the
+exact headline text that is put into the format, but may have any
+TODO state, priority and tags.")
 (make-variable-buffer-local 'org-complex-heading-regexp-format)
 (defvar org-todo-line-tags-regexp nil
   "Matches a headline and puts TODO state into group 2 if present.
 Also put tags into group 4 if tags are present.")
 (make-variable-buffer-local 'org-todo-line-tags-regexp)
-(defvar org-nl-done-regexp nil
-  "Matches newline followed by a headline with the DONE keyword.")
-(make-variable-buffer-local 'org-nl-done-regexp)
-(defvar org-looking-at-done-regexp nil
-  "Matches the DONE keyword a point.")
-(make-variable-buffer-local 'org-looking-at-done-regexp)
 (defvar org-ds-keyword-length 12
   "Maximum length of the Deadline and SCHEDULED keywords.")
 (make-variable-buffer-local 'org-ds-keyword-length)
@@ -4550,7 +4560,9 @@ means to push this value onto the list in the variable.")
 		     (assoc (car e) org-tag-alist))
 		(push e org-tag-alist)))))
 
-      ;; Compute the regular expressions and other local variables
+      ;; Compute the regular expressions and other local variables.
+      ;; Using `org-outline-regexp-bol' would complicate them much,
+      ;; because of the fixed white space at the end of that string.
       (if (not org-done-keywords)
 	  (setq org-done-keywords (and org-todo-keywords-1
 				       (list (org-last org-todo-keywords-1)))))
@@ -4565,49 +4577,48 @@ means to push this value onto the list in the variable.")
 	    org-not-done-keywords
 	    (org-delete-all org-done-keywords (copy-sequence org-todo-keywords-1))
 	    org-todo-regexp
-	    (concat "\\<\\(" (mapconcat 'regexp-quote org-todo-keywords-1
-					"\\|") "\\)\\>")
+	    (concat "\\("
+		    (mapconcat 'regexp-quote org-todo-keywords-1 "\\|")
+		    "\\)")
 	    org-not-done-regexp
-	    (concat "\\<\\("
+	    (concat "\\("
 		    (mapconcat 'regexp-quote org-not-done-keywords "\\|")
-		    "\\)\\>")
+		    "\\)")
+	    org-heading-regexp
+	    "\\(\\*+\\)\\(?: +\\(.*?\\)\\)?[ \t]*$"
+	    org-heading-keyword-regexp-format
+	    "\\(\\*+\\)\\(?: +%s\\)\\(?: +\\(.*?\\)\\)?[ \t]*$"
+	    org-heading-keyword-maybe-regexp-format
+	    "\\(\\*+\\)\\(?: +%s\\)?\\(?: +\\(.*?\\)\\)?[ \t]*$"
 	    org-not-done-heading-regexp
-	    (concat "^\\(\\*+\\)[ \t]+\\("
-		    (mapconcat 'regexp-quote org-not-done-keywords "\\|")
-		    "\\)[ \t]+")
+	    (format org-heading-keyword-regexp-format org-not-done-regexp)
 	    org-todo-line-regexp
-	    (concat "^\\(\\*+\\)[ \t]+\\(?:\\("
-		    (mapconcat 'regexp-quote org-todo-keywords-1 "\\|")
-		    "\\)[ \t]+\\)?\\(.*\\)")
+	    (format org-heading-keyword-maybe-regexp-format org-todo-regexp)
 	    org-complex-heading-regexp
-	    (concat "^\\(\\*+\\)[ \t]+\\(?:\\("
-		    (mapconcat 'regexp-quote org-todo-keywords-1 "\\|")
-		    "\\)[ \t]+\\)?"
-		    "\\(?:\\(\\[#.\\]\\)[ \t]+\\)?"
-		    "\\(.*?\\)"
-		    "\\(?:[ \t]+\\(:[[:alnum:]_@#%:]+:\\)\\)?[ \t]*$")
+	    (concat "\\(\\*+\\)"
+		    "\\(?: +" org-todo-regexp "\\)?"
+		    "\\(?: +\\(\\[#.\\]\\)\\)?"
+		    "\\( +:\\(.*?\\)\\)?"
+		    (org-re "\\(?:[ \t]+\\(:[[:alnum:]_@#%:]+:\\)\\)?")
+		    "[ \t]*$")
 	    org-complex-heading-regexp-format
-	    (concat "^\\(\\*+\\)[ \t]+\\(?:\\("
-		    (mapconcat 'regexp-quote org-todo-keywords-1 "\\|")
-		    "\\)[ \t]+\\)?"
-		    "\\(?:\\(\\[#.\\]\\)[ \t]+\\)?"
-		    "\\(?:\\(?:\\[[0-9%%/]+\\]\\)[ \t]+\\)?" ; stats cookie
-		    "\\(%s\\)[ \t]*"
-		    "\\(?:\\[[0-9%%/]+\\]\\)?" ; stats cookie
-		    (org-re "\\(?:[ \t]+\\(:[[:alnum:]_@#%%:]+:\\)\\)?[ \t]*$"))
-	    org-nl-done-regexp
-	    (concat "\n\\*+[ \t]+"
-		    "\\(?:" (mapconcat 'regexp-quote org-done-keywords "\\|")
-		    "\\)" "[ \t]+")
+	    (concat "\\(\\*+\\)"
+		    "\\(?: +" org-todo-regexp "\\)?"
+		    "\\(?: +\\(\\[#.\\]\\)\\)?"
+		    "\\(?: +"
+		    ;; Stats cookies can be sticked to body.
+		    "\\(?:\\[[0-9%%/]+\\] *\\)?"
+		    "\\(%s\\)"
+		    "\\(?: *\\[[0-9%%/]+\\]\\)?"
+		    "\\)"
+		    (org-re "\\(?:[ \t]+\\(:[[:alnum:]_@#%%:]+:\\)\\)?")
+		    "[ \t]*$")
 	    org-todo-line-tags-regexp
-	    (concat "^\\(\\*+\\)[ \t]+\\(?:\\("
-		    (mapconcat 'regexp-quote org-todo-keywords-1 "\\|")
-		    "\\)[ \t]+\\)"
-		    (org-re "\\(.*?\\([ \t]:[[:alnum:]:_@#%]+:[ \t]*\\)?$\\)"))
-	    org-looking-at-done-regexp
-	    (concat "^" "\\(?:"
-		    (mapconcat 'regexp-quote org-done-keywords "\\|") "\\)"
-		    "[ \t]+")
+	    (concat "\\(\\*+\\)"
+		    "\\(?: +" org-todo-regexp "\\)?"
+		    "\\( +:\\(.*?\\)\\)?"
+		    (org-re "\\(?:[ \t]+\\(:[[:alnum:]:_@#%]+:\\)\\)?")
+		    "[ \t]*$")
 	    org-deadline-regexp (concat "\\<" org-deadline-string)
 	    org-deadline-time-regexp
 	    (concat "\\<" org-deadline-string " *<\\([^>]+\\)>")
@@ -5658,13 +5669,16 @@ needs to be inserted at a specific position in the font-lock sequence.")
 	   '("^&?%%(.*\\|<%%([^>\n]*?>" (0 'org-sexp-date t))
 	   '(org-hide-wide-columns (0 nil append))
 	   ;; TODO lines
-	   (list (concat "^\\*+[ \t]+" org-todo-regexp "\\([ \t]\\|$\\)")
-		 '(1 (org-get-todo-face 1) t))
+	   (list (format org-heading-keyword-regexp-format
+			 org-todo-regexp)
+		 '(2 (org-get-todo-face 2) t))
 	   ;; DONE
 	   (if org-fontify-done-headline
-	       (list (concat "^[*]+ +\\<\\("
-			     (mapconcat 'regexp-quote org-done-keywords "\\|")
-			     "\\)\\(.*\\)")
+	       (list (format org-heading-keyword-regexp-format
+			     (concat
+			      "\\("
+			      (mapconcat 'regexp-quote org-done-keywords "\\|")
+			      "\\)"))
 		     '(2 'org-headline-done t))
 	     nil)
 	   ;; Priorities
@@ -5702,8 +5716,8 @@ needs to be inserted at a specific position in the font-lock sequence.")
 	   ;; Code
 	   '(org-activate-code (1 'org-code t))
 	   ;; COMMENT
-	   (list (concat "^\\*+[ \t]+\\<\\(" org-comment-string
-			 "\\|" org-quote-string "\\)\\>")
+	   (list (format org-heading-keyword-regexp-format
+			 "\\(" org-comment-string "\\|" org-quote-string "\\)")
 		 '(1 'org-special-keyword t))
 	   '("^#.*" (0 'font-lock-comment-face t))
 	   ;; Blocks and meta lines
@@ -7027,14 +7041,15 @@ When NO-TODO is non-nil, don't include TODO keywords."
       (looking-at org-complex-heading-regexp)
       (match-string 4))
      (no-tags
-      (looking-at "\\*+[ \t]+\\([^\n\r]*?\\)\\([ \t]+:[[:alnum:]:_@#%]+:[ \t]*\\)?$")
+      (looking-at (concat org-outline-regexp
+			  "\\(.*?\\)"
+			  "\\(?:[ \t]+:[[:alnum:]:_@#%]+:\\)?[ \t]*$"))
       (match-string 1))
      (no-todo
-      (looking-at (concat "\\*+[ \t]+" org-todo-regexp " +"
-			  "\\([^\n\r]*?[ \t]+:[[:alnum:]:_@#%]+:[ \t]*\\)?$"))
-      (match-string 2))
-     (t (looking-at "\\*+[ \t]+\\([^\r\n]*\\)")
-	(match-string 1)))))
+      (looking-at org-todo-line-regexp)
+      (match-string 3))
+     (t (looking-at org-heading-regexp)
+	(match-string 2)))))
 
 (defun org-heading-components ()
   "Return the components of the current heading.
@@ -10980,13 +10995,16 @@ expands them."
   (save-excursion
     (org-back-to-heading)
     (let (case-fold-search)
-      (if (looking-at (concat org-outline-regexp
-			      "\\( *\\<" org-comment-string "\\>[ \t]*\\)"))
-	  (replace-match "" t t nil 1)
-	(if (looking-at org-outline-regexp)
-	    (progn
-	      (goto-char (match-end 0))
-	      (insert org-comment-string " ")))))))
+      (cond
+       ((looking-at (format org-heading-keyword-regexp-format
+			    org-comment-string))
+	(goto-char (match-end 1))
+	(looking-at (concat " +" org-comment-string))
+	(replace-match "" t t)
+	(when (eolp) (insert " ")))
+       ((looking-at org-outline-regexp)
+	(goto-char (match-end 0))
+	(insert org-comment-string " "))))))
 
 (defvar org-last-todo-state-is-todo nil
   "This is non-nil when the last TODO state change led to a TODO state.
@@ -11068,8 +11086,8 @@ For calling through lisp, arg is also interpreted in the following way:
       (catch 'exit
 	(org-back-to-heading t)
 	(if (looking-at org-outline-regexp) (goto-char (1- (match-end 0))))
-	(or (looking-at (concat " +" org-todo-regexp "\\( +\\|$\\)"))
-	    (looking-at " *"))
+	(or (looking-at (concat org-todo-regexp "\\( +\\|[ \t]*$\\)"))
+	    (looking-at "\\( *\\|[ \t]*$\\)"))
 	(let* ((match-data (match-data))
 	       (startpos (point-at-bol))
 	       (logging (save-match-data (org-entry-get nil "LOGGING" t t)))
@@ -16285,7 +16303,8 @@ When a buffer is unmodified, it is just killed.  When modified, it is saved
 		  (if (org-on-heading-p t)
 		      (add-text-properties (point-at-bol) (org-end-of-subtree t) pa))))
 	      (goto-char (point-min))
-	      (setq re (concat org-outline-regexp-bol "+" org-comment-string "\\>"))
+	      (setq re (format org-heading-keyword-regexp-format
+			       org-comment-string))
 	      (while (re-search-forward re nil t)
 		(add-text-properties
 		 (match-beginning 0) (org-end-of-subtree t) pc)))
@@ -19836,13 +19855,16 @@ this line is also exported in fixed-width font."
 	    (forward-line 1)))
       (save-excursion
 	(org-back-to-heading)
-	(if (looking-at (concat org-outline-regexp
-				"\\( *\\<" org-quote-string "\\>[ \t]*\\)"))
-	    (replace-match "" t t nil 1)
-	  (if (looking-at org-outline-regexp)
-	      (progn
-		(goto-char (match-end 0))
-		(insert org-quote-string " "))))))))
+	(cond
+	 ((looking-at (format org-heading-keyword-regexp-format
+			      org-quote-string))
+	  (goto-char (match-end 1))
+	  (looking-at (concat " +" org-quote-string))
+	  (replace-match "" t t)
+	  (when (eolp) (insert " ")))
+	 ((looking-at org-outline-regexp)
+	  (goto-char (match-end 0))
+	  (insert org-quote-string " ")))))))
 
 (defun org-reftex-citation ()
   "Use reftex-citation to insert a citation into the buffer.
@@ -20158,8 +20180,9 @@ empty."
        (save-excursion
          (beginning-of-line 1)
 	 (let ((case-fold-search nil))
-	   (looking-at (concat "^\\(\\*+\\)[ \t]+\\(" org-todo-regexp
-			       "\\)?[ \t]*$"))))))
+	   (looking-at org-todo-line-regexp)
+	   (not (match-string 3))))))
+
 (defun org-at-heading-or-item-p ()
   (or (org-on-heading-p) (org-at-item-p)))
 
-- 
1.7.6