proposal to have ignoreheading tags/properties

proposal to have ignoreheading tags/properties

[Mark Edgington]

In using org-mode, there is one problem that has always irked me (and
is apparently also closely related to the FAQ "How do I ignore a
headline?").  When I am writing something, I sometimes want to group
things by concept or by work to be done, or any other number of
groupings.  BUT I do not want these groupings to be part of the
exported document itself.  In fact, I would like it to be as if the
grouping did not exist at all (i.e. a headline that is ignored).

The problem with using an ignored headline for grouping things is that
it still does have an effect on exported document structure, in that
all of the elements contained "inside / in the scope of" the ignored
headline still keep their depth (one level deeper than the level of
the ignored headline).

What I want is for the nested items to have their levels all promoted
by one, so that it's truly as if the ignored headline wasn't there at
all, and that it invisibly wrapped around a group of items without
requiring them to have a deeper level.  Perhaps this could be done by
use of an "ignoreheading" and an "ignoreheadingpromote" tag (one
promotes the level of contained items, another doesn't), or some
equivalent set of properties that could be set on a headline.

Would there be any chance that something like this could be built in
to org-mode?  I think it would make it far more flexible in terms of
organizing things, making this organization process orthogonal to the
selection of sections/subsections of a document.

Re: proposal to have ignoreheading tags/properties

[Thorsten Jolitz]

Mark Edgington <edgimar@gmail.com> writes:

> In using org-mode, there is one problem that has always irked me (and
> is apparently also closely related to the FAQ "How do I ignore a
> headline?").  When I am writing something, I sometimes want to group
> things by concept or by work to be done, or any other number of
> groupings.  BUT I do not want these groupings to be part of the
> exported document itself.  In fact, I would like it to be as if the
> grouping did not exist at all (i.e. a headline that is ignored).
>
> The problem with using an ignored headline for grouping things is that
> it still does have an effect on exported document structure, in that
> all of the elements contained "inside / in the scope of" the ignored
> headline still keep their depth (one level deeper than the level of
> the ignored headline).
>
> What I want is for the nested items to have their levels all promoted
> by one, so that it's truly as if the ignored headline wasn't there at
> all, and that it invisibly wrapped around a group of items without
> requiring them to have a deeper level.  Perhaps this could be done by
> use of an "ignoreheading" and an "ignoreheadingpromote" tag (one
> promotes the level of contained items, another doesn't), or some
> equivalent set of properties that could be set on a headline.
>
> Would there be any chance that something like this could be built in
> to org-mode?  I think it would make it far more flexible in terms of
> organizing things, making this organization process orthogonal to the
> selection of sections/subsections of a document.

In a tree structure, when ignoring the parent node, it seems only
logical that the siblings are ignored too. 

You seem to use the wrong tool for the task (headlines), this looks like
a perfect use case for TAGS, i.e. define your (concept) groups as
tags. If these tags are not part of `org-export-exclude-tags' they won't
affect exporting, but you can still use them to build your agenda or a
sparse tree or so.

-- 
cheers,
Thorsten

Re: proposal to have ignoreheading tags/properties

[Ken Mankoff]

+1 to the OP.

On 2014-06-12 at 13:32, Thorsten Jolitz wrote:
>
> In a tree structure, when ignoring the parent node, it seems only
> logical that the siblings are ignored too.

I'm found myself in all of the following situations: 
  1. I want the heading and everything below not-exported
  2. I want the heading not-exported, but the content exported. 
     1. I don't care if the sub-headings are kept or moved up a level
     2. I do care - I want the sub-headings to remain as-is.
     3. I do care - I want the sub-headings to be promoted.
  3. I want the heading and the heading-level content not-exported, but
     sub-heading and their content exported

> You seem to use the wrong tool for the task (headlines), this looks
> like a perfect use case for TAGS, i.e. define your (concept) groups as
> tags. If these tags are not part of `org-export-exclude-tags' they
> won't affect exporting, but you can still use them to build your
> agenda or a sparse tree or so.

A simple example where tags don't work, nothing to do with agenda. Some
paper formats (Nature, for example) do not allow headings or
sub-headings. It is just 2 pages of text. But it is certainly nice to
write the paper and organize thoughts in sections and sub-sections. This
is case 2.0 above. Each heading is tagged ":noexport" and it is not
exported, but the content (text, figures, lists) below the heading is
exported. I'm not sure how tags would help here, other than the tag of
":noexport". 

Another common (for me) example is to have a heading called "* Appendix"
in a paper, and then the LaTeX "\appendix" command. The Org heading is
just for me. It should not be exported. All headings below the \appendix
command are Org sub-sections but should be promoted to \section in the
final document.

  -k.

Re: proposal to have ignoreheading tags/properties

[Mark Edgington]

Thorsten Jolitz <tjolitz <at> gmail.com> writes:

> 
> In a tree structure, when ignoring the parent node, it seems only
> logical that the siblings are ignored too. 
> 
> You seem to use the wrong tool for the task (headlines), this looks like
> a perfect use case for TAGS, i.e. define your (concept) groups as
> tags. If these tags are not part of `org-export-exclude-tags' they won't
> affect exporting, but you can still use them to build your agenda or a
> sparse tree or so.
> 

Why do you suppose this is the "wrong tool"?  It is a quite natural and
sensible tool, because it allows grouping, folding, and nesting collections
of items together.  I cannot do that with tags.  If you don't like the idea
of having a headline serve this purpose, then perhaps we can invent a new
kind of "pseudo-headline" which behaves in this way.  How would you propose
to use tags alone to do something like the following which allows folding
and unfolding the contents, without a lot of extra work? -- for example:

* Chapters about Topic A  :pseudo:
** Chapter 1 Title
** Chapter 2 Title
* Chapters about Topic B  :pseudo:
** Chapter 3 Title
List of interesting things:
*** items relevant to X   :pseudo:
- item 1
- item 2
- item 3
*** items relevant to Y   :pseudo:
- item 4
- item 5


Another example would be, say, if you wanted to divide up some kind of
text-file (e.g. source code, or prose), dividing it into groupings that make
sense to you, but not wanting to actually bring these changes into the
document's exported structure.  Here's an example of a letter:

* Addresses / date   :pseudo:
123 Cherry Lane
City, ST 12345

October 5, 2014

Ms. Jane Doe
Accounts Payable
....

* Greeting           :pseudo:
Dear Ms. Johnson:

* Body               :pseudo:
It has come to my attention that ......

* Closing            :pseudo:
Sincerely,


John Doe

* Postscript         :pseudo:
P.S. .....


Perhaps one could make it so that when a headline bullet (sequence) has a
'#' character tacked on after the sequence, it is no longer a headline, but
a "summary node" having the property that it promotes the levels of all its
children.  It doesn't much matter to me *how* one makes such a node, but I
think the availability of nodes/headlines like this is important.

In any case, it's not clear that this is "the wrong tool".  I would use it,
and for me (and presumably others) it would be the right (kind of) tool. 
Furthermore tags are limited by their brevity -- with a pseudo headline I
can describe a concept or category with much more detail / clarity.

Re: proposal to have ignoreheading tags/properties

[Thorsten Jolitz]

Ken Mankoff <mankoff@gmail.com> writes:

> Another common (for me) example is to have a heading called "* Appendix"
> in a paper, and then the LaTeX "\appendix" command. The Org heading is
> just for me. It should not be exported. All headings below the \appendix
> command are Org sub-sections but should be promoted to \section in the
> final document.

then IMO it should be 

,----------------------------
| * Appendix
| ** Latex Command :noexport:
| \appendix
| ** Subsec1
| ** Subsec2 ...
`----------------------------

This is very common 

,-----------
| * Level 1
| text
| ** Level 2
| text
| ** Level 2
`-----------

but often not very good. The text under Level 1 might deserve its own
subsection, and then things become easier and more consistent.

-- 
cheers,
Thorsten

Re: proposal to have ignoreheading tags/properties

[Eric Schulte]

Mark Edgington <edgimar@gmail.com> writes:

> In using org-mode, there is one problem that has always irked me (and
> is apparently also closely related to the FAQ "How do I ignore a
> headline?").  When I am writing something, I sometimes want to group
> things by concept or by work to be done, or any other number of
> groupings.  BUT I do not want these groupings to be part of the
> exported document itself.  In fact, I would like it to be as if the
> grouping did not exist at all (i.e. a headline that is ignored).
>
> The problem with using an ignored headline for grouping things is that
> it still does have an effect on exported document structure, in that
> all of the elements contained "inside / in the scope of" the ignored
> headline still keep their depth (one level deeper than the level of
> the ignored headline).
>
> What I want is for the nested items to have their levels all promoted
> by one, so that it's truly as if the ignored headline wasn't there at
> all, and that it invisibly wrapped around a group of items without
> requiring them to have a deeper level.  Perhaps this could be done by
> use of an "ignoreheading" and an "ignoreheadingpromote" tag (one
> promotes the level of contained items, another doesn't), or some
> equivalent set of properties that could be set on a headline.
>
> Would there be any chance that something like this could be built in
> to org-mode?  I think it would make it far more flexible in terms of
> organizing things, making this organization process orthogonal to the
> selection of sections/subsections of a document.
>

I just ran across this need myself, and updated an old solution to work
with the new exporter.  See the thread and my solution at the following.

http://lists.gnu.org/archive/html/emacs-orgmode/2014-06/msg00238.html

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D (see https://u.fsf.org/yw)

Re: proposal to have ignoreheading tags/properties

[Ken Mankoff]

On 2014-06-12 at 14:11, Thorsten Jolitz wrote:
> Ken Mankoff <mankoff@gmail.com> writes:
>
>> Another common (for me) example is to have a heading called "*
>> Appendix" in a paper, and then the LaTeX "\appendix" command. The Org
>> heading is just for me. It should not be exported. All headings below
>> the \appendix command are Org sub-sections but should be promoted to
>> \section in the final document.
>
> then IMO it should be 
>
> ,----------------------------
> | * Appendix
> | ** Latex Command :noexport:
> | \appendix
> | ** Subsec1
> | ** Subsec2 ...
> `----------------------------

I'm confused how this creates a correct LaTeX document. The "\appendix"
command is not exported, since that section was tagged ":noexport:", in
which case, what is the point of having "\appendix"?

I *need* the LaTeX command exported. It changes things in the remainder
of the LaTeX document. A section titled "Appendix", which is what your
text above creates, does not create a correct LaTeX
document. Furthermore, Subsec1 and Subsec2 need to be promoted, either
in Org or in the export.

,----------------------------
| * Appendix :noexportheading:promotesubheading:
| \appendix
| ** Subsec1
| ** Subsec2
`----------------------------

The above works, assuming that "noexportheading" doesn't export the
heading, but does export the content ("\appendix"), and the
"promotesubheading" does what you would expect.

  -k.

Re: proposal to have ignoreheading tags/properties

[Aaron Ecay]

Hi Eric, hi all,

2014ko ekainak 12an, Eric Schulte-ek idatzi zuen:
> 
> I just ran across this need myself, and updated an old solution to work
> with the new exporter.  See the thread and my solution at the following.
> 
> http://lists.gnu.org/archive/html/emacs-orgmode/2014-06/msg00238.html

It’s really wonderful that you and the other members of the org
community have devoted lots of careful effort to this problem.  See
also:
- http://orgmode.org/worg/org-hacks.html#ignoreheadline
- http://mid.gmane.org/87lhvvjoo0.fsf@gmx.us
- http://mid.gmane.org/87zjii42k9.fsf@ucl.ac.uk

Many of the solutions work approximately, but not in all circumstances
– see the discussion on worg.  Searching my email archives for the
string “ignoreheading” in 2014, I see threads from May 14, May 9, May
6, March 27, March 17, and Jan 7 on this topic. It seems reasonable to
ask whether this constitutes enough user interest to make this a core
feature.  The benefits would include:
- the code can be made robust by the combined effort of those
  interested
- the feature can be incorporated into the test suite, so it won’t
  accidentally break in the future
- it will be more “discoverable” in the org info manual (which almost
  certainly gets a larger audience than worg)

What do you think?

-- 
Aaron Ecay

Re: proposal to have ignoreheading tags/properties

[Nicolas Girard]

2014-06-12 20:54 GMT+02:00 Aaron Ecay <aaronecay@gmail.com>:
> Hi Eric, hi all,
>
> 2014ko ekainak 12an, Eric Schulte-ek idatzi zuen:
>>
>> I just ran across this need myself, and updated an old solution to work
>> with the new exporter.  See the thread and my solution at the following.
>>
>> http://lists.gnu.org/archive/html/emacs-orgmode/2014-06/msg00238.html
>
> It’s really wonderful that you and the other members of the org
> community have devoted lots of careful effort to this problem.  See
> also:
> - http://orgmode.org/worg/org-hacks.html#ignoreheadline
> - http://mid.gmane.org/87lhvvjoo0.fsf@gmx.us
> - http://mid.gmane.org/87zjii42k9.fsf@ucl.ac.uk
>
> Many of the solutions work approximately, but not in all circumstances
> – see the discussion on worg.  Searching my email archives for the
> string “ignoreheading” in 2014, I see threads from May 14, May 9, May
> 6, March 27, March 17, and Jan 7 on this topic. It seems reasonable to
> ask whether this constitutes enough user interest to make this a core
> feature.  The benefits would include:
> - the code can be made robust by the combined effort of those
>   interested
> - the feature can be incorporated into the test suite, so it won’t
>   accidentally break in the future
> - it will be more “discoverable” in the org info manual (which almost
>   certainly gets a larger audience than worg)
>


I also ran across this need. What I had in mind was that certain todo
types would be treated as inline.
For instance, if "P" was a member of this hypothetical
org-inline-todo-keywords, then

* P a paragraph
  some contents

would be rendered as

some contents

by the exporter, no matter the backend.

Such a feature is more generic and would be useful in other contexts ;
and the LaTeX-related issues discussed in this thread would be solved
using something like

* INLINE appendix
  \appendix
* Appendix 1


Cheers,

Nicolas

Re: proposal to have ignoreheading tags/properties

[Ken Mankoff]

On 2014-06-12 at 15:21, Nicolas Girard wrote:
> I also ran across this need. What I had in mind was that certain todo
> types would be treated as inline.  
> ...
> Such a feature is more generic and would be useful in other contexts ;
> and the LaTeX-related issues discussed in this thread would be solved
> using something like

> * INLINE appendix
>   \appendix
> * Appendix 1

In your example, I cannot collapse "* Appendix 1 "under "INLINE
appendix". I need to add a "*" to it, and it is now at the incorrect
level when exporting. There is no point in having the "INLINE appendix"
here.

I'm not sure how this feature using TODO keywords is "more generic"
(also, more generic than what?). TODO types have a very specific
functionality, and I think one can only have 1 TODO type. So if you
wanted to have

* INLINE TODO foo
or
* WAITING INLINE bar

It would not work.

Implementing this feature as tags allows one to use existing TODO
items. Tags are already a more generic item so adding a new behavior
isn't that dramatic (and is it new? :noexport: already exists), and
multiple tags are supported. It seems a better place for the
implementation.

I vote for the following tags:

+ :noexport: Does not export item, content, and children.
+ :ignoreheading: Does not export heading. Exports content and children.
+ :ignorecontent: Does not export heading or content. Does export children.
+ :ignorebranch: Does not export heading, content, or children.
+ :promotesubheadings: Promotes children headings, regardless of
  exporting this heading or not

Note that :ignorebranch: is the same as ":noexport" but is a more
consistent naming scheme. Ignoring and promotion are two separate items
and can be used together or exclusively for maximum number of export
behaviors. 

In this case, a modified version of your example might look like this:

* Appendix      :ignoreheading:promotesubheadings:
  \appendix
** Appendix 1
** Appendix 2

  -k.

Re: proposal to have ignoreheading tags/properties

[Nicolas Girard]

2014-06-12 21:21 GMT+02:00 Nicolas Girard <girard.nicolas@gmail.com>:
> 2014-06-12 20:54 GMT+02:00 Aaron Ecay <aaronecay@gmail.com>:
>> Hi Eric, hi all,
>
>
> I also ran across this need. What I had in mind was that certain todo
> types would be treated as inline.
> For instance, if "P" was a member of this hypothetical
> org-inline-todo-keywords, then
>
> * P a paragraph
>   some contents
>
> would be rendered as
>
> some contents
>
> by the exporter, no matter the backend.
>
> Such a feature is more generic and would be useful in other contexts ;
> and the LaTeX-related issues discussed in this thread would be solved
> using something like
>
> * INLINE appendix
>   \appendix
> * Appendix 1
>
>


Or, even better: rendering *both* the headline *and* the body of
lnline nodes as inline contents too:

* INLINE My inline
contents

--> My inline contents

* INLINE \appendix

--> \appendix

Re: proposal to have ignoreheading tags/properties

[Nicolas Girard]

2014-06-12 21:26 GMT+02:00 Ken Mankoff <mankoff@gmail.com>:
>
> On 2014-06-12 at 15:21, Nicolas Girard wrote:

>
> I vote for the following tags:
>
> + :noexport: Does not export item, content, and children.
> + :ignoreheading: Does not export heading. Exports content and children.
> + :ignorecontent: Does not export heading or content. Does export children.
> + :ignorebranch: Does not export heading, content, or children.
> + :promotesubheadings: Promotes children headings, regardless of
>   exporting this heading or not
>


Agreed. That would rock !

Re: proposal to have ignoreheading tags/properties

[Eric Schulte]

[-- Attachment #1: Type: text/plain, Size: 757 bytes --]

>
> I also ran across this need. What I had in mind was that certain todo
> types would be treated as inline.
> For instance, if "P" was a member of this hypothetical
> org-inline-todo-keywords, then
>
> * P a paragraph
>   some contents
>
> would be rendered as
>
> some contents
>
> by the exporter, no matter the backend.
>
> Such a feature is more generic and would be useful in other contexts ;
> and the LaTeX-related issues discussed in this thread would be solved
> using something like
>
> * INLINE appendix
>   \appendix
> * Appendix 1
>

Why TODO types rather than a tag?  IMO using a TODO type would conflate
task management and document structuring.  What do you think about the
attached patch which should add this functionality to the core.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-export-removes-INLINE-heading-and-promotes-subtree.patch --]
[-- Type: text/x-diff, Size: 2596 bytes --]

From 5a41eae2af24097ec9c1507926af6f6fab8f2628 Mon Sep 17 00:00:00 2001
From: Eric Schulte <schulte.eric@gmail.com>
Date: Thu, 12 Jun 2014 16:11:04 -0400
Subject: [PATCH] export removes INLINE heading and promotes subtree

* lisp/ox.el (org-export-remove-and-promote-children-of-inline-headlines):
  A new function.
  (org-export-as): Include
  `org-export-remove-and-promote-children-of-inline-headlines' in the
  export process.
---
 lisp/ox.el | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/lisp/ox.el b/lisp/ox.el
index 4bfef52..961d795 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -2320,6 +2320,29 @@ tree is modified by side effect and returned by the function."
 		(plist-get info prop)
 		info))))
 
+(defun org-export-remove-and-promote-children-of-inline-headlines (data info)
+  "Remove inline headlines and promote their children.
+DATA is the parse tree.  INFO is a plist containing export
+options.  Each headline tagged as INLINE will be removed
+retaining its contents, and promoting any children headlines by a
+single level."
+  (org-element-map data org-element-all-elements
+    (lambda (object)
+      (when (and (equal 'headline (org-element-type object))
+                 (or (member "inline" (org-element-property :tags object))
+		     (member "INLINE" (org-element-property :tags object))))
+        (mapc (lambda (el)
+                ;; recursively promote all nested headlines
+                (org-element-map el 'headline
+                  (lambda (el)
+                    (when (equal 'headline (org-element-type el))
+                      (org-element-put-property el
+                        :level (1- (org-element-property :level el))))))
+                (org-element-insert-before el object))
+              (cddr object))
+        (org-element-extract-element object)))
+    info nil org-element-all-elements))
+
 (defun org-export--remove-uninterpreted-data-1 (data info)
   "Change uninterpreted elements back into Org syntax.
 DATA is a parse tree or a secondary string.  INFO is a plist
@@ -3124,6 +3147,9 @@ Return code as a string."
 	 ;; Handle left-over uninterpreted elements or objects in
 	 ;; parse tree and communication channel.
 	 (org-export-remove-uninterpreted-data tree info)
+	 ;; Remove headlines tagged as inline and promote their
+	 ;; children.
+	 (org-export-remove-and-promote-children-of-inline-headlines tree info)
 	 ;; Call options filters and update export options.  We do not
 	 ;; use `org-export-filter-apply-functions' here since the
 	 ;; arity of such filters is different.
-- 
2.0.0


[-- Attachment #3: Type: text/plain, Size: 117 bytes --]


>
>
> Cheers,
>
> Nicolas

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D (see https://u.fsf.org/yw)

Re: proposal to have ignoreheading tags/properties

[Nicolas Girard]

2014-06-12 22:13 GMT+02:00 Eric Schulte <schulte.eric@gmail.com>:
>>
>
> Why TODO types rather than a tag?  IMO using a TODO type would conflate
> task management and document structuring.

Agreed.

> What do you think about the
> attached patch which should add this functionality to the core.
>

Seems fine to me !

I created a public repository containing a Makefile and a bunch of
unit tests here: https://github.com/ngirard/org-inline-headings-tests.


Cheers,

Nicolas

Re: proposal to have ignoreheading tags/properties

[Eric Schulte]

Nicolas Girard <girard.nicolas@gmail.com> writes:

> 2014-06-12 22:13 GMT+02:00 Eric Schulte <schulte.eric@gmail.com>:
>>>
>>
>> Why TODO types rather than a tag?  IMO using a TODO type would conflate
>> task management and document structuring.
>
> Agreed.
>
>> What do you think about the
>> attached patch which should add this functionality to the core.
>>
>
> Seems fine to me !
>

Great, I'll leave actual application and possible adjustment to Nicolas
Goaziou as he knows best how the exporter should grow.

>
> I created a public repository containing a Makefile and a bunch of
> unit tests here: https://github.com/ngirard/org-inline-headings-tests.
>

This looks great, perhaps you could fold the unit tests into the testing
directory Org-mode repository and then share them as a patch?

Best,
Eric

>
>
> Cheers,
>
> Nicolas

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D (see https://u.fsf.org/yw)

Re: proposal to have ignoreheading tags/properties

[Ken Mankoff]

On 2014-06-12 at 16:13, Eric Schulte wrote:
> What do you think about the attached patch which should add this
> functionality to the core.

Why "inline"? 

Org already has "inline TODO items" which is a different thing. 

I don't think the word "inline" signifies that a heading will or won't
be exported and/or its children promoted.

  -k.

Re: proposal to have ignoreheading tags/properties

[Eric Schulte]

Ken Mankoff <mankoff@gmail.com> writes:

> On 2014-06-12 at 16:13, Eric Schulte wrote:
>> What do you think about the attached patch which should add this
>> functionality to the core.
>
> Why "inline"? 
>
> Org already has "inline TODO items" which is a different thing. 
>
> I don't think the word "inline" signifies that a heading will or won't
> be exported and/or its children promoted.
>

Can you suggest a more intuitive/appropriate tag name?  I'm not
personally partial to "inline", it was just the first thing that
occurred to me.  Previous implementations of similar behavior used the
tag "prelim".

Thanks,
Eric

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D (see https://u.fsf.org/yw)

Re: proposal to have ignoreheading tags/properties

[Samuel Wales]

if i were a newcomer i'd wonder how these affected babel, agenda, etc.

maybe noexport, noexportheading, noexportcontent, exportpromotechildren?


On 6/12/14, Ken Mankoff <mankoff@gmail.com> wrote:
> + :noexport: Does not export item, content, and children.
> + :ignoreheading: Does not export heading. Exports content and children.
> + :ignorecontent: Does not export heading or content. Does export children.
> + :ignorebranch: Does not export heading, content, or children.
> + :promotesubheadings: Promotes children headings, regardless of
>   exporting this heading or not

Re: proposal to have ignoreheading tags/properties

[Ken Mankoff]

Hi Eric,  On 2014-06-12 at 20:46, Eric Schulte wrote: 
> Can you suggest a more intuitive/appropriate tag name?  I'm not 
> personally partial to "inline", it was just the first thing that 
> occurred to me.  Previous implementations of similar behavior 
> used the tag "prelim". 
 
I posted the following before. I think you might not be getting 
all the emails I post to the list. For example, I commented that 
INLINE as a TODO keyword didn't make sense to me using a similar 
explanation to your reply to that same email.   I have used your 
implementation from last week using "prelim" but changed the word, 
because as with "inline", I don't associate the word "prelim" with 
the behavior being implemented. 
 
  -k. 
 
I vote for the following tags:

+ :noexport: Does not export item, content, and children.
+ :ignoreheading: Does not export heading. Exports content and 
children.
+ :ignorecontent: Does not export heading or content. Does export 
children.
+ :ignorebranch: Does not export heading, content, or children.
+ :promotesubheadings: Promotes children headings, regardless of
  exporting this heading or not

Note that :ignorebranch: is the same as ":noexport" but is a more
consistent naming scheme. Ignoring and promotion are two separate 
items
and can be used together or exclusively for maximum number of 
export
behaviors.

Re: proposal to have ignoreheading tags/properties

[Eric Abrahamsen]

Mark Edgington <edgimar@gmail.com> writes:

> In using org-mode, there is one problem that has always irked me (and
> is apparently also closely related to the FAQ "How do I ignore a
> headline?").  When I am writing something, I sometimes want to group
> things by concept or by work to be done, or any other number of
> groupings.  BUT I do not want these groupings to be part of the
> exported document itself.  In fact, I would like it to be as if the
> grouping did not exist at all (i.e. a headline that is ignored).
>
> The problem with using an ignored headline for grouping things is that
> it still does have an effect on exported document structure, in that
> all of the elements contained "inside / in the scope of" the ignored
> headline still keep their depth (one level deeper than the level of
> the ignored headline).
>
> What I want is for the nested items to have their levels all promoted
> by one, so that it's truly as if the ignored headline wasn't there at
> all, and that it invisibly wrapped around a group of items without
> requiring them to have a deeper level.  Perhaps this could be done by
> use of an "ignoreheading" and an "ignoreheadingpromote" tag (one
> promotes the level of contained items, another doesn't), or some
> equivalent set of properties that could be set on a headline.
>
> Would there be any chance that something like this could be built in
> to org-mode?  I think it would make it far more flexible in terms of
> organizing things, making this organization process orthogonal to the
> selection of sections/subsections of a document.

It looks like a groundswell for remove-andor-promote tags for headlines,
but for the sake of argument let me propose the use of blocks. It seems
to me that something like a "generic block" (a block that does nothing
but delete its begin/end delimiters on export) fits the use-case better:

#+BEGIN_BLOCK Here's a list of reasons why blocks make sense

1. They group together content that should go together, but don't
   constitute a part of document structure. They're a little like
   notecards in Scrivener

2. You can fold them like headlines, and re-order them with
   org-meta(up|down)

3. You could put arbitrary text after the begin block declaration, as if
   it were ignored headline text.

4. I can't think of any more reasons, except that this seems like the
   right "weight" for this kind of feature.

#+END_BLOCK

Anyway, just being argumentative...

E

Re: proposal to have ignoreheading tags/properties

[Mark Edgington]

Eric Schulte <schulte.eric <at> gmail.com> writes:

> Ken Mankoff <mankoff <at> gmail.com> writes:
> >
> > I don't think the word "inline" signifies that a heading will or won't
> > be exported and/or its children promoted.
> >
> 
> Can you suggest a more intuitive/appropriate tag name?

Would it be possible / sensible to allow user-specified aliases for the
different "canonical" tags (whatever they turn out to be), just like it's
possible to specify tags other than "noexport" which will not be exported?

Re: proposal to have ignoreheading tags/properties

[Mark Edgington]

Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
> 
> It looks like a groundswell for remove-andor-promote tags for headlines,
> but for the sake of argument let me propose the use of blocks. It seems
> to me that something like a "generic block" (a block that does nothing
> but delete its begin/end delimiters on export) fits the use-case better:
> 

Consider the following:

* Test
#+begin_block abc
* test2
#+begin_block def
* test3
** test4 
#+end_block
** test5
#+end_block


Some remarks on this, and on blocks vs. headlines:

- For me, the above example ends up being indented very poorly with
org-indent-mode active.  Also folding the nested headlines swallows up the
end-block lines.

- I find that it's difficult to identify what belongs to what block, and the
need to have both start and end lines to delineate the blocks is a bit more
noisy and can be a pain to work with (what if I want to remove the "abc
frame" -- I will need not only to delete the begin line, but also to locate
where the corresponding end line is, and delete it as well).

- creating a block (manually, at least) requires more effort than creating a
headline (= more RSI).

- It also may be convenient at times to be able to remove the :promote:,
etc. tags in order to have the exporter include the grouping as part of the
exported document's structure.

- Likewise, what if I want to add a :noexport: tag so that all of the
content is ignored -- easy with headlines, more work to do the same thing
with a block.

Re: proposal to have ignoreheading tags/properties

[Eric Abrahamsen]

Mark Edgington <edgimar@gmail.com> writes:

> Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
>> 
>> It looks like a groundswell for remove-andor-promote tags for headlines,
>> but for the sake of argument let me propose the use of blocks. It seems
>> to me that something like a "generic block" (a block that does nothing
>> but delete its begin/end delimiters on export) fits the use-case better:
>> 
>
> Consider the following:
>
> * Test
>
> #+begin_block abc
> * test2
> #+begin_block def
> * test3
> ** test4 
> #+end_block
>
> ** test5
> #+end_block

FWIW, I think the above example is actually illegal Org syntax -- it's
going to result in breakage in many ways, not just this case.

> Some remarks on this, and on blocks vs. headlines:
>
> - For me, the above example ends up being indented very poorly with
> org-indent-mode active.  Also folding the nested headlines swallows up the
> end-block lines.
>
> - I find that it's difficult to identify what belongs to what block, and the
> need to have both start and end lines to delineate the blocks is a bit more
> noisy and can be a pain to work with (what if I want to remove the "abc
> frame" -- I will need not only to delete the begin line, but also to locate
> where the corresponding end line is, and delete it as well).
>
> - creating a block (manually, at least) requires more effort than creating a
> headline (= more RSI).
>
> - It also may be convenient at times to be able to remove the :promote:,
> etc. tags in order to have the exporter include the grouping as part of the
> exported document's structure.
>
> - Likewise, what if I want to add a :noexport: tag so that all of the
> content is ignored -- easy with headlines, more work to do the same thing
> with a block.

Yup, I don't really have any argument against these above points -- I
think it's mostly a matter of use-case.

The inconvenience of the block approach could easily be remedied with a
few extra functions: something like org-wrap-region-in-block and
org-delete-enclosing-block would probably do it.

But your second two objections are just a matter of a difference in
use-case: you want something more flexible and powerful. My suggestion
is just a lighter-weight alternate if (it's a possibility) the tag-based
approach gets nixed...

E

Re: proposal to have ignoreheading tags/properties

[Eric Schulte]

Ken Mankoff <mankoff@gmail.com> writes:

> Hi Eric,  On 2014-06-12 at 20:46, Eric Schulte wrote: 
>> Can you suggest a more intuitive/appropriate tag name?  I'm not
>> personally partial to "inline", it was just the first thing that
>> occurred to me.  Previous implementations of similar behavior used
>> the tag "prelim". 
>
> I posted the following before. I think you might not be getting all
> the emails I post to the list. For example, I commented that INLINE as
> a TODO keyword didn't make sense to me using a similar explanation to
> your reply to that same email.   I have used your implementation from
> last week using "prelim" but changed the word, because as with
> "inline", I don't associate the word "prelim" with the behavior being
> implemented. 
>

Sounds like I'm missing something.

>
>  -k.
>
> I vote for the following tags:
>
> + :noexport: Does not export item, content, and children.
> + :ignoreheading: Does not export heading. Exports content and
> children.
> + :ignorecontent: Does not export heading or content. Does export
> children.
> + :ignorebranch: Does not export heading, content, or children.
> + :promotesubheadings: Promotes children headings, regardless of
>  exporting this heading or not
>
> Note that :ignorebranch: is the same as ":noexport" but is a more
> consistent naming scheme. Ignoring and promotion are two separate
> items
> and can be used together or exclusively for maximum number of export
> behaviors.

This feels excessive to me.  "ignorecontent" could more easily be
implemented with a leading ignored heading, and the common use case of
promoting a heading's children while ignoring the heading itself now
requires two tags with long names.

Best,
Eric

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D (see https://u.fsf.org/yw)

Re: proposal to have ignoreheading tags/properties

[Rasmus]

Eric Schulte <schulte.eric@gmail.com> writes:

>>
>> I also ran across this need. What I had in mind was that certain todo
>
>> types would be treated as inline.
>> For instance, if "P" was a member of this hypothetical
>> org-inline-todo-keywords, then
>>
>> * P a paragraph
>>   some contents
>>
>> would be rendered as
>>
>> some contents
>>
>> by the exporter, no matter the backend.
>>
>> Such a feature is more generic and would be useful in other contexts ;
>> and the LaTeX-related issues discussed in this thread would be solved
>> using something like
>>
>> * INLINE appendix
>>   \appendix
>> * Appendix 1
>>
>
> Why TODO types rather than a tag?  IMO using a TODO type would conflate
> task management and document structuring.  What do you think about the
> attached patch which should add this functionality to the core.

I would be happy to see this change in core, as I use
ignoreheading-like tags all of the time.  On the other hand it's not
particularly elegant, but it's already in ox-beamer.el, so that's
probably OK.

Thanks,
Rasmus

-- 
Er du tosset for noge' lårt!

Re: proposal to have ignoreheading tags/properties

[Rasmus]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> Ken Mankoff <mankoff@gmail.com> writes:
>
>> Another common (for me) example is to have a heading called "* Appendix"
>> in a paper, and then the LaTeX "\appendix" command. The Org heading is
>> just for me. It should not be exported. All headings below the \appendix
>> command are Org sub-sections but should be promoted to \section in the
>> final document.
>
> then IMO it should be 
>
> ,----------------------------
> | * Appendix
> | ** Latex Command :noexport:
> | \appendix
> | ** Subsec1
> | ** Subsec2 ...
> `----------------------------

Your code example is broken, but let's not care about that for now.

So the appendix case is kind of special.  Really what we want is to
inject a command before the insertion of the appendix.  Due to the
nature of LaTeX this is easy.  I often do something like

* appendix pre :ignoreheading:
#+LATEX: \appendix
* my appendix
  content

However, perhaps a property specifying a command to be inserted before
the headline is more appropriate.  Then, we could write the above as

* my appendix 
  :PROPERTIES:
  :LATEX_PRE_HEADING: \appendix
  :HTML_PRE_HEADING: <appendix>
  :HTML_POST_HEADING: </appendix>
  :END:
  content

which would export

   \appendix
   \section{my appendix}
   content
   
Of course, in the above example it would be better to set the property
HTML_CONTAINER to appendix for html export.

—Rasmus
 
-- 
Together we'll stand, divided we'll fall

Re: proposal to have ignoreheading tags/properties

[Thorsten Jolitz]

Rasmus <rasmus@gmx.us> writes:

> Thorsten Jolitz <tjolitz@gmail.com> writes:
>
>> Ken Mankoff <mankoff@gmail.com> writes:
>>
>>> Another common (for me) example is to have a heading called "* Appendix"
>>> in a paper, and then the LaTeX "\appendix" command. The Org heading is
>>> just for me. It should not be exported. All headings below the \appendix
>>> command are Org sub-sections but should be promoted to \section in the
>>> final document.
>>
>> then IMO it should be 
>>
>> ,----------------------------
>> | * Appendix
>> | ** Latex Command :noexport:
>> | \appendix
>> | ** Subsec1
>> | ** Subsec2 ...
>> `----------------------------
>
> Your code example is broken, but let's not care about that for now.

Yes, I (unfortunately) did not care about the LaTeX side of this special
case, just wanted to say that this

,-------------
| * 1st Level
| text
| ** 2nd Level
| text
| ** 2nd Level
| text
`-------------

often causes headaches that can easily avoided, e.g by 

,-------------
| * 1st Level
| ** 2nd Level
| text
| ** 2nd Level
| text
| ** 2nd Level
| text
`-------------

or (your solution)

,-------------
| * 1st Level
| text
| * 1st Level
| ** 2nd Level
| text
| ** 2nd Level
| text
`-------------

> So the appendix case is kind of special.  Really what we want is to
> inject a command before the insertion of the appendix.  Due to the
> nature of LaTeX this is easy.  I often do something like
>
> * appendix pre :ignoreheading:
> #+LATEX: \appendix
> * my appendix
>   content

-- 
cheers,
Thorsten

Re: proposal to have ignoreheading tags/properties

[Nicolas Goaziou]

Hello,

Eric Schulte <schulte.eric@gmail.com> writes:

> Why TODO types rather than a tag?  IMO using a TODO type would conflate
> task management and document structuring.  What do you think about the
> attached patch which should add this functionality to the core.

Thank you. Unfortunately, in many cases this code will make the parse
tree invalid. Consider the example below:

  * H1
    Text1
  ** H2 :inline:
    Text2

A simplified version of the parse tree is:

  (headline
   (section
    (paragraph "Text1"))
   (headline
    (section
     (paragraph "Text2"))))

With your function, it becomes

  (headline
   (section
    (paragraph "Text1"))
   (section
    (paragraph "Text2")))

which is invalid, as a given headline is not expected to have more than
one section.

Of course, it is possible to add code in order to merge both sections
and get

  (headline
   (section
    (paragraph "Text1")
    (paragraph "Text2")))

which is exactly what you obtain in the first answer of the FAQ, along
with its limitations (see the :noexport: example in the same question).

Actually, the problem is deeper than that. This :inline: tag is just
a convoluted way to ask for a positive answer to another FAQ: « Can
I close an outline section without starting a new section? »
(http://orgmode.org/worg/org-faq.html#closing-outline-sections). Indeed,
allowing :include: tags is equivalent to allowing to close sections
before the next one, at least at the export level:

  * Section one

  Some text

  ** Subsection one

  Some text

  ** Subsection two

  Some text

  ** end Subsection Two                                                     :inline:

  Continue text in section one.

This is not possible and goes against so many assumptions in Org that it
will always introduce problems, as your function does.

Since it cannot work in the general case, I do not think it should go in
core. Fortunately, a simple function in `org-export-before-parsing-hook'
provides a decent solution already. Users requiring more sophistication
can always implement their own function and add it to that hook.

OTOH, the situation could be improved wrt :export: and :noexport: tags.
We could allow nesting :export: tags within :noexport: tags with the
following rule: the :export: headline with the lowest level within
the :noexport: tree gets promoted to the root of the tree.
Other :export: headlines have their level set relatively to this one.
Thus:

  Text before first headline
  * H1
  Body1
  ** H2 :noexport:
  Body2
  *** H3
  Body3
  *** H4 :export:
  Body4
  **** H5
  Body5

will be seen as

  Text before first headline
  * H1
  Body1
  ** H4
  Body4
  *** H5
  Body5

This is not inlining, since "Body2" is lost anyway, and it may require
to tag all children of a given headline, but it's already better than
the current situation, is robust enough for inclusion in core, and
doesn't require introducing a plethora of new special tags.

The reverse situation (with swapped :noexport: and :export: tags) is
already handled by the exporter.

Also, I recall that it is possible to exclude from export some parts of
the document with drawers and an appropriate value for
`org-export-with-drawers'. The only limitation is that you cannot have
a headline or another drawer within a drawer.


Regards,

-- 
Nicolas Goaziou

Re: proposal to have ignoreheading tags/properties

[Mark Edgington]

Nicolas Goaziou <mail <at> nicolasgoaziou.fr> writes:

> Actually, the problem is deeper than that. This :inline: tag is just
> a convoluted way to ask for a positive answer to another FAQ: « Can
> I close an outline section without starting a new section? »
> (http://orgmode.org/worg/org-faq.html#closing-outline-sections). Indeed,
> allowing :include: tags is equivalent to allowing to close sections
> before the next one, at least at the export level:
> 
>   * Section one
> 
>   Some text
> 
>   ** Subsection one
> 
>   Some text
> 
>   ** Subsection two
> 
>   Some text
> 
>   ** end Subsection Two                                                  
  :inline:
> 
>   Continue text in section one.
>

If I understand your example correctly, it seems like you are assuming that
the :inline: tag should promote a section's contents to the level *above*
the level of the section having the :inline: tag.  To me this behavior
doesn't make sense, and that's also not what I would expect such a tag to do
-- instead, the section's text (anything which comes before the next
headline at any level) should be merged with the text of the nearest
preceding headline.  Then all nested headlines contained in the :inline:
section should be promoted.

It is true that this could sometimes be confusing.  For example:

  * A
  text1
  ** B
  text2
  * C  :inline:
  text3
  ** D
  text 4

would get treated like:

  * A
  text1
  ** B
  text2
  text3
  * D
  text 4

In this case, one would likely omit 'text3' from the first part of the
example, since it doesn't make much sense to have it there.  For the most
part, though, it would be a behavior that makes sense (e.g. if "* C" were
replaced with "** C" in the example).

It may be that "inline" isn't the best word to describe this behavior, which
is why something with "ignore" or "promotechildren" has been mentioned.

Re: proposal to have ignoreheading tags/properties

[Aaron Ecay]

Hi Nicolas,

Thanks for your thoughts.

2014ko ekainak 14an, Nicolas Goaziou-ek idatzi zuen:
> OTOH, the situation could be improved wrt :export: and :noexport: tags.
> We could allow nesting :export: tags within :noexport: tags with the
> following rule: the :export: headline with the lowest level within
> the :noexport: tree gets promoted to the root of the tree.
> Other :export: headlines have their level set relatively to this one.
> Thus:
> 
>   Text before first headline
>   * H1
>   Body1
>   ** H2 :noexport:
>   Body2
>   *** H3
>   Body3
>   *** H4 :export:
>   Body4
>   **** H5
>   Body5
> 
> will be seen as
> 
>   Text before first headline
>   * H1
>   Body1
>   ** H4
>   Body4
>   *** H5
>   Body5

I’m confused.  In the text, you say “promoted to the root level of the
tree”, which I expect to mean promotion to a top-level headline.  In the
example, though, H4 is promoted to second-level.  Do you mean “promoted
to the level of the highest dominating :noexport: headline”?  That seems
correct to me (but I have not thought about it extensively).

Regardless, I like it very much.

> 
> This is not inlining, since "Body2" is lost anyway, and it may require
> to tag all children of a given headline, but it's already better than
> the current situation, is robust enough for inclusion in core, and
> doesn't require introducing a plethora of new special tags.

I think under your proposal it would be possible to add a single special
tag which is equivalent (by definition) to tagging a headline noexport
and all its children export.  This could be implemented as a parse tree
transformation adding the (no)export tags at an early stage in the export
code.

I agree with Mark and others that “ignore” (or some variant like
“exportignore” or “ignoreheading”) are better names for this tag than
“inline”.  My vote, FWIW, is that the level of sustained user interest
justifies the inclusion of this new tag.  But I hope it will not be too
burdensome, since it just serves as a sort of syntactic sugar and
doesn’t require any substantial new semantics (beyond your proposal for
the extension of (no)export).  To borrow your example, this would look
like:

  Text before first headline
  * H1
  Body1
  ** H2 :ignore:
  Body2
  *** H3
  Body3
  *** H4
  Body4
  **** H5
  Body5

exports to:

  Text before first headline
  * H1
  Body1
  ** H3
  Body3
  ** H4
  Body4
  *** H5
  Body5

(Note lack of “Text2”)

-- 
Aaron Ecay

Re: proposal to have ignoreheading tags/properties

[Aaron Ecay]

Hi Mark,

2014ko ekainak 14an, Mark Edgington-ek idatzi zuen:

[...]


> It is true that this could sometimes be confusing.  For example:
> 
>   * A
>   text1
>   ** B
>   text2
>   * C  :inline:
>   text3
>   ** D
>   text 4
> 
> would get treated like:
> 
>   * A
>   text1
>   ** B
>   text2
>   text3
>   * D
>   text 4

The problem with this example which Nicolas alludes to arises if B is
tagged noexport.  Then text3 will also be not exported, even though it
seems like it should be.

-- 
Aaron Ecay

Re: proposal to have ignoreheading tags/properties

[Nicolas Goaziou]

Hello,

Mark Edgington <edgimar@gmail.com> writes:

> If I understand your example correctly, it seems like you are assuming that
> the :inline: tag should promote a section's contents to the level *above*
> the level of the section having the :inline: tag.

I'm always assuming the worst.

> To me this behavior doesn't make sense, and that's also not what
> I would expect such a tag to do -- instead, the section's text
> (anything which comes before the next headline at any level) should be
> merged with the text of the nearest preceding headline. Then all
> nested headlines contained in the :inline: section should be promoted.
>
> It is true that this could sometimes be confusing.  For example:
>
>   * A
>   text1
>   ** B
>   text2
>   * C  :inline:
>   text3
>   ** D
>   text 4
>
> would get treated like:
>
>   * A
>   text1
>   ** B
>   text2
>   text3
>   * D
>   text 4
>
> In this case, one would likely omit 'text3' from the first part of the
> example, since it doesn't make much sense to have it there.  For the most
> part, though, it would be a behavior that makes sense (e.g. if "* C" were
> replaced with "** C" in the example).

As I said, you cannot solve that confusing situation in the general
case. If we allow the confusing part (i.e "text3") altogether, then my
suggestion still holds:

  - if you don't need to nest headlines, use a drawer.
  - if you do need sub-headings, extending :export: and :noexport: is
    sufficient.

Example:

  * A
  text1
  ** B
  text2
  * C :noexport:
  text3
  ** D :export:
  text

If you really need the exact behaviour that you describe, I suggest to
extend the first answer to the relative FAQ so that it also promotes
sub-headings besides removing the headline.

> It may be that "inline" isn't the best word to describe this behavior, which
> is why something with "ignore" or "promotechildren" has been
> mentioned.

The fact that it's difficult to find a good descriptive name for that
feature is a good indication that it isn't meant for general
consumption. ;)


Regards,

-- 
Nicolas Goaziou

Re: proposal to have ignoreheading tags/properties

[Nicolas Goaziou]

Hello,

Aaron Ecay <aaronecay@gmail.com> writes:

> I’m confused.  In the text, you say “promoted to the root level of the
> tree”, which I expect to mean promotion to a top-level headline.  In the
> example, though, H4 is promoted to second-level.  Do you mean “promoted
> to the level of the highest dominating :noexport: headline”?  That seems
> correct to me (but I have not thought about it extensively).

Exactly. I meant "promoted to the root of the :noexport: tree", which is
basically what you describe.

> I think under your proposal it would be possible to add a single special
> tag which is equivalent (by definition) to tagging a headline noexport
> and all its children export.  This could be implemented as a parse tree
> transformation adding the (no)export tags at an early stage in the export
> code.

I'd rather not introduce more special tags, if possible.

I think that this feature should be implemented with :export:/:noexport:
before we start pondering about the relevance of some syntactic sugar.

There are other ways to handle "same tag on all children", e.g., an
interactive function.


Regards,

-- 
Nicolas Goaziou

Re: proposal to have ignoreheading tags/properties

[Aaron Ecay]

Hi Nicolas,

2014ko ekainak 14an, Nicolas Goaziou-ek idatzi zuen:
> 
> Hello,
> 
> Aaron Ecay <aaronecay@gmail.com> writes:
> 
>> I’m confused.  In the text, you say “promoted to the root level of the
>> tree”, which I expect to mean promotion to a top-level headline.  In the
>> example, though, H4 is promoted to second-level.  Do you mean “promoted
>> to the level of the highest dominating :noexport: headline”?  That seems
>> correct to me (but I have not thought about it extensively).
> 
> Exactly. I meant "promoted to the root of the :noexport: tree", which is
> basically what you describe.
> 
>> I think under your proposal it would be possible to add a single special
>> tag which is equivalent (by definition) to tagging a headline noexport
>> and all its children export.  This could be implemented as a parse tree
>> transformation adding the (no)export tags at an early stage in the export
>> code.
> 
> I'd rather not introduce more special tags, if possible.
> 
> I think that this feature should be implemented with :export:/:noexport:
> before we start pondering about the relevance of some syntactic sugar.

I understand your point about gradualism.

> 
> There are other ways to handle "same tag on all children", e.g., an
> interactive function.

I think a parse tree filter might work better, since then there is no
possibility of the tags inadvertently getting out of sync.  But either
way, it will be easy to implement based on the extension to (no)export
functionality you proposed.

Thanks,

-- 
Aaron Ecay

Re: proposal to have ignoreheading tags/properties

[Eric Schulte]

[-- Attachment #1: Type: text/plain, Size: 5789 bytes --]

Hi,

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Hello,
>
> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> Why TODO types rather than a tag?  IMO using a TODO type would conflate
>> task management and document structuring.  What do you think about the
>> attached patch which should add this functionality to the core.
>
> Thank you. Unfortunately, in many cases this code will make the parse
> tree invalid. Consider the example below:
>
>   * H1
>     Text1
>   ** H2 :inline:
>     Text2
>
> A simplified version of the parse tree is:
>
>   (headline
>    (section
>     (paragraph "Text1"))
>    (headline
>     (section
>      (paragraph "Text2"))))
>
> With your function, it becomes
>
>   (headline
>    (section
>     (paragraph "Text1"))
>    (section
>     (paragraph "Text2")))
>
> which is invalid, as a given headline is not expected to have more than
> one section.
>
> Of course, it is possible to add code in order to merge both sections
> and get
>
>   (headline
>    (section
>     (paragraph "Text1")
>     (paragraph "Text2")))
>
> which is exactly what you obtain in the first answer of the FAQ, along
> with its limitations (see the :noexport: example in the same question).
>
> Actually, the problem is deeper than that. This :inline: tag is just
> a convoluted way to ask for a positive answer to another FAQ: « Can
> I close an outline section without starting a new section? »
> (http://orgmode.org/worg/org-faq.html#closing-outline-sections). Indeed,
> allowing :include: tags is equivalent to allowing to close sections
> before the next one, at least at the export level:
>
>   * Section one
>
>   Some text
>
>   ** Subsection one
>
>   Some text
>
>   ** Subsection two
>
>   Some text
>
>   ** end Subsection Two                                                     :inline:
>
>   Continue text in section one.
>
> This is not possible and goes against so many assumptions in Org that it
> will always introduce problems, as your function does.
>

Thanks Nicolas.  Point clearly stated.  All of the structural issues
hinge upon the inclusion of the contents of an inlined (ignored)
headline, so the solution is to change the behavior s.t. the non-subtree
contents of ignored headlines are also removed from export.  This is
implemented below.

>
> Since it cannot work in the general case, I do not think it should go in
> core. Fortunately, a simple function in `org-export-before-parsing-hook'
> provides a decent solution already. Users requiring more sophistication
> can always implement their own function and add it to that hook.
>
> OTOH, the situation could be improved wrt :export: and :noexport: tags.
> We could allow nesting :export: tags within :noexport: tags with the
> following rule: the :export: headline with the lowest level within
> the :noexport: tree gets promoted to the root of the tree.
> Other :export: headlines have their level set relatively to this one.
> Thus:
>
>   Text before first headline
>   * H1
>   Body1
>   ** H2 :noexport:
>   Body2
>   *** H3
>   Body3
>   *** H4 :export:
>   Body4
>   **** H5
>   Body5
>
> will be seen as
>
>   Text before first headline
>   * H1
>   Body1
>   ** H4
>   Body4
>   *** H5
>   Body5
>

In my opinion the manual interleaving of "noexport" and "export" tags is
overly cumbersome and is non-obvious.  The obscure nature of this
solution is evidenced by multiple discussions and implementations of
filter functions to handle situations which could be covered by this
noexport/export pattern.

I think the attached patch should be applied to the core.  It includes
the following.

- a single new export function which removes the headlines and contents
  (the "section" element) of headlines tagged "ignoreexport", then
  retains and promotes all sub-headlines contained therein.

- a single new export tag named "ignoreexport" (ignore seems to be the
  crowd favorite, and adding "export" reduces possible conflicts with
  existing personal tags)

- some tests protecting this new functionality

- I'd like to add documentation of this tag to the manual to improve
  visibility beyond the existing FAQ entries (which have failed to
  prevent multiple on list discussions and re-implementations of this
  feature).  It is not clear where to document such a tag.  The "export"
  and "noexport" tags are only documented as a side effect of
  documentation for the SELECT_TAGS keyword.  I think it would be
  beneficial to add a "selective export" section or paragraph (which
  existed in the old manual) to the new manual.

From the included tests, the effect of this patch is to convert a tree
like the following,

,----
| * H1
| Text1
| ** H2                                                          :ignoreexport:
| Text2
| *** H3
| Text3
| **** H4
| Text4
| *** H5
| Text5
| **** H6                                                        :ignoreexport:
| Text6
| ***** H7
| Text7
| ***** H8
| Text8
`----

on export to a tree like the following.

,----
| 1 H1
| ====
| 
|   Text1
| 
| 1.1 H3
| ~~~~~~
| 
|   Text3
| 
| 1.1.1 H4
| --------
| 
|   Text4
| 
| 1.2 H5
| ~~~~~~
| 
|   Text5
| 
| 1.2.1 H7
| --------
| 
|   Text7
| 
| 1.2.2 H8
| --------
| 
|   Text8
`----

I'm sympathetic to arguments about maintaining simplicity of the core,
and even more so to arguments about maintaining structural validity of
trees during export.  I believe that this revised patch fully maintains
valid tree structures, and I'd suggest that the increase in complexity
of a single keyword is justified by the demonstrated user demand for
this functionality.

Thanks,


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-export-remove-ignore-headline-and-promote-children.patch --]
[-- Type: text/x-diff, Size: 5437 bytes --]

From 9d95ad617f890142fe0d31ae355d205ef40ce6f3 Mon Sep 17 00:00:00 2001
From: Eric Schulte <schulte.eric@gmail.com>
Date: Sun, 15 Jun 2014 19:46:31 -0400
Subject: [PATCH 1/2] export remove ignore headline and promote children

* lisp/ox.el (org-export-ignore-headlines-retain-and-promoting-children):
  A new function.
  (org-export-as): Include
  `org-export-ignore-headlines-retain-and-promoting-children' into the
  export process.

* testing/lisp/test-ox.el (test-org-export/ignored-headlines-text):
  Example org-mode file for ignoreexport headline tests.
  (test-org-export/handle-ignored-headlines-system): System tests for
  org-export-ignore-headlines-retain-and-promoting-children.
  (test-org-export/handle-ignored-headlines-unit): Unit tests for
  org-export-ignore-headlines-retain-and-promoting-children.
---
 lisp/ox.el              | 31 ++++++++++++++++++++++++++++
 testing/lisp/test-ox.el | 54 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 85 insertions(+)

diff --git a/lisp/ox.el b/lisp/ox.el
index 4bfef52..ae3a11c 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -2320,6 +2320,34 @@ tree is modified by side effect and returned by the function."
 		(plist-get info prop)
 		info))))
 
+(defun org-export-ignore-headlines-retain-and-promoting-children (data info)
+  "Remove headlines tagged \"ignoreexport\" retaining sub-headlines.
+DATA is the parse tree.  INFO is a plist containing export
+options.  Each headline tagged \"ignoreexport\" will be removed
+removing its contents but retaining and promoting any children
+headlines by a single level."
+  (org-element-map data org-element-all-elements
+    (lambda (object)
+      (when (and (equal 'headline (org-element-type object))
+                 (member "ignoreexport" (org-element-property :tags object)))
+        (mapc (lambda (el)
+                ;; recursively promote all nested headlines
+                (org-element-map el 'headline
+                  (lambda (el)
+                    (when (equal 'headline (org-element-type el))
+                      (org-element-put-property el
+                        :level (1- (org-element-property :level el))))))
+                ;; insert back into parse tree
+                (org-element-insert-before el object))
+              ;; drop first three elements of headline
+              ;; 1. headline tag
+              ;; 2. properties
+              ;; 3. section
+              (cdddr object))
+        (org-element-extract-element object)))
+    info nil)
+  data)
+
 (defun org-export--remove-uninterpreted-data-1 (data info)
   "Change uninterpreted elements back into Org syntax.
 DATA is a parse tree or a secondary string.  INFO is a plist
@@ -3124,6 +3152,9 @@ Return code as a string."
 	 ;; Handle left-over uninterpreted elements or objects in
 	 ;; parse tree and communication channel.
 	 (org-export-remove-uninterpreted-data tree info)
+	 ;; Remove headlines tagged "ignoreexport" and promote their
+	 ;; children.
+	 (org-export-ignore-headlines-retain-and-promoting-children tree info)
 	 ;; Call options filters and update export options.  We do not
 	 ;; use `org-export-filter-apply-functions' here since the
 	 ;; arity of such filters is different.
diff --git a/testing/lisp/test-ox.el b/testing/lisp/test-ox.el
index 234032e..2ed0af9 100644
--- a/testing/lisp/test-ox.el
+++ b/testing/lisp/test-ox.el
@@ -1782,6 +1782,60 @@ Paragraph[fn:1]"
 		(org-export-as (org-test-default-backend)
 			       nil nil nil '(:with-tasks nil))))))))
 
+(defvar test-org-export/ignored-headlines-text
+  "* H1
+Text1
+** H2                                                          :ignoreexport:
+Text2
+*** H3
+Text3
+**** H4
+Text4
+*** H5
+Text5
+**** H6                                                        :ignoreexport:
+Text6
+***** H7
+Text7
+***** H8
+Text8\n")
+
+(ert-deftest test-org-export/handle-ignored-headlines-system ()
+  "Test `org-export-ignore-headlines-retain-and-promoting-children'."
+  (let ((exported
+	 (org-test-with-temp-text test-org-export/ignored-headlines-text
+	   (org-export-as (org-test-default-backend) nil nil nil nil))))
+    ;; ensure ignored headlines and contents are not present
+    (mapc (lambda (s) (should-not (string-match (regexp-quote s) exported)))
+	  (list "H2" "Text2" "H6" "Text6"))
+    ;; ensure all other headlines and contents are present
+    (mapc (lambda (s) (should (string-match (regexp-quote s) exported)))
+	  ;; should not be included
+	  (list "H1" "Text1" "H3" "Text3" "H4" "Text4"
+		"H5" "Text5" "H7" "Text7" "H8" "Text8"))))
+
+(ert-deftest test-org-export/handle-ignored-headlines-unit ()
+  "Test `org-export-ignore-headlines-retain-and-promoting-children'."
+  (let ((data (org-export-ignore-headlines-retain-and-promoting-children
+	       (org-test-with-temp-text test-org-export/ignored-headlines-text
+		 (org-element-parse-buffer))
+	       nil)))
+    (flet ((level-of (name)
+		     (let (out)
+		       (org-element-map data 'headline
+			 (lambda (el)
+			   (when (string= name
+					  (org-element-property :raw-value el))
+			     (setq out (org-element-property :level el)))))
+		       out)))
+      ;; ensure ignored headlines and contents are not present
+      (mapc (lambda (pair) (should (= (car pair) (level-of (cdr pair)))))
+	    '((1 . "H1")
+	      (2 . "H3")
+	      (3 . "H4")
+	      (2 . "H5")
+	      (3 . "H7")
+	      (3 . "H8"))))))
 
 \f
 ;;; Keywords
-- 
2.0.0


[-- Attachment #3: Type: text/plain, Size: 90 bytes --]


-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D (see https://u.fsf.org/yw)

Re: proposal to have ignoreheading tags/properties

[Nicolas Goaziou]

Hello,

Eric Schulte <schulte.eric@gmail.com> writes:

> In my opinion the manual interleaving of "noexport" and "export" tags is
> overly cumbersome and is non-obvious.

It is as non-obvious as the task it achieves.

> The obscure nature of this solution is evidenced by multiple
> discussions and implementations of filter functions to handle
> situations which could be covered by this noexport/export pattern.

I don't know what you are referring to, since this "noexport/export"
pattern isn't implemented yet. Do you have an example of such
a discussion that dismissed said pattern?

> I think the attached patch should be applied to the core.  It includes
> the following.
>
> - a single new export function which removes the headlines and contents
>   (the "section" element) of headlines tagged "ignoreexport", then
>   retains and promotes all sub-headlines contained therein.
>
> - a single new export tag named "ignoreexport" (ignore seems to be the
>   crowd favorite, and adding "export" reduces possible conflicts with
>   existing personal tags)

Thank you for the patch.

Again, my concern is that it introduces another tag for a task that
could be possible without it. "export", "noexport", "ignoreexport": this
all gets confusing. Furthermore, this tag should, at the bare minimum,
be configurable so it also introduces another keyword on par with
SELECT_TAGS and EXCLUDE_TAGS.

Moreover, that task is highly specific; I'm not convinced we should have
a dedicated keyword for each of them. I'd rather have a simple solution
for selective export problems, even if, as a generic solution, it may
look clumsier.

> From the included tests, the effect of this patch is to convert a tree
> like the following,
>
> ,----
> | * H1
> | Text1
> | ** H2                                                          :ignoreexport:
> | Text2
> | *** H3
> | Text3
> | **** H4
> | Text4
> | *** H5
> | Text5
> | **** H6                                                        :ignoreexport:
> | Text6
> | ***** H7
> | Text7
> | ***** H8
> | Text8
> `----

Note that my suggestion is

  ,----
  | * H1
  | Text1
  | ** H2                                                          :noexport:
  | Text2
  | *** H3                                                           :export:
  | Text3
  | **** H4
  | Text4
  | *** H5                                                           :export:
  | Text5
  | **** H6                                                        :noexport:
  | Text6
  | ***** H7                                                         :export:
  | Text7
  | ***** H8                                                         :export:
  | Text8
  `----

In this case, it is more verbose. But the opposite would happen if you
needed to also ignore children within the "ignoreexport" headlines.
In order to get

  * H1
  ** H4

compare


  * H1
  ** H2   :ignoreexport:
  *** H3      :noexport:
  *** H4

with

  * H1
  ** H2       :noexport:
  *** H3
  *** H4        :export:

This time, the brevity of :ignoreexport: is not so clear. Also, it will
not prevent the need to nest tags at some points. IOW, it is only
efficient for one very specific use case. Outside of this case, the new
tag would just be an annoyance and you would fallback
to :noexport:/:export: nesting, if implemented. And if we cannot avoid
tags nesting, I'd take 2 "nesteable" tags over 3 any day.

Another point to consider is that :noexport:/:export has the advantage
to mark explicitly what will be included in the export process. As such,
I don't find less obvious than your proposal: in both cases, you have to
understand the rules applied behind the keywords anyway. And these rules
are well-known for "export" and "noexport" tags.

> I'm sympathetic to arguments about maintaining simplicity of the core,
> and even more so to arguments about maintaining structural validity of
> trees during export.  I believe that this revised patch fully maintains
> valid tree structures, and I'd suggest that the increase in complexity
> of a single keyword is justified by the demonstrated user demand for
> this functionality.

It mostly preserves tree structure, indeed. More comments about this
below.

Just to be clear: I'm not against implementing the functionality. We can
implement it without increasing complexity, so the demand is hardly
a justification for this added complexity.

Making :export: and :noexport: tags symmetrical is a good thing anyway:
why could we add :noexport: within :export: trees but not the other way?
And, as I demonstrated earlier, combination of both permits more
patterns than :ignoreexport:. Therefore, we should implement it in any
case. As a bonus, it provides a solution for the problem at hand.

If this solution happens to be inadequate, then we can discuss about
better solutions, this time with more options (a user-defined hook,
a macro, a command, even a new tag if we cannot escape it...).

Anyway, some comments follow about your code.

> +(defun org-export-ignore-headlines-retain-and-promoting-children (data info)
> +  "Remove headlines tagged \"ignoreexport\" retaining sub-headlines.
> +DATA is the parse tree.  INFO is a plist containing export
> +options.  Each headline tagged \"ignoreexport\" will be removed
> +removing its contents but retaining and promoting any children
> +headlines by a single level."
> +  (org-element-map data org-element-all-elements

Since you're only interested in headlines, org-element-all-elements -->
headline

> +    (lambda (object)
> +      (when (and (equal 'headline (org-element-type object))

This check is not necessary per the suggestion above.

> +                 (member "ignoreexport" (org-element-property :tags object)))
> +        (mapc (lambda (el)
> +                ;; recursively promote all nested headlines
> +                (org-element-map el 'headline
> +                  (lambda (el)
> +                    (when (equal 'headline (org-element-type el))

You don't need this `mapc' + `org-element-map' + `org-element-type'
construct. More on this below.

> +                      (org-element-put-property el
> +                        :level (1- (org-element-property :level el))))))

Actually, hard-coding `1-' will not always work, because it assumes that
children differ from their parent by exactly one level. This is not
always the case.

 * 1
 *** 1.0.1
 ***** 1.0.1.0.1
 ***** 1.0.1.0.2

Of course, this is a contrived example. Nevertheless, just to make it
robust, promoting children should put the lowest level children to the
level of their parent and preserve relative level differences between
other children.

> +                ;; insert back into parse tree
> +                (org-element-insert-before el object))
> +              ;; drop first three elements of headline
> +              ;; 1. headline tag
> +              ;; 2. properties
> +              ;; 3. section
> +              (cdddr object))

`cddr' is really `org-element-contents'. Also, some headlines do not
have a section, which is not mandatory. In this case `cdddr' would also
remove the first headline, which is wrong.

This can be solved by replacing `mapc' + `org-element-map' + `cdddr'
with:

  (org-element-map (org-element-contents object) 'headline
    (lambda (el) ...))


I appreciate your work on this, but let me stress it again: I'm
convinced that :ignoreexport: tag is only a partial solution to the
problem, and, as such, a dead-end. :noexport:/:export: combination is
more versatile, at the price of some verbosity in some cases, but
without the added complexity another tag would introduce.


Regards,

-- 
Nicolas Goaziou

Re: proposal to have ignoreheading tags/properties

[Mark Edgington]

Nicolas Goaziou <mail <at> nicolasgoaziou.fr> writes:
 
> Moreover, that task is highly specific; I'm not convinced we should have
> a dedicated keyword for each of them. I'd rather have a simple solution
> for selective export problems, even if, as a generic solution, it may
> look clumsier.

Hi Nicolas,

I agree that the nested use of :export: and :noexport: symmetric is a good
feature to implement.  I think the main point of contention that I and
others may have is that even if the :ignoreexport: (or whatever you wish to
call it) tag is "highly specific" w.r.t. the variety of export/noexport
arrangements one can have, it is also highly useful.

While implementing a specific tag for this task may not be wise to implement
at this time, there should be *some* way that an org-mode user can avoid
being required to manually add export tags to every child of a noexport node
in order to achieve this "summary node" functionality.  In order to make it
possible to add a non-exported summary-node (i.e. a node which we might have
marked as :ignoreexport:) at any location in the tree without needing to add
N more tags for all of the children, one would need to somehow "by default"
have :export: tags implicit on all nodes, unless a :noexport: tag was on a
node which would override the behavior of the :export: tag.

Maybe there needs to be a way to decide whether a specific tag (i.e. on a
specific headline) will be applied recursively or not -- e.g. if the tag has
a "*" suffix appended to it, then it is applied to all descendants
recursively, but otherwise it applies only to the single headline it is on.
 Ultimately I see the problem of having to tag all of the children as
stemming from this default behavior of tags being recursive.  Having control
of their "recursivitiy" would alleviate the problem.

Regards,

Mark

Re: proposal to have ignoreheading tags/properties

[Eric Schulte]

[-- Attachment #1: Type: text/plain, Size: 4475 bytes --]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Hello,
>
> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> In my opinion the manual interleaving of "noexport" and "export" tags is
>> overly cumbersome and is non-obvious.
>
> It is as non-obvious as the task it achieves.
>
>> The obscure nature of this solution is evidenced by multiple
>> discussions and implementations of filter functions to handle
>> situations which could be covered by this noexport/export pattern.
>
> I don't know what you are referring to, since this "noexport/export"
> pattern isn't implemented yet. Do you have an example of such
> a discussion that dismissed said pattern?
>
>> I think the attached patch should be applied to the core.  It includes
>> the following.
>>
>> - a single new export function which removes the headlines and contents
>>   (the "section" element) of headlines tagged "ignoreexport", then
>>   retains and promotes all sub-headlines contained therein.
>>
>> - a single new export tag named "ignoreexport" (ignore seems to be the
>>   crowd favorite, and adding "export" reduces possible conflicts with
>>   existing personal tags)
>
> Thank you for the patch.
>
> Again, my concern is that it introduces another tag for a task that
> could be possible without it. "export", "noexport", "ignoreexport": this
> all gets confusing. Furthermore, this tag should, at the bare minimum,
> be configurable so it also introduces another keyword on par with
> SELECT_TAGS and EXCLUDE_TAGS.
>
> Moreover, that task is highly specific; I'm not convinced we should have
> a dedicated keyword for each of them. I'd rather have a simple solution
> for selective export problems, even if, as a generic solution, it may
> look clumsier.
>
>> From the included tests, the effect of this patch is to convert a tree
>> like the following,
>>
>> ,----
>> | * H1
>> | Text1
>> | ** H2                                                          :ignoreexport:
>> | Text2
>> | *** H3
>> | Text3
>> | **** H4
>> | Text4
>> | *** H5
>> | Text5
>> | **** H6                                                        :ignoreexport:
>> | Text6
>> | ***** H7
>> | Text7
>> | ***** H8
>> | Text8
>> `----
>
> Note that my suggestion is
>
>   ,----
>   | * H1
>   | Text1
>   | ** H2                                                          :noexport:
>   | Text2
>   | *** H3                                                           :export:
>   | Text3
>   | **** H4
>   | Text4
>   | *** H5                                                           :export:
>   | Text5
>   | **** H6                                                        :noexport:
>   | Text6
>   | ***** H7                                                         :export:
>   | Text7
>   | ***** H8                                                         :export:
>   | Text8
>   `----
>
> In this case, it is more verbose. But the opposite would happen if you
> needed to also ignore children within the "ignoreexport" headlines.
> In order to get
>
>   * H1
>   ** H4
>
> compare
>
>
>   * H1
>   ** H2   :ignoreexport:
>   *** H3      :noexport:
>   *** H4
>
> with
>
>   * H1
>   ** H2       :noexport:
>   *** H3
>   *** H4        :export:
>
> This time, the brevity of :ignoreexport: is not so clear. Also, it will
> not prevent the need to nest tags at some points. IOW, it is only
> efficient for one very specific use case. Outside of this case, the new
> tag would just be an annoyance and you would fallback
> to :noexport:/:export: nesting, if implemented. And if we cannot avoid
> tags nesting, I'd take 2 "nesteable" tags over 3 any day.
>
> Another point to consider is that :noexport:/:export has the advantage
> to mark explicitly what will be included in the export process. As such,
> I don't find less obvious than your proposal: in both cases, you have to
> understand the rules applied behind the keywords anyway. And these rules
> are well-known for "export" and "noexport" tags.
>

This is a specific tag for a specific but common use case.  I'll defer
to your judgment on the exporting code and instead share a filter
function which may be added to personal configurations (I've added this
to the FAQ as well [1]).

Having looked more closely at my use cases (documented in the attached
implementation), including the contents of ignored headlines is required
for most practical applications (which may make the parse tree invalid,
but which works well for every export backend I've used thus far).


[-- Attachment #2: ignore-headline.el --]
[-- Type: application/emacs-lisp, Size: 2603 bytes --]

[-- Attachment #3: Type: text/plain, Size: 4963 bytes --]


>
>> I'm sympathetic to arguments about maintaining simplicity of the core,
>> and even more so to arguments about maintaining structural validity of
>> trees during export.  I believe that this revised patch fully maintains
>> valid tree structures, and I'd suggest that the increase in complexity
>> of a single keyword is justified by the demonstrated user demand for
>> this functionality.
>
> It mostly preserves tree structure, indeed. More comments about this
> below.
>
> Just to be clear: I'm not against implementing the functionality. We can
> implement it without increasing complexity, so the demand is hardly
> a justification for this added complexity.
>
> Making :export: and :noexport: tags symmetrical is a good thing anyway:
> why could we add :noexport: within :export: trees but not the other way?
> And, as I demonstrated earlier, combination of both permits more
> patterns than :ignoreexport:. Therefore, we should implement it in any
> case. As a bonus, it provides a solution for the problem at hand.
>
> If this solution happens to be inadequate, then we can discuss about
> better solutions, this time with more options (a user-defined hook,
> a macro, a command, even a new tag if we cannot escape it...).
>
> Anyway, some comments follow about your code.
>
>> +(defun org-export-ignore-headlines-retain-and-promoting-children (data info)
>> +  "Remove headlines tagged \"ignoreexport\" retaining sub-headlines.
>> +DATA is the parse tree.  INFO is a plist containing export
>> +options.  Each headline tagged \"ignoreexport\" will be removed
>> +removing its contents but retaining and promoting any children
>> +headlines by a single level."
>> +  (org-element-map data org-element-all-elements
>
> Since you're only interested in headlines, org-element-all-elements -->
> headline
>

Done, thanks.

>
>> +    (lambda (object)
>> +      (when (and (equal 'headline (org-element-type object))
>
> This check is not necessary per the suggestion above.
>

Yes.

>
>> +                 (member "ignoreexport" (org-element-property :tags object)))
>> +        (mapc (lambda (el)
>> +                ;; recursively promote all nested headlines
>> +                (org-element-map el 'headline
>> +                  (lambda (el)
>> +                    (when (equal 'headline (org-element-type el))
>
> You don't need this `mapc' + `org-element-map' + `org-element-type'
> construct. More on this below.
>

This includes the section contents, and ensures the order in which the
children headlines are mapped, so that they are inserted with
`org-element-insert-before' in the correct order.  This is both because
I'm not sure what exactly is guaranteed WRT mapping order by
org-element-map, and also so the code is explicit about the ordering of
side effects with element insertion.

>
>> +                      (org-element-put-property el
>> +                        :level (1- (org-element-property :level el))))))
>
> Actually, hard-coding `1-' will not always work, because it assumes that
> children differ from their parent by exactly one level. This is not
> always the case.
>
>  * 1
>  *** 1.0.1
>  ***** 1.0.1.0.1
>  ***** 1.0.1.0.2
>
> Of course, this is a contrived example. Nevertheless, just to make it
> robust, promoting children should put the lowest level children to the
> level of their parent and preserve relative level differences between
> other children.
>

Good catch.  I've made this change.

>
>> +                ;; insert back into parse tree
>> +                (org-element-insert-before el object))
>> +              ;; drop first three elements of headline
>> +              ;; 1. headline tag
>> +              ;; 2. properties
>> +              ;; 3. section
>> +              (cdddr object))
>
> `cddr' is really `org-element-contents'. Also, some headlines do not
> have a section, which is not mandatory. In this case `cdddr' would also
> remove the first headline, which is wrong.
>
> This can be solved by replacing `mapc' + `org-element-map' + `cdddr'
> with:
>
>   (org-element-map (org-element-contents object) 'headline
>     (lambda (el) ...))
>

Thanks, better to stick with the org-element API.

>
>
> I appreciate your work on this, but let me stress it again: I'm
> convinced that :ignoreexport: tag is only a partial solution to the
> problem, and, as such, a dead-end. :noexport:/:export: combination is
> more versatile, at the price of some verbosity in some cases, but
> without the added complexity another tag would introduce.
>

We disagree here.  I think that the particular use case addressed by
this functionality is frequent enough to justify the tag, but I
understand your points especially given that most useful implementations
will require including the content of the ignored headline.

Best,

>
>
> Regards,


Footnotes: 
[1]  http://orgmode.org/worg/org-faq.html#ignoreheadline

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D (see https://u.fsf.org/yw)

Re: proposal to have ignoreheading tags/properties

[Aaron Ecay]

Hi Eric,

Thanks for your work on this code.  Partially inspired by this discussion,
I’ve just created contrib/lisp/ox-extras.el, which I hope will become a
home for useful export hook functions like this one.  Would you like to
add your code there (or would you mind if I did so)?

Thanks,

-- 
Aaron Ecay

Re: proposal to have ignoreheading tags/properties

[Eric Schulte]

Aaron Ecay <aaronecay@gmail.com> writes:

> Hi Eric,
>
> Thanks for your work on this code.  Partially inspired by this discussion,
> I’ve just created contrib/lisp/ox-extras.el, which I hope will become a
> home for useful export hook functions like this one.  Would you like to
> add your code there (or would you mind if I did so)?
>

Please feel free to add this to ox-extras.  Thanks -- Eric

>
> Thanks,

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D (see https://u.fsf.org/yw)

Re: proposal to have ignoreheading tags/properties

[Bastien]

Hi Mark, Aaron, Eric and Nicolas,

Eric Schulte <schulte.eric@gmail.com> writes:

> Please feel free to add this to ox-extras.

I suggest renaming ox-extra.el to ox-filters-extra.el and to have
org-mode/lisp/ox-filters.el for filters that are important enough
to be in core.

I think Eric's filter is important enough to be in core, together
with an option to let users decide what tag should be used instead
of "ignore" (with "ignore" as a default).

Filters are nice and not used enough, having some filters in core
can help teaching people how to use them.

What do you all think?

-- 
 Bastien

Re: proposal to have ignoreheading tags/properties

[Mark Edgington]

[-- Attachment #1: Type: text/plain, Size: 426 bytes --]

Hi Bastien,

On Sun, Jul 27, 2014 at 1:21 PM, Bastien <bzg@gnu.org> wrote:

>
> I suggest renaming ox-extra.el to ox-filters-extra.el and to have
> org-mode/lisp/ox-filters.el for filters that are important enough
> to be in core.
>

Your suggestion sounds sensible, but of course I'm biased, as I've been
using Eric's filter daily now (thanks Eric!), and would be pleased to have
something like this in core.

Regards,

Mark

[-- Attachment #2: Type: text/html, Size: 792 bytes --]

Re: proposal to have ignoreheading tags/properties

[Rasmus]

Bastien <bzg@gnu.org> writes:

> Hi Mark, Aaron, Eric and Nicolas,
>
> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> Please feel free to add this to ox-extras.
>
> I suggest renaming ox-extra.el to ox-filters-extra.el and to have
> org-mode/lisp/ox-filters.el for filters that are important enough
> to be in core.
>
> I think Eric's filter is important enough to be in core, together
> with an option to let users decide what tag should be used instead
> of "ignore" (with "ignore" as a default).

How about "ignoreheading" as this is already the appropriate tag for a
similar type of action in ox-beamer.el?  See e.g. the variable
org-beamer-environments-special and consider this awkward headline:

   * my ignored headline :ignored:ignoreheading:
     Ignored in Beamer and in LaTeX

—Rasmus

-- 
May the Force be with you

Re: proposal to have ignoreheading tags/properties

[Mark Edgington]

Hi Rasmus,

Rasmus <rasmus <at> gmx.us> writes:

> 
> Bastien <bzg <at> gnu.org> writes:
> >
> > I think Eric's filter is important enough to be in core, together
> > with an option to let users decide what tag should be used instead
> > of "ignore" (with "ignore" as a default).
> 
> How about "ignoreheading" as this is already the appropriate tag for a
> similar type of action in ox-beamer.el?  See e.g. the variable
> org-beamer-environments-special and consider this awkward headline:
> 
>    * my ignored headline :ignored:ignoreheading:
>      Ignored in Beamer and in LaTeX
> 

I am partial to "ignore" because it is simpler, and if manually typing in,
requires less effort.  But anyway, wouldn't it be the case that you would
only need to use "ignore" (and not also "ignoreheading"), since it would
filter out such headlines and ox-beamer wouldn't even see them?  If this
isn't the case, it would be sensible for things to work that way nonetheless.

Regards,

Mark

Re: proposal to have ignoreheading tags/properties

[Nicolas Goaziou]

Hello,

Bastien <bzg@gnu.org> writes:

> I suggest renaming ox-extra.el to ox-filters-extra.el and to have
> org-mode/lisp/ox-filters.el for filters that are important enough
> to be in core.
>
> I think Eric's filter is important enough to be in core, together
> with an option to let users decide what tag should be used instead
> of "ignore" (with "ignore" as a default).
>
> Filters are nice and not used enough, having some filters in core
> can help teaching people how to use them.
>
> What do you all think?

Filters are _not_ meant to be in core since they are hardly a generic
solution for a class of problem. They are entry points for user-level
hacking. Generic patches should operate at the parse tree level, not
using regexps.

Eric's filter, like any other filter, has flaws that cannot be fixed.
Useful filters ought to be published in Worg, not included in core.


Regards,

-- 
Nicolas Goaziou

Re: proposal to have ignoreheading tags/properties

[Rasmus]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Hello,
>
> Bastien <bzg@gnu.org> writes:
>
>> I suggest renaming ox-extra.el to ox-filters-extra.el and to have
>> org-mode/lisp/ox-filters.el for filters that are important enough
>> to be in core.
>>
>> I think Eric's filter is important enough to be in core, together
>> with an option to let users decide what tag should be used instead
>> of "ignore" (with "ignore" as a default).
>>
>> Filters are nice and not used enough, having some filters in core
>> can help teaching people how to use them.
>>
>> What do you all think?
>
> Filters are _not_ meant to be in core since they are hardly a generic
> solution for a class of problem. They are entry points for user-level
> hacking. Generic patches should operate at the parse tree level, not
> using regexps.
>
> Eric's filter, like any other filter, has flaws that cannot be fixed.
> Useful filters ought to be published in Worg, not included in core.

I agree with Nicolas, though I thought the inclusion was settled by
now.  Repeating myself upon the announcement of ox-extra.el¹:

    If it's helpful (enough) it should be ox.el, I guess, or
    ox-ENGINGE.el. . .  Parallel, semi-official extension to ox itself
    doesn't sound nice.

Worg → ox-filters is better IMO.  This would only become more true (in
my mind) if that — again IMO, horrendous — idea of splitting off
contrib is made into reality.

Cheers,
Rasmus

Footnotes: 
¹   http://lists.gnu.org/archive/html/emacs-orgmode/2014-06/msg00813.html

-- 
I feel emotional landscapes they puzzle me

Re: proposal to have ignoreheading tags/properties

[Bastien]

Hi Nicolas,

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Filters are _not_ meant to be in core since they are hardly a generic
> solution for a class of problem. They are entry points for user-level
> hacking. Generic patches should operate at the parse tree level, not
> using regexps.
>
> Eric's filter, like any other filter, has flaws that cannot be fixed.
> Useful filters ought to be published in Worg, not included in core.

Fair enough.

Still, can someone add Eric's solution to contrib/lisp/ox-extra.el?

Thanks,

-- 
 Bastien

Re: proposal to have ignoreheading tags/properties

[Mark Edgington]

[-- Attachment #1: Type: text/plain, Size: 847 bytes --]

Hi Bastien,

I've attached a patch for ox-extra which doesn't yet include the
option for choosing specific tag names (the 'ignore' tag is currently
hard-coded).  Feel free to modify / commit it.

Regards,

Mark


On Tue, Jul 29, 2014 at 10:31 AM, Bastien <bzg@gnu.org> wrote:
> Hi Nicolas,
>
> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
>> Filters are _not_ meant to be in core since they are hardly a generic
>> solution for a class of problem. They are entry points for user-level
>> hacking. Generic patches should operate at the parse tree level, not
>> using regexps.
>>
>> Eric's filter, like any other filter, has flaws that cannot be fixed.
>> Useful filters ought to be published in Worg, not included in core.
>
> Fair enough.
>
> Still, can someone add Eric's solution to contrib/lisp/ox-extra.el?
>
> Thanks,
>
> --
>  Bastien

[-- Attachment #2: 0001-ox-extra.el-add-ignore-headlines-filter.patch --]
[-- Type: text/x-patch, Size: 4541 bytes --]

From 7b60eefcb21c2a62b1ab7f248f6a0b993d89cc4d Mon Sep 17 00:00:00 2001
From: Mark Edgington <edgimar@gmail.com>
Date: Sat, 2 Aug 2014 00:32:29 -0400
Subject: [PATCH] * ox-extra.el: add ignore-headlines filter

---
 contrib/lisp/ox-extra.el | 82 +++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 81 insertions(+), 1 deletion(-)

diff --git a/contrib/lisp/ox-extra.el b/contrib/lisp/ox-extra.el
index f4f0b76..01368cb 100644
--- a/contrib/lisp/ox-extra.el
+++ b/contrib/lisp/ox-extra.el
@@ -23,6 +23,12 @@
 ;; are not part of org's core.  Call `ox-extras-activate' passing a
 ;; list of symbols naming extras, which will be installed globally in
 ;; your org session.
+;;
+;; For example, you could include the following in your .emacs file:
+;;
+;;    (require 'ox-extra)
+;;    (ox-extras-activate '(latex-header-blocks ignore-headlines))
+;;
 
 ;; Currently available extras:
 
@@ -35,6 +41,12 @@
 ;;   ...
 ;; #+end_latex
 
+;; - `ignore-headlines' -- allow a headline (but not its children) to
+;; be ignored.  Any headline tagged with the 'ignore' tag will be
+;; ignored (i.e. will not be included in the export), but any child
+;; headlines will not be ignored (unless explicitly tagged to be
+;; ignored), and will instead have their levels promoted by one.
+
 ;; TODO:
 ;; - add a function to org-mode-hook that looks for a ox-extras local
 ;;   variable and activates the specified extras buffer-locally
@@ -75,8 +87,76 @@
 	    ;; earlier in the file
 	    (reverse positions)))))
 
+
+;; During export headlines which have the "ignore" tag are removed
+;; from the parse tree.  Their contents are retained (leading to a
+;; possibly invalid parse tree, which nevertheless appears to function
+;; correctly with most export backends) all children headlines are
+;; retained and are promoted to the level of the ignored parent
+;; headline.
+;;
+;; This makes it possible to add structure to the original Org-mode
+;; document which does not effect the exported version, such as in the
+;; following examples.
+;;
+;; Wrapping an abstract in a headline
+;;
+;;     * Abstract                        :ignore:
+;;     #+LaTeX: \begin{abstract}
+;;     #+HTML: <div id="abstract">
+;;
+;;     ...
+;;
+;;     #+HTML: </div>
+;;     #+LaTeX: \end{abstract}
+;;
+;; Placing References under a headline (using ox-bibtex in contrib)
+;;
+;;     * References                     :ignore:
+;;     #+BIBLIOGRAPHY: dissertation plain
+;;
+;; Inserting an appendix for LaTeX using the appendix package.
+;;
+;;     * Appendix                       :ignore:
+;;     #+LaTeX: \begin{appendices}
+;;     ** Reproduction
+;;     ...
+;;     ** Definitions
+;;     #+LaTeX: \end{appendices}
+;;
+(defun org-export-ignore-headlines (data backend info)
+  "Remove headlines tagged \"ignore\" retaining contents and promoting children.
+Each headline tagged \"ignore\" will be removed retaining its
+contents and promoting any children headlines to the level of the
+parent."
+  (org-element-map data 'headline
+    (lambda (object)
+      (when (member "ignore" (org-element-property :tags object))
+        (let ((level-top (org-element-property :level object))
+              level-diff)
+          (mapc (lambda (el)
+                  ;; recursively promote all nested headlines
+                  (org-element-map el 'headline
+                    (lambda (el)
+                      (when (equal 'headline (org-element-type el))
+                        (unless level-diff
+                          (setq level-diff (- (org-element-property :level el)
+                                              level-top)))
+                        (org-element-put-property el
+                          :level (- (org-element-property :level el)
+                                    level-diff)))))
+                  ;; insert back into parse tree
+                  (org-element-insert-before el object))
+                (org-element-contents object)))
+        (org-element-extract-element object)))
+    info nil)
+  data)
+
+;(add-hook 'org-export-filter-parse-tree-functions 'org-export-ignore-headlines)
+
 (defconst ox-extras
-  '((latex-header-blocks org-latex-header-blocks-filter org-export-before-parsing-hook))
+  '((latex-header-blocks org-latex-header-blocks-filter org-export-before-parsing-hook)
+    (ignore-headlines org-export-ignore-headlines org-export-filter-parse-tree-functions))
   "A list of org export extras that can be enabled.
 
 Should be a list of items of the form (NAME FN HOOK).  NAME is a
-- 
1.9.2

Re: proposal to have ignoreheading tags/properties

[Aaron Ecay]

Hi Mark,

2014ko abuztuak 2an, Mark Edgington-ek idatzi zuen:
> 
> Hi Bastien,
> 
> I've attached a patch for ox-extra which doesn't yet include the
> option for choosing specific tag names (the 'ignore' tag is currently
> hard-coded).  Feel free to modify / commit it.
> 
> Regards,
> 
> Mark

Thanks for the patch.  It is committed (with light modifications).

-- 
Aaron Ecay