[patch] Support CUSTOM_ID property in latex export

[patch] Support CUSTOM_ID property in latex export

[Richard Lawrence]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Patch to support CUSTOM_ID property in latex export --]
[-- Type: text/x-diff, Size: 2537 bytes --]

From 81115d0884c165778520aa1b4d4fa83580417e1c Mon Sep 17 00:00:00 2001
From: Richard Lawrence <richard.lawrence@berkeley.edu>
Date: Sat, 15 Feb 2014 11:59:44 -0800
Subject: [PATCH] LaTeX export: support CUSTOM_ID property in section labels
 and link refs

* lisp/ox-latex.el (org-latex-headline): when exporting a headline, if
  it has a CUSTOM_ID property, use that value as the associated
  label for a section (or whatever)
(org-latex-link): when exporting a link, if the
  destination is a headline with a CUSTOM_ID property, use that value
  in the referencing command

Fixes an issue raised at: http://thread.gmane.org/gmane.emacs.orgmode/54039

TINYCHANGE
---
 lisp/ox-latex.el |   25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 5815874..cbca0a5 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -1372,11 +1372,14 @@ holding contextual information."
 	   (full-text (funcall org-latex-format-headline-function
 			       todo todo-type priority text tags))
 	   ;; Associate \label to the headline for internal links.
+	   ;; Use the value of :CUSTOM_ID: property as label if it is defined.
 	   (headline-label
-	    (format "\\label{sec-%s}\n"
-		    (mapconcat 'number-to-string
-			       (org-export-get-headline-number headline info)
-			       "-")))
+	    (let ((custom-label (org-element-property :CUSTOM_ID headline)))
+	      (if custom-label (format "\\label{%s}\n" custom-label)
+		(format "\\label{sec-%s}\n"
+			(mapconcat 'number-to-string
+				   (org-export-get-headline-number headline info)
+				   "-")))))
 	   (pre-blanks
 	    (make-string (org-element-property :pre-blank headline) 10)))
       (if (or (not section-fmt) (org-export-low-level-p headline info))
@@ -1846,11 +1849,15 @@ INFO is a plist holding contextual information.  See
 	  ;; title.
 	  (headline
 	   (let ((label
-		  (format "sec-%s"
-			  (mapconcat
-			   'number-to-string
-			   (org-export-get-headline-number destination info)
-			   "-"))))
+		  (or
+		   ;; Case 1: headline has a CUSTOM_ID property; use that as label
+		   (org-element-property :CUSTOM_ID destination)
+		   ;; Case 2: headline has no CUSTOM_ID; use default numbering 
+		   (format "sec-%s"
+			   (mapconcat
+			    'number-to-string
+			    (org-export-get-headline-number destination info)
+			    "-")))))
 	     (if (and (plist-get info :section-numbers) (not desc))
 		 (format "\\ref{%s}" label)
 	       (format "\\hyperref[%s]{%s}" label
-- 
1.7.10.4


[-- Attachment #2: Type: text/plain, Size: 1940 bytes --]


Hi Orgsters,

Here is a patch to add support for using CUSTOM_ID properties for labels
and refs in the LaTeX exporter.

The patch uses the value of CUSTOM_ID when exporting a headline for the
associated \label, and when exporting a link for the associated \ref
command (or whatever).

For example, from this Org file:

===============================================================================
* Headline 1
  :PROPERTIES:
  :CUSTOM_ID: sec:headline1
  :END:
  Links to headlines which have no CUSTOM_ID still work normally, like
  this one: [[Headline 2]].

* Headline 2
  Links to headlines which have a CUSTOM_ID property will use this
  value to refer to them:

  This link refers to Headline 1 using the builtin syntax for
  CUSTOM_ID: [[#sec:headline1]].

  This one uses the fuzzy search on the headline text: [[Headline 1]].
===============================================================================

the relevant section is now exported as:

===============================================================================
\section{Headline 1}
\label{sec:headline1}
Links to headlines which have no CUSTOM\(_{\text{ID}}\) still work normally, like
this one: \ref{sec-2}.
\section{Headline 2}
\label{sec-2}
Links to headlines which have a CUSTOM\(_{\text{ID}}\) property will use this
value to refer to them:

This link refers to Headline 1 using the builtin syntax for
CUSTOM\(_{\text{ID}}\): \ref{sec:headline1}.

This one uses the fuzzy search on the headline text: \ref{sec:headline1}.
===============================================================================

Previously, the label for Headline 1 would have been \label{sec-1}, and
the links under Headline 2 would have been exported as \ref{sec-1}. 

This addresses an issue that was raised here, but got no response:
http://thread.gmane.org/gmane.emacs.orgmode/54039

I also need this behavior, which is why I wrote this.

Hope that's helpful!

Best,
Richard

Re: [patch] Support CUSTOM_ID property in latex export

[Nicolas Goaziou]

Hello,

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> Here is a patch to add support for using CUSTOM_ID properties for labels
> and refs in the LaTeX exporter.

Thank you for the patch. Though, I don't understand why it is needed.

> The patch uses the value of CUSTOM_ID when exporting a headline for the
> associated \label, and when exporting a link for the associated \ref
> command (or whatever).

[...]

> This addresses an issue that was raised here, but got no response:
> http://thread.gmane.org/gmane.emacs.orgmode/54039

The issue no longer applies since \label{custom-id} are not generated
anymore. Moreover, the "bug" is probably a misunderstanding of
`org-export-solidify-link-text' (actually, its 7.8 counterpart).

> I also need this behavior, which is why I wrote this.

Would you mind explaining your use case?

From a user's perspective, CUSTOM_ID allows an user to refer to
a particular headline with a specific internal link type. How this is
done internally is something different, which belong to the
implementation level.


Regards,

-- 
Nicolas Goaziou

Re: [patch] Support CUSTOM_ID property in latex export

[Richard Lawrence]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Richard Lawrence <richard.lawrence@berkeley.edu> writes:
>
>> Here is a patch to add support for using CUSTOM_ID properties for labels
>> and refs in the LaTeX exporter.
>
> Thank you for the patch. Though, I don't understand why it is needed.
>
>> I also need this behavior, which is why I wrote this.
>
> Would you mind explaining your use case?
>
> From a user's perspective, CUSTOM_ID allows an user to refer to
> a particular headline with a specific internal link type. How this is
> done internally is something different, which belong to the
> implementation level.

Sure.  I apologize -- I should have said more about this in my previous
email than just referring to that old post.

I am presently using Org to write my dissertation, which I compile using
the LaTeX exporter.  This means I am writing long documents with
embedded LaTeX blocks.  (Maybe this is ill-advised, but it has worked
great so far!)

Basically, I want control over how the labeling is done in the exported
document for two reasons:

1) Sometimes I need to refer to a section from within an embedded LaTeX
block.  In that case, I need to know the appropriate label to use at the
LaTeX level, not just in Org.  For example:

* A headline
  :PROPERTIES:
  :CUSTOM_ID: sec:a-headline
  :END:

# ... stuff ...

#+BEGIN_LATEX
% ... more stuff ...
(see section~\ref{sec:a-headline})
#+END_LATEX

This is not possible with the present section labeling in LaTeX export,
because I have no way of forcing Org to use a particular label for a
section.

2) I hope this doesn't happen, but there may come a time when I need to
move away from Org and just use straight LaTeX.  Having control over the
labeling will make this transition much easier, because it means I won't
have to worry about manually changing the labels in a long document from
Org's default "sec-..." numbering to my own semantic labels.

Right now, I am at a risk of losing a lot of information if I have to
make this transition, because my CUSTOM_ID values don't have any effect
on the generated LaTeX.  Being able to set section labels as needed from
within Org would reduce this risk and therefore allow me to put this
transition off longer.

Another reason (not mine, but someone else might care) is:

3) This will make the LaTeX exporter's behavior more consistent with the
HTML exporter's behavior.  The HTML exporter will use CUSTOM_ID if it is
supplied to construct the id attributes of headlines and divs.  If
someone is relying on this behavior of the HTML exporter, they might be
unpleasantly surprised by the LaTeX exporter's behavior.

Does all that seem reasonable?
  
Best,
Richard

Re: [patch] Support CUSTOM_ID property in latex export

[Nicolas Goaziou]

Hello,

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> 1) Sometimes I need to refer to a section from within an embedded LaTeX
> block.  In that case, I need to know the appropriate label to use at the
> LaTeX level, not just in Org.  For example:
>
> * A headline
>   :PROPERTIES:
>   :CUSTOM_ID: sec:a-headline
>   :END:
>
> # ... stuff ...
>
> #+BEGIN_LATEX
> % ... more stuff ...
> (see section~\ref{sec:a-headline})
> #+END_LATEX

I don't think this is a good idea, as the character set allowed in
\label{...} macros is only a subset of the character set allowed in
custom id value. Hence the `org-export-solidify-link-text' function.

If you are cautious, this will not be a problem, but it could bite users
with little LaTeX knowledge.

> This is not possible with the present section labeling in LaTeX export,
> because I have no way of forcing Org to use a particular label for a
> section.

  * A headline
    #+latex: \label{my-section}

    #+BEGIN_LATEX
    % ... more stuff ...
    (see section~\ref{my-section})
    #+END_LATEX

It also seems more consistent to me: since you want to explicitly write
the \ref{...}, you are also expected to explicitly write the \label{...}
part.

> 2) I hope this doesn't happen, but there may come a time when I need to
> move away from Org and just use straight LaTeX.  Having control over the
> labeling will make this transition much easier, because it means I won't
> have to worry about manually changing the labels in a long document from
> Org's default "sec-..." numbering to my own semantic labels.

See above. You can even automate that with a hook (i.e., get the custom
id value and add a corresponding label at the beginning of the
headline).

> 3) This will make the LaTeX exporter's behavior more consistent with the
> HTML exporter's behavior.  The HTML exporter will use CUSTOM_ID if it is
> supplied to construct the id attributes of headlines and divs.  If
> someone is relying on this behavior of the HTML exporter, they might be
> unpleasantly surprised by the LaTeX exporter's behavior.

One relying on an implementation detail instead of the actual
specifications has to be prepared for surprises.

What do you think?


Regards,

-- 
Nicolas Goaziou

Re: [patch] Support CUSTOM_ID property in latex export

[Richard Lawrence]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Richard Lawrence <richard.lawrence@berkeley.edu> writes:
>
>> 1) Sometimes I need to refer to a section from within an embedded LaTeX
>> block.  In that case, I need to know the appropriate label to use at the
>> LaTeX level, not just in Org.  For example:
>>
>> * A headline
>>   :PROPERTIES:
>>   :CUSTOM_ID: sec:a-headline
>>   :END:
>>
>> # ... stuff ...
>>
>> #+BEGIN_LATEX
>> % ... more stuff ...
>> (see section~\ref{sec:a-headline})
>> #+END_LATEX
>
> I don't think this is a good idea, as the character set allowed in
> \label{...} macros is only a subset of the character set allowed in
> custom id value. Hence the `org-export-solidify-link-text' function.
>
> If you are cautious, this will not be a problem, but it could bite users
> with little LaTeX knowledge.

Well, I don't quite see the force of this.  After all, isn't there a
general rule to the effect that "if you override Org's default behavior,
you're on your own, so be careful"?  Users are expected to make sure
that CUSTOM_ID is unique, for example.

It seems to me that if you explicitly specify CUSTOM_ID with the intent
of overriding Org's default labeling, you ought to have some idea what
can go in a \label, and be prepared to debug your LaTeX compilation if
there's an error.  If you're not prepared to do that, you should limit
yourself to the default behavior.  But if you *are* prepared to do that,
why should Org prevent you? 

>> This is not possible with the present section labeling in LaTeX export,
>> because I have no way of forcing Org to use a particular label for a
>> section.
>
>   * A headline
>     #+latex: \label{my-section}
>
>     #+BEGIN_LATEX
>     % ... more stuff ...
>     (see section~\ref{my-section})
>     #+END_LATEX
>
> It also seems more consistent to me: since you want to explicitly write
> the \ref{...}, you are also expected to explicitly write the \label{...}
> part.

But then I would not be able to take advantage of referring to that
label from within plain Org text (i.e., outside an embedded LaTeX block)
using links.  The whole point of this is that I want to be able to refer
to a *single*, unambiguous label from within both contexts.

The strategy you suggest would result in multiple labels in the same
location in the exported document.  This is bad because it introduces
ambiguity and is thus fragile.  The exported document could have two sets
of \refs which point to two different \labels.  Initially, LaTeX
would compile them to the same thing, but if one of the labels got moved
or deleted, one set of refs would break.  

>> 2) I hope this doesn't happen, but there may come a time when I need to
>> move away from Org and just use straight LaTeX.  Having control over the
>> labeling will make this transition much easier, because it means I won't
>> have to worry about manually changing the labels in a long document from
>> Org's default "sec-..." numbering to my own semantic labels.
>
> See above. You can even automate that with a hook (i.e., get the custom
> id value and add a corresponding label at the beginning of the
> headline).

I realize that I could automate this if necessary.  That in fact is why
I wrote the patch: I'm trying to do the work of automating it now,
rather than waiting until a moment when I realize in desperation that I
have to convert to LaTeX. :)

For the reasons I cited above, I wouldn't want to use multiple labels,
which is why I prefer to implement my automation this way.  Is it
possible to use a hook to do what this patch does? i.e., to *replace*
Org's default labeling with labels generated from CUSTOM_ID, and have
links exported to \refs using that value instead of the default
labeling?

I sent the patch because I saw on the list that at least one other
person wanted a feature like this.  If it is not accepted, that's OK; I
will work around it, probably using a derived export backend.  But I
have to imagine that other people are doing something similar. I know
there are other people on this list using Org to write long documents
that they compile with LaTeX, and they are probably facing a similar
problem.

>> 3) This will make the LaTeX exporter's behavior more consistent with the
>> HTML exporter's behavior.  The HTML exporter will use CUSTOM_ID if it is
>> supplied to construct the id attributes of headlines and divs.  If
>> someone is relying on this behavior of the HTML exporter, they might be
>> unpleasantly surprised by the LaTeX exporter's behavior.
>
> One relying on an implementation detail instead of the actual
> specifications has to be prepared for surprises.

Maybe so, but that's actually sort of my point.  At the moment, my
options are:
  1) Use multiple labeling schemes, one accessible to Org, one
     accessible to LaTeX, and use the former in Org text and the latter
     in embedded LaTeX
  2) Avoid using Org's labeling/linking entirely, and just explicitly specify
     all my \labels and \refs
  3) Rely on my understanding of how Org will produce section labels
     when I \ref sections inside embedded LaTeX blocks

Option 1 creates ambiguity, is fragile, and is thus not ideal.  Option 2
means giving up the advantages of using Org references instead of LaTeX.

Option 3 is clearly relying on an implementation detail in a problematic
way: it will break as soon as my section numbering changes.  

The patch allows me to *avoid* relying on an implementation detail in
this problematic way, while still getting the advantages of Org's
labeling and linking capabilities.

Having Org pass CUSTOM_ID through to \label does in a sense mean the
user is relying on an implementation detail of the exporter, but in an
explicit and predictable way, which makes it unproblematic.  Consider an
analogy: users who specify :options in an #+ATTR_LATEX declaration are
also relying on the implementation details of the exporter (they are
assuming it will export their options text unchanged), but this is not
problematic because they are explicitly requesting that the default
behavior (don't use options, or use some default options) be overridden.
Isn't overriding labeling with CUSTOM_ID pretty much the same thing?

Best,
Richard


(If possible, please encrypt your reply to me using my PGP key:
Key ID: CF6FA646
Fingerprint: 9969 43E1 CF6F A646.
See http://www.ocf.berkeley.edu/~rwl/encryption.html for more information.)

Re: [patch] Support CUSTOM_ID property in latex export

[Nicolas Goaziou]

Hello,

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> It seems to me that if you explicitly specify CUSTOM_ID with the intent
> of overriding Org's default labeling, you ought to have some idea what
> can go in a \label, and be prepared to debug your LaTeX compilation if
> there's an error.  If you're not prepared to do that, you should limit
> yourself to the default behavior.  But if you *are* prepared to do that,
> why should Org prevent you?

This is the problem. At the moment, CUSTOM_ID has no limitation about
the characters it can use. As long as the value is unique, Org will
create a valid label for it.

OTOH, you patch introduces a limitation and could force users to debug
LaTeX compilation, even if they didn't want to mess with Org's default
labeling in the first place. If you are *not* prepared, why Org should
force you?

So, this is not a net benefit in the general case.

> The strategy you suggest would result in multiple labels in the same
> location in the exported document.  This is bad because it introduces
> ambiguity and is thus fragile.  The exported document could have two sets
> of \refs which point to two different \labels.  Initially, LaTeX
> would compile them to the same thing, but if one of the labels got moved
> or deleted, one set of refs would break.

Sorry for being dense, but I fail to see where is the "ambiguity". Org
will not get confused with its own internal labels, neither will you
with yours. Do you have a real worrisome situation in mind?

>>> 2) I hope this doesn't happen, but there may come a time when I need to
>>> move away from Org and just use straight LaTeX.  Having control over the
>>> labeling will make this transition much easier, because it means I won't
>>> have to worry about manually changing the labels in a long document from
>>> Org's default "sec-..." numbering to my own semantic labels.

Here, I understand the problem. There is a solution, but it is not
trivial.

You can write a parse-tree filter that collects associations between
custom ID (obtained with `org-element-property') and headline numbers
(obtained with `org-export-get-headline-number'). You can store this
alist in the info channel. Then, you write a link and headline filter
that replaces "sec-..." labels and refs with their custom ID equivalent.

> Maybe so, but that's actually sort of my point.  At the moment, my
> options are:
>   1) Use multiple labeling schemes, one accessible to Org, one
>      accessible to LaTeX, and use the former in Org text and the latter
>      in embedded LaTeX
>   2) Avoid using Org's labeling/linking entirely, and just explicitly specify
>      all my \labels and \refs
>   3) Rely on my understanding of how Org will produce section labels
>      when I \ref sections inside embedded LaTeX blocks
>
> Option 1 creates ambiguity, is fragile, and is thus not ideal.

"Not ideal" is not necessarily "wrong". Also, as explained above, your
patch is not ideal either. I just think the current implementation is
(slightly) better.

Now, if you can improve your suggestion and solve my concerns about it,
I'm still all ears.

> Having Org pass CUSTOM_ID through to \label does in a sense mean the
> user is relying on an implementation detail of the exporter, but in an
> explicit and predictable way, which makes it unproblematic.  Consider an
> analogy: users who specify :options in an #+ATTR_LATEX declaration are
> also relying on the implementation details of the exporter (they are
> assuming it will export their options text unchanged), but this is not
> problematic because they are explicitly requesting that the default
> behavior (don't use options, or use some default options) be overridden.
> Isn't overriding labeling with CUSTOM_ID pretty much the same thing?

No it isn't. Exporting :options value unchanged is part of its
specifications. It is even written in the manual.

CUSTOM_ID specifications require an export back-end to provide a way to
link to a headline with some specific syntax. We happen to disagree on
how this should be done. This is an implementation detail.


Regards,

-- 
Nicolas Goaziou

Re: [patch] Support CUSTOM_ID property in latex export

[Richard Lawrence]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> Richard Lawrence <richard.lawrence@berkeley.edu> writes:
>
>> It seems to me that if you explicitly specify CUSTOM_ID with the intent
>> of overriding Org's default labeling, you ought to have some idea what
>> can go in a \label, and be prepared to debug your LaTeX compilation if
>> there's an error.  If you're not prepared to do that, you should limit
>> yourself to the default behavior.  But if you *are* prepared to do that,
>> why should Org prevent you?
>
> This is the problem. At the moment, CUSTOM_ID has no limitation about
> the characters it can use. As long as the value is unique, Org will
> create a valid label for it.
>
> OTOH, you patch introduces a limitation and could force users to debug
> LaTeX compilation, even if they didn't want to mess with Org's default
> labeling in the first place. If you are *not* prepared, why Org should
> force you?
>
> So, this is not a net benefit in the general case.

OK, I can understand this.  There are people who are using CUSTOM_ID
already, and they shouldn't have to worry about debugging LaTeX if they
weren't counting on it.  (In my case, I'm not using this property for
anything else, so this wasn't an issue, and using CUSTOM_ID provided a
handy way to use the [[#link]] syntax to introduce \refs with the label
I intended.)

Would using a different property---say, LATEX_LABEL---resolve your
concerns?  This property could be explicitly documented as overriding
Org's default labeling, with the value passed down directly to LaTeX.
During link resolution, a headline would export with a "\label{VAL}",
and a link to a headline with this property would export to "\ref{VAL}",
where "VAL" is the value of this property.

Thus, e.g.,

** A headline
   :PROPERTIES:
   :CUSTOM_ID: foo
   :LATEX_LABEL: bar
   :END:
Some text ... this is section [[#foo]].

would become:

\subsection{A headline}
\label{bar}
Some text \ldots this is section \ref{bar}.
 
That would meet all my needs, I think.

In my case it would also be handy to have some way to link to headlines
based on the LATEX_LABEL property directly (say, like [[label:bar]]).
But that's easy enough to add.

>> The strategy you suggest would result in multiple labels in the same
>> location in the exported document.  This is bad because it introduces
>> ambiguity and is thus fragile.  The exported document could have two sets
>> of \refs which point to two different \labels.  Initially, LaTeX
>> would compile them to the same thing, but if one of the labels got moved
>> or deleted, one set of refs would break.
>
> Sorry for being dense, but I fail to see where is the "ambiguity". Org
> will not get confused with its own internal labels, neither will you
> with yours. Do you have a real worrisome situation in mind?

The worrisome situation I have in mind is if I find that I eventually
need to move away from Org to straight LaTeX.  I would want to start
with an Org export to LaTeX, and then continue from that point by
editing the exported .tex file.  In that case, one label could
eventually get deleted, or they could drift apart, and then one set of
\refs could subtly break (say, if I put a new \subsection in between
them).  To avoid this, I want the exported .tex file to just use one set
of labels.

Best,
Richard


(If possible, please encrypt your reply to me using my PGP key:
Key ID: CF6FA646
Fingerprint: 9969 43E1 CF6F A646.
See http://www.ocf.berkeley.edu/~rwl/encryption.html for more information.)

Re: [patch] Support CUSTOM_ID property in latex export

[Nicolas Goaziou]

Hello,

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> Would using a different property---say, LATEX_LABEL---resolve your
> concerns?  This property could be explicitly documented as overriding
> Org's default labeling, with the value passed down directly to LaTeX.

I'd rather have a variable, e.g., `org-latex-custom-id-as-label'. When
this variable is non-nil, Org uses raw custom ID value instead of
auto-generated value for labels.

Its docstring should explain the limitations that are introduced when
using this variable, and in which cases it is interesting to enable it
(i.e, your use-case). IOW the docstring should be informative about the
trade-off.

So, it's basically your patch with an additional variable and its
docstring. Do you want to take care of it?

> The worrisome situation I have in mind is if I find that I eventually
> need to move away from Org to straight LaTeX.

OK. Since you had developed this idea in another paragraph, I thought
there was another reason. Never mind then.


Regards,

-- 
Nicolas Goaziou

Re: [patch] Support CUSTOM_ID property in latex export

[Richard Lawrence]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Richard Lawrence <richard.lawrence@berkeley.edu> writes:
>
>> Would using a different property---say, LATEX_LABEL---resolve your
>> concerns?  This property could be explicitly documented as overriding
>> Org's default labeling, with the value passed down directly to LaTeX.
>
> I'd rather have a variable, e.g., `org-latex-custom-id-as-label'. When
> this variable is non-nil, Org uses raw custom ID value instead of
> auto-generated value for labels.
>
> Its docstring should explain the limitations that are introduced when
> using this variable, and in which cases it is interesting to enable it
> (i.e, your use-case). IOW the docstring should be informative about the
> trade-off.

Ah, yes, that is more elegant.

> So, it's basically your patch with an additional variable and its
> docstring. Do you want to take care of it?

Sure, I can do this in the next couple of days.

Best,
Richard


(If possible, please encrypt your reply to me using my PGP key:
Key ID: CF6FA646
Fingerprint: 9969 43E1 CF6F A646.
See http://www.ocf.berkeley.edu/~rwl/encryption.html for more information.)

Re: [patch] Support CUSTOM_ID property in latex export

[Richard Lawrence]

[-- Attachment #1: Type: text/plain, Size: 773 bytes --]

Hi Nicolas and all,

Here's a new patch that adds a variable org-latex-custom-id-as-label to
control whether CUSTOM_ID should be used to generate labels during LaTeX
export.

Let me know what you think.  In particular, I wasn't sure if I should
provide more information in the defcustom statement beyond :group and
:type (like :package-version?).  Also, does the docstring represent the
trade-offs of using this variable well enough?

I wasn't sure how to get git format-patch to generate a single patch for
the changes between my branch and master (since there are now two
commits on my branch), so this was generated with git diff --patch.  If
you want me to send the commit message, etc. can you let me know how to
do this in whatever way is most convenient for you?  


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: custom-id.patch --]
[-- Type: text/x-diff, Size: 3292 bytes --]

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 5815874..df22768 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -375,6 +375,47 @@ which format headlines like for Org version prior to 8.0."
   :package-version '(Org . "8.0")
   :type 'function)
 
+(defcustom org-latex-custom-id-as-label nil
+   "Toggle use of CUSTOM_ID properties for generating section labels.
+
+If non-nil, Org will use the value of a headline's CUSTOM_ID
+property as the argument to the \label command for the LaTeX
+section corresponding to the headline.
+
+Setting this variable gives you control over how Org generates
+labels for sections during LaTeX export.  One reason to do this
+is that it allows you to refer to headlines using a single label
+both in Org's link syntax and in embedded LaTeX code.
+
+For example, when this variable is non-nil, a headline like this:
+
+  ** Some section
+     :PROPERTIES:
+     :CUSTOM_ID: sec:foo
+     :END:
+  This is section [[#sec:foo]].
+  #+BEGIN_LATEX
+  And this is still section \ref{sec:foo}.
+  #+END_LATEX
+
+will be exported to LaTeX as:
+
+  \subsection{Some section}
+  \label{sec:foo}
+  This is section \ref{sec:foo}.
+  And this is still section \ref{sec:foo}.
+
+Note, however, that when a headline defines a value for
+CUSTOM_ID, Org simply passes this value to \label unchanged.  You
+are responsible for ensuring that the value is a valid LaTeX
+\label key, that it is unique throughout the generated document,
+etc.
+ 
+For headlines that do not define the CUSTOM_ID property, Org will
+continue to use its default labeling scheme to generate labels
+and resolve links into section references."
+  :group 'org-export-latex
+  :type 'boolean)
 
 ;;;; Footnotes
 
@@ -1373,10 +1414,14 @@ holding contextual information."
 			       todo todo-type priority text tags))
 	   ;; Associate \label to the headline for internal links.
 	   (headline-label
-	    (format "\\label{sec-%s}\n"
-		    (mapconcat 'number-to-string
-			       (org-export-get-headline-number headline info)
-			       "-")))
+	    (let ((custom-label (and org-latex-custom-id-as-label
+				     (org-element-property :CUSTOM_ID headline))))
+	      (if custom-label
+		  (format "\\label{%s}\n" custom-label)
+		(format "\\label{sec-%s}\n"
+			(mapconcat 'number-to-string
+				   (org-export-get-headline-number headline info)
+				   "-")))))
 	   (pre-blanks
 	    (make-string (org-element-property :pre-blank headline) 10)))
       (if (or (not section-fmt) (org-export-low-level-p headline info))
@@ -1845,12 +1890,16 @@ INFO is a plist holding contextual information.  See
 	  ;; number.  Otherwise, display description or headline's
 	  ;; title.
 	  (headline
-	   (let ((label
-		  (format "sec-%s"
-			  (mapconcat
-			   'number-to-string
-			   (org-export-get-headline-number destination info)
-			   "-"))))
+	   (let* ((custom-label (and org-latex-custom-id-as-label
+				    (org-element-property :CUSTOM_ID destination)))
+		  (label
+		   (or
+		    custom-label
+		    (format "sec-%s"
+			    (mapconcat
+			     'number-to-string
+			     (org-export-get-headline-number destination info)
+			     "-")))))
 	     (if (and (plist-get info :section-numbers) (not desc))
 		 (format "\\ref{%s}" label)
 	       (format "\\hyperref[%s]{%s}" label

[-- Attachment #3: Type: text/plain, Size: 209 bytes --]



Best,
Richard


(If possible, please encrypt your reply to me using my PGP key:
Key ID: CF6FA646
Fingerprint: 9969 43E1 CF6F A646.
See http://www.ocf.berkeley.edu/~rwl/encryption.html for more information.)

Re: [patch] Support CUSTOM_ID property in latex export

[Nicolas Goaziou]

Hello,

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> Here's a new patch that adds a variable org-latex-custom-id-as-label to
> control whether CUSTOM_ID should be used to generate labels during LaTeX
> export.

Thank you for this.

> Let me know what you think.  In particular, I wasn't sure if I should
> provide more information in the defcustom statement beyond :group and
> :type (like :package-version?).

I think we should provide temporary (fake) values so we remember to
update them during the next merge.

   :version "24.5"
   :package-version '(Org . "8.3")

> Also, does the docstring represent the trade-offs of using this
> variable well enough?

The docstring is quite complete. I added a few suggestions. Also, you
need to escape backslashes (e.g., \\label).

> I wasn't sure how to get git format-patch to generate a single patch for
> the changes between my branch and master (since there are now two
> commits on my branch), so this was generated with git diff --patch.  If
> you want me to send the commit message, etc. can you let me know how to
> do this in whatever way is most convenient for you?

You can first merge the two commits on your branch into a single one
with "git rebase -i". Within Magit, it boils down to use key E on the
first of the two commits, then use key s on the second one and
eventually C-c C-c. This will merge them and give you a chance to edit
the final commit message.

Then you can send it with git format-patch.

> diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
> index 5815874..df22768 100644
> --- a/lisp/ox-latex.el
> +++ b/lisp/ox-latex.el
> @@ -375,6 +375,47 @@ which format headlines like for Org version prior to 8.0."
>    :package-version '(Org . "8.0")
>    :type 'function)
>  
> +(defcustom org-latex-custom-id-as-label nil
> +   "Toggle use of CUSTOM_ID properties for generating section labels.
> +
> +If non-nil, Org will use the value of a headline's CUSTOM_ID
> +property as the argument to the \label command for the LaTeX
> +section corresponding to the headline.

I may be useful to add a note about the default behaviour:

  By default, Org generates unique labels for all headlines.  When this
  variable is non-nil...

> +Setting this variable gives you control over how Org generates
> +labels for sections during LaTeX export.  One reason to do this
> +is that it allows you to refer to headlines using a single label
> +both in Org's link syntax and in embedded LaTeX code.
> +
> +For example, when this variable is non-nil, a headline like this:
> +
> +  ** Some section
> +     :PROPERTIES:
> +     :CUSTOM_ID: sec:foo
> +     :END:
> +  This is section [[#sec:foo]].
> +  #+BEGIN_LATEX
> +  And this is still section \ref{sec:foo}.
> +  #+END_LATEX
> +
> +will be exported to LaTeX as:
> +
> +  \subsection{Some section}
> +  \label{sec:foo}
> +  This is section \ref{sec:foo}.
> +  And this is still section \ref{sec:foo}.
> +
> +Note, however, that when a headline defines a value for
> +CUSTOM_ID, Org simply passes this value to \label unchanged.  You
> +are responsible for ensuring that the value is a valid LaTeX
> +\label key, that it is unique throughout the generated document,
> +etc.

I think you can remove "that it is unique throughout the generated
document", as it is already explained in the manual, and not specific to
this variable.

OTOH, it would be nice to specify what is a valid \label key (e.g.,
forbidden characters) and that the default value doesn't have this
limitation (otherwise, that wouldn't be much of a trade-off).

> +	    (let ((custom-label (and org-latex-custom-id-as-label
> +				     (org-element-property :CUSTOM_ID headline))))

There is one thing to consider here. We can define the new variable as
a back-end options, i.e., add

  (:latex-manual-id nil nil org-latex-custom-id-as-label)

in the back-end definition (the name of the property doesn't matter
much, you can change it).

This is not strictly necessary, but it allows, for example, to change
its value for specific projects (in the publishing sense) without
setting the variable globally.

If you think it is useful to do so,

  (and org-latex-custom-id-as-label

should become

  (and (plist-get info :latex-manual-id)

> +	   (let* ((custom-label (and org-latex-custom-id-as-label
> +				    (org-element-property :CUSTOM_ID destination)))

Ditto.


What do you think?


Regards,

-- 
Nicolas Goaziou

Re: [patch] Support CUSTOM_ID property in latex export

[Richard Lawrence]

Hi Nicolas and all,

Thanks for your feedback.  I have a couple of questions about your
comments:

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> I think you can remove "that it is unique throughout the generated
> document", as it is already explained in the manual, and not specific to
> this variable.

True, it is explained that CUSTOM_ID must be unique, but not that the
generated label must be.  I have changed this to:

"You are responsible for ensuring that the value is a valid LaTeX
\\label key, and that no other \\label commands with the same key appear
elsewhere in your document."

That seems clearer to me; it forbids e.g. introducing labels with the
same key on a #+LATEX: line.  Sound good?

> OTOH, it would be nice to specify what is a valid \label key (e.g.,
> forbidden characters) and that the default value doesn't have this
> limitation (otherwise, that wouldn't be much of a trade-off).
>
>> +	    (let ((custom-label (and org-latex-custom-id-as-label
>> +				     (org-element-property :CUSTOM_ID headline))))
>

I can't actually find a clear explanation anywhere of exactly what is
and isn't allowed in a label key.  All the LaTeX documentation seems to
just say:

"A key can consist of any sequence of letters, digits, or punctuation
characters. Upper and lowercase letters are different."

But clearly, the issue is what sort of "punctuation" is allowed.  ":"
and "_" are OK, but "%" and "$" aren't...is there a definitive list
somewhere I should refer to?  Maybe I should just say the user should
have a look at the regexp in org-export-solidify-link-text?

> There is one thing to consider here. We can define the new variable as
> a back-end options, i.e., add
>
>   (:latex-manual-id nil nil org-latex-custom-id-as-label)
>
>
> in the back-end definition (the name of the property doesn't matter
> much, you can change it).

>
> This is not strictly necessary, but it allows, for example, to change
> its value for specific projects (in the publishing sense) without
> setting the variable globally.
>
> If you think it is useful to do so,
>
>   (and org-latex-custom-id-as-label
>
> should become
>
>   (and (plist-get info :latex-manual-id)
>
>> +	   (let* ((custom-label (and org-latex-custom-id-as-label
>> +				    (org-element-property :CUSTOM_ID destination)))
>
> Ditto.
>

OK, that sounds like a good idea, but are these the only changes that
would be necessary?  Where should the name of the back-end option and
its relationship to this variable be documented?

Best,
Richard


(If possible, please encrypt your reply to me using my PGP key:
Key ID: CF6FA646
Fingerprint: 9969 43E1 CF6F A646.
See http://www.ocf.berkeley.edu/~rwl/encryption.html for more information.)

Re: [patch] Support CUSTOM_ID property in latex export

[Nicolas Goaziou]

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> True, it is explained that CUSTOM_ID must be unique, but not that the
> generated label must be.  I have changed this to:
>
> "You are responsible for ensuring that the value is a valid LaTeX
> \\label key, and that no other \\label commands with the same key appear
> elsewhere in your document."
>
> That seems clearer to me; it forbids e.g. introducing labels with the
> same key on a #+LATEX: line.  Sound good?

Fair enough.

> I can't actually find a clear explanation anywhere of exactly what is
> and isn't allowed in a label key.  All the LaTeX documentation seems to
> just say:
>
> "A key can consist of any sequence of letters, digits, or punctuation
> characters. Upper and lowercase letters are different."
>
> But clearly, the issue is what sort of "punctuation" is allowed.  ":"
> and "_" are OK, but "%" and "$" aren't...is there a definitive list
> somewhere I should refer to?

I don't know any.

> Maybe I should just say the user should have a look at the regexp in
> org-export-solidify-link-text?

Besides alphanumeric characters, this function allows "_", ".", "-" and
":". I think it is safe to assume only these punctuation characters are
allowed.

Also, the docstring should insist on the fact that this limitation only
applies when the variable is non-nil.

>> There is one thing to consider here. We can define the new variable as
>> a back-end options, i.e., add
>>
>>   (:latex-manual-id nil nil org-latex-custom-id-as-label)
>>
>>
>> in the back-end definition (the name of the property doesn't matter
>> much, you can change it).
>
>>
>> This is not strictly necessary, but it allows, for example, to change
>> its value for specific projects (in the publishing sense) without
>> setting the variable globally.
>>
>> If you think it is useful to do so,
>>
>>   (and org-latex-custom-id-as-label
>>
>> should become
>>
>>   (and (plist-get info :latex-manual-id)
>>
>>> +	   (let* ((custom-label (and org-latex-custom-id-as-label
>>> +				    (org-element-property :CUSTOM_ID destination)))
>>
>> Ditto.
>>
>
> OK, that sounds like a good idea, but are these the only changes that
> would be necessary?

Yes.

> Where should the name of the back-end option and its relationship to
> this variable be documented?

Probably in:

  (info "(org) Publishing options")

Unfortunately, only generic and html-specific options are described
there. We could add a LaTeX section (but it wouldn't contain much).


Regards,

-- 
Nicolas Goaziou

Re: [patch] Support CUSTOM_ID property in latex export

[Richard Lawrence]

[-- Attachment #1: Type: text/plain, Size: 204 bytes --]

Hi Nicolas and all,

OK, I think I've got a patch now that addresses everything you asked
for.  It is attached.  This has been quite a learning experience!  Let
me know if other changes are necessary.  


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Support using CUSTOM_ID to generate section labels --]
[-- Type: text/x-diff, Size: 5855 bytes --]

From 07bfc34a48858aa386c0416e592082610c913ef3 Mon Sep 17 00:00:00 2001
From: Richard Lawrence <richard.lawrence@berkeley.edu>
Date: Fri, 21 Feb 2014 11:22:08 -0800
Subject: [PATCH] Support using CUSTOM_ID property to generate section labels
 during LaTeX export
To: emacs-orgmode@gnu.org
Cc: n.goaziou@gmail.com

This allows the user to control how Org generates label keys for
headlines during LaTeX export, and thus to know their keys in advance.
This is useful for e.g. using \ref commands inside of embedded LaTeX
blocks.

* lisp/ox-latex.el:
  (defcustom org-latex-custom-id-as-label): Variable to control using
    CUSTOM_ID values as labels during export

  Backend definition: Add :latex-custom-id-labels option to backend's
    :options-alist, using value of org-latex-custom-id-as-label to
    define its behavior

  (org-latex-headline): Optionally generate label keys based on CUSTOM_ID,
    depending on value of :latex-custom-id-labels option

  (org-latex-link): Optionally generate refs based on CUSTOM_ID,
    depending on value of :latex-custom-id-labels option

This change was discussed in the following thread:
http://thread.gmane.org/gmane.emacs.orgmode/82392
---
 lisp/ox-latex.el |   84 +++++++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 73 insertions(+), 11 deletions(-)

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 5815874..4ae4bf4 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -106,7 +106,8 @@
 		   (:latex-class-options "LATEX_CLASS_OPTIONS" nil nil t)
 		   (:latex-header "LATEX_HEADER" nil nil newline)
 		   (:latex-header-extra "LATEX_HEADER_EXTRA" nil nil newline)
-		   (:latex-hyperref-p nil "texht" org-latex-with-hyperref t))
+		   (:latex-hyperref-p nil "texht" org-latex-with-hyperref t)
+		   (:latex-custom-id-labels nil nil org-latex-custom-id-as-label))
   :filters-alist '((:filter-options . org-latex-math-block-options-filter)
 		   (:filter-parse-tree . org-latex-math-block-tree-filter)))
 
@@ -375,6 +376,59 @@ which format headlines like for Org version prior to 8.0."
   :package-version '(Org . "8.0")
   :type 'function)
 
+(defcustom org-latex-custom-id-as-label nil
+   "Toggle use of CUSTOM_ID properties for generating section labels.
+
+When this variable is non-nil, Org will use the value of a
+headline's CUSTOM_ID property as the key for the \\label command
+for the LaTeX section corresponding to the headline.
+
+By default, Org generates its own internal section labels for all
+headlines during LaTeX export.  This process ensures that the
+\\label keys are unique and valid, but it means the keys are not
+available in advance of the export process.
+
+Setting this variable gives you control over how Org generates
+labels for sections during LaTeX export, so that you may know
+their keys in advance.  One reason to do this is that it allows
+you to refer to headlines using a single label both in Org's link
+syntax and in embedded LaTeX code.
+
+For example, when this variable is non-nil, a headline like this:
+
+  ** Some section
+     :PROPERTIES:
+     :CUSTOM_ID: sec:foo
+     :END:
+  This is section [[#sec:foo]].
+  #+BEGIN_LATEX
+  And this is still section \\ref{sec:foo}.
+  #+END_LATEX
+
+will be exported to LaTeX as:
+
+  \\subsection{Some section}
+  \\label{sec:foo}
+  This is section \\ref{sec:foo}.
+  And this is still section \\ref{sec:foo}.
+
+Note, however, that setting this variable introduces a limitation
+on the possible values for CUSTOM_ID.  When this variable is
+non-nil and a headline defines a CUSTOM_ID value, Org simply
+passes this value to \\label unchanged.  You are responsible for
+ensuring that the value is a valid LaTeX \\label key, and that no
+other \\label commands with the same key appear elsewhere in your
+document.  (Keys may contain letters, numbers, and the following
+punctuation: '_' '.' '-' ':'.)  There are no such limitations on
+CUSTOM_ID when this variable is nil.
+ 
+For headlines that do not define the CUSTOM_ID property, Org will
+continue to use its default labeling scheme to generate labels
+and resolve links into section references."
+  :group 'org-export-latex
+  :type 'boolean
+  :version "24.5"
+  :package-version '(Org . "8.3"))
 
 ;;;; Footnotes
 
@@ -1373,10 +1427,14 @@ holding contextual information."
 			       todo todo-type priority text tags))
 	   ;; Associate \label to the headline for internal links.
 	   (headline-label
-	    (format "\\label{sec-%s}\n"
-		    (mapconcat 'number-to-string
-			       (org-export-get-headline-number headline info)
-			       "-")))
+	    (let ((custom-label (and (plist-get info :latex-custom-id-labels)
+				     (org-element-property :CUSTOM_ID headline))))
+	      (if custom-label
+		  (format "\\label{%s}\n" custom-label)
+		(format "\\label{sec-%s}\n"
+			(mapconcat 'number-to-string
+				   (org-export-get-headline-number headline info)
+				   "-")))))
 	   (pre-blanks
 	    (make-string (org-element-property :pre-blank headline) 10)))
       (if (or (not section-fmt) (org-export-low-level-p headline info))
@@ -1845,12 +1903,16 @@ INFO is a plist holding contextual information.  See
 	  ;; number.  Otherwise, display description or headline's
 	  ;; title.
 	  (headline
-	   (let ((label
-		  (format "sec-%s"
-			  (mapconcat
-			   'number-to-string
-			   (org-export-get-headline-number destination info)
-			   "-"))))
+	   (let* ((custom-label (and (plist-get info :latex-custom-id-labels)
+				     (org-element-property :CUSTOM_ID destination)))
+		  (label
+		   (or
+		    custom-label
+		    (format "sec-%s"
+			    (mapconcat
+			     'number-to-string
+			     (org-export-get-headline-number destination info)
+			     "-")))))
 	     (if (and (plist-get info :section-numbers) (not desc))
 		 (format "\\ref{%s}" label)
 	       (format "\\hyperref[%s]{%s}" label
-- 
1.7.10.4


[-- Attachment #3: Type: text/plain, Size: 208 bytes --]


Best,
Richard


(If possible, please encrypt your reply to me using my PGP key:
Key ID: CF6FA646
Fingerprint: 9969 43E1 CF6F A646.
See http://www.ocf.berkeley.edu/~rwl/encryption.html for more information.)

Re: [patch] Support CUSTOM_ID property in latex export

[Nicolas Goaziou]

Hello,

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> OK, I think I've got a patch now that addresses everything you asked
> for.  It is attached.  This has been quite a learning experience!  Let
> me know if other changes are necessary.

It looks good. I applied it.

Thank you for your work.


Regards,

-- 
Nicolas Goaziou

Re: [patch] Support CUSTOM_ID property in latex export

[Achim Gratz]

Richard Lawrence writes:
> OK, I think I've got a patch now that addresses everything you asked
> for.  It is attached.  This has been quite a learning experience!  Let
> me know if other changes are necessary.

http://orgmode.org/worg/org-contribute.html#sec-5

Please use complete sentences in the changelog and do not add empty
lines when you continue the entry (or start them off again with a star
and the file part).  Read some of the existing commit messages to see
how this is done.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs