Bug: HTML export ignoring CUSTOM_ID properties

Bug: HTML export ignoring CUSTOM_ID properties

[T.F. Torrey]

Hello list,

With the latest from Git master, the HTML export ignores CUSTOM_ID
properties for subtrees.  I've seen list traffic that the names of the
export ID's are being changed, but this is not intentional, right?

Emacs : GNU Emacs 24.4.1 (i586-pc-linux-gnu, GTK+ Version 3.14.5) of
 2015-03-07 on binet, modified by Debian Package: Org-mode version
 8.3beta (release_8.3beta-1045-gd8494b @
 /home/tftorrey/T/org-mode/lisp/)

All the best,
Terry
-- 
T.F. Torrey

Re: Bug: HTML export ignoring CUSTOM_ID properties

[Rasmus]

Hi,

tftorrey@tftorrey.com (T.F. Torrey) writes:

> With the latest from Git master, the HTML export ignores CUSTOM_ID
> properties for subtrees.  I've seen list traffic that the names of the
> export ID's are being changed, but this is not intentional, right?

It doesn't ignore it, but it is translate to a generic anchor as needed.
Previously, both a generic anchor and the custom_id would be inserted in
html.  E.g. for a headline with id "h1":

    <h2 id="h1"><a id="sec-1" name="sec-1"></a>

The problem is that we don't know that h1 is unique.

However, in html custom_id serves as an important measure to facilitate
css customization, e.g. on a section level-basis.

Thus, I think it is a bug, unless there is a better way to allow
per-section css. I will look at this later unless somebody beets me to it.

—Rasmus

-- 
Send from my Emacs

Re: Bug: HTML export ignoring CUSTOM_ID properties

[T.F. Torrey]

Hi Rasmus,

As someone with all my personal and work files in Org, I sincerely
appreciate all the work you do to improve Org.  But...

Rasmus <rasmus@gmx.us> writes:

>> With the latest from Git master, the HTML export ignores CUSTOM_ID
>> properties for subtrees.  I've seen list traffic that the names of the
>> export ID's are being changed, but this is not intentional, right?
>
> It doesn't ignore it, but it is translate to a generic anchor as
> needed.

Isn't translating it to a generic anchor the same as ignoring it?
Without a CUSTOM_ID you get a generic anchor.  With a CUSTOM_ID you get
a generic anchor.

> Previously, both a generic anchor and the custom_id would be inserted in
> html.  E.g. for a headline with id "h1":
>
>     <h2 id="h1"><a id="sec-1" name="sec-1"></a>

Yes, that was a little clunky, but it did work, and the name attributes
could be suppressed.

> The problem is that we don't know that h1 is unique.
>
> However, in html custom_id serves as an important measure to facilitate
> css customization, e.g. on a section level-basis.

Yes, the user was responsible for making sure CUSTOM_ID's were unique,
but in practice that never amounted to what could be described as "the"
problem.

However, CUSTOM_ID's are not just CSS targets, they are link targets.

Consider this typical usage:

I have a page of my favorite presidents:

#+BEGIN_SRC org
* Favorite Presidents

** George Washington
:PROPERTIES:
:CUSTOM_ID: washington
:END:

He was tall.

** Bill Clinton
:PROPERTIES:
:CUSTOM_ID: clinton
:END:

He liked the ladies.
#+END_SRC

Because I know (knew) the id of the section about Clinton, I could link
to my page from another document outside Org with a link to
presidents.html#clinton.

This is how all the links between documents are done on my website, and
it's mostly how internal links in the HTML that becomes e-books is done.

However, with the new export code, many, perhaps all, of my links are
broken, because what comes out of the HTML exporter is this:

#+BEGIN_HTML
<div id="outline-container-sec:4" class="outline-2">
<h2 id="sec:4"><span class="section-number-2">1</span> Favorite Presidents</h2>
<div class="outline-text-2" id="text-1">
</div><div id="outline-container-sec:1" class="outline-3">
<h3 id="sec:1"><span class="section-number-3">1.1</span> George Washington</h3>
<div class="outline-text-3" id="text-washington">
<p>
He was tall.
</p>
</div>
</div>

<div id="outline-container-sec:2" class="outline-3">
<h3 id="sec:2"><span class="section-number-3">1.2</span> Bill Clinton</h3>
<div class="outline-text-3" id="text-clinton">
<p>
He liked the ladies.
</p>
</div>
</div>
#+END_HTML

Notice how my CUSTOM_ID's are no longer ID's at all.  And simply adding
"text-" to my CUSTOM_ID's is not an answer.  For one thing, CUSTOM_ID
exports should not change on the breeze of developer whims.  For
another, the ID should be attached to the heading, not the body of the
text; otherwise, a person following the link would have no idea if it
went to the Clinton section or not.

Note that this also breaks any CSS styling for the section with the
CUSTOM_ID (which I also use).  If I used a CUSTOM_ID because wanted a
swanky background for the heading saying "Bill Clinton", the current
export not only doesn't use that ID, it doesn't encompass the heading
with his name in.

> Thus, I think it is a bug, unless there is a better way to allow
> per-section css. I will look at this later unless somebody beets me to it.

Given the lack of outcry, I may be the only one using CUSTOM_ID's for
HTML export.  However, if usage is widespread enough and accidental
duplicates are a problem enough that this needs to be addressed,
wouldn't it be better for the exporter to simply report duplicate ID's
as they are found?

Finally, given that this doesn't appear to work at all in any form of
its intended usage, how did this even get committed to master?  Sure,
code in master may have bugs, but this is more than a bug; this is
unusable code that breaks code that worked.  Shouldn't it be developed
on a feature branch or in someone's private repo until it actually
works?

Unless there is a quick fix that restores external (non-Org-generated)
links to sections with CUSTOM_ID's, please revert these changes until
the development reaches a usable state.

All the best,
Terry
-- 
T.F. Torrey

Re: Bug: HTML export ignoring CUSTOM_ID properties

[Rasmus]

Hi,

tftorrey@tftorrey.com (T.F. Torrey) writes:

>>> With the latest from Git master, the HTML export ignores CUSTOM_ID
>>> properties for subtrees.  I've seen list traffic that the names of the
>>> export ID's are being changed, but this is not intentional, right?
>>
>> It doesn't ignore it, but it is translate to a generic anchor as
>> needed.
>
> Isn't translating it to a generic anchor the same as ignoring it?
> Without a CUSTOM_ID you get a generic anchor.  With a CUSTOM_ID you get
> a generic anchor.

You click it and it still works.  It's not ignored.  Within the syntax it
does the right thing.

> Because I know (knew) the id of the section about Clinton, I could link
> to my page from another document outside Org with a link to
> presidents.html#clinton.

Presumably you could use presidents.org::#clinton still.


> Notice how my CUSTOM_ID's are no longer ID's at all.  And simply adding
> "text-" to my CUSTOM_ID's is not an answer.  For one thing, CUSTOM_ID
> exports should not change on the breeze of developer whims.  For
> another, the ID should be attached to the heading, not the body of the
> text; otherwise, a person following the link would have no idea if it
> went to the Clinton section or not.
>
> Note that this also breaks any CSS styling for the section with the
> CUSTOM_ID (which I also use).  If I used a CUSTOM_ID because wanted a
> swanky background for the heading saying "Bill Clinton", the current
> export not only doesn't use that ID, it doesn't encompass the heading
> with his name in.

I have a half-baked patch that restores the old behavior, but I have to
think a bit more about it, and I won't have time today.  Nicolas might
also see it in the coming days.  E.g. ox-latex has org-latex--label.  The
question is whether there should be a solution in ox or whether each
backend should have org-BACKEND--label.

>> Thus, I think it is a bug, unless there is a better way to allow
>> per-section css. I will look at this later unless somebody beets me to it.
>
> Given the lack of outcry, I may be the only one using CUSTOM_ID's for
> HTML export.  However, if usage is widespread enough and accidental
> duplicates are a problem enough that this needs to be addressed,
> wouldn't it be better for the exporter to simply report duplicate ID's
> as they are found?

It was changed this week.

> Finally, given that this doesn't appear to work at all in any form of
> its intended usage, how did this even get committed to master?  Sure,
> code in master may have bugs, but this is more than a bug; this is
> unusable code that breaks code that worked.  Shouldn't it be developed
> on a feature branch or in someone's private repo until it actually
> works?

Master is where we develop things.  Nicolas made the change and he's off
this week.  Feel free to use Org 8.2.

> Unless there is a quick fix that restores external (non-Org-generated)
> links to sections with CUSTOM_ID's, please revert these changes until
> the development reaches a usable state.

Won't happen.

—Rasmus

-- 
I hear there's rumors on the, uh, Internets. . .

Re: Bug: HTML export ignoring CUSTOM_ID properties

[T.F. Torrey]

Hello Rasmus,

Rasmus <rasmus@gmx.us> writes:

>>>> With the latest from Git master, the HTML export ignores CUSTOM_ID
>>>> properties for subtrees.  I've seen list traffic that the names of the
>>>> export ID's are being changed, but this is not intentional, right?
>>>
>>> It doesn't ignore it, but it is translate to a generic anchor as
>>> needed.
>>
>> Isn't translating it to a generic anchor the same as ignoring it?
>> Without a CUSTOM_ID you get a generic anchor.  With a CUSTOM_ID you get
>> a generic anchor.
>
> You click it and it still works.  It's not ignored.  Within the syntax it
> does the right thing.
>
>> Because I know (knew) the id of the section about Clinton, I could link
>> to my page from another document outside Org with a link to
>> presidents.html#clinton.
>
> Presumably you could use presidents.org::#clinton still.

Links may work from inside Org, but the original intent of CUSTOM_ID was
to produce a stable ID for the HTML export that could be linked to from
outside Org.  With the changed functionality, all existing links to
presidents.html#clinton are broken, because the usage of CUSTOM_ID has
changed.  Any custom CSS is also broken, for the same reason.

With the new usage, CUSTOM_ID and ID have virtually the same
functionality.

>> Notice how my CUSTOM_ID's are no longer ID's at all.  And simply adding
>> "text-" to my CUSTOM_ID's is not an answer.  For one thing, CUSTOM_ID
>> exports should not change on the breeze of developer whims.  For
>> another, the ID should be attached to the heading, not the body of the
>> text; otherwise, a person following the link would have no idea if it
>> went to the Clinton section or not.
>>
>> Note that this also breaks any CSS styling for the section with the
>> CUSTOM_ID (which I also use).  If I used a CUSTOM_ID because wanted a
>> swanky background for the heading saying "Bill Clinton", the current
>> export not only doesn't use that ID, it doesn't encompass the heading
>> with his name in.
>
> I have a half-baked patch that restores the old behavior, but I have to
> think a bit more about it, and I won't have time today.  Nicolas might
> also see it in the coming days.  E.g. ox-latex has org-latex--label.  The
> question is whether there should be a solution in ox or whether each
> backend should have org-BACKEND--label.

As the current state is unusable, there would be no additional harm in
applying your half-baked patch.

>>> Thus, I think it is a bug, unless there is a better way to allow
>>> per-section css. I will look at this later unless somebody beets me to it.
>>
>> Given the lack of outcry, I may be the only one using CUSTOM_ID's for
>> HTML export.  However, if usage is widespread enough and accidental
>> duplicates are a problem enough that this needs to be addressed,
>> wouldn't it be better for the exporter to simply report duplicate ID's
>> as they are found?
>
> It was changed this week.

Either no one is using it according to its original intended
functionality, no one has republished their HTML documents with the
latest from git, or people have simply not noticed that their links are
broken.

>> Finally, given that this doesn't appear to work at all in any form of
>> its intended usage, how did this even get committed to master?  Sure,
>> code in master may have bugs, but this is more than a bug; this is
>> unusable code that breaks code that worked.  Shouldn't it be developed
>> on a feature branch or in someone's private repo until it actually
>> works?
>
> Master is where we develop things.

The code on master is *supposed* to work as advertised.  It identifies
itself as 8.3 beta.  Changes that break functionality in exploratory
ways are alpha changes.

> Nicolas made the change and he's off this week.  Feel free to use Org
> 8.2.

Ha ha.  There are many tools capable of exporting plain-text documents
to HTML with predictable and stable ID's.  If Org 8.3 is not going to be
one of them, using Org 8.2 is not a solution.

>> Unless there is a quick fix that restores external (non-Org-generated)
>> links to sections with CUSTOM_ID's, please revert these changes until
>> the development reaches a usable state.
>
> Won't happen.

As you said, they aren't your changes and it isn't your decision.

Best,
Terry
-- 
T.F. Torrey

Re: Bug: HTML export ignoring CUSTOM_ID properties

[Aaron Ecay]

Hi Terry,

I don’t think it’s necessary to be combative.  Rasmus is doing his best,
as are all the org developers.  Sometimes things break by accident –
that is the nature of beta software.  The problems you raised are
important.  Let’s try to figure out how to fix them.

You wrote:

> Links may work from inside Org, but the original intent of CUSTOM_ID was
> to produce a stable ID for the HTML export that could be linked to from
> outside Org.

I think this is true.  Looking at the pages in Worg, for example,
provides ample evidence of this strategy in action.

CUSTOM_ID is also sometimes needed for latex export
(cf. org-latex-prefer-user-labels).  It is important for IDs to be
unique, and to conform to certain format restrictions.  What if
CUSTOM_ID properties were checked for these requirements when exporting,
raising an error if they are not suitable and otherwise passing through
to the export output?  This would maintain CUSTOM_ID as an interface to
labeling systems outside org (latex \ref{}, html #anchor links, ...),
but would also make export more robust.  It’s also in line with recent
changes to raise export errors for undefined macros, unresolvable links,
etc.

--
Aaron Ecay

Re: Bug: HTML export ignoring CUSTOM_ID properties

[Nicolas Goaziou]

Hello,

tftorrey@tftorrey.com (T.F. Torrey) writes:

> Links may work from inside Org, but the original intent of CUSTOM_ID was
> to produce a stable ID for the HTML export that could be linked to from
> outside Org.  With the changed functionality, all existing links to
> presidents.html#clinton are broken, because the usage of CUSTOM_ID has
> changed.  Any custom CSS is also broken, for the same reason.
>
> With the new usage, CUSTOM_ID and ID have virtually the same
> functionality.

Good point.

> The code on master is *supposed* to work as advertised.  It identifies
> itself as 8.3 beta.  Changes that break functionality in exploratory
> ways are alpha changes.

"beta" is indeed misleading. I suggest to ignore it.

As Rasmus pointed out, master is where development happens. Some changes
introduced here may break Org. If one such change makes Org unusable for
you, you can easily revert Org to an earlier, working, commit, without
fuss. Of course, we appreciate if you report the problem encountered
beforehand.

> Ha ha.  There are many tools capable of exporting plain-text documents
> to HTML with predictable and stable ID's.

Considering that not any string is eligible as HTML id (e.g., forbidden
characters), either these tools are putting restrictions on chosen IDs
or they are lying.

> As you said, they aren't your changes and it isn't your decision.

I overlooked the problem in HTML and made a mistake. It happens, more
often than I would like. However, you are not required to be obnoxious
about it. It helps no one.

The problem should be fixed in 0449b785b4b58ec16e1aac126634de70eee519a4.
Thank you for reporting it.


Regards,

-- 
Nicolas Goaziou

Re: Bug: HTML export ignoring CUSTOM_ID properties

[T.F. Torrey]

Hello Aaron,

Aaron Ecay <aaronecay@gmail.com> writes:

> You wrote:
>
>> Links may work from inside Org, but the original intent of CUSTOM_ID was
>> to produce a stable ID for the HTML export that could be linked to from
>> outside Org.
>
> I think this is true.  Looking at the pages in Worg, for example,
> provides ample evidence of this strategy in action.

If this functionality were not provided by CUSTOM_ID, we would need to
invent a different property to serve the function.

> CUSTOM_ID is also sometimes needed for latex export
> (cf. org-latex-prefer-user-labels).  It is important for IDs to be
> unique, and to conform to certain format restrictions.  What if
> CUSTOM_ID properties were checked for these requirements when exporting,
> raising an error if they are not suitable and otherwise passing through
> to the export output?  This would maintain CUSTOM_ID as an interface to
> labeling systems outside org (latex \ref{}, html #anchor links, ...),
> but would also make export more robust.  It’s also in line with recent
> changes to raise export errors for undefined macros, unresolvable links,
> etc.

This is what I suggested in an earlier e-mail (which was unreasonably
cordial, yet summarily dismissed in a way that made me angry, and which
was sent in response to the summary dismissal of my polite bug report).

Does it warrant an error, though?  I've been using the functionality
extensively for years, and I can remember only one time that I had an
inadvertent problem caused by a duplicate CUSTOM_ID.  A simple warning
would have headed that off.

On the contrary, if the show is going to
stop with an error, I will have to make sure my documents meet someone
else's idea of what is useful.  For instance, many of my documents get
exported together (web pages with #news sections, for instance), and it
would be unnecessarily inconvenient making the CUSTOM_ID unique across
all agenda files.  Someone else, however, might want that.

I'm wondering why this is being addressed at all.  Is this actually a
problem for someone?  Or is this just another attempt to save
hypothetical users' feet?  Or is it just a side-effect of other
refactoring?

Best,
Terry
-- 
T.F. Torrey

Re: Bug: HTML export ignoring CUSTOM_ID properties

[T.F. Torrey]

Hello Nicolas,

First, thank you for your incredible work on Org.  I hope you enjoyed
your days off, but I have to admit that your announcement that you were
taking the week off worried me.  I seem to remember we lost Carsten and
Bastien soon after they took a week off.  When (if?) you finally get
burned out and leave, all Org users will feel the loss.

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> "beta" is indeed misleading. I suggest to ignore it.
>
> As Rasmus pointed out, master is where development happens. Some changes
> introduced here may break Org. If one such change makes Org unusable for
> you, you can easily revert Org to an earlier, working, commit, without
> fuss. Of course, we appreciate if you report the problem encountered
> beforehand.

Yes, changes on master can and do occasionally break Org, but they are
*supposed* to work.  You wouldn't leave the spreadsheet functionality in
an unusable state and just tell people to use 8.2.

But yes, it should be a simple matter to revert the commit that caused
the problem for me until the problem can be addressed.  That was the
second thing I looked at.  However, the place where this change happened
is not obvious in the git logs.  I still don't know where it came from.

I did see that most (maybe all) of your changes are accompanied by
tests.  I'm not very familiar with the testing.  Are the tests
restricted to merely checking if the code explodes?  Or is there a
possibility to test whether the code does what it is supposed to do?
Presumably this change that broke functionality passed its tests just
fine.

>> Ha ha.  There are many tools capable of exporting plain-text documents
>> to HTML with predictable and stable ID's.
>
> Considering that not any string is eligible as HTML id (e.g., forbidden
> characters), either these tools are putting restrictions on chosen IDs
> or they are lying.

They aren't lying because they don't claim to allow only valid ID's.
They produce valid ID's on their own, but when a user calls for a
specific ID (the {#clinton} construct in Markdown comes to mind), they
just do what the user tells them to do.  Which is a good thing.

I sense there is a difference of philosophy at play here.

In my view, the purpose of tools such as Org that convert documents to
HTML is to do what the user tells them to do, even if that means
creating invalid HTML.  On many occasions in the past, and probably some
in the present and the future, I have used conversion tools to produce
technically invalid HTML as in intermediate format to be further
processed by XSLT to a final product.  A tool that refused to produce
invalid HTML would be no help at all.  In fact, I'm not aware of any
tool that disallows that except maybe for some beginner level things.

On the contrary, the slant of Org's development lately seems to be first
to make sure users don't make any mistakes, and then to follow their
instructions.

>> As you said, they aren't your changes and it isn't your decision.
>
> I overlooked the problem in HTML and made a mistake. It happens, more
> often than I would like. However, you are not required to be obnoxious
> about it. It helps no one.

Your mistakes are very rare, and your work is sincerely appreciated.  I
think your comment about my response is out of context, and I'm not sure
your final statement is true.  My polite comments were summarily
dismissed, but now anyone who depended on CUSTOM_ID has been helped.

> The problem should be fixed in 0449b785b4b58ec16e1aac126634de70eee519a4.
> Thank you for reporting it.

Thank you for your prompt action, but can I ask what you mean by
"fixed"?  Have you decided to revert CUSTOM_ID to its previous
functionality?  Are you still planning on changing its functionality
and/or meaning?  Are you still planning on throwing warnings or errors in
the event of duplicate or invalid CUSTOM_ID's?

Best,
Terry
-- 
T.F. Torrey

Re: Bug: HTML export ignoring CUSTOM_ID properties

[Nicolas Goaziou]

Hello,

tftorrey@tftorrey.com (T.F. Torrey) writes:

> Yes, changes on master can and do occasionally break Org, but they are
> *supposed* to work.  You wouldn't leave the spreadsheet functionality in
> an unusable state and just tell people to use 8.2.

CUSTOM_ID was *supposed* to work after this change. Actually, it worked,
except in two use cases (custom CSS and linking from outside Org) in one
specific export back-end.

> But yes, it should be a simple matter to revert the commit that caused
> the problem for me until the problem can be addressed.  That was the
> second thing I looked at.  However, the place where this change happened
> is not obvious in the git logs.  I still don't know where it came
> from.

AFAICT, a single commit affected "ox-html" significantly lately:
459033265295723cbfb0fccb3577acbfdc9d0285. Anyway, you may also bisect to
find the problematic commit; doing so might also improve the bug report.

> I did see that most (maybe all) of your changes are accompanied by
> tests.  I'm not very familiar with the testing.  Are the tests
> restricted to merely checking if the code explodes?

You cannot test everything. Moreover, we do not write tests specific to
export back-ends (i.e., "ox.el" is extensively tested but not
"ox-latex.el" or, in this case "ox-html.el").

> They aren't lying because they don't claim to allow only valid ID's.
> They produce valid ID's on their own, but when a user calls for a
> specific ID (the {#clinton} construct in Markdown comes to mind), they
> just do what the user tells them to do.  Which is a good thing.

No, it isn't a good thing in all cases. 

In some circumstances, the user creates an ID possibly without knowing
about it, e.g. with targets and radio targets. How Org handles these
objects is an implementation detail, and shouldn't be thrown at the face
of users.

However, it happened recently to some user (see
<http://permalink.gmane.org/gmane.emacs.orgmode/96841>). The culprit was
`org-export-solidify-link-text', a very wrong function. I didn't explain
the problem because of its internal nature, but it seems I should have.

For the record, `org-export-solidify-link-text' was just turning any
character non alphanumerical and not among "_.-:" into a hyphen. This
function was also applied to many things, including CUSTOM_ID. IOW,
"clinton" became "clinton", but also both "clintén" and "clintàn" became
"clint-n". So, basically CUSTOM_ID were already broken for anyone using
non ASCII characters.

Of course, if it had been only about CUSTOM_ID, the solution would have
been to simply remove the call to `org-export-solidify-link-text' and
let the user handle it. But there are also radio targets, and to
a lesser extent, targets, which are expected to be human-readable. E.g.,
if, for some reason, I need to write <<<éé>>> and <<<àà>>> in some
document, I certainly don't want them to both refer to "--" ID. Note
they could also be <<<some "text">>> or <<<a&b%c>>>, and could be
exported through HTML, LaTeX, etc. all with different expectations for
their IDs.

Since there is no reason to impose restrictions about them on the user
and the fact they are Org specific features, using an internal reference
is fine in this case. However, `org-export-solidify-link-text' is not
the answer, as it is not bijective. Therefore, I implemented
`org-export-get-reference', which relies on a very basic and portable
set of characters (alphanumeric ones) while still ensuring stability of
references. Of course, predictability is not achieved, but it wasn't
before either, excepted in the most simple cases.

Furthermore it isn't a problem in practice since users are not expected
to (and shouldn't) rely on these references externally. Here, CUSTOM_ID
in the context of HTML export is the exception, not the rule.

I hope this clarifies the purpose of the change.

> In my view, the purpose of tools such as Org that convert documents to
> HTML is to do what the user tells them to do, even if that means
> creating invalid HTML.  On many occasions in the past, and probably some
> in the present and the future, I have used conversion tools to produce
> technically invalid HTML as in intermediate format to be further
> processed by XSLT to a final product.  A tool that refused to produce
> invalid HTML would be no help at all.  In fact, I'm not aware of any
> tool that disallows that except maybe for some beginner level things.
>
> On the contrary, the slant of Org's development lately seems to be first
> to make sure users don't make any mistakes, and then to follow their
> instructions.

Again, this change isn't about protecting users, but fixing an incorrect
function, alas very much used across code base and affecting many users.

>> I overlooked the problem in HTML and made a mistake. It happens, more
>> often than I would like. However, you are not required to be obnoxious
>> about it. It helps no one.
>
> Your mistakes are very rare, and your work is sincerely appreciated.  I
> think your comment about my response is out of context, and I'm not sure
> your final statement is true.  My polite comments were summarily
> dismissed, but now anyone who depended on CUSTOM_ID has been helped.

Communication, and especially written one, is sometimes misleading,
nevertheless I found the tone of your answer unnecessarily harsh.

In any case, your last message didn't trigger the fix. The first one
would have been sufficient, but I needed to get back online first.

> Thank you for your prompt action, but can I ask what you mean by
> "fixed"?  Have you decided to revert CUSTOM_ID to its previous
> functionality?  Are you still planning on changing its functionality
> and/or meaning?

I didn't plan to change CUSTOM_ID functionality in the first place.
I fixed a deeply internal function. This fix implied invasive changes,
and CUSTOM_ID was merely affected by side effect.

> Are you still planning on throwing warnings or errors in the event of
> duplicate or invalid CUSTOM_ID's?

I have implemented something related recently, but I'll comment about
this in another thread.


Regards,

-- 
Nicolas Goaziou

Re: Bug: HTML export ignoring CUSTOM_ID properties

[Rasmus]

Hi,

tftorrey@tftorrey.com (T.F. Torrey) writes:

> First, thank you for your incredible work on Org.  I hope you enjoyed
> your days off, but I have to admit that your announcement that you were
> taking the week off worried me.  I seem to remember we lost Carsten and
> Bastien soon after they took a week off.  When (if?) you finally get
> burned out and leave, all Org users will feel the loss.

This remark is uncalled for and very unfair.

> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
>> "beta" is indeed misleading. I suggest to ignore it.
>>
>> As Rasmus pointed out, master is where development happens. Some changes
>> introduced here may break Org. If one such change makes Org unusable for
>> you, you can easily revert Org to an earlier, working, commit, without
>> fuss. Of course, we appreciate if you report the problem encountered
>> beforehand.
>
> Yes, changes on master can and do occasionally break Org, but they are
> *supposed* to work.  You wouldn't leave the spreadsheet functionality in
> an unusable state and just tell people to use 8.2.

To the best of my knowledge nobody makes a living from developing Org.
While everybody tries hard to make Org great, you cannot expect people to
put in work at the time *you* would prefer.  That being said, you are free
to hire somebody to make changes for you.  That way you could gain control
over the timing of particular fixes.

If you use git version of Org it is advisable to learn about "git
checkout".  In the following message I mention the range of changes that
also affect the change that has enraged you so greatly:

       http://permalink.gmane.org/gmane.emacs.orgmode/96890

> But yes, it should be a simple matter to revert the commit that caused
> the problem for me until the problem can be addressed.  That was the
> second thing I looked at.  However, the place where this change happened
> is not obvious in the git logs.  I still don't know where it came from.

See above.  In addition to the tips Nicolas provided you, it is good to
familiarize yourself with recent activity on the list.

> On the contrary, the slant of Org's development lately seems to be first
> to make sure users don't make any mistakes, and then to follow their
> instructions.

{{Citation needed}}

>>> As you said, they aren't your changes and it isn't your decision.
>>
>> I overlooked the problem in HTML and made a mistake. It happens, more
>> often than I would like. However, you are not required to be obnoxious
>> about it. It helps no one.
>
> Your mistakes are very rare, and your work is sincerely appreciated.  I
> think your comment about my response is out of context, and I'm not sure
> your final statement is true.  My polite comments were summarily
> dismissed, but now anyone who depended on CUSTOM_ID has been helped.

Nobody dismissed your "polite" comments:

    However, in html custom_id serves as an important measure to
    facilitate css customization, e.g. on a section level-basis.

    Thus, I think it is a bug, unless there is a better way to allow
    per-section css. I will look at this later unless somebody beets me to
    it.
    
    <http://permalink.gmane.org/gmane.emacs.orgmode/96932>

>> The problem should be fixed in 0449b785b4b58ec16e1aac126634de70eee519a4.
>> Thank you for reporting it.
>
> Thank you for your prompt action, but can I ask what you mean by
> "fixed"?  Have you decided to revert CUSTOM_ID to its previous
> functionality?  Are you still planning on changing its functionality
> and/or meaning?  Are you still planning on throwing warnings or errors in
> the event of duplicate or invalid CUSTOM_ID's?

Other commands that you may study further are "git log" and "git show".
I often use the following page as well:

   http://orgmode.org/cgit.cgi/org-mode.git

I hope this helps.

—Rasmus

-- 
Spil noget med Slayer!

Re: Bug: HTML export ignoring CUSTOM_ID properties

[T.F. Torrey]

Hello Nicolas,

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

[... 77 lines omitted ...]

> I hope this clarifies the purpose of the change.

It does.  Thank you very much for taking the time.

[... 41 lines omitted ...]

>> Are you still planning on throwing warnings or errors in the event of
>> duplicate or invalid CUSTOM_ID's?
>
> I have implemented something related recently, but I'll comment about
> this in another thread.

I look forward to reading it.

Best,
Terry
-- 
T.F. Torrey

Re: Bug: HTML export ignoring CUSTOM_ID properties

[Daniel Clemente]

El Sat, 18 Apr 2015 15:57:15 +0200 Rasmus va escriure:
> >
> > Note that this also breaks any CSS styling for the section with the
> > CUSTOM_ID (which I also use).  If I used a CUSTOM_ID because wanted a
> > swanky background for the heading saying "Bill Clinton", the current
> > export not only doesn't use that ID, it doesn't encompass the heading
> > with his name in.
> 

I was also bit by that CUSTOM_ID change. I'm not using CSS to style an element, like #clinton in your example, but in some places I want to define an ID #clinton so that I can build a URL like myweb.com/presidents/#clinton (instead of, say, /presidents/#orgheadline515).


I also saw this change (diff format):

-<div id="outline-container-sec-1-4-3-1-2" class="outline-6">
-<h6 id="sec-1-4-3-1-2"><span class="section-number-6">1.4.3.1.2</span> tercer error con stash</h6>
+<div id="outline-container-orgheadline129" class="outline-6">
+<h6 id="orgheadline129"><span class="section-number-6">1.4.3.1.2</span> tercer error con stash</h6>

The #sec-1-4-3-1-2 format was better. If I delete section 1.4.3.1.2, section 1.5 is still called 1.5, that's good. With a flat numbering like #orgheadline129, deleting any headline would change the IDs of all headers after it. And what's the use of IDs if they're not permanent?

Re: Bug: HTML export ignoring CUSTOM_ID properties

[Nicolas Goaziou]

Hello,

Daniel Clemente <n142857@gmail.com> writes:

> I also saw this change (diff format):
>
> -<div id="outline-container-sec-1-4-3-1-2" class="outline-6">
> -<h6 id="sec-1-4-3-1-2"><span class="section-number-6">1.4.3.1.2</span> tercer error con stash</h6>
> +<div id="outline-container-orgheadline129" class="outline-6">
> +<h6 id="orgheadline129"><span class="section-number-6">1.4.3.1.2</span> tercer error con stash</h6>
>
> The #sec-1-4-3-1-2 format was better. If I delete section 1.4.3.1.2,
> section 1.5 is still called 1.5, that's good.

And if you delete section 1.4, section 1.5 is no longer called 1.5. So
you need to update IDs most times you change headline numbering. I don't
think it is really better than the current state.

> With a flat numbering like #orgheadline129, deleting any headline
> would change the IDs of all headers after it.

True. This was already the case before, as explained above, or when
using unnumbered headlines.

> And what's the use of IDs if they're not permanent?

The point is that Org knows the ID associated to a given headline, and
provides tools to access them (`org-export-get-reference' for back-end
developers, [[*tercer error con stash]] for users).

I'm not against re-using CUSTOM_ID as in "outline-container-CUSTOM_ID",
but in the general case, where no CUSTOM_ID is defined, the current
state is fine by me.


Regards,

-- 
Nicolas Goaziou

Re: Bug: HTML export ignoring CUSTOM_ID properties

[Daniel Clemente]

El Tue, 21 Apr 2015 09:25:13 +0200 Nicolas Goaziou va escriure:
> 
> Hello,
> 
> Daniel Clemente <n142857@gmail.com> writes:
> 
> > I also saw this change (diff format):
> >
> > -<div id="outline-container-sec-1-4-3-1-2" class="outline-6">
> > -<h6 id="sec-1-4-3-1-2"><span class="section-number-6">1.4.3.1.2</span> tercer error con stash</h6>
> > +<div id="outline-container-orgheadline129" class="outline-6">
> > +<h6 id="orgheadline129"><span class="section-number-6">1.4.3.1.2</span> tercer error con stash</h6>
> >
> > The #sec-1-4-3-1-2 format was better. If I delete section 1.4.3.1.2,
> > section 1.5 is still called 1.5, that's good.
> 
> And if you delete section 1.4, section 1.5 is no longer called 1.5. So
> you need to update IDs most times you change headline numbering. I don't
> think it is really better than the current state.
> 

It's good to minimize the number of changes after each export. I prefer to review 10 changes rather than 200. I cannot just forget and let the automatic export work its way; I need to review it because in every export I find many different export bugs or unexpected features.

While the two systems work, I don't see either why the #orgheadline129 system is better than the #sec-1-4-3-1-2. Not important enough to justify a breaking change either.


> > And what's the use of IDs if they're not permanent?
> 
> The point is that Org knows the ID associated to a given headline, and
> provides tools to access them (`org-export-get-reference' for back-end
> developers, [[*tercer error con stash]] for users).
> 

IDs are not only internal. CSS knows about it, JS too, and URLs can be built and shared through the web that include org IDs (myweb.com/somedoc.html#someid).

Re: Bug: HTML export ignoring CUSTOM_ID properties

[Nicolas Goaziou]

Daniel Clemente <n142857@gmail.com> writes:

> While the two systems work, I don't see either why the #orgheadline129
> system is better than the #sec-1-4-3-1-2.

Actually is is better because it is consistent. "sec-1-4-3-1-2" is
almost as fragile and doesn't play nice with unnumbered headlines.

However, this is not the reason why this change was introduced.
I explained it already.

> Not important enough to justify a breaking change either.

This is not really a breaking change. AFAIK, "sec-x.x.x" labelling was
never made public. However, you can use CUSTOM_ID to enforce some label
and then refer to it.

> IDs are not only internal. CSS knows about it, JS too, and URLs can be
> built and shared through the web that include org IDs
> (myweb.com/somedoc.html#someid).

ID not specified by the user are internal. "sec-x-x" is an internal
scheme. You can still use CUSTOM_ID for that.

Of course, it is also possible to add an anchor with id "sec-x-x" to
numbered headlines only (maybe as an opt-in so newcomers can at least
read about it). However, I suspect that a filter can do as well.


Regards,