Multi-line links

Multi-line links

[Tobias Getzner]

Dear org-mode list,

It appears that multi-line links in org-mode will only work with if
they span no more than two lines:

http://comments.gmane.org/gmane.emacs.orgmode/19919

I was wondering if this hard-coded limit could be removed in future
versions, also given the fact that the syntax spec. does not preclude
newline characters from appearing within links.

In my case, I was trying to set-up the org-ref extension so that it
would handle multi-line links like the following, which makes them more
legible than a single long line.

[[citet:green1994hybridreasoningmodel,
  green1994generatingindirectanswers,
  green1992conversationalimplicaturesindirect]]

The problem with this is that org-mode will only invoke the link
handler for the first to lines, apparently due to the hard-coded limit,
and for the third line, org-mode will only report «no link found».

Since org-ref parses the links internally (i. e., the link-open action
is different for each bibkey), this cannot be worked around by clicking
on the first two lines.

Best regards,
Tobias

Re: Multi-line links

[Tobias Getzner]

On Wed, 2014-07-02 at 12:39 +0200, Tobias Getzner wrote:
> It appears that multi-line links in org-mode will only work with if
> they span no more than two lines:
> 
> http://comments.gmane.org/gmane.emacs.orgmode/19919

One additional issue I’ve hit upon is that the link :path returned by
org-mode is also truncated after a newline, while the :raw-link is
correctly returned in full.

Regards,
T.

Re: Multi-line links

[Nicolas Goaziou]

Hello,

Tobias Getzner <tobias.getzner@gmx.de> writes:

> On Wed, 2014-07-02 at 12:39 +0200, Tobias Getzner wrote:
>> It appears that multi-line links in org-mode will only work with if
>> they span no more than two lines:
>> 
>> http://comments.gmane.org/gmane.emacs.orgmode/19919
>
> One additional issue I’ve hit upon is that the link :path returned by
> org-mode is also truncated after a newline, while the :raw-link is
> correctly returned in full.

Could you provide an example?

Note that Org links do not allow newline characters in the "link" part.
Is it really an issue or a limitation?


Regards,

-- 
Nicolas Goaziou

Re: Multi-line links

[Tobias Getzner]

Hello Nicolas,

On Sun, 2014-07-06 at 09:23 +0200, Nicolas Goaziou wrote:
> Tobias Getzner <tobias.getzner@gmx.de> writes:
> > On Wed, 2014-07-02 at 12:39 +0200, Tobias Getzner wrote:
> > One additional issue I’ve hit upon is that the link :path returned by
> > org-mode is also truncated after a newline, while the :raw-link is
> > correctly returned in full.
> 
> Could you provide an example?

My original example also applies to this issue.

[[citet:green1994hybridreasoningmodel,
  green1994generatingindirectanswers,
  green1992conversationalimplicaturesindirect]]

Clicking on the first line in the above link, the returned :path will
be just be «green1994hybridreasoningmodel,», i. e., the :path is
truncated after the newline. The raw-path will be correct, however,
i. e., «citet:green1994hybrid…,\ngreen1994generating…,\ngreen1992…».

Apart from the truncation issue, the link itself can be triggered only
via the first two lines, while the last line yields «no link found»,
despite the fact that the last line is syntax-highlighted and
clickable.

> Note that Org links do not allow newline characters in the "link" part.
> Is it really an issue or a limitation?

According to the syntax spec, newlines should be allowed both in the
link path and in the description, but I realize the syntax spec is kind
of after-the-fact. On the other hand, the implementation seems to
already support n-line links for the arbitrary value of 2 (with the
caveat that the :path value currently breaks).

The reason why I consider it a limitation is cases like the one I
initially outlined. An extension like org-ref might want to internally
parse the link path, and it makes sense to allow newlines as separators
here, so that the links remain legible and so one can have hard
line-breaks at 80 cols. In particular, BibTeX-based links can become
very long. If one prefers hard line-breaks, these links will impair the
readability of the document.

On the other hand, it’s an inconsistency that these links will be
correctly syntax-highlighted and clickable on every line, with the
handler then breaking on lines > 2 and with a truncated :path. 

Since link paths and descriptions cannot contain square brackets, would
it be possible to rely just on these to terminate link parsing?

In an earlier post of yours, you seem to have identified the relevant
code responsible for this:

http://lists.gnu.org/archive/html/emacs-orgmode/2011-03/msg00532.html

If there is some strong reason for a hard-coded limit, would it be
possible to expose the limit as a user-definable variable, and to fix
the :path truncation issue?

Best regards,
Tobias

Re: Multi-line links

[Nicolas Goaziou]

Hello,

Tobias Getzner <tobias.getzner@gmx.de> writes:

> My original example also applies to this issue.
>
> [[citet:green1994hybridreasoningmodel,
>   green1994generatingindirectanswers,
>   green1992conversationalimplicaturesindirect]]
>
> Clicking on the first line in the above link, the returned :path will
> be just be «green1994hybridreasoningmodel,», i. e., the :path is
> truncated after the newline. The raw-path will be correct, however,
> i. e., «citet:green1994hybrid…,\ngreen1994generating…,\ngreen1992…».

This is because `org-element-link-parser' uses internally
`org-link-re-with-space3' for analysis.

> According to the syntax spec, newlines should be allowed both in the
> link path and in the description, but I realize the syntax spec is kind
> of after-the-fact.

Actually, it's a mistake in the syntax specifications. No newlines
characters are allowed in paths (see for example `org-non-link-chars')
at the moment.

> Since link paths and descriptions cannot contain square brackets, would
> it be possible to rely just on these to terminate link parsing?

Links are not limited to square bracket syntax. There are also angle
links (e.g., <http://orgmode.org> and plain links: http://orgmode.org).

> If there is some strong reason for a hard-coded limit, would it be
> possible to expose the limit as a user-definable variable, and to fix
> the :path truncation issue?

I don't think there is a strong reason for that limitation.

RFC 3986 (Appendix C) suggests how to handle multi-lines URI. We could
use it to handle such links.

Note that there is more than "org-element.el" to change though (e.g.,
`org-make-link-regexps') and some parts of Org relying only on regexps
to extract the path, may not work properly with such links.

Do you want to work on such a patch?


Regards,

-- 
Nicolas Goaziou

Re: Multi-line links

[Tobias Getzner]

Nicolas, thank you for reply. Due to some health issues I’m only
responding now.

On So, 2014-07-06 at 21:28 +0200, Nicolas Goaziou wrote:
> Tobias Getzner <tobias.getzner@gmx.de> writes:
> 
> > If there is some strong reason for a hard-coded limit, would it be
> > possible to expose the limit as a user-definable variable, and to fix
> > the :path truncation issue?
> 
> I don't think there is a strong reason for that limitation.
> 
> RFC 3986 (Appendix C) suggests how to handle multi-lines URI. We could
> use it to handle such links.
> 
> Note that there is more than "org-element.el" to change though (e.g.,
> `org-make-link-regexps') and some parts of Org relying only on regexps
> to extract the path, may not work properly with such links.
> 
> Do you want to work on such a patch?

I’m afraid given my nigh complete inexperience with elisp it is
unlikely I’ll be able to provide a good, generic solution for this. My
original hope was that for someone familiar with the code, there
wouldn’t be many changes needed, since multi-line descriptions seemed
to be working (though only for < 3 lines) and since the raw-path seemed
to contain the needed line-breaks already. But if I understand you
correctly, the complication is that the path parsing is the same across
different link types, and so one cannot simply fix it up so that :path
is equivalent to :raw-path, just with the link prefix stripped of?

When I experimented with making multi-line BibTeX links work, I tried
working around the issue of the path being truncated by using raw-path
instead. But it seems that this breaks when doing export, since the
handler seems to only be given the normal (truncated) path as an
argument. Is there any way to make the exporter call the export handler
with the raw-path as an argument, currently?

Best regards,
T.

Re: Multi-line links

[Nicolas Goaziou]

Hello,

Tobias Getzner <tobias.getzner@gmx.de> writes:

> I’m afraid given my nigh complete inexperience with elisp it is
> unlikely I’ll be able to provide a good, generic solution for this. My
> original hope was that for someone familiar with the code, there
> wouldn’t be many changes needed,

In the long run, I'm pretty sure the project will benefit more if you
bring in your own, temporary, inexperience and start hacking from there.

> since multi-line descriptions seemed to be working (though only for <
> 3 lines) and since the raw-path seemed to contain the needed
> line-breaks already. But if I understand you correctly, the
> complication is that the path parsing is the same across different
> link types, and so one cannot simply fix it up so that :path is
> equivalent to :raw-path, just with the link prefix stripped of?

I fixed it in maint. "org-element.el" used an inadequate regexp to
analyze the path. Could you confirm it?


Regards,

-- 
Nicolas Goaziou

Re: Multi-line links

[Tobias Getzner]

Hello Nicolas,

On Sa, 2014-07-26 at 15:32 +0200, Nicolas Goaziou wrote:
> In the long run, I'm pretty sure the project will benefit more if you
> bring in your own, temporary, inexperience and start hacking from there.

I’ll definitely look into this once I get a hang of Elisp. When I
(unsuccessfully) tried to figure out how links are handled, it occurred
to me some more inline docs would be useful; but then again maybe once
one is used to Elisp the code becomes more transparent. When I have
some time for another attempt to wrap my head around the code, maybe
I could try adding in some comments as I go along.

> > since multi-line descriptions seemed to be working (though only for <
> > 3 lines) and since the raw-path seemed to contain the needed
> > line-breaks already. But if I understand you correctly, the
> > complication is that the path parsing is the same across different
> > link types, and so one cannot simply fix it up so that :path is
> > equivalent to :raw-path, just with the link prefix stripped of?
> 
> I fixed it in maint. "org-element.el" used an inadequate regexp to
> analyze the path. Could you confirm it?

If I understand correctly, this change addresses the issue of truncated
«path» and «raw-link»? These seem to return the expected results for
multi-line links now. Wonderful! Thank you very much for this!

The syntax highlighting seems to also work, though it requires
refreshing the buffer display.

The only remaining issue would be that these links can only be
triggered from the first line. Clicking on another line will yield «No
link found», so I cannot get org-ref to return the BibTeX entry
appropriate for the line from which the action is triggered.

Strangely, when I inspect the other lines using org-element-context,
the link type is correctly shown, even though it’s a few lines before
point. But when issuing org-open-at-point, it seems no handler is found
when point is not on the first line of a multi-line link.

Best regards,
Tobias

Re: Multi-line links

[Nicolas Goaziou]

Hello,

Tobias Getzner <tobias.getzner@gmx.de> writes:

> The only remaining issue would be that these links can only be
> triggered from the first line. Clicking on another line will yield «No
> link found», so I cannot get org-ref to return the BibTeX entry
> appropriate for the line from which the action is triggered.

`org-open-at-point' was rewritten in master branch a while ago, so it
should be fixed in Org 8.3. The fix will not be backported to Org 8.2,
though, as it implies a non-trivial amount of changes.


Regards,

-- 
Nicolas Goaziou