link interfering with brackets when abbreviated

link interfering with brackets when abbreviated

[Michael Brand]

Hi all

The recent bug report below reminds me of a comparable situation with
a bug that I wanted to report at some time:

--------------------------------
#+LINK: link-abbreviation http://www.orgmode.org/#

1) [ ] [[http://www.orgmode.org/#docs]]
2) [[link-abbreviation:docs]]
3) [ ] [[link-abbreviation:docs]]

1 bla bla [ ] [[http://www.orgmode.org/#docs]]

2 bla bla [[link-abbreviation:docs]]

3 bla bla [ ] [[link-abbreviation:docs]]

| 1 | [ ] | [[http://www.orgmode.org/#docs]] |
| 2 |     | [[link-abbreviation:docs]]       |
| 3 | [ ] | [[link-abbreviation:docs]]       |
--------------------------------

C-c C-o with point on "1" or "2" opens the browser, on "3" I expect
the same but an error happens.

Michael

On Thu, Oct 3, 2013 at 12:56 PM, Christoph LANGE
<math.semantic.web@gmail.com> wrote:
subject: C-c C-c doesn't tick check box when pressed on a hyperlink in a
list item
> this is a bug in Org 8.2 on Emacs 24.3.  I can't use
> org-submit-bug-report right now (see previous mail), so let me try this way.
>
> I have a check list like this
>
>  * [X] item
>  * [ ] item
>
> and some of the items contain hyperlinks.  When I am on one such
> hyperlink and press C-c C-c it doesn't tick the check box but says "C-c
> C-c can do nothing useful at this location".

Re: link interfering with brackets when abbreviated

[Michael Brand]

Hi Nicolas

There is a recent regression that I hope I can use to revive an old
unanswered thread that, as it seems to me, is perfectly related.

On Thu, Oct 3, 2013 at 2:11 PM, Michael Brand
<michael.ch.brand@gmail.com> wrote:
> Hi all
>
> The recent bug report below reminds me of a comparable situation with
> a bug that I wanted to report at some time:
>
> --------------------------------
> #+LINK: link-abbreviation http://www.orgmode.org/#
>
> 1) [ ] [[http://www.orgmode.org/#docs]]
> 2) [[link-abbreviation:docs]]
> 3) [ ] [[link-abbreviation:docs]]
>
> 1 bla bla [ ] [[http://www.orgmode.org/#docs]]
>
> 2 bla bla [[link-abbreviation:docs]]
>
> 3 bla bla [ ] [[link-abbreviation:docs]]
>
> | 1 | [ ] | [[http://www.orgmode.org/#docs]] |
> | 2 |     | [[link-abbreviation:docs]]       |
> | 3 | [ ] | [[link-abbreviation:docs]]       |
> --------------------------------
>
> C-c C-o with point on "1" or "2" opens the browser, on "3" I expect
> the same but an error happens.
>
> Michael
>
> On Thu, Oct 3, 2013 at 12:56 PM, Christoph LANGE
> <math.semantic.web@gmail.com> wrote:
> subject: C-c C-c doesn't tick check box when pressed on a hyperlink in a
> list item
>> this is a bug in Org 8.2 on Emacs 24.3.  I can't use
>> org-submit-bug-report right now (see previous mail), so let me try this way.
>>
>> I have a check list like this
>>
>>  * [X] item
>>  * [ ] item
>>
>> and some of the items contain hyperlinks.  When I am on one such
>> hyperlink and press C-c C-c it doesn't tick the check box but says "C-c
>> C-c can do nothing useful at this location".

The above old issue with "3" remains the same on
release_8.2.5h-645-g3f55b45.

The recent regression is from release_8.2.5h-647-gfc9ce86

    commit fc9ce86cfc1ecf7e86028027a12875a26500e774
    Author: Nicolas Goaziou <n.goaziou@gmail.com>
    Date:   Sun Feb 23 11:35:34 2014 +0100

        Rewrite `org-open-at-point' using Elements

and leads to the error "No link found", on all of "1", "2" and "3".

And once again thank you for your work in reimplementing more and more
by using org-element.

Michael

Re: link interfering with brackets when abbreviated

[Nicolas Goaziou]

Hello,

Michael Brand <michael.ch.brand@gmail.com> writes:

> The above old issue with "3" remains the same on
> release_8.2.5h-645-g3f55b45.
>
> The recent regression is from release_8.2.5h-647-gfc9ce86
>
>     commit fc9ce86cfc1ecf7e86028027a12875a26500e774
>     Author: Nicolas Goaziou <n.goaziou@gmail.com>
>     Date:   Sun Feb 23 11:35:34 2014 +0100
>
>         Rewrite `org-open-at-point' using Elements
>
> and leads to the error "No link found", on all of "1", "2" and "3".

With the following Org document:

--8<---------------cut here---------------start------------->8---
#+LINK: link-abbreviation http://www.orgmode.org/#

1) [ ] [[http://www.orgmode.org/#docs]]
2) [[link-abbreviation:docs]]
3) [ ] [[link-abbreviation:docs]]

1 bla bla [ ] [[http://www.orgmode.org/#docs]]

2 bla bla [[link-abbreviation:docs]]

3 bla bla [ ] [[link-abbreviation:docs]]

| 1 | [ ] | [[http://www.orgmode.org/#docs]] |
| 2 |     | [[link-abbreviation:docs]]       |
| 3 | [ ] | [[link-abbreviation:docs]]       |
--8<---------------cut here---------------end--------------->8---

I do not get any error, i.e, every link is opened in the browser.

Did you refresh your local set-up with C-c C-c on the LINK keyword?


Regards,

-- 
Nicolas Goaziou

Re: link interfering with brackets when abbreviated

[Michael Brand]

Hi Nicolas

On Wed, Feb 26, 2014 at 4:25 PM, Nicolas Goaziou <n.goaziou@gmail.com> wrote:
> I do not get any error, i.e, every link is opened in the browser.

With point exactly on all variants of "1", "2" and "3"? _On_ is also
important to see the difference of "1" and "2" vs.  "3" for the old
issue in release_8.2.5h-645-g3f55b45, please check that too.

> Did you refresh your local set-up with C-c C-c on the LINK keyword?

Yes, also with C-c C-c. Anyway I used

    git checkout master && git pull && make cleanall uncompiled && \\
    /x/arch/emacs/src/emacs -Q -L /x/git/org-mode/lisp /z/x.org

with GNU Emacs 24.3.2 and now also 24.1.1 and 23.4.1.

Michael

Re: link interfering with brackets when abbreviated

[Nicolas Goaziou]

Michael Brand <michael.ch.brand@gmail.com> writes:

> With point exactly on all variants of "1", "2" and "3"? _On_ is also
> important to see the difference of "1" and "2" vs.  "3" for the old
> issue in release_8.2.5h-645-g3f55b45, please check that too.

I don't understand. Using C-c C-o on "1" "2" or "3" will not open any
link since point is not on a link anyway. So you will consistently get
"No link found".


Regards,

-- 
Nicolas Goaziou

Re: link interfering with brackets when abbreviated

[Michael Brand]

Hi Nicolas

On Wed, Feb 26, 2014 at 4:54 PM, Nicolas Goaziou <n.goaziou@gmail.com> wrote:
> I don't understand. Using C-c C-o on "1" "2" or "3" will not open any
> link since point is not on a link anyway. So you will consistently get
> "No link found".

Aha, now as I see that you removed the following from the docstring of
org-open-at-point I start to get it:

    If there is no link at point, this function will search forward up
    to the end of the current line.

Before I found it _very_ useful that after I navigated to the right
line at last, I didn't have to navigate further within this line. That
the position of point within a line with an Org list item matters for
structure editing I can follow. But for org-open-at-point I do not yet
understand this requirement. What is the benefit of removing the
search on the same line?

Michael

Re: link interfering with brackets when abbreviated

[Nicolas Goaziou]

Hello,

Michael Brand <michael.ch.brand@gmail.com> writes:

> What is the benefit of removing the search on the same line?

`org-element-context' returns the context under point, not on the other
side of the line. Your are on a link, C-c C-o opens it, otherwise, it
doesn't. I find it very predictable.

The "feature" you are missing probably made sense in the previous
implementation, since it already searched forward for a link, but it
would be confusing (and more costly) in the new one.


Regards,

-- 
Nicolas Goaziou

Re: link interfering with brackets when abbreviated

[Michael Brand]

Hi Nicolas

On Wed, Feb 26, 2014 at 5:41 PM, Nicolas Goaziou <n.goaziou@gmail.com> wrote:
>
> Michael Brand <michael.ch.brand@gmail.com> writes:
>
>> What is the benefit of removing the search on the same line?
>
> `org-element-context' returns the context under point, not on the other
> side of the line. Your are on a link, C-c C-o opens it, otherwise, it
> doesn't. I find it very predictable.
>
> The "feature" you are missing probably made sense in the previous
> implementation, since it already searched forward for a link, but it
> would be confusing (and more costly) in the new one.

I expect some users to go through the same surprise than me. Maybe
that there will be enough voices to get the searching on the same line
for "C-c C-o" (org-open-at-point) back.

Michael

Re: link interfering with brackets when abbreviated

[Bastien]

Hi Michael,

Michael Brand <michael.ch.brand@gmail.com> writes:

> I expect some users to go through the same surprise than me. Maybe
> that there will be enough voices to get the searching on the same line
> for "C-c C-o" (org-open-at-point) back.

Count me in -- this is a regression that needs to be fixed.

Nicolas, any stronger objection than the one your already
expressed?

-- 
 Bastien

Re: link interfering with brackets when abbreviated

[Bastien]

Hi Michael and Nicolas,

Michael Brand <michael.ch.brand@gmail.com> writes:

> And once again thank you for your work in reimplementing more and more
> by using org-element.

A quick note on this.

Here are the reasons why we *want* to rewrite some functions using
org-element:

- *bug fixing*: the rewrite fixes bugs.

- *speed*: the rewrite provides a faster implementation;

- *predictability*: the rewrite provides a more predictable
   implementation;

- *completeness*: the rewrite covers more use-cases than the
  previous /ad hoc/ implementation.

Now here are the reasons why we *don't want* to use org-element:

- *high ceiling*: if the rewrite is less readable for potential
  developers.

- *complexity*: if the rewritten version is more complex.

- *rigidity*: if the rewritten version adds unwanted constraints.

Of course, the decision is always a trade-off.

I'm particularily sensitive to the "high ceiling" point above: we
don't want all contributors to learn about Org elements before they
can submit a patch -- of course, many will do, and we can encourage
them to do so, but let's not close too many doors.

IOW, even though org-element.el is great (and I hope my comments above
will not be taken as criticism), "rewriting using org-element" is not
a goal /per se/, just one great possibility to use when needed.

-- 
 Bastien

Re: link interfering with brackets when abbreviated

[Nicolas Goaziou]

Hello,

Bastien <bzg@gnu.org> writes:

> Count me in -- this is a regression that needs to be fixed.
>
> Nicolas, any stronger objection than the one your already
> expressed?

As I said, the "end of line" is not a structural unit. Implementing that
"feature", which, I must admit, I find cheesy, back will be fragile and
confusing.

For example, white spaces after an object still belong to an object. So
in the following case:

  [[http://orgmode.org]]   [[http://duckduckgo.com]]

using C-c C-o between the two links will open the first one, but there:

  [[http://orgmode.org]] and [[http://duckduckgo.com]]

C-c C-o on the "and" will open the second one.

Also in the following example:

  [fn:1] This is some text [[http://orgmode.org]]

C-c C-o on "some" currently triggers `org-footnote-action' since point
is in a footnote definition. But with the behaviour you describe, it
would be hard to predict whether it should move to the link or still
open the footnote.

There are many other examples. This "convenient feature" is
unpredictable and not worth implementing back (not counting the fact
that it wouldn't be totally trivial to do properly).


Regards,

-- 
Nicolas Goaziou

Re: link interfering with brackets when abbreviated

[Nicolas Goaziou]

Bastien <bzg@gnu.org> writes:

> Here are the reasons why we *want* to rewrite some functions using
> org-element:

I don't know who "we" is. Apparently, I'm not in.

> - *bug fixing*: the rewrite fixes bugs.
>
> - *speed*: the rewrite provides a faster implementation;
>
> - *predictability*: the rewrite provides a more predictable
>    implementation;
>
> - *completeness*: the rewrite covers more use-cases than the
>   previous /ad hoc/ implementation.
>
> Now here are the reasons why we *don't want* to use org-element:
>
> - *high ceiling*: if the rewrite is less readable for potential
>   developers.
>
> - *complexity*: if the rewritten version is more complex.
>
> - *rigidity*: if the rewritten version adds unwanted constraints.
>
> Of course, the decision is always a trade-off.
>
> I'm particularily sensitive to the "high ceiling" point above: we
> don't want all contributors to learn about Org elements before they
> can submit a patch -- of course, many will do, and we can encourage
> them to do so, but let's not close too many doors.
>
> IOW, even though org-element.el is great (and I hope my comments above
> will not be taken as criticism), "rewriting using org-element" is not
> a goal /per se/, just one great possibility to use when needed.

I think at least one of us is missing the point. I do not "rewrite using
Elements" for any of the reasons above. But before I explain my reasons,
I will comment "we"'s a bit.

I cannot talk about bug fixing and completeness as it depends on the
function rewritten. Though, don't hold your breath, implementation with
Elements will /always/ be slower than the current one. I'm working on
the cache to make that fact bearable, but I don't possess pixie dust.
Also, I'm pretty sure it will never be less rigid. A parser is
inherently very rigid. But this will sure make it more predictable.

Moreover, learning about "org-element.el" is not that hard. There are
4 major functions: `org-element-at-point', `org-element-context',
`org-element-property' and `org-element-type'. Of course, it can help to
also know about properties attached to each element type. That's
documented in Worg and pretty explicit in "org-element.el".

Anyway, back to my reasons to "rewrite using Elements".

Org consists of mostly (but not totally) independent parts, each one
implementing its own Org parser, similar to the others, but sometimes
slightly different. This is, to say the least, highly inefficient. It is
also a potential source of bugs.

Worse, it can impede further improvements. Indeed, I doubt that anyone
knows Org's full code base. So any slight change to syntax could break
some unknown parts of Org. Therefore, basically no structural
improvement can happen today without tremendous efforts and pain (for
the very same reasons, no new export back-end could be created
yesterday).

A function rewritten using Elements shares the same knowledge about Org
syntax with other rewritten functions. If we improve Elements, all of
them benefit from the improvement. If we modify it, all of them get the
modification. I'm exaggerating it a bit, but I'm sure you get the idea.
It is no panacea, but IMO, Org will be a better place when all of its
parts agree on a common, unambiguous, syntax.

So, in a nutshell, rewriting using org-element is almost a goal /per
se/. It comes with a price, which is that a re-implementation is not
a perfect copy of the original function, but the reward is higher. If
"we" disagree with that, then it would nice be if "we" could let me
know. Better late than sorry, I guess.


Regards,

-- 
Nicolas Goaziou

Re: link interfering with brackets when abbreviated

[Bastien]

Hi Nicolas,

"we" was short for "we, Org contributors", and it was a request for
comment, so I'm glad you did.

I understand your point very well: structural consistency favors ease
of maintainance and evolutivity.  The new export engine is a perfect
example of this: without a clean parser, it would not exist.

But this is also a perfect example of what I want to insist on: the
benefit of relying on the parser has to be very clear.  It is not
something you can systematically and blindly taken for granted.

For example, if we were to rely on the parser for fontification now,
it would be certainly too slow, and the cost for the user would be too
high.  I know this rewrite is somewhere on the roadmap, and I know you
will consider it only when the cost of the slowdown will not be too
high -- so the trade-off strategy I was describing is something I
guess we (as in "you and me") agree on.

Thanks,

-- 
 Bastien

Re: link interfering with brackets when abbreviated

[Bastien]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> As I said, the "end of line" is not a structural unit. Implementing that
> "feature", which, I must admit, I find cheesy, back will be fragile and
> confusing.
>
> For example, white spaces after an object still belong to an
> object.

Well, this is counterintuitive.

> So in the following case:
>
>   [[http://orgmode.org]]   [[http://duckduckgo.com]]
>
> using C-c C-o between the two links will open the first one, but there:
>
>   [[http://orgmode.org]] and [[http://duckduckgo.com]]
>
> C-c C-o on the "and" will open the second one.

This current behavior is surprising too here, and only predictable for
users who know that whitespaces are part of the previous object -- i.e.
nobody.

> Also in the following example:
>
>   [fn:1] This is some text [[http://orgmode.org]]
>
> C-c C-o on "some" currently triggers `org-footnote-action' since point
> is in a footnote definition.

Which is counterintuitive too!

> But with the behaviour you describe, it would be hard to predict
> whether it should move to the link or still open the footnote.

Let me describe the behavior I favor:

  C-c C-o opens the link at point (i.e. "the link that the cursor is
  visibly on") or the next link on the same line.

  When on a headline and if there are several links on the same line,
  prompt the user for which one she wants to visit.

I find it very simple and predictable.

> There are many other examples. This "convenient feature" is
> unpredictable and not worth implementing back (not counting the fact
> that it wouldn't be totally trivial to do properly).

Sorry, but the current behavior feels just too wrong.

-- 
 Bastien

Re: link interfering with brackets when abbreviated

[Nicolas Goaziou]

Hello,

Bastien <bzg@gnu.org> writes:

>> For example, white spaces after an object still belong to an
>> object.
>
> Well, this is counterintuitive.

So they should belong to the next object? I don't find it more
intuitive. Anyway, it's an internal representation for white spaces so
it doesn't matter here. See next answer.

>> So in the following case:
>>
>>   [[http://orgmode.org]]   [[http://duckduckgo.com]]
>>
>> using C-c C-o between the two links will open the first one, but there:
>>
>>   [[http://orgmode.org]] and [[http://duckduckgo.com]]
>>
>> C-c C-o on the "and" will open the second one.
>
> This current behavior is surprising too here, and only predictable for
> users who know that whitespaces are part of the previous object -- i.e.
> nobody.

That's not a problem, it is easy to remove this. C-c C-o will do nothing
on white spaces after an object.

>> Also in the following example:
>>
>>   [fn:1] This is some text [[http://orgmode.org]]
>>
>> C-c C-o on "some" currently triggers `org-footnote-action' since point
>> is in a footnote definition.
>
> Which is counterintuitive too!

It was part of the specs of the _previous_ implementation. I didn't
change anything here.

But it can be removed.

>> But with the behaviour you describe, it would be hard to predict
>> whether it should move to the link or still open the footnote.
>
> Let me describe the behavior I favor:
>
>   C-c C-o opens the link at point (i.e. "the link that the cursor is
>   visibly on")

This is already the case (minus the trailing spaces situation)

> or the next link on the same line.

Not really possible, as explained before. And not intuitive, IMO.

>   When on a headline

This is the case.

> and if there are several links on the same line, prompt the user for
> which one she wants to visit.

Come on. This wasn't done even in the previous implementation.

> I find it very simple and predictable.

The only really predictable behaviour is: "open the link under point".
Everything else is arguable.


Regards,

-- 
Nicolas Goaziou

Re: link interfering with brackets when abbreviated

[Bastien]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> So they should belong to the next object? I don't find it more
> intuitive. Anyway, it's an internal representation for white spaces so
> it doesn't matter here. See next answer.

I've no problem with this internal representation.

> That's not a problem, it is easy to remove this. C-c C-o will do nothing
> on white spaces after an object.

Yes, I think that would be better.

>>> Also in the following example:
>>>
>>>   [fn:1] This is some text [[http://orgmode.org]]
>>>
>>> C-c C-o on "some" currently triggers `org-footnote-action' since point
>>> is in a footnote definition.
>>
>> Which is counterintuitive too!
>
> It was part of the specs of the _previous_ implementation. I didn't
> change anything here.

Yes.  There was an inconsistency in the previous implementation (as
just tested from the maint branch): when point is in between two
non-footnote links, C-c C-o opens the one on the right, while
between [fn:1] and a http:// link, C-c C-o calls org-footnote-action
iff point is within the footnote...

> But it can be removed.

Yes, this should be removed IMO.

>>> But with the behaviour you describe, it would be hard to predict
>>> whether it should move to the link or still open the footnote.
>>
>> Let me describe the behavior I favor:
>>
>>   C-c C-o opens the link at point (i.e. "the link that the cursor is
>>   visibly on")
>
> This is already the case (minus the trailing spaces situation)
>
>> or the next link on the same line.
>
> Not really possible, as explained before. And not intuitive, IMO.

I don't understand why this is not possible.  The fact that the end of
the line is not a structural element from Org's parser point of view
should not prevent features that rely on some notion of "end of the
line".

>>   When on a headline
>
> This is the case.
>
>> and if there are several links on the same line, prompt the user for
>> which one she wants to visit.
>
> Come on. This wasn't done even in the previous implementation.

When I meant is this:

* Visit http://orgmode.org and http://www.gnu.org
  ^

When point is on the headline, the current implementation in maint is
to prompt the user whether he wants to visit one of the two links.

The new implementation does this too, I mentioned it for the sake of
completeness -- so maybe there is a misunderstanding.

>> I find it very simple and predictable.
>
> The only really predictable behaviour is: "open the link under point".
> Everything else is arguable.

Then I argue that the previous behavior, which is to open the next
link on the same line, is very convenient and very human-predictable
when encoutered at least once.

If I'm alone in thinking this, I'm fine surrending, but I hope we
can give this another chance :)

-- 
 Bastien

Re: link interfering with brackets when abbreviated

[Michael Brand]

Hi Nicolas

On Thu, Feb 27, 2014 at 11:28 AM, Nicolas Goaziou <n.goaziou@gmail.com> wrote:
> The only really predictable behaviour is: "open the link under point".
> Everything else is arguable.

What for me is a conflict between ...

    1) There are arguments to change - as you recently did in master -
       org-open-at-point to open only if there is a link _at point_.
       Just as its function name tells.

    2) As a user I would like to have the possibility to open a link
       between point and end of line without having to navigate to the
       exact column. Also in modes other than Org.

... I solve for myself with the following function bound to "C-c o"
(not "C-c C-o"). It moves to and opens the first link between point
and end of line by simply trying point by point until it works the
current org-open-at-point from master that opens only when point is on
a link:

#+BEGIN_SRC emacs-lisp
  (defun f-open-link-between-point-and-eol ()
    "Move to and open first link between point and end of line.
  As long as not yet at end of line and as long as
  `org-open-at-point' or `browse-url-at-point' gives an error
  advance point by one character. For Org and other major modes."
    (interactive)
    (let ((p (point)) (err t))
      (while (and (not (eolp))
                  (setq err (not (ignore-errors
                                   (or (cond
                                        ;; Org mode
                                        ((eq major-mode 'org-mode)
                                         (org-open-at-point))
                                        ;; Maybe more major modes here
                                        ;; [...]
                                        ;; All other major modes
                                        (t
                                         (browse-url-at-point)))
                                       t)))))
        (forward-char))
      (when err
        (goto-char p)
        (user-error "No link between point and end of line"))))
#+END_SRC

The above function does not only work for Org link and URL in Org mode
but also for URL in any other major mode. The definition of what
should be the alternative link when there is no link at the starting
point is simply delegated to org-open-at-point and
browse-url-at-point. This simplification makes it slow for links
towards the end or not present in long lines. There is plenty of room
for a better implementation than I did with this very simple first
approach.

In the case that, depending on user feedback, even "C-c C-o" itself
should move to the first link between point and end of line - or
whatever other link - also when there is no link at the starting
point, I suggest to keep org-open-at-point to open only when point is
on a link and to wrap this move into a new function, named e. g.
org-open-between-point-and-eol to be bound to "C-c C-o".

The simplified solution with f-open-link-between-point-and-eol already
covers all my use cases of the old org-open-at-point. The
predictability of what will be the alternative link remains arguable.
I find its predictability at least better than with the old
org-open-at-point. And last but not least also the original issue

    #+LINK: link-abbreviation http://www.orgmode.org/#
    1) [ ] [[http://www.orgmode.org/#docs]]
    2) [[link-abbreviation:docs]]
    3) [ ] [[link-abbreviation:docs]]

of this thread with point on the character "3" is of course solved for
me.

Michael

Re: link interfering with brackets when abbreviated

[Bastien]

Hi Nicolas,

you just updated `org-open-at-point' without reimplementing the
previous behavior -- is this work in progress?

If it is not, I suggest to discuss the change before implementing it.
Nobody ever complained about the previous behavior, and both Michael
and me are suppporting it.

Thanks for considering this,

-- 
 Bastien

Re: link interfering with brackets when abbreviated

[Nicolas Goaziou]

Hello,

Bastien <bzg@altern.org> writes:

> you just updated `org-open-at-point' without reimplementing the
> previous behavior -- is this work in progress?

No, it isn't. I fixed the bugs we discussed, and one reported on the ML.

> If it is not, I suggest to discuss the change before implementing it.
> Nobody ever complained about the previous behavior, and both Michael
> and me are suppporting it.

I didn't remove that non-essential feature for my pleasure, but because
it didn't fit in the new internal model. As I already said, implementing
it back is a bit of work and will probably not be very clean. Why
bother?

Anyway, I don't understand why there is so much fuss about this. As you
well know, Org provides `org-next-link' (which is bound to C-c C-x C-n),
and Emacs provides incremental search. Do you, or Michael, honestly open
so many links that an additional C-s ... RET to move on each of them is
too much?

There is also `org-open-at-point-functions', which can probably be used
here.

I think that the coolness of the feature eludes me for all I can see is
a crude hack.


Regards,

-- 
Nicolas Goaziou

Re: link interfering with brackets when abbreviated

[Yasushi SHOJI]

Hi Nicolas,

At Fri, 28 Feb 2014 00:43:19 +0100,
Nicolas Goaziou wrote:
> 
> Bastien <bzg@altern.org> writes:
> 
> > If it is not, I suggest to discuss the change before implementing it.
> > Nobody ever complained about the previous behavior, and both Michael
> > and me are suppporting it.
> 
> I didn't remove that non-essential feature for my pleasure, but because
> it didn't fit in the new internal model. As I already said, implementing
> it back is a bit of work and will probably not be very clean. Why
> bother?

Let's separate those internal parser thing and interactive commands,
while we discus. I agree to Nicolas that parser and internal functions
should NOT be ambiguous nor confusing.

However, we humans are not machines nor slave of computers.  We tell
computers what we want, or even, we want to make computers think and
do what we are thinking. That's the reason why we, these days, have
*-dwim commands.  We don't want to make our users to adopt how
computers work.

> Anyway, I don't understand why there is so much fuss about this.

That's because a) the commands have been working, b) many other
commands _do_ work even if it's not right on the elements. ie. S-right
right after a timestamp, C-c C-c on checkbox list.  Are you planning
to remove these features, too?

> I think that the coolness of the feature eludes me for all I can see is
> a crude hack.

What if we create org-open-at-point-dwim and map to C-c C-o.  Nicolas,
do you object?

Anyway, thank you Nicolas for your work.  We all appreciate your great
work.

Thanks,
-- 
            yashi

Re: link interfering with brackets when abbreviated

[Yasushi SHOJI]

Hi Nicolas,

At Fri, 28 Feb 2014 00:43:19 +0100,
Nicolas Goaziou wrote:
> 
> Bastien <bzg@altern.org> writes:
> 
> > If it is not, I suggest to discuss the change before implementing it.
> > Nobody ever complained about the previous behavior, and both Michael
> > and me are suppporting it.
> 
> I didn't remove that non-essential feature for my pleasure, but because
> it didn't fit in the new internal model. As I already said, implementing
> it back is a bit of work and will probably not be very clean. Why
> bother?

Let's separate those internal parser thing and interactive commands,
while we discus. I agree to Nicolas that parser and internal functions
should NOT be ambiguous nor confusing.

However, we humans are not machines nor slave of computers.  We tell
computers what we want, or even, we want to make computers think and
do what we are thinking. That's the reason why we, these days, have
*-dwim commands.  We don't want to make our users to adopt how
computers work.

> Anyway, I don't understand why there is so much fuss about this.

That's because a) the commands have been working, b) many other
commands _do_ work even if it's not right on the elements. ie. S-right
right after a timestamp, C-c C-c on checkbox list.  Are you planning
to remove these features, too?

> I think that the coolness of the feature eludes me for all I can see is
> a crude hack.

What if we create org-open-at-point-dwim and map to C-c C-o.  Nicolas,
do you object?

Anyway, thank you Nicolas for your work.  We all appreciate your great
work.

Thanks,
-- 
            yashi

Re: link interfering with brackets when abbreviated

[Nicolas Goaziou]

Hello,

Yasushi SHOJI <yashi@atmark-techno.com> writes:

> However, we humans are not machines nor slave of computers.  We tell
> computers what we want, or even, we want to make computers think and
> do what we are thinking. That's the reason why we, these days, have
> *-dwim commands.  We don't want to make our users to adopt how
> computers work.

This is one of the things that annoy me: opening next link on the same
line is not, IMO, "dwim". See below.

>> Anyway, I don't understand why there is so much fuss about this.
>
> That's because a) the commands have been working

This is not a sufficient reason. We are discussing a minor feature.
Removing it doesn't remove any functionality to Org, as the "thing" just
saves a few keystrokes, on a good day.

While re-implementing the function, it appears that the feature just
doesn't fit. So this is a good time to ponder about its real usefulness,
and, if it is worth bending the new function to add it back. I think it
isn't.

As I already said, opening the next link in the same line is dubious. In
the following example, with point between the links, the previous
behaviour was to open "link2":

  [[link1]]  [[link2]]

Now consider the following case, where point is before the "a":

  [[link1]] a very ... very long line of text [[link2]]

The previous behaviour implied to also open "link2". This is not really
straightforward. Worse, if `visual-line-mode' is on, [[link2]] can be
many lines below. In the following case, with point still before the
first "a", opening [[link2]] is just odd:

  [[link1]] a very ... very long line
  which spans over many visual lines
  of text [[link2]].

It is odd because in the same situation, without `visual-line-mode' but
with `auto-fill-mode' on, C-c C-o will report "No link found".

So, why do we care about the same line? We could as well open the next
link in the same paragraph (or verse block, table cell). I'm not arguing
for the latter, but I insist on the fact that opening "next link on the
same line" is arbitrary and not really "dwim".

OTOH, I'm sure Emacs users know (beginners aside, but they'll learn
soon) how to move quickly point wherever they want, without even
thinking about it.

> b) many other commands _do_ work even if it's not right on the
> elements. ie. S-right right after a timestamp, C-c C-c on checkbox
> list. Are you planning to remove these features, too?

C-c C-c already uses Elements. I'll leave S-right for another day.

>> I think that the coolness of the feature eludes me for all I can see is
>> a crude hack.
>
> What if we create org-open-at-point-dwim and map to C-c C-o.  Nicolas,
> do you object?

Implementing it in `org-open-at-point' or `org-open-at-point-whatever'
is still implementing it in Org. I still think it's not worth it.


Regards,

-- 
Nicolas Goaziou

Re: link interfering with brackets when abbreviated

[Bastien]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> I insist on the fact that opening "next link on the
> same line" is arbitrary and not really "dwim".

And we insist on keeping the previous behavior, please trust us.

Right now the speedy command `o' is broken.  This is a pattern I
use very frequently: use `n' to navigate to the next headline,
then `o' to open the link there.

Also `org-agenda-open-link' is now broken.

Can you have a look and fix this later issue?

I will then re-add the previous behavior on `org-open-at-point'.

Thanks,

-- 
 Bastien

Re: link interfering with brackets when abbreviated

[Bastien]

Hi again,

Bastien <bzg@gnu.org> writes:

> Right now the speedy command `o' is broken.  This is a pattern I
> use very frequently: use `n' to navigate to the next headline,
> then `o' to open the link there.

Forget about this -- some draft code of mine interfered.

> Also `org-agenda-open-link' is now broken.

This one is really broken.

Thanks,

-- 
 Bastien

Re: link interfering with brackets when abbreviated

[Nicolas Goaziou]

Bastien <bzg@gnu.org> writes:

> And we insist on keeping the previous behavior, please trust us.

This is not a matter of trust. I asked about use-cases to understand why
this feature was needed, and all I got was "because it was here".

> Also `org-agenda-open-link' is now broken.
>
> Can you have a look and fix this later issue?

Would you mind elaborating a bit about it?

> I will then re-add the previous behavior on `org-open-at-point'.

I guess that closes the discussion then.


Regards,

-- 
Nicolas Goaziou

Re: link interfering with brackets when abbreviated

[Bastien]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Bastien <bzg@gnu.org> writes:
>
>> And we insist on keeping the previous behavior, please trust us.
>
> This is not a matter of trust. I asked about use-cases to understand why
> this feature was needed, and all I got was "because it was here".

More precisely, the answer was: because we use it and find it useful.

>> Also `org-agenda-open-link' is now broken.
>>
>> Can you have a look and fix this later issue?
>
> Would you mind elaborating a bit about it?

C-c C-o throws a "No link found" message when hit on a link in an
agenda view.

>> I will then re-add the previous behavior on `org-open-at-point'.
>
> I guess that closes the discussion then.

We can raise again the discussion about suppressing this feature any
time but I personally think this is a waste of time.

Thanks,

-- 
 Bastien

Re: link interfering with brackets when abbreviated

[Nicolas Goaziou]

Bastien <bzg@gnu.org> writes:

>> This is not a matter of trust. I asked about use-cases to understand why
>> this feature was needed, and all I got was "because it was here".
>
> More precisely, the answer was: because we use it and find it useful.

Thank you for the precision. Now, what about caring to give me one (or
more) use case?

>>> Also `org-agenda-open-link' is now broken.
>>>
>>> Can you have a look and fix this later issue?
>>
>> Would you mind elaborating a bit about it?
>
> C-c C-o throws a "No link found" message when hit on a link in an
> agenda view.

Sorry for not being clear. I did try, I didn't get any error. My dummy
entry was:

  * TODO [[http://orgmode.org]]

in a block agenda and

  * [[http://orgmode.org]]
    DEADLINE: <2014-03-01 sam.>

in regular agenda.

Both times, I could open the link. So, could you send me a dummy entry
where the bug can be reproduced? It will save me a lot of time.


Regards,

-- 
Nicolas Goaziou

Re: link interfering with brackets when abbreviated

[Yasushi SHOJI]

Hi Nicolas,

Thanks for your time.

At Sat, 01 Mar 2014 21:20:18 +0100,
Nicolas Goaziou wrote:
> 
> >> Anyway, I don't understand why there is so much fuss about this.
> >
> > That's because a) the commands have been working
> 
> This is not a sufficient reason. We are discussing a minor feature.
> Removing it doesn't remove any functionality to Org, as the "thing" just
> saves a few keystrokes, on a good day.

Ok.  If this is yet another bickshed, I'll drop from the discussion.

> While re-implementing the function, it appears that the feature just
> doesn't fit. So this is a good time to ponder about its real usefulness,
> and, if it is worth bending the new function to add it back. I think it
> isn't.
> 
> As I already said, opening the next link in the same line is dubious. In
> the following example, with point between the links, the previous
> behaviour was to open "link2":
> 
>   [[link1]]  [[link2]]
> 
> Now consider the following case, where point is before the "a":
> 
>   [[link1]] a very ... very long line of text [[link2]]
> 
> The previous behaviour implied to also open "link2". This is not
> really straightforward.

If the point is before the "a", that means the point is right after
the link, it should open `link1' instead of `link2', IMNSHO.  This
isn't even the previous behavior, I admit, but if you move the pointer
to the end of the line (that's right after the link2), it _opened_
links2. This behavior works quite well with Emacs' cursor movement.

;; uga, `forward-word' doesn't work as I expected on
;; [[http://google.com][google]].  It stops at the first `o'.

>                          Worse, if `visual-line-mode' is on,
> [[link2]] can be many lines below. In the following case, with point
> still before the first "a", opening [[link2]] is just odd:
> 
>   [[link1]] a very ... very long line
>   which spans over many visual lines
>   of text [[link2]].
> 
> It is odd because in the same situation, without `visual-line-mode' but
> with `auto-fill-mode' on, C-c C-o will report "No link found".

Both should report "No link found".  `org-end-of-line' takes care of
`visual-line-mode', why not `org-open-at-link'?
-- 
              yashi

Re: link interfering with brackets when abbreviated

[Nicolas Goaziou]

Hello,

Yasushi SHOJI <yashi@atmark-techno.com> writes:

> At Sat, 01 Mar 2014 21:20:18 +0100,
> Nicolas Goaziou wrote:
>> 
>> This is not a sufficient reason. We are discussing a minor feature.
>> Removing it doesn't remove any functionality to Org, as the "thing" just
>> saves a few keystrokes, on a good day.
>
> Ok.  If this is yet another bickshed, I'll drop from the discussion.

This whole thread is about bikeshedding, no "if" involved.

Though, I'm struggling to get a constructive discussion. I asked
a couple of times already in what cases that feature was useful, in
order to understand what some users were missing. I even gave examples
about the inconsistencies in the previous implementation.

And all I got so far was drama.

>> Now consider the following case, where point is before the "a":
>> 
>>   [[link1]] a very ... very long line of text [[link2]]
>> 
>> The previous behaviour implied to also open "link2". This is not
>> really straightforward.
>
> If the point is before the "a", that means the point is right after
> the link, it should open `link1' instead of `link2', IMNSHO.  This
> isn't even the previous behavior, I admit,

... which is my point: the previous behaviour was wrong.

> but if you move the pointer to the end of the line (that's right after
> the link2), it _opened_ links2. This behavior works quite well with
> Emacs' cursor movement.

The rewrite did it a few commits ago, but then I was asked to ignore
white spaces after a link, which include the first position after the
link.

I agree I should make a special case here.

> ;; uga, `forward-word' doesn't work as I expected on
> ;; [[http://google.com][google]].  It stops at the first `o'.

I do not understand this.

>>                          Worse, if `visual-line-mode' is on,
>> [[link2]] can be many lines below. In the following case, with point
>> still before the first "a", opening [[link2]] is just odd:
>> 
>>   [[link1]] a very ... very long line
>>   which spans over many visual lines
>>   of text [[link2]].
>> 
>> It is odd because in the same situation, without `visual-line-mode' but
>> with `auto-fill-mode' on, C-c C-o will report "No link found".
>
> Both should report "No link found".  `org-end-of-line' takes care of
> `visual-line-mode', why not `org-open-at-link'?

I don't know. I just remark that the previous implementation didn't take
care of `visual-line-mode'.

Again, I'm asking to think again the "feature" because it is ill-defined
and doesn't make sense at the moment. Without a correct specification,
it is not worth re-implementing. 

With examples, I could understand the real scope involved (even though
your message gives me indications about it) and the type of links we're
talking about (it is easy and quite cheap to find the next link of
a given type, much expensive to look after every possible type).


Regards,

-- 
Nicolas Goaziou

Re: link interfering with brackets when abbreviated

[Bastien]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> And all I got so far was drama.

Please: everyone is showing great respect for your work, it is not
helpful to dismiss our contributions as "drama".  Your time is highly
valuable and so is ours.  I don't think Michael, Yasushi or me would
care replying if it was just for drama.

Here is my use-case: I often use C-c C-o as a shortcut for C-c C-x
C-n C-c C-o -- that's 2.5 shorter.  This is convenient, and I would
need a good reason for removing something that convenient.

Now there are really two issues: this one above, which can easily be
fixed by making `org-open-at-point' more flexible.  And the general
issue of when and how Org features should rely on Org syntax, as
defined by the parser.

(Whether links in comments should be treated as links by C-c C-o
belongs to the second issue, this is what I'd like to discuss now.)

To my knowledge, the first raison d'être of the parser was to create
a stable and reliable infrastructure for the export engine.  In this
case, a parser is needed, because that's the only way to consistently
have a common denominator for all export backends.

Now the parser can also be used for other features.  You already
explained the reasons very well: features will be easier to maintain,
and the whole set of features relying on the parser will evolve more
smoothly.

That said, Org syntax is not the only valid representation of an
org-mode buffer.

It is the only useful one when exporting a buffer to a certain format,
because we need to map the structure of the original contents to the
structure of the target one.  But another representation of an org-mode
buffer is the one that a user has in mind when manipulating it, part
of this representation depends on Org syntax, part of it depends on
Emacs general facilities.

For example, Emacs and the user have a notion of `end-of-line', but
Org syntax does not.  Org syntax says whitespaces after an object
belong to the object but my immediate representation says they do not.
Or we do have a notion of visual indentation (with org-indent-mode
turned on), but this does not correspond to any real content, and /a
fortiori/ to an Org syntactic element, since this is just visual
sugar.

(At this point, I feel like I'm "lecturing" the list -- I am not,
I'm just trying to express myself clearly, because I really care about
removing emotional distortion from this thread, and would like to make
my point very simple, so please bare with me.)

There is a minimalistic view of Org as the combination of a syntax and
a set of features to manipulate this syntax.  There is a maximalistic
view of Org as a syntax, a set of features to manipulate it, and a set
of Emacs facilities to manipulate the mental representation of an Org
buffer, which will *never* be the same than the parser representation.

If Org were to be a Ruby gem, then the minimalistic view would be the
right one, because libraries need to stick to common denominator and
favor predictability.  For example, in such a library, we don't care
about "\n" characters by themselves in paragraphs, because the end of
lines is irrelevant here, syntactictly speaking.

But Org is no library: it's the Emacs way to manipulate Org files.
The users' representations of their Org buffer and the affordances
that are based on them are as important as the parser representation.

Hence the case for links in comment, for example: the user read them
as links, there is no value in preventing the users to open them with
C-c C-o, and it is convenient to allow them to do so.

Long story short: when users ask for to keep a feature they find
convenient, this is no drama or conservative position, this is the
expression that their interaction with an Org buffer will be less
smooth if they have to constrain their representation to that of the
parser.

Finding the correct balance here does not impact the value of the
parser at all, quite on the contrary.  The better the parser, the
easier it will be to draw the line between the minimalistic and the
maximalistic views, even in the code.  For example, a comment in
`org-open-at-point' can clearly explain why opening the next link on
the same line is allowed, even if it makes `org-open-at-point' unpure
syntactic-wise.

In a sense, the value of the parser grows with the number of functions
that depend on it, hence the temptation to use it everywhere possible.
But IMO this does not capture the whole truth: the value of the parser
also depends on the number of decisions it helps us make regarding
these minimalistic/maximalistic trade-offs.  It is a unvaluable tool
to clearly think about features, precisely because it empowers the
developers to think in terms of syntactic elements, and see where this
thinking comes short.

I'm done: please don't stop working on the parser just because the
parser is not the only way to think about feature, and please just
remember all participants on this thread are goodwilling users who
try to make a point.

Thanks,

-- 
 Bastien

Re: link interfering with brackets when abbreviated

[Bastien]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Sorry for not being clear. I did try, I didn't get any error. My dummy
> entry was:
>
>   * TODO [[http://orgmode.org]]
>
> in a block agenda and
>
>   * [[http://orgmode.org]]
>     DEADLINE: <2014-03-01 sam.>
>
> in regular agenda.
>
> Both times, I could open the link. So, could you send me a dummy entry
> where the bug can be reproduced? It will save me a lot of time.

Sorry, all my time has been swallowed by my rant, I will try to
provide a reproducible recipe later on -- but the error is real.

It works for small files I tested, but not for my real agenda files.
I suspect maybe the lazy parsing of open-but-not-yet-visible agenda
files somehow prevents the links to get opened.

Maybe someone else can also report if C-c C-o works on his agendas.

Thanks,

-- 
 Bastien

Re: link interfering with brackets when abbreviated

[Nicolas Goaziou]

Bastien <bzg@gnu.org> writes:

> Here is my use-case: I often use C-c C-o as a shortcut for C-c C-x
> C-n C-c C-o -- that's 2.5 shorter.  This is convenient, and I would
> need a good reason for removing something that convenient.

This is not a use case. A use case would explain me why (or, better,
where) you need to use C-c C-x C-n C-c C-o. No worries, though, as I do
not expect to get an answer anymore.

> Now there are really two issues: this one above, which can easily be
> fixed by making `org-open-at-point' more flexible.  And the general
> issue of when and how Org features should rely on Org syntax, as
> defined by the parser.
>
> (Whether links in comments should be treated as links by C-c C-o
> belongs to the second issue, this is what I'd like to discuss now.)
>
> To my knowledge, the first raison d'être of the parser was to create
> a stable and reliable infrastructure for the export engine.  In this
> case, a parser is needed, because that's the only way to consistently
> have a common denominator for all export backends.

The goal of the parser was to define properly Org syntax, by merging
concepts from different parts of Org.  The exporter was only a proof of
concept for the parser, and also a way to exercise the choices made in
the syntax.

> Now the parser can also be used for other features.  You already
> explained the reasons very well: features will be easier to maintain,
> and the whole set of features relying on the parser will evolve more
> smoothly.
>
> That said, Org syntax is not the only valid representation of an
> org-mode buffer.

It should be, or the whole concept is moot. If "Org syntax" is only
valid during export, if fontification interprets Org differently, if
users see Org differently too, there is no point in defining a syntax.
Just let everyone implement its own private Org.

Note that this is what happened recently. One would like to use UTF-8
characters instead of stars for headlines. Another one would like to use
## for emphasis...

> It is the only useful one when exporting a buffer to a certain format,
> because we need to map the structure of the original contents to the
> structure of the target one.

Again, this will not work if we do not agree on the structure. This will
raise questions like "How come that my document is exported this way,
even though I interpret it that way?".

> But another representation of an org-mode buffer is the one that
> a user has in mind when manipulating it, part of this representation
> depends on Org syntax, part of it depends on Emacs general facilities.
>
> For example, Emacs and the user have a notion of `end-of-line', but
> Org syntax does not.  Org syntax says whitespaces after an object
> belong to the object but my immediate representation says they do not.
> Or we do have a notion of visual indentation (with org-indent-mode
> turned on), but this does not correspond to any real content, and /a
> fortiori/ to an Org syntactic element, since this is just visual
> sugar.

You are mixing subjects here. For example, `org-end-of-line' is backed
up with Elements already. This has nothing to do with Org syntax.

> There is a minimalistic view of Org as the combination of a syntax and
> a set of features to manipulate this syntax.  There is a maximalistic
> view of Org as a syntax, a set of features to manipulate it, and a set
> of Emacs facilities to manipulate the mental representation of an Org
> buffer, which will *never* be the same than the parser representation.

Of course, but the "maximalistic" view can only be a superset of the
"minimalistic" one. The former can see more than the latter, but it
cannot disagree on what the latter sees.

> But Org is no library: it's the Emacs way to manipulate Org files.

And Org files are expressed in an Org format. Org syntax tries to define
it.

> The users' representations of their Org buffer and the affordances
> that are based on them are as important as the parser representation.

Their Org buffer is still expressed in the Org format.

> Hence the case for links in comment, for example: the user read them
> as links, there is no value in preventing the users to open them with
> C-c C-o, and it is convenient to allow them to do so.

Sorry, this is not convenient. This is just nonsense.

For example, Org prevents a user from inserting a footnote reference in
a comment (for good reasons), but links should be allowed? Can't every
part of Org agree?

AFAICT, a comment is a comment, and it is to be expected that comments
only contain raw text. According to the manual:

  Lines starting with zero or more whitespace characters followed by one
  '#' and a whitespace are treated as comments and will never be exported.

Note that "are treated as comments" is different from "will never be
exported" since we write both. The parser simply treats comments as
comments. Until recently, some parts of Org were careless and didn't
treat them as such. This is a fix.

[...]

> I'm done: please don't stop working on the parser just because the
> parser is not the only way to think about feature, and please just
> remember all participants on this thread are goodwilling users who
> try to make a point.

IMO, there is a single representation for the Org format, and it must be
defined clearly. Org syntax is an attempt to do so (but I never said it
was definitive) and Org elements implements it.

I started to work on the parser because it was high time to give Org
one. From the beginning, I wanted all core functions to rely on it, for
reasons I already explained. And for the same reasons, anything less is
not worthy, as it would become yet another part of Org interpreting Org
its own way. I never pretended to think or act otherwise.

Some users apparently don't want a parser, i.e. a global consistent
definition of Org syntax, for reasons that I cannot understand. I think
it is a major mistake, but I'm fine with it.

So, there is no middle path. Either the project continues towards its
goal, or it stops here. Obviously, I'd rather have the maintainer's
opinion on this.


Regards,

-- 
Nicolas Goaziou

Re: link interfering with brackets when abbreviated

[Bastien]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

>> That said, Org syntax is not the only valid representation of an
>> org-mode buffer.
>
> It should be, or the whole concept is moot. If "Org syntax" is only
> valid during export, if fontification interprets Org differently, if
> users see Org differently too, there is no point in defining a syntax.
> Just let everyone implement its own private Org.

You are pushing things to the extreme.

Even if the syntax is used for 57% of the functions, there is a point
in defining one, and there would be no point for all users to define
their own.

>> It is the only useful one when exporting a buffer to a certain format,
>> because we need to map the structure of the original contents to the
>> structure of the target one.
>
> Again, this will not work if we do not agree on the structure. This will
> raise questions like "How come that my document is exported this way,
> even though I interpret it that way?".

We agree here: a proper syntax is needed for exporting.

>> But another representation of an org-mode buffer is the one that
>> a user has in mind when manipulating it, part of this representation
>> depends on Org syntax, part of it depends on Emacs general facilities.
>>
>> For example, Emacs and the user have a notion of `end-of-line', but
>> Org syntax does not.  Org syntax says whitespaces after an object
>> belong to the object but my immediate representation says they do not.
>> Or we do have a notion of visual indentation (with org-indent-mode
>> turned on), but this does not correspond to any real content, and /a
>> fortiori/ to an Org syntactic element, since this is just visual
>> sugar.
>
> You are mixing subjects here. For example, `org-end-of-line' is backed
> up with Elements already. This has nothing to do with Org syntax.

I'm sorry you don't see the point: `org-end-of-line' being backed by
Org syntax does not mean "the end of a line" has a meaning for the Org
parser.  My point is: it does not have a meaning for the parser while
is has one for the user.  This illustrates the fact there are several
representations, which I need for my reasoning: if there was a unique
representation, there would be no point in trying to balance several
of them when defining features.

>> There is a minimalistic view of Org as the combination of a syntax and
>> a set of features to manipulate this syntax.  There is a maximalistic
>> view of Org as a syntax, a set of features to manipulate it, and a set
>> of Emacs facilities to manipulate the mental representation of an Org
>> buffer, which will *never* be the same than the parser representation.
>
> Of course, but the "maximalistic" view can only be a superset of the
> "minimalistic" one. The former can see more than the latter, but it
> cannot disagree on what the latter sees.

Ideally, yes.

>> But Org is no library: it's the Emacs way to manipulate Org files.
>
> And Org files are expressed in an Org format. Org syntax tries to define
> it.

Agreed.

>> Hence the case for links in comment, for example: the user read them
>> as links, there is no value in preventing the users to open them with
>> C-c C-o, and it is convenient to allow them to do so.
>
> Sorry, this is not convenient. This is just nonsense.

Let's ping the users about this particular nonsense.

> For example, Org prevents a user from inserting a footnote reference in
> a comment (for good reasons), but links should be allowed? Can't every
> part of Org agree?
>
> AFAICT, a comment is a comment

Er.. shall I quote Alice in Wonderland here?

You seem to consider Org comments as comments in programming languages.
But Org is not a programming language, it is used for any kind of text.

> IMO, there is a single representation for the Org format, and it must be
> defined clearly. Org syntax is an attempt to do so (but I never said it
> was definitive) and Org elements implements it.
>
> I started to work on the parser because it was high time to give Org
> one. From the beginning, I wanted all core functions to rely on it, for
> reasons I already explained. And for the same reasons, anything less is
> not worthy, as it would become yet another part of Org interpreting Org
> its own way. I never pretended to think or act otherwise.

This is not a all-or-nothing issue, and all-or-less is still different
than all-or-nothing.

> Some users apparently don't want a parser, i.e. a global consistent
> definition of Org syntax, for reasons that I cannot understand.

Nobody ever said anything coming close to that.

> So, there is no middle path.

I can see plenty of them!

> Either the project continues towards its goal, or it stops
> here. Obviously, I'd rather have the maintainer's opinion on this.

Yes.  In the meantime, other users' voices can help us step back and
see things differently.

-- 
 Bastien

Re: link interfering with brackets when abbreviated

[Thorsten Jolitz]

Bastien <bzg@gnu.org> writes:

> Yes.  In the meantime, other users' voices can help us step back and
> see things differently.

I used to have outcommented (w3m-browse-url ...) links in my init.el
file, and I could evaluate them when I wanted to look-up something
although they were outcommented:

#+begin_src emacs-lisp
;; *** Package Manager
;; (w3m-browse-url "https://github.com/dimitri/el-get#readme")
[...]
#+end_src

So, although I understand Nicolas reasoning very well and agree with
him for the most part, from a practical point of view I would like
links to work even when commented out (like Bastien and others). But
if the price for this would be Nicolas abandoning the parser/exporter
development I would say that this would be a VERY BAD DEAL for
Org-mode.

Just my 2c

-- 
cheers,
Thorsten

Re: link interfering with brackets when abbreviated

[Gustav Wikström]

[-- Attachment #1: Type: text/plain, Size: 1277 bytes --]

Hi, a "user" signing in. Although not involved in the development of this
piece of software I'm taking the opportunity to chime in anyway.

I'd like to give Nicolas Goaziou my support in this issue. It makes it much
simpler to understand, use, develop and maintain the software if it is
congruent. A well defined syntax, and tools that respect the rules of how
to parse it, will IMO be of big importance moving forward.

About the issue of two links on the same line.. From my perspective (for
what it's worth); Trying to open a link when not being inside a link with
the mark should give the same behaviour as trying to open a link when on a
headline. It is not certain which link is intended to be opened, so why not
give the user the options available instead of guessing? Set the scope to
parse to the current paragraph, to make a difference from calling C-c C-o
from the headline. That, to me, is the intuitive behaviour.

About the issue of links in comments (My opinion, for what it's worth):
It's a comment.. Expect it to behave as one. Worst case: copy the link and
paste it in the browser.

About the issue of links in properties: Wouldn't it be nice to allow this?
Maybe a future functionality to consider?


Best regards

Gustav Wikström

[-- Attachment #2: Type: text/html, Size: 1355 bytes --]

Re: link interfering with brackets when abbreviated

[Ista Zahn]

Another user here, chiming in to support Nicolas's position. From my
perspective orgmode is so vast and complicated that the number one
thing we need (even from a user perspective) is predictability. I'd
rather see minor conveniences removed in favor of a constancy and a
logical interface.

Best,
Ista

On Sun, Mar 2, 2014 at 4:16 PM, Gustav Wikström <gustav.erik@gmail.com> wrote:
> Hi, a "user" signing in. Although not involved in the development of this
> piece of software I'm taking the opportunity to chime in anyway.
>
> I'd like to give Nicolas Goaziou my support in this issue. It makes it much
> simpler to understand, use, develop and maintain the software if it is
> congruent. A well defined syntax, and tools that respect the rules of how to
> parse it, will IMO be of big importance moving forward.
>
> About the issue of two links on the same line.. From my perspective (for
> what it's worth); Trying to open a link when not being inside a link with
> the mark should give the same behaviour as trying to open a link when on a
> headline. It is not certain which link is intended to be opened, so why not
> give the user the options available instead of guessing? Set the scope to
> parse to the current paragraph, to make a difference from calling C-c C-o
> from the headline. That, to me, is the intuitive behaviour.
>
> About the issue of links in comments (My opinion, for what it's worth): It's
> a comment.. Expect it to behave as one. Worst case: copy the link and paste
> it in the browser.
>
> About the issue of links in properties: Wouldn't it be nice to allow this?
> Maybe a future functionality to consider?
>
>
> Best regards
>
> Gustav Wikström

Re: link interfering with brackets when abbreviated

[Josiah Schwab]

Hi All,

> Yes.  In the meantime, other users' voices can help us step back and
> see things differently.

(For reference: I have been using org-mode -- for TODO lists and note
taking -- for a few years now, but have not contributed code.)

I imagine myself as a naive user (which does not take too much) who does
not know the internal structure of org and its syntax.

If I execute org-open-at-point at the start of a line with a link and
get the message "No link at point", I think "Ah, I need to move point
onto the link."  I do and all is good.

So I find myself in Nicholas' camp in regards to the discussion of
org-open-at-point.  I think any pains -- which seem to be quite minor --
associated with the change in behavior are worth the gains in uniformity
and clarity.  And here I am thinking here not only about my day-to-day
use of org-mode, but also of my (slow) journey towards understanding
more about how it works.

But if my point is on a link in a comment, and I do org-open-at-point,
and get the message "No link at point", then I think, "But why?  I am on
something that looks like a link".

So in regards to the discussion of links in comments, I see more clearly
the argument Bastien, Michael, etc. are making.  But I am unsure here
how to judge whether the benefits outweigh the costs.

Perhaps that is helpful; perhaps not!

Best Regards,
Josiah

Re: link interfering with brackets when abbreviated

[Michael Brand]

Hi all

On Sun, Mar 2, 2014 at 4:49 PM, Bastien <bzg@gnu.org> wrote:
> In the meantime, other users' voices can help us step back and
> see things differently.

May I ask at least Nicolas and Bastien:

When you carefully reread my last post (Thursday)
http://lists.gnu.org/archive/html/emacs-orgmode/2014-02/msg00991.html
of this thread: Is it clear that when point is after the character "x"

    - x y [2014-03-03 Mon] z t http://orgmode.org

I want to keep "M-x org-open-at-point" to result in the error "No link
found", in any case?


The other reason for this post is an update of my function
f-open-link-between-point-and-eol to deal with links in Org mode that
occur in a place that is not a link according to Org syntax (currently
two cases in discussion). I bind this function still to "C-c o". Not
to "C-c C-o", because I want to use f-open-link-between-point-and-eol
also outside of Org and because I want to have the possibility to use
"C-c C-o" to find out on which point not and on which point
org-open-at-point results in the error "No link found", for example to
learn more about Org syntax and how to better cooperate with it.

#+BEGIN_SRC emacs-lisp
  (defun f-open-link-between-point-and-eol ()
    "Move to and open first link between point and end of line.
  As long as not yet at end of line and as long as
  `org-open-at-point' and `browse-url-at-point' result in an error
  advance point by one character. For Org and other major modes."
    (interactive)
    (let ((p (point)) opened)
      (while (not (or (eolp)
                      (progn (ignore-errors
                               (cond
                                ;; Org mode
                                ((eq major-mode 'org-mode)
                                 (org-open-at-point)
                                 (setq opened 'org-open-at-point))
                                ;; Maybe more major modes that have an
                                ;; open function specific to their
                                ;; syntax
                                ))
                             (unless opened
                               (ignore-errors
                                 (browse-url-at-point)
                                 (setq opened 'browse-url-at-point)))
                             opened)))
        (forward-char))
      (if opened
          (message "Link opened with %s" opened)
        (goto-char p)
        (user-error "No link between point and end of line"))))
#+END_SRC

Here f-open-link-between-point-and-eol is with "(org-open-at-point)"
but actually I'm using "(org-open-at-point 1)" instead.

Michael

Re: link interfering with brackets when abbreviated

[Christian Moe]

+1  -- Another user chiming in, broadly in agreement with Gustav,
details below.

Gustav Wikström writes:

> Hi, a "user" signing in. Although not involved in the development of this
> piece of software I'm taking the opportunity to chime in anyway.
>
> I'd like to give Nicolas Goaziou my support in this issue. It makes it much
> simpler to understand, use, develop and maintain the software if it is
> congruent. A well defined syntax, and tools that respect the rules of how
> to parse it, will IMO be of big importance moving forward.

I think we all agree on that. And I think the *presumption* should be
that incongruent features will have to go. Still, Org is about letting
users organize stuff as conveniently and flexibly as possible, and if
some very convenient feature relies on some ad-hoc solution, it should
be possible, on a case-by-case basis, to consider keeping it. 

> About the issue of two links on the same line.. From my perspective (for
> what it's worth); Trying to open a link when not being inside a link with
> the mark should give the same behaviour as trying to open a link when on a
> headline. It is not certain which link is intended to be opened, so why not
> give the user the options available instead of guessing? Set the scope to
> parse to the current paragraph, to make a difference from calling C-c C-o
> from the headline. That, to me, is the intuitive behaviour.

+1? Sounds right to me. (This would also alert a user who *accidentally*
hits C-c C-o, instead of unexpectedly moving point to a target he didn't
mean to visit. Not sure if it's ever happened to me, but it could.)

BTW, in years of using Org I never ever realized that you *could* use
C-c C-o for anything outside a link...

> About the issue of links in comments (My opinion, for what it's worth):
> It's a comment.. Expect it to behave as one. Worst case: copy the link and
> paste it in the browser.

+1. I do have links in # comments. It's convenient, but I'd be OK with
the inconvenience of giving them up, if it helps make Org easier to
maintain. I can always put stuff in drawers.

> About the issue of links in properties: Wouldn't it be nice to allow this?
> Maybe a future functionality to consider?

+1? As a user, I've never been quite sure if it's good practice to put
Org links -- or timestamps -- in properties. But since it does actually
work (still does in 8.2.3), I do put them there. And once they are
there, I find it very convenient to be able to visit the link, and
manipulate the timestamps in all the ways Org enables.

Yours,
Christian

Re: link interfering with brackets when abbreviated

[Matt Lundin]

Bastien <bzg@gnu.org> writes:

> Hi Nicolas,
>
> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>
>> Sorry for not being clear. I did try, I didn't get any error. My dummy
>> entry was:
>>
>>   * TODO [[http://orgmode.org]]
>>
>> in a block agenda and
>>
>>   * [[http://orgmode.org]]
>>     DEADLINE: <2014-03-01 sam.>
>>
>> in regular agenda.
>>
>> Both times, I could open the link. So, could you send me a dummy entry
>> where the bug can be reproduced? It will save me a lot of time.
>
> Sorry, all my time has been swallowed by my rant, I will try to
> provide a reproducible recipe later on -- but the error is real.
>
> It works for small files I tested, but not for my real agenda files.
> I suspect maybe the lazy parsing of open-but-not-yet-visible agenda
> files somehow prevents the links to get opened.
>
> Maybe someone else can also report if C-c C-o works on his agendas.

I cannot reproduce this (though I do have trouble opening escaped links
-- see bug report in another thread).

Matt

Re: link interfering with brackets when abbreviated

[Sebastien Vauban]

Gustav Wikström wrote:
> About the issue of links in comments (My opinion, for what it's worth):
> It's a comment.. Expect it to behave as one. Worst case: copy the link and
> paste it in the browser.

Or use `M-x ffap' (find-file-at-point)...

Best regards,
  Seb

-- 
Sebastien Vauban

Re: link interfering with brackets when abbreviated

[Samuel Wales]

tldr, and wary of bikeshedding, but a fool so i rush in:

  1] currently in maint the awesome package org-mouse.el activates
links in comments.  RET does not.  Perhaps this could be made more
consistent or optional?
  2] currently in maint links are not fontified in comments or
footnote definitions.  Perhaps they could be if they are active to
avoid major surprises, or optionally for those who like me prefer [as
a foolish bikeshedding user] to fontify them.

then again, i am the guy who insists on multiple paragraphs in inline
footnote definitions, including for fontification if the fontification
ever gets parserified.  how crazy can you get?!

btw, the nuclear power plant MUST be painted blue.

Re: link interfering with brackets when abbreviated

[Samuel Wales]

if and only if it is not desirable to highlight links, then perhaps
they could be un-collapsed so that you
[[http://dangerous-place.com][know they are links]].

Re: link interfering with brackets when abbreviated

[Sebastien Vauban]

Samuel Wales wrote:
> if and only if it is not desirable to highlight links, then perhaps
> they could be un-collapsed so that you
> [[http://dangerous-place.com][know they are links]].

M-x visible-mode

Best regards,
  Seb

-- 
Sebastien Vauban

Re: link interfering with brackets when abbreviated

[Samuel Wales]

On 3/3/14, Sebastien Vauban <sva-news@mygooglest.com> wrote:
>> [[http://dangerous-place.com][know they are links]].
>
> M-x visible-mode

the whole point is that comments and footnote definitions obscure the
fact that there is a link there.

who wants to run visible-mode all the time?  that would defeat the
purpose of link collapsing.

nobody is going to run it if there is no indication that there is a link there.

Re: link interfering with brackets when abbreviated

[Robert Horn]

I'm a user who doesn't much care about link following command behavior,
but Bastien's point about context is important.  The behavior of a
command needs to depend upon much more than just syntax.

Two really dramatic examples are region narrowing and outline folding.
When operating on a narrowed region there are a great many differences
in how commands behave.  Similarly, when a headline is folded, commands
behave very differently.

So be very careful to include consideration of the context when defining
commands. Some context is much more subtle.

My one link related comment is that I'm very puzzled by those who think
that links in comments should not be followed.  In programs I make heavy
use of links in comments so that the comment can include a "see this
[document]" as part of the comment.  It's a link that other programmers
want to follow.  I don't often put comments into my org files, but I
would expect to follow links in them also.  In programming a comment
means "don't try to compile or execute this".  It doesn't mean
destruction of all other semantic value.  It means a highly selective
removal of semantics.

I would expect links in comments to still be followable.

R Horn

Re: link interfering with brackets when abbreviated

[Sebastien Vauban]

Samuel Wales wrote:
> On 3/3/14, Sebastien Vauban <sva-news-D0wtAvR13HarG/iDocfnWg@public.gmane.org> wrote:
>>> [[http://dangerous-place.com][know they are links]].
>>
>> M-x visible-mode
>
> the whole point is that comments and footnote definitions obscure the
> fact that there is a link there.
>
> who wants to run visible-mode all the time?  that would defeat the
> purpose of link collapsing.

OK. I thought you were asking for "conscious", on-the-fly appearance of
links.

> nobody is going to run it if there is no indication that there is
> a link there.

What type of indication do you have in mind?

Best regards,
  Seb

-- 
Sebastien Vauban

Re: link interfering with brackets when abbreviated

[Samuel Wales]

hi sebastien,

as i wrote, my preference is for links to be fontified in comments and
inline footnote definitions the same way as everywhere else.

samuel

On 3/4/14, Sebastien Vauban <sva-news@mygooglest.com> wrote:
> What type of indication do you have in mind?

Re: link interfering with brackets when abbreviated

[Bastien]

Samuel Wales <samologist@gmail.com> writes:

> btw, the nuclear power plant MUST be painted blue.

FWIW, I strongly disagree.

-- 
 Bastien