Bug: Tildes in URL impact visible link text [9.3 (release_9.3 @ /usr/share/emacs/27.1/lisp/org/)]

Bug: Tildes in URL impact visible link text [9.3 (release_9.3 @ /usr/share/emacs/27.1/lisp/org/)]

[Chris Hunt]

I'm trying to create a link in an org file with this URL:

    https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#metricsV2:graph=~(view~'timeSeries~stacked~false~metrics~(~(~'CWAgent~'backup_time~'host~'desktop~'metric_type~'timing))~region~'us-east-1);query=~'*7bCWAgent*2chost*2cmetric_type*7d

and this description: "metrics".

To do it, I followed these steps:

1. run `emacs -Q test.org`, where test.org is not an existing file
2. emacs GUI displays, type `M-x org-insert-link`
3. a "Link" prompt is displayed, paste the link and press return
4. a "Description" prompt is displayed, type "metrics" and press return

At this point I expect to see only the text "metrics" in the buffer, with normal
link decoration (underline and highlight).

Instead, I see "~(~'CWAgent~metrics" with some mix of link decoration
and some other style, as shown in the following image:

https://i.imgur.com/vb9vE43.png

The actual text from the file is

    [[https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#metricsV2:graph=~(view~'timeSeries~stacked~false~metrics~(~(~'CWAgent~'backup_time~'host~'desktop~'metric_type~'timing))~region~'us-east-1);query=~'*7bCWAgent*2chost*2cmetric_type*7d][metrics]]

This is emacs 27.1 from http://ppa.launchpad.net/kelleyk/emacs/ubuntu,
with org version 9.3.

Emacs  : GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version
3.22.30, cairo version 1.15.10)
 of 2020-09-19
Package: Org mode version 9.3 (release_9.3 @ /usr/share/emacs/27.1/lisp/org/)

current state:
==============
(setq
 org-src-mode-hook '(org-src-babel-configure-edit-buffer
             org-src-mode-configure-edit-buffer)
 org-link-shell-confirm-function 'yes-or-no-p
 org-metadown-hook '(org-babel-pop-to-session-maybe)
 org-clock-out-hook '(org-clock-remove-empty-clock-drawer)
 org-mode-hook '(#[0 "\300\301\302\303\304$\207"
           [add-hook change-major-mode-hook org-show-all append local]
           5]
         #[0 "\300\301\302\303\304$\207"
           [add-hook change-major-mode-hook org-babel-show-result-all
            append local]
           5]
         org-babel-result-hide-spec org-babel-hide-all-hashes)
 org-archive-hook '(org-attach-archive-delete-maybe)
 org-confirm-elisp-link-function 'yes-or-no-p
 org-agenda-before-write-hook '(org-agenda-add-entry-text)
 org-metaup-hook '(org-babel-load-in-session-maybe)
 org-bibtex-headline-format-function #[257 "\300 \236A\207" [:title] 3
"\n\n(fn ENTRY)"]
 org-babel-pre-tangle-hook '(save-buffer)
 org-tab-first-hook '(org-babel-hide-result-toggle-maybe
              org-babel-header-arg-expand)
 org-occur-hook '(org-first-headline-recenter)
 org-cycle-hook '(org-cycle-hide-archived-subtrees org-cycle-show-empty-lines
          org-optimize-window-after-visibility-change)
 org-speed-command-hook '(org-speed-command-activate
              org-babel-speed-command-activate)
 org-confirm-shell-link-function 'yes-or-no-p
 org-link-parameters '(("attachment" :follow org-attach-open-link :export
            org-attach-export-link :complete
            org-attach-complete-link)
               ("id" :follow org-id-open)
               ("eww" :follow eww :store org-eww-store-link)
               ("rmail" :follow org-rmail-open :store
            org-rmail-store-link)
               ("mhe" :follow org-mhe-open :store org-mhe-store-link)
               ("irc" :follow org-irc-visit :store org-irc-store-link
            :export org-irc-export)
               ("info" :follow org-info-open :export org-info-export
            :store org-info-store-link)
               ("gnus" :follow org-gnus-open :store
            org-gnus-store-link)
               ("docview" :follow org-docview-open :export
            org-docview-export :store org-docview-store-link)
               ("bibtex" :follow org-bibtex-open :store
            org-bibtex-store-link)
               ("bbdb" :follow org-bbdb-open :export org-bbdb-export
            :complete org-bbdb-complete-link :store
            org-bbdb-store-link)
               ("w3m" :store org-w3m-store-link) ("file+sys")
               ("file+emacs") ("shell" :follow org-link--open-shell)
               ("news" :follow
            #[257 "\301\300\302 Q!\207" ["news" browse-url ":"] 5
              "\n\n(fn URL)"]
            )
               ("mailto" :follow
            #[257 "\301\300\302 Q!\207" ["mailto" browse-url ":"]
              5 "\n\n(fn URL)"]
            )
               ("https" :follow
            #[257 "\301\300\302 Q!\207" ["https" browse-url ":"]
              5 "\n\n(fn URL)"]
            )
               ("http" :follow
            #[257 "\301\300\302 Q!\207" ["http" browse-url ":"] 5
              "\n\n(fn URL)"]
            )
               ("ftp" :follow
            #[257 "\301\300\302 Q!\207" ["ftp" browse-url ":"] 5
              "\n\n(fn URL)"]
            )
               ("help" :follow org-link--open-help)
               ("file" :complete org-link-complete-file)
               ("elisp" :follow org-link--open-elisp)
               ("doi" :follow org-link--open-doi))
 org-link-elisp-confirm-function 'yes-or-no-p
 )

Re: Bug: Tildes in URL impact visible link text [9.3 (release_9.3 @ /usr/share/emacs/27.1/lisp/org/)]

[Juan Manuel Macías]

Hello,

I think the problem comes because in that url the tilde does not have an
escape character. If it's just that case, you can try replacing each
tilde with %7E (see
https://www.w3schools.com/tags/ref_urlencode.asp). That way the
link description would have to be formatted correctly, without spurious
characters.

Regards

Juan Manuel 

Chris Hunt <chrahunt@gmail.com> writes:

> I'm trying to create a link in an org file with this URL:
>
>     https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#metricsV2:graph=~(view~'timeSeries~stacked~false~metrics~(~(~'CWAgent~'backup_time~'host~'desktop~'metric_type~'timing))~region~'us-east-1);query=~'*7bCWAgent*2chost*2cmetric_type*7d
>
> and this description: "metrics".
>
> To do it, I followed these steps:
>
> 1. run `emacs -Q test.org`, where test.org is not an existing file
> 2. emacs GUI displays, type `M-x org-insert-link`
> 3. a "Link" prompt is displayed, paste the link and press return
> 4. a "Description" prompt is displayed, type "metrics" and press return
>
> At this point I expect to see only the text "metrics" in the buffer, with normal
> link decoration (underline and highlight).
>
> Instead, I see "~(~'CWAgent~metrics" with some mix of link decoration
> and some other style, as shown in the following image:
>
> https://i.imgur.com/vb9vE43.png
>
> The actual text from the file is
>
>     [[https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#metricsV2:graph=~(view~'timeSeries~stacked~false~metrics~(~(~'CWAgent~'backup_time~'host~'desktop~'metric_type~'timing))~region~'us-east-1);query=~'*7bCWAgent*2chost*2cmetric_type*7d][metrics]]
>
> This is emacs 27.1 from http://ppa.launchpad.net/kelleyk/emacs/ubuntu,
> with org version 9.3.
>
> Emacs  : GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version
> 3.22.30, cairo version 1.15.10)
>  of 2020-09-19
> Package: Org mode version 9.3 (release_9.3 @ /usr/share/emacs/27.1/lisp/org/)
>
> current state:
> ==============
> (setq
>  org-src-mode-hook '(org-src-babel-configure-edit-buffer
>              org-src-mode-configure-edit-buffer)
>  org-link-shell-confirm-function 'yes-or-no-p
>  org-metadown-hook '(org-babel-pop-to-session-maybe)
>  org-clock-out-hook '(org-clock-remove-empty-clock-drawer)
>  org-mode-hook '(#[0 "\300\301\302\303\304$\207"
>            [add-hook change-major-mode-hook org-show-all append local]
>            5]
>          #[0 "\300\301\302\303\304$\207"
>            [add-hook change-major-mode-hook org-babel-show-result-all
>             append local]
>            5]
>          org-babel-result-hide-spec org-babel-hide-all-hashes)
>  org-archive-hook '(org-attach-archive-delete-maybe)
>  org-confirm-elisp-link-function 'yes-or-no-p
>  org-agenda-before-write-hook '(org-agenda-add-entry-text)
>  org-metaup-hook '(org-babel-load-in-session-maybe)
>  org-bibtex-headline-format-function #[257 "\300 \236A\207" [:title] 3
> "\n\n(fn ENTRY)"]
>  org-babel-pre-tangle-hook '(save-buffer)
>  org-tab-first-hook '(org-babel-hide-result-toggle-maybe
>               org-babel-header-arg-expand)
>  org-occur-hook '(org-first-headline-recenter)
>  org-cycle-hook '(org-cycle-hide-archived-subtrees org-cycle-show-empty-lines
>           org-optimize-window-after-visibility-change)
>  org-speed-command-hook '(org-speed-command-activate
>               org-babel-speed-command-activate)
>  org-confirm-shell-link-function 'yes-or-no-p
>  org-link-parameters '(("attachment" :follow org-attach-open-link :export
>             org-attach-export-link :complete
>             org-attach-complete-link)
>                ("id" :follow org-id-open)
>                ("eww" :follow eww :store org-eww-store-link)
>                ("rmail" :follow org-rmail-open :store
>             org-rmail-store-link)
>                ("mhe" :follow org-mhe-open :store org-mhe-store-link)
>                ("irc" :follow org-irc-visit :store org-irc-store-link
>             :export org-irc-export)
>                ("info" :follow org-info-open :export org-info-export
>             :store org-info-store-link)
>                ("gnus" :follow org-gnus-open :store
>             org-gnus-store-link)
>                ("docview" :follow org-docview-open :export
>             org-docview-export :store org-docview-store-link)
>                ("bibtex" :follow org-bibtex-open :store
>             org-bibtex-store-link)
>                ("bbdb" :follow org-bbdb-open :export org-bbdb-export
>             :complete org-bbdb-complete-link :store
>             org-bbdb-store-link)
>                ("w3m" :store org-w3m-store-link) ("file+sys")
>                ("file+emacs") ("shell" :follow org-link--open-shell)
>                ("news" :follow
>             #[257 "\301\300\302 Q!\207" ["news" browse-url ":"] 5
>               "\n\n(fn URL)"]
>             )
>                ("mailto" :follow
>             #[257 "\301\300\302 Q!\207" ["mailto" browse-url ":"]
>               5 "\n\n(fn URL)"]
>             )
>                ("https" :follow
>             #[257 "\301\300\302 Q!\207" ["https" browse-url ":"]
>               5 "\n\n(fn URL)"]
>             )
>                ("http" :follow
>             #[257 "\301\300\302 Q!\207" ["http" browse-url ":"] 5
>               "\n\n(fn URL)"]
>             )
>                ("ftp" :follow
>             #[257 "\301\300\302 Q!\207" ["ftp" browse-url ":"] 5
>               "\n\n(fn URL)"]
>             )
>                ("help" :follow org-link--open-help)
>                ("file" :complete org-link-complete-file)
>                ("elisp" :follow org-link--open-elisp)
>                ("doi" :follow org-link--open-doi))
>  org-link-elisp-confirm-function 'yes-or-no-p
>  )
>

Re: Bug: Tildes in URL impact visible link text [9.3 (release_9.3 @ /usr/share/emacs/27.1/lisp/org/)]

[tomas]

[-- Attachment #1: Type: text/plain, Size: 547 bytes --]

On Sun, Dec 27, 2020 at 07:12:42PM +0100, Juan Manuel Macías wrote:
> Hello,
> 
> I think the problem comes because in that url the tilde does not have an
> escape character. If it's just that case, you can try replacing each
> tilde with %7E (see
> https://www.w3schools.com/tags/ref_urlencode.asp). That way the
> link description would have to be formatted correctly, without spurious
> characters.

This would be a bug: tilde is an allowed URI character [1]

Cheers

[1] https://tools.ietf.org/html/rfc3986#section-2.3

 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Bug: Tildes in URL impact visible link text [9.3 (release_9.3 @ /usr/share/emacs/27.1/lisp/org/)]

[Juan Manuel Macías]

Hello,

<tomas@tuxteam.de> writes:

> This would be a bug: tilde is an allowed URI character [1]

You're right. I have also noticed that if I write links like these:

[[*foo ~foo~ foo]][[foo]]

[[*foo =foo= foo]][[foo]]

There is an anomaly in the description similar to what Chris comments in
his email...

Regards,

Juan Manuel 

Re: Bug: Tildes in URL impact visible link text [9.3 (release_9.3 @ /usr/share/emacs/27.1/lisp/org/)]

[Chris Hunt]

> If it's just that case, you can try replacing each
> tilde with %7E (see
> https://www.w3schools.com/tags/ref_urlencode.asp). That way the
> link description would have to be formatted correctly, without spurious
> characters.

That's a good workaround, thank you. Manually replacing "~" in the URL
with "%7E" results in the expected
description ("metrics"). Using that link in Firefox navigates to the
page successfully, too.

I looked at the problem some more. I think the underlying behavior
leading to the bug is that `org-activate-links` sets an
`invisible` text property on the URL and brackets of link text.
`org-activate-code`, which runs afterwards, identifies
"code" in a way that isn't aware of links, then proceeds to drop the
"invisible" property. That behavior makes sense,
since in the usual case we're removing properties so the text is
displayed verbatim, but not when the whole text is contained
in the URL section of a link.