Okay, back to link escaping.

What this is about:

Current implementation of percent escaping URIs uses a whitelist
approach, e.g. only percent escapes characters that are in
`org-link-escape-chars' or in a user supplied list.  This is a problem
because using this function requires knowledge about all possible
characters that could occur in a URI -- and URIs are limited to plain
ASCII, meaning a call to the function must list literally all possible
characters and their escapings to get a properly percent escaped
string.

To solve this problem the behavior of the function is changed to
percent escape every character that is an ASCII controll character or
not an ASCII character.  Subsequently the unescaping function is
changed accordingly to handle percent encoded multibyte unicode
characters.

1/ I did some testing with the new proposed org-link-escape and the
modified `org-protocol-unhex-string': Create a random string with
ASCII and multibyte unicode characters, randomly taken from
(ucs-names); perform escape-unescape; compare the result with the
original string.  Works perfect.  Testing randomly created string with
old escaping of non-ascii strings is on the list.

2/ Of course there could still be the problem, that a user had created
a sequence of old escapes that the new unescaping function will
interpret wrongly.  Not sure how likely this is, but in theory this
could happen.  Personally I think we should risk breaking peoples'
links in this way.

3/ I highly suggest changing the syntax of `org-link-escape-chars'.
Currently it is a list of cons with the character in car and the
replacement string in cdr.  Using such a table in escaping is easy
(assq char table), but in the unescaping process it might get tricky.

Moreover if the function should do percent escaping, the escpae
sequence is already determined by the string to replace.  The new
syntax would be simply a list of characters to escape in addition to
the rule mentioned above (< 32 and > 126).

This would break compatibility with functions that have used
org-link-escape/unescape for something else than percent escaping
(e.g. replace ] by %FF and not %5D and such).  But this again is
bearable: Although it the docstring talks about escaping things that
are problematic, the only way to do such escaping in a standardized
way is percent escaping.

4/ If all agree that breaking backward incompatibility in the case
mentioned above (or did I forgot one?) is bearable, I would go ahead
and perform the necessary changes:

  1. Use the new algorithm in `org-link-escape'
  2. Modify Syntax of `org-link-escape-chars'
  3. Issue a warning if someone calls `org-link-escape' with a table
     of the old syntax.
  4. Move the unescaping functions from org-protocol.el to org.el and
     rename them.
  5. Declare `org-protocol-unhex-string' and
     `org-protocol-unhex-compound' obsolete (make-obsolete).
  6. Drop a message to the list informing about these changes.
  7. Wait some months and purge the obsolete functions.

Best,
  -- David
--
OpenPGP... 0x99ADB83B5A4478E6
Jabber.... dmjena@jabber.org
Email..... dmaus@ictsoc.de