From: Sebastian Rose <sebastian_rose@gmx.de>
To: "Jan Böcker" <jan.boecker@jboecker.de>
Cc: emacs-orgmode@gnu.org, dmg@uvic.ca
Subject: Re: Re: org-protocol: non-ASCII characters
Date: Mon, 08 Feb 2010 12:30:09 +0100 [thread overview]
Message-ID: <878wb4rmm6.fsf@gmx.de> (raw)
In-Reply-To: <4B6D7E1A.3030807@jboecker.de> ("Jan Böcker"'s message of "Sat, 06 Feb 2010 15:35:06 +0100")
Jan Böcker <jan.boecker@jboecker.de> writes:
> On 06.02.2010 14:50, Jan Böcker wrote:
>> AFAIK, your current approach is correct.
>
> I was wrong. The attached patch fixes a bug in the encode_uri function.
> That fixes the non-ASCII characters problem in xournal for me.
>
> The gchar type is just typedef'd to char, which means it is signed. To
> get the byte value, it must be cast to unsigned int first.
>
> - Jan
Hi Jan and Daniel!
Sorry for answering with that long delay. I read Daniel's mail last
week, but I had to think about the answer.
I'll just describe, what the `org-protocol-unhex-string' functions do
here, and what they expect as arguments.
Basically, it is OK to url-encode each character who's binary
representation start with 1 (i.e., the value of the character is higher
than 127). The text to be url-encoded should be UTF-8 ideally.
If you use glib::ustring, it's easy to transform any iso-8859 string to
utf-8. Each character, whos binary representation start with a 1, has to
be url-encoded as well as the `%' character [1], but you could as
url-encode the entire utf-8 string.
The function that does the decoding is `org-protocol-unhex-string' which
in turn uses `org-protocol-unhex-compound'.
`man utf-8` shows, how org-protocol tries to decode characters.
The JavaScript-Funktion `encodeURIComponent()' returns exactly what we
need. It recodes a string to utf-8 and then encodes all characters,
except digits, ASCII letters and these punctuation characters: -_.!~*'()
See ECMA-262 Standard, Section 15.1.3
(http://bclary.com/2004/11/07/ecma-262.html#a-15.1.3 [2]):
"The character is first transformed into a sequence of octets using
the UTF-8 transformation..."
Again, note, that the decoding mechanism relies on the fact, that the
sequence to decode is url-encoded UTF-8.
Example:
The url-encoded unicode representation of the German umlaut `ö' is
`%C3%B6'. Thus
(org-protocol-unhex-string "%C3%B6")
gives you "ö".
In iso-8859-1, the url-encoded representation of the same character `ö' was
`%F6'. But
(org-protocol-unhex-string "%F6")
gives you "" - the empty string. There is no utf-8 character with this binary
representation, since every byte starting with a 1 (i.e. is bigger than 127)
starts a multibyte sequence (2 or more bytes).
But:
(org-protocol-unhex-string "%2F%3C")
gives you, as expected, "/<" which shows, that you could savely
url-encode each and every character of a utf-8 encoded string.
== Footnotes:
[1] The percent character `%' has to be encoded, if followed by
[0-9A-Fa-f]{2}, because org-protocol will assume, that a sequence
matching "\\(%[0-9a-f][0-9a-f]\\)+" is an encoded character. That
said, a `%' has to be url-encoded, since one will hardly ever
know for sure, that a `%' is never followed by "[0-9a-f][0-9a-f]".
[2] Get a PDF version of ECMA-262 third edition here:
http://www.ecma-international.org/publications/standards/Ecma-262.htm
next prev parent reply other threads:[~2010-02-08 11:30 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-01-31 8:39 org-remember support in xournal D M German
2010-01-31 8:45 ` dmg
2010-02-03 9:19 ` Jan Böcker
2010-02-03 9:22 ` dmg
2010-02-04 18:43 ` org-protocol: non-ASCII characters D M German
2010-02-06 13:50 ` Jan Böcker
2010-02-06 14:35 ` Jan Böcker
2010-02-08 11:30 ` Sebastian Rose [this message]
2010-02-12 22:23 ` dmg
2010-02-13 13:22 ` Jan Böcker
2010-02-15 9:41 ` Sebastian Rose
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=878wb4rmm6.fsf@gmx.de \
--to=sebastian_rose@gmx.de \
--cc=dmg@uvic.ca \
--cc=emacs-orgmode@gnu.org \
--cc=jan.boecker@jboecker.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).