unicode chars in org-emphasis-regexp-components

unicode chars in org-emphasis-regexp-components

[Eric Abrahamsen]

Hi, I do a lot of plain text writing in org, and would love to be able
to allow emphasis markup when it's up flush against m-dashes (x2014),
n-dashes (x2013), and ellipses (x2026), possibly among other things.
When I try to add these characters to org-emphasis-regexp-components via
the customize interface, emacs complains about "junk at end of
expression: —…–" and won't let me save it. Unicode's not junk!

I know this is an emacs issue, not an org issue, but if there's a
workaround, I would love to know about it! Setting that variable outside
of customize looks hairy…

Thanks!
Eric

Re: unicode chars in org-emphasis-regexp-components

[Bastien]

Hi Eric,

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Hi, I do a lot of plain text writing in org, and would love to be able
> to allow emphasis markup when it's up flush against m-dashes (x2014),
> n-dashes (x2013), and ellipses (x2026), possibly among other things.
> When I try to add these characters to org-emphasis-regexp-components via
> the customize interface, emacs complains about "junk at end of
> expression: —…–" and won't let me save it. Unicode's not junk!

Can you share the value of your org-emphasis-regexp-components variable
so that we can test it?  Also please tell us what version of Org/Emacs
you are using, especially if this turns out to be an Emacs issue.

Thanks,

-- 
 Bastien

Re: unicode chars in org-emphasis-regexp-components

[Eric Abrahamsen]

On Sat, Feb 19 2011, Bastien wrote:

> Hi Eric,
>
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Hi, I do a lot of plain text writing in org, and would love to be able
>> to allow emphasis markup when it's up flush against m-dashes (x2014),
>> n-dashes (x2013), and ellipses (x2026), possibly among other things.
>> When I try to add these characters to org-emphasis-regexp-components via
>> the customize interface, emacs complains about "junk at end of
>> expression: —…–" and won't let me save it. Unicode's not junk!
>
> Can you share the value of your org-emphasis-regexp-components variable
> so that we can test it?  Also please tell us what version of Org/Emacs
> you are using, especially if this turns out to be an Emacs issue.
>
> Thanks,

I don't think I've changed this from the default:

org-emphasis-regexp-components:
(" 	('\"{" "- 	.,:!?;'\")}\\" " 	\n,\"'" "." 5)

org-version:
Org-mode version 7.4 (release_7.4.400.gc7700)

emacs-version:
GNU Emacs 23.2.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.22.0) of
2010-10-19

I'd kind of suspect the customize interface here, but I really don't
know how to go about setting this variable properly with setq.

Thanks!
Eric

Re: Re: unicode chars in org-emphasis-regexp-components

[Bastien]

Hi Eric,

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> I don't think I've changed this from the default:
>
> org-emphasis-regexp-components:
> (" 	('\"{" "- 	.,:!?;'\")}\\" " 	\n,\"'" "." 5)
>
> org-version:
> Org-mode version 7.4 (release_7.4.400.gc7700)
>
> emacs-version:
> GNU Emacs 23.2.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.22.0) of
> 2010-10-19
>
> I'd kind of suspect the customize interface here, 

Maybe -- don't hesitate to report it to Emacs developers.

> but I really don't know how to go about setting this variable properly
> with setq.

Well, C-h v org-emphasis-regexp-components RET is quite informative.
Don't be afraid by this meduse-like list of strings... these are just
four strings and a number :)

Best,

-- 
 Bastien

Re: unicode chars in org-emphasis-regexp-components

[Eric Abrahamsen]

On Sun, Feb 27 2011, Bastien wrote:

> Hi Eric,
>
>> but I really don't know how to go about setting this variable properly
>> with setq.
>
> Well, C-h v org-emphasis-regexp-components RET is quite informative.
> Don't be afraid by this meduse-like list of strings... these are just
> four strings and a number :)

Okay, I was getting confused by the escaped double quotes. I've set it
properly, so far as I can tell, but it's still failing to recognize
emphasis markers when they're next to my unicode characters,
specifically—the m-dash and ellipsis…

Is this somehow debuggable, say in an elisp REPL using internal
functions? Can I provide any more useful information?

Eric

Re: Re: unicode chars in org-emphasis-regexp-components

[Bastien]

Hi Eric,

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Okay, I was getting confused by the escaped double quotes. I've set it
> properly, so far as I can tell, but it's still failing to recognize
> emphasis markers when they're next to my unicode characters,
> specifically—the m-dash and ellipsis…
>
> Is this somehow debuggable, say in an elisp REPL using internal
> functions? Can I provide any more useful information?

Yes - you may share the new value of org-emphasis-regexp-components
(with unicode characters) so that we can try ourselves.  Should be a
matter of using the right representation of these chars when setting 
the variable.

Thanks,

-- 
 Bastien

Re: Re: unicode chars in org-emphasis-regexp-components

[Eric Abrahamsen]

On Wed, Mar 02 2011, Bastien wrote:
> Hi Eric,
>
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Okay, I was getting confused by the escaped double quotes. I've set it
>> properly, so far as I can tell, but it's still failing to recognize
>> emphasis markers when they're next to my unicode characters,
>> specifically—the m-dash and ellipsis…
>>
>> Is this somehow debuggable, say in an elisp REPL using internal
>> functions? Can I provide any more useful information?
>
> Yes - you may share the new value of org-emphasis-regexp-components
> (with unicode characters) so that we can try ourselves.  Should be a
> matter of using the right representation of these chars when setting 
> the variable.

Here goes. Hopefully the tabs and newlines and invisible characters and
whatnot will come through properly. I expect gnus will try to strip
unprintable characters. At any rate, all I've added is the m-dash and
ellipsis to the first two strings (and changed the number of allowed
newlines to 5).

org-emphasis-regexp-components
(" 	('\"{…—" "…—- 	.,:!?;'\")}\\" " 	
,\"'" "." 5)


Thanks!

Re: Re: unicode chars in org-emphasis-regexp-components

[Eric Abrahamsen]

On Wed, Mar 02 2011, Bastien wrote:

> Hi Eric,
>
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Okay, I was getting confused by the escaped double quotes. I've set it
>> properly, so far as I can tell, but it's still failing to recognize
>> emphasis markers when they're next to my unicode characters,
>> specifically—the m-dash and ellipsis…
>>
>> Is this somehow debuggable, say in an elisp REPL using internal
>> functions? Can I provide any more useful information?
>
> Yes - you may share the new value of org-emphasis-regexp-components
> (with unicode characters) so that we can try ourselves.  Should be a
> matter of using the right representation of these chars when setting 
> the variable.

I just realized that many LaTeX-style macros will export correctly as
HTML entities, so in a sense this isn't strictly necessary. If "---"
turns into an m-dash in both HTML and LaTeX then the problem is more or
less solved, though on principle I'd definitely prefer that unicode work
correctly. Also, there doesn't seem to be a working macro for curly
quotes to HTML…

Thanks again,
Eric

Re: Re: unicode chars in org-emphasis-regexp-components

[Eric Abrahamsen]

On Wed, Mar 02 2011, Bastien wrote:

> Hi Eric,
>
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Okay, I was getting confused by the escaped double quotes. I've set it
>> properly, so far as I can tell, but it's still failing to recognize
>> emphasis markers when they're next to my unicode characters,
>> specifically—the m-dash and ellipsis…
>>
>> Is this somehow debuggable, say in an elisp REPL using internal
>> functions? Can I provide any more useful information?
>
> Yes - you may share the new value of org-emphasis-regexp-components
> (with unicode characters) so that we can try ourselves.  Should be a
> matter of using the right representation of these chars when setting 
> the variable.

I just looked at the full listing for org-entities, will now shut up
until I've finished reading all the documentation, sorry.