Org markup and non-ASCII punctuation (was: org parser and priorities of inline elements)

[Ihor Radchenko <yantar92@posteo.net>]

Max Nikulin <manikulin@gmail.com> writes:

> On 21/11/2021 16:28, Ihor Radchenko wrote:
>> 
>> Also, is there any reason why we are not simply using punctuation
>> character class instead of listing punctuation chars explicitly (and
>> only for English)? What about "_你叫什么名字_？"
>
> It seems punctuation character class is too broad. E.g.
>      ¿ INVERTED QUESTION MARK
> normally appears before words, while "?" is usually after them. I do not 
> see anything special in
>      (category-set-mnemonics (char-category-set ?¿))
> that may help to discriminate such cases.

The last resort is define-category where we can manage exceptions.
But I think that even without distinguishing ?¿, we can improve the
situation for CJK users a lot.

We can probably split character categories into "left", "right", and
"neutral" with "(" being "left" example, ")" being "right" example, and
" " being "neutral" example.
We start from using the information we can extract from Unicode data and
modify it as necessary.

Then, emphasis will be defined as PRE MARKER ... MARKER POST with
PRE = left+neutral category
POST = right+neutral category

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>