* Inconsistent text markup handling when double-nesting markers
@ 2023-10-09 23:02 Tom Alexander
2023-10-10 12:07 ` Ihor Radchenko
0 siblings, 1 reply; 9+ messages in thread
From: Tom Alexander @ 2023-10-09 23:02 UTC (permalink / raw)
To: emacs-orgmode
I used the following test document:
```
__foo__
**foo**
```
I'd expect the two to behave the same but the first one parses as:
```
(paragraph
"_"
(subscript "foo")
"__"
)
```
Whereas the second parses as:
```
(paragraph
(bold
(bold
"foo"
)
)
)
```
This pattern happens in worg at [2]
Looking at the description for text markup in the syntax document[1], I don't see any reason the first wouldn't be parsed as an underline:
1. PRE: valid because it is the beginning of a line
2. MARKER: valid underscore
3. CONTENTS: valid. Series of objects from standard set includes both subscript and text markup, so regardless of how we parse the interior, its valid. Also cannot begin or end with whitespace but there is no whitespace in the CONTENTS.
4. MARKER: valid underscore
5. POST: Only valid if we extend the underline to the 2nd underscore so it ends at the end of the line. But the 2nd line shows us that having copies of the marker inside the CONTENTS is fine so I see two possible expected parses of the CONTENTS:
4a. (underline "foo")
4b. ((subscript "foo") (plain-text "_"))
I also ran the following test document to further prove that having copies of the marker inside the CONTENTS is fine:
```
*foo*bar*
```
which parses as (bold "foo*bar")
So the only way the top line would fail to parse as an underline is if it matched the first closing underscore as closing the underline, but that would be invalid because underscore is not a valid POST character and invalid copies of the closing marker are ignored as proven by both "**foo**" and "*foo*bar*".
[1] https://orgmode.org/worg/org-syntax.html#Emphasis_Markers
[2] https://git.sr.ht/~bzg/worg/tree/ba6cda890f200d428a5d68e819eef15b5306055f/org-contrib/babel/intro.org#L117
--
Tom Alexander
pgp: https://fizz.buzz/pgp.asc
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Inconsistent text markup handling when double-nesting markers
2023-10-09 23:02 Inconsistent text markup handling when double-nesting markers Tom Alexander
@ 2023-10-10 12:07 ` Ihor Radchenko
2023-10-11 2:23 ` Max Nikulin
0 siblings, 1 reply; 9+ messages in thread
From: Ihor Radchenko @ 2023-10-10 12:07 UTC (permalink / raw)
To: Tom Alexander; +Cc: emacs-orgmode
"Tom Alexander" <tom@fizz.buzz> writes:
> I used the following test document:
> ```
> __foo__
>
> **foo**
> ```
>
> I'd expect the two to behave the same but the first one parses as:
> ```
> (paragraph
> "_"
> (subscript "foo")
> "__"
> )
> ```
Fixed, on main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=fe23bec60
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Inconsistent text markup handling when double-nesting markers
2023-10-10 12:07 ` Ihor Radchenko
@ 2023-10-11 2:23 ` Max Nikulin
2023-10-11 9:15 ` Ihor Radchenko
0 siblings, 1 reply; 9+ messages in thread
From: Max Nikulin @ 2023-10-11 2:23 UTC (permalink / raw)
To: Tom Alexander; +Cc: emacs-orgmode
On 10/10/2023 19:07, Ihor Radchenko wrote:
> "Tom Alexander" writes:
>
>> I used the following test document:
>> ```
>> __foo__
>>
>> **foo**
>> ```
>
> Fixed, on main.
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=fe23bec60
Isn't nested bold for "**bold**" a bug? Generally it is not allowed and
*b1 *b2* b3*
is parsed as bold only for "b1 *b2".
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Inconsistent text markup handling when double-nesting markers
2023-10-11 2:23 ` Max Nikulin
@ 2023-10-11 9:15 ` Ihor Radchenko
2023-10-11 12:16 ` Max Nikulin
0 siblings, 1 reply; 9+ messages in thread
From: Ihor Radchenko @ 2023-10-11 9:15 UTC (permalink / raw)
To: Max Nikulin; +Cc: Tom Alexander, emacs-orgmode
Max Nikulin <manikulin@gmail.com> writes:
> Isn't nested bold for "**bold**" a bug? Generally it is not allowed and
>
> *b1 *b2* b3*
>
> is parsed as bold only for "b1 *b2".
No, **bold** it is not a bug. The parser is recursive with inner markup
not "seeing" its parent. So, we first parse the outer bold and then
continue parsing the contents separately, as *bold*.
Be it another way, /*bold italic*/ would also not be allowed as
we demand bol, whitespace, -, (, {, ', or " before the markup:
https://orgmode.org/worg/org-syntax.html#Emphasis_Markers
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Inconsistent text markup handling when double-nesting markers
2023-10-11 9:15 ` Ihor Radchenko
@ 2023-10-11 12:16 ` Max Nikulin
2023-10-11 12:26 ` Ihor Radchenko
0 siblings, 1 reply; 9+ messages in thread
From: Max Nikulin @ 2023-10-11 12:16 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: Tom Alexander, emacs-orgmode
On 11/10/2023 16:15, Ihor Radchenko wrote:
> Max Nikulin <manikulin@gmail.com> writes:
>
>> Isn't nested bold for "**bold**" a bug? Generally it is not allowed and
>>
>> *b1 *b2* b3*
>>
>> is parsed as bold only for "b1 *b2".
>
> No, **bold** it is not a bug. The parser is recursive with inner markup
> not "seeing" its parent. So, we first parse the outer bold and then
> continue parsing the contents separately, as *bold*.
I just find the following rather confusing:
(org-export-string-as "**bold**" 'html t)
"<p>\n<b><b>bold</b></b></p>\n"
(org-export-string-as "**inner* outer*" 'html t)
"<p>\n<b>*inner</b> outer*</p>\n"
(org-export-string-as "*outer *inner**" 'html t)
"<p>\n<b>outer <b>inner</b></b></p>\n"
(org-export-string-as "*begin *inner* end*" 'html t)
"<p>\n<b>begin *inner</b> end*</p>\n"
> Be it another way, /*bold italic*/ would also not be allowed as
> we demand bol, whitespace, -, (, {, ', or " before the markup:
> https://orgmode.org/worg/org-syntax.html#Emphasis_Markers
Certainly /*b*/ should work, but nested bold was a surprise for me. I
believed that nesting is strictly prohibited. The case of underscores is
even more tricky due to ambiguity of underline and subscript.
P.S. Juan Manuel at certain moment discovered that pandoc allows nesting
for *b1 *b2* b3*.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Inconsistent text markup handling when double-nesting markers
2023-10-11 12:16 ` Max Nikulin
@ 2023-10-11 12:26 ` Ihor Radchenko
2023-10-11 14:40 ` Tom Alexander
2023-10-12 10:23 ` Max Nikulin
0 siblings, 2 replies; 9+ messages in thread
From: Ihor Radchenko @ 2023-10-11 12:26 UTC (permalink / raw)
To: Max Nikulin; +Cc: Tom Alexander, emacs-orgmode
Max Nikulin <manikulin@gmail.com> writes:
>> No, **bold** it is not a bug. The parser is recursive with inner markup
>> not "seeing" its parent. So, we first parse the outer bold and then
>> continue parsing the contents separately, as *bold*.
>
> I just find the following rather confusing:
>
> (org-export-string-as "**bold**" 'html t)
> "<p>\n<b><b>bold</b></b></p>\n"
> (org-export-string-as "**inner* outer*" 'html t)
> "<p>\n<b>*inner</b> outer*</p>\n"
> (org-export-string-as "*outer *inner**" 'html t)
> "<p>\n<b>outer <b>inner</b></b></p>\n"
> (org-export-string-as "*begin *inner* end*" 'html t)
> "<p>\n<b>begin *inner</b> end*</p>\n"
Maybe. It is indeed one of the edge cases. But it is following the
parser logic, which is (1) first matching markup is parser; (2) parsing
recursive contents is isolated.
>> Be it another way, /*bold italic*/ would also not be allowed as
>> we demand bol, whitespace, -, (, {, ', or " before the markup:
>> https://orgmode.org/worg/org-syntax.html#Emphasis_Markers
>
> Certainly /*b*/ should work, but nested bold was a surprise for me. I
> believed that nesting is strictly prohibited. The case of underscores is
> even more tricky due to ambiguity of underline and subscript.
It is not strictly prohibited on purpose. It is just a consequence of
how the parser works that nesting <end> constructs is almost impossible,
except certain edge cases like **b**.
> P.S. Juan Manuel at certain moment discovered that pandoc allows nesting
> for *b1 *b2* b3*.
Which is a bug in pandoc.
I think we discussed this topic a number of times in the past - our
markup is a compromise between simplicity for users and simplicity of
the parser. This works in many simple cases, but edge cases become
problematic.
Workarounds have been discussed as well. For example, creole markup and
generic inline markup constructs (your idea with direct AST and the idea
with inline special blocks).
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Inconsistent text markup handling when double-nesting markers
2023-10-11 12:26 ` Ihor Radchenko
@ 2023-10-11 14:40 ` Tom Alexander
2023-10-12 10:23 ` Max Nikulin
1 sibling, 0 replies; 9+ messages in thread
From: Tom Alexander @ 2023-10-11 14:40 UTC (permalink / raw)
To: Ihor Radchenko, Max Nikulin; +Cc: emacs-orgmode
> Fixed, on main.
Thanks!
--
Tom Alexander
pgp: https://fizz.buzz/pgp.asc
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Inconsistent text markup handling when double-nesting markers
2023-10-11 12:26 ` Ihor Radchenko
2023-10-11 14:40 ` Tom Alexander
@ 2023-10-12 10:23 ` Max Nikulin
2023-10-12 12:04 ` Ihor Radchenko
1 sibling, 1 reply; 9+ messages in thread
From: Max Nikulin @ 2023-10-12 10:23 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: Tom Alexander, emacs-orgmode
On 11/10/2023 19:26, Ihor Radchenko wrote:
> Max Nikulin writes:
>
>> P.S. Juan Manuel at certain moment discovered that pandoc allows nesting
>> for *b1 *b2* b3*.
>
> Which is a bug in pandoc.
>
> I think we discussed this topic a number of times in the past - our
> markup is a compromise between simplicity for users and simplicity of
> the parser. This works in many simple cases, but edge cases become
> problematic.
I have no intention to raise discussions of changing patterns to
recognize beginning and end of objects or extending of syntax.
My guess is that pandoc may use bottom-up, not top-down approach. I
admit, my opinion may be biased by reading complains concerning
unexpected behavior of current implementation. Perhaps besides
advantages pandoc parser has downsides. I would not be surprised if
bottom up parser is unbearable without some tool that generates code for
provided rules.
By the way, is it explicitly specified that within an element namely
top-down strategy must be used to recognize objects?
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Inconsistent text markup handling when double-nesting markers
2023-10-12 10:23 ` Max Nikulin
@ 2023-10-12 12:04 ` Ihor Radchenko
0 siblings, 0 replies; 9+ messages in thread
From: Ihor Radchenko @ 2023-10-12 12:04 UTC (permalink / raw)
To: Max Nikulin; +Cc: Tom Alexander, emacs-orgmode
Max Nikulin <manikulin@gmail.com> writes:
> By the way, is it explicitly specified that within an element namely
> top-down strategy must be used to recognize objects?
https://orgmode.org/worg/org-syntax.html has it, I think.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2023-10-12 12:04 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-10-09 23:02 Inconsistent text markup handling when double-nesting markers Tom Alexander
2023-10-10 12:07 ` Ihor Radchenko
2023-10-11 2:23 ` Max Nikulin
2023-10-11 9:15 ` Ihor Radchenko
2023-10-11 12:16 ` Max Nikulin
2023-10-11 12:26 ` Ihor Radchenko
2023-10-11 14:40 ` Tom Alexander
2023-10-12 10:23 ` Max Nikulin
2023-10-12 12:04 ` Ihor Radchenko
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).