Org emphasis markers inside src blocks causes incorrect fontification

Org emphasis markers inside src blocks causes incorrect fontification

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 1332 bytes --]

Hello,

I increase the number of consecutive lines to be parsed for Org emphasis to
20 lines using:

=====
 (setcar (nthcdr 4 org-emphasis-regexp-components) 20) ;Up to 20 lines,
 (org-set-emph-re 'org-emphasis-regexp-components
org-emphasis-regexp-components)
=====

With that set, I see that equal signs in source blocks falsely trigger the
org-verbatim fontification. And I see this[1]. Notice the fontification in
the attached image (link at bottom of this email).. The equal signs in the
first src block initiate the org-verbatim fontification and that goes on
till the equal signs in the next code block.. so Heading 2 shows up in
org-verbatim face.

Here is the test file:

=====
* Heading 1
#+BEGIN_SRC nim
let foo = 0
doAssert foo == 0
#+END_SRC
** Heading 2
#+BEGIN_SRC nim
let bar = 0
doAssert bar == 0
#+END_SRC
=====

Shouldn't the presence of org emphasis markers be ignored when inside src
blocks?

Org mode version 9.1.2 (release_9.1.2-202-g04e053 @
/home/kmodi/usr_local/apps/6/emacs/emacs-26/share/emacs/site-lisp/org/)

I don't see this issue (or rather that issue is masked) if I evaluate:

=====
 (setcar (nthcdr 4 org-emphasis-regexp-components) 1) ;Back to default
 (org-set-emph-re 'org-emphasis-regexp-components
org-emphasis-regexp-components)
=====

[1]: https://i.imgur.com/6TmUoTS.png
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 2106 bytes --]

Re: Org emphasis markers inside src blocks causes incorrect fontification

[Nicolas Goaziou]

Hello,

Kaushal Modi <kaushal.modi@gmail.com> writes:

> I increase the number of consecutive lines to be parsed for Org emphasis to
> 20 lines using:
>
> =====
>  (setcar (nthcdr 4 org-emphasis-regexp-components) 20) ;Up to 20 lines,
>  (org-set-emph-re 'org-emphasis-regexp-components
> org-emphasis-regexp-components)
> =====
>
> With that set, I see that equal signs in source blocks falsely trigger the
> org-verbatim fontification. And I see this[1]. Notice the fontification in
> the attached image (link at bottom of this email).. The equal signs in the
> first src block initiate the org-verbatim fontification and that goes on
> till the equal signs in the next code block.. so Heading 2 shows up in
> org-verbatim face.

[...]

> Shouldn't the presence of org emphasis markers be ignored when inside src
> blocks?

It should. 

Note that current fontification system only approximates the syntax of
the Org document. There are certainly more bugs like this one lurking in
the shadows.

At the moment, my suggestion would be to not mess with
`org-emphasis-regexp-components'.

Regards,

-- 
Nicolas Goaziou

Re: Org emphasis markers inside src blocks causes incorrect fontification

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 858 bytes --]

On Thu, Nov 16, 2017 at 4:47 PM Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:

> At the moment, my suggestion would be to not mess with
> `org-emphasis-regexp-components'.
>

I stayed away from messing with that for the longest time, until a bug got
into me to always auto-fill the Org files (as it makes it easier to read
them in browser, etc.)

As I started auto-filling, I noticed that stuff like:

=====
Line 1 starts with /three words
italicized/, but the third word ends
up on the 2nd line.
=====

It wasn't always in my control to always begin and end emphasis on the same
line, and so resorted to tweaking org-emphasis-regexp-components. Overall
it works great, except for cases like these.

Again it's a font-locking bug, and thankfully doesn't impact the empasis
detection in exporters.

Would you please look into fixing this?
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 1413 bytes --]

Re: Org emphasis markers inside src blocks causes incorrect fontification

[Nicolas Goaziou]

[-- Attachment #1: Type: text/plain, Size: 750 bytes --]

Hello,

Kaushal Modi <kaushal.modi@gmail.com> writes:

> Again it's a font-locking bug, and thankfully doesn't impact the empasis
> detection in exporters.
>
> Would you please look into fixing this?

I cannot. Fixing it would require to rewrite the whole thing to use the
parser, so that fontification matches the syntax of the document. This,
in turn, would require to repair the parser's cache so that it doesn't
freeze when the wind is north, northwest. Ah, well.

Meanwhile, I pile up ad-hoc rules to avoid mis-fontification, until the
next pathological case is reported.

IOW, I'm only mildly interested in fixing fontification bugs.

You may want to test the following rules and report if it fits your use
case.

Regards,

-- 
Nicolas Goaziou

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Small-fix-to-emphasis-fontification.patch --]
[-- Type: text/x-diff, Size: 1841 bytes --]

From 52e4c5e1e69b0aaef1795289d8cf9456e76716f1 Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou <mail@nicolasgoaziou.fr>
Date: Sat, 18 Nov 2017 00:19:35 +0100
Subject: [PATCH] Small fix to emphasis fontification

* lisp/org.el (org-do-emphasis-faces): Do not span over paragraph
  separators.

Reported-by: Kaushal Modi <kaushal.modi@gmail.com>
<http://lists.gnu.org/archive/html/emacs-orgmode/2017-11/msg00202.html>
---
 lisp/org.el | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index f873f1021..861ce31db 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5705,20 +5705,16 @@ This should be called after the variable `org-link-parameters' has changed."
 	  (when (save-excursion
 		  (goto-char (match-beginning 0))
 		  (and
-		   ;; Do not match headline stars.  Do not consider
-		   ;; stars of a headline as closing marker for bold
-		   ;; markup either.
-		   (not (and (equal marker "*")
-			     (save-excursion
-			       (forward-char)
-			       (skip-chars-backward "*")
-			       (looking-at-p org-outline-regexp-bol))))
 		   ;; Do not match table hlines.
 		   (not (and (equal marker "+")
 			     (org-match-line
-			      "^[ \t]*\\(|[-+]+|?\\|\\+[-+]+\\+\\)[ \t]*$")))
+			      "[ \t]*\\(|[-+]+|?\\|\\+[-+]+\\+\\)[ \t]*$")))
+		   ;; Match full emphasis markup regexp.
 		   (looking-at (if verbatim? org-verbatim-re org-emph-re))
-		   ;; At a table row, do not cross cell boundaries.
+		   ;; Do not span over paragraph boundaries.
+		   (not (string-match-p org-element-paragraph-separate
+					(match-string 2)))
+		   ;; Do not span over cells in table rows.
 		   (not (and (save-match-data (org-match-line "[ \t]*|"))
 			     (string-match-p "|" (match-string 4))))))
 	    (pcase-let ((`(,_ ,face ,_) (assoc marker org-emphasis-alist)))
-- 
2.14.3

Re: Org emphasis markers inside src blocks causes incorrect fontification

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 218 bytes --]

On Fri, Nov 17, 2017 at 6:30 PM Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:

>
> You may want to test the following rules and report if it fits your use
> case.
>

That patch works great! Thanks!
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 604 bytes --]

Re: Org emphasis markers inside src blocks causes incorrect fontification

[Nicolas Goaziou]

Hello,

Kaushal Modi <kaushal.modi@gmail.com> writes:

> That patch works great! Thanks!

Applied. Thank you.

Regards,

-- 
Nicolas Goaziou