Org mode fontification error in #<buffer myfile.org> in python and ipython source blocks

Org mode fontification error in #<buffer myfile.org> in python and ipython source blocks

[Sebastian Gimeno]

[-- Attachment #1: Type: text/plain, Size: 917 bytes --]

Hi,

I am using emacs 27.1 and org-plus-contrib 20201026.

I am having problems with the fontification of python and ipython source
blocks when the code contains curly brackets "{}" (other course blocks are
ok). For instance, the following snippet

#+BEGIN_SRC python :results drawer
import matplotlib.pyplot as plt
plt.plot([1,2,3])
plt.show()
a=1
print("a={}".format{a})
#+END_SRC

does not fontify correctly in either python or ipython source blocks.

As a consequence, when exporting the org file to HTML (C-c C-e h h), it
fails with the following message:

font-lock-fontify-keywords-region: Invalid function: #<subr progn>

If changing the code snippet to:

#+BEGIN_SRC python :results drawer
import matplotlib.pyplot as plt
plt.plot([1,2,3])
plt.show()
a=1
print("a=%s" % a)
#+END_SRC

the block fontifies and the file is exported to HTML correctly.

Can you reproduce this behaviour?

Many thanks in advance!

[-- Attachment #2: Type: text/html, Size: 1566 bytes --]

Re: Org mode fontification error in #<buffer myfile.org> in python and ipython source blocks

[Jack Kamm]

Hi Sebastian --

> I am having problems with the fontification of python and ipython source
> blocks when the code contains curly brackets "{}" (other course blocks are
> ok). For instance, the following snippet
>
> #+BEGIN_SRC python :results drawer
> import matplotlib.pyplot as plt
> plt.plot([1,2,3])
> plt.show()
> a=1
> print("a={}".format{a})
> #+END_SRC
>
> does not fontify correctly in either python or ipython source blocks.
>
> As a consequence, when exporting the org file to HTML (C-c C-e h h), it
> fails with the following message:
>
> font-lock-fontify-keywords-region: Invalid function: #<subr progn>

I'm unable to reproduce this behavior on emacs 27.1 and git
master. Fontification and export work fine with this code block for me.

Does the error still happen when you use emacs -q ? Or if you use git
master?

Jack

Re: Org mode fontification error in #<buffer myfile.org> in python and ipython source blocks

[Sebastian Gimeno]

[-- Attachment #1: Type: text/plain, Size: 1156 bytes --]

Hi Jack,

thanks for replying!

The error does not happen using "emacs -q" (built-in package: org 9.3). I
haven't tried with the git version yet. i will and let you know.

Cheers,
Sebastian

On Sun, Nov 1, 2020 at 9:19 PM Jack Kamm <jackkamm@gmail.com> wrote:

> Hi Sebastian --
>
> > I am having problems with the fontification of python and ipython source
> > blocks when the code contains curly brackets "{}" (other course blocks
> are
> > ok). For instance, the following snippet
> >
> > #+BEGIN_SRC python :results drawer
> > import matplotlib.pyplot as plt
> > plt.plot([1,2,3])
> > plt.show()
> > a=1
> > print("a={}".format{a})
> > #+END_SRC
> >
> > does not fontify correctly in either python or ipython source blocks.
> >
> > As a consequence, when exporting the org file to HTML (C-c C-e h h), it
> > fails with the following message:
> >
> > font-lock-fontify-keywords-region: Invalid function: #<subr progn>
>
> I'm unable to reproduce this behavior on emacs 27.1 and git
> master. Fontification and export work fine with this code block for me.
>
> Does the error still happen when you use emacs -q ? Or if you use git
> master?
>
> Jack
>
>

[-- Attachment #2: Type: text/html, Size: 1726 bytes --]

Re: Org mode fontification error in #<buffer myfile.org> in python and ipython source blocks

[Sebastian Gimeno]

[-- Attachment #1: Type: text/plain, Size: 1583 bytes --]

Hi Jack,

I have cloned the git master. Running without configuration ("make
vanilla"), emacs correctly fontifies the source block and it also gets
exported to HTML. It seems that it is a configuration problem. Sorry for
not double checking first!

Many thanks for your help!

Cheers,
Sebastian


On Mon, Nov 2, 2020 at 12:43 AM Sebastian Gimeno <changimeno@gmail.com>
wrote:

> Hi Jack,
>
> thanks for replying!
>
> The error does not happen using "emacs -q" (built-in package: org 9.3). I
> haven't tried with the git version yet. i will and let you know.
>
> Cheers,
> Sebastian
>
> On Sun, Nov 1, 2020 at 9:19 PM Jack Kamm <jackkamm@gmail.com> wrote:
>
>> Hi Sebastian --
>>
>> > I am having problems with the fontification of python and ipython source
>> > blocks when the code contains curly brackets "{}" (other course blocks
>> are
>> > ok). For instance, the following snippet
>> >
>> > #+BEGIN_SRC python :results drawer
>> > import matplotlib.pyplot as plt
>> > plt.plot([1,2,3])
>> > plt.show()
>> > a=1
>> > print("a={}".format{a})
>> > #+END_SRC
>> >
>> > does not fontify correctly in either python or ipython source blocks.
>> >
>> > As a consequence, when exporting the org file to HTML (C-c C-e h h), it
>> > fails with the following message:
>> >
>> > font-lock-fontify-keywords-region: Invalid function: #<subr progn>
>>
>> I'm unable to reproduce this behavior on emacs 27.1 and git
>> master. Fontification and export work fine with this code block for me.
>>
>> Does the error still happen when you use emacs -q ? Or if you use git
>> master?
>>
>> Jack
>>
>>

[-- Attachment #2: Type: text/html, Size: 2524 bytes --]

Re: Org mode fontification error in #<buffer myfile.org> in python and ipython source blocks

[stardiviner]

[-- Attachment #1: Type: text/plain, Size: 1500 bytes --]

I have same problem sometimes in different babel languages.
I would like to know what caused this problem too.
Can you send me a message after you solved problem? Thanks in advance. :)
smile

[stardiviner]           <Hack this world!>      GPG key ID: 47C32433
IRC(freeenode): stardiviner                     Twitter:  @numbchild
Key fingerprint = 9BAA 92BC CDDD B9EF 3B36  CB99 B8C4 B8E5 47C3 2433
Blog: http://stardiviner.github.io/


On Mon, Nov 2, 2020 at 2:26 AM Sebastian Gimeno <changimeno@gmail.com>
wrote:

> Hi,
>
> I am using emacs 27.1 and org-plus-contrib 20201026.
>
> I am having problems with the fontification of python and ipython source
> blocks when the code contains curly brackets "{}" (other course blocks are
> ok). For instance, the following snippet
>
> #+BEGIN_SRC python :results drawer
> import matplotlib.pyplot as plt
> plt.plot([1,2,3])
> plt.show()
> a=1
> print("a={}".format{a})
> #+END_SRC
>
> does not fontify correctly in either python or ipython source blocks.
>
> As a consequence, when exporting the org file to HTML (C-c C-e h h), it
> fails with the following message:
>
> font-lock-fontify-keywords-region: Invalid function: #<subr progn>
>
> If changing the code snippet to:
>
> #+BEGIN_SRC python :results drawer
> import matplotlib.pyplot as plt
> plt.plot([1,2,3])
> plt.show()
> a=1
> print("a=%s" % a)
> #+END_SRC
>
> the block fontifies and the file is exported to HTML correctly.
>
> Can you reproduce this behaviour?
>
> Many thanks in advance!
>

[-- Attachment #2: Type: text/html, Size: 2722 bytes --]

Re: Org mode fontification error in #<buffer myfile.org> in python and ipython source blocks

[Sebastian Gimeno]

[-- Attachment #1: Type: text/plain, Size: 2452 bytes --]

Dear Stardiviner,

I used the "scimax" configuration. Scimax's maintainer, John Kitchin,
helped me out to find the root cause of the problem, i.e. the following
addition to " python-font-lock-keywords":

(setq python-font-lock-keywords
      (append python-font-lock-keywords
     '(;; this is the full string.
;; group 1 is the quote type and a closing quote is matched
;; group 2 is the string part
(
;; "f\\(['\"]\\{1,3\\}\\)\\([^\\1]+?\\)\\1"
"f\\(['\"]\\{1,3\\}\\)\\(.+?\\)\\1"
;; these are the {keywords}
("{[^}]*?}"
 ;; Pre-match form
 (progn (goto-char (match-beginning 0)) (match-end 0))
 ;; Post-match form
 (goto-char (match-end 0))
 ;; face for this match
 (0 font-lock-variable-name-face t))))))

Commenting out this code snippet, the fontfying / HTML-export problem goes
away.

Cheers,
sebastian




On Tue, Nov 3, 2020 at 6:50 AM stardiviner <numbchild@gmail.com> wrote:

> I have same problem sometimes in different babel languages.
> I would like to know what caused this problem too.
> Can you send me a message after you solved problem? Thanks in advance. :)
> smile
>
> [stardiviner]           <Hack this world!>      GPG key ID: 47C32433
> IRC(freeenode): stardiviner                     Twitter:  @numbchild
> Key fingerprint = 9BAA 92BC CDDD B9EF 3B36  CB99 B8C4 B8E5 47C3 2433
> Blog: http://stardiviner.github.io/
>
>
> On Mon, Nov 2, 2020 at 2:26 AM Sebastian Gimeno <changimeno@gmail.com>
> wrote:
>
>> Hi,
>>
>> I am using emacs 27.1 and org-plus-contrib 20201026.
>>
>> I am having problems with the fontification of python and ipython source
>> blocks when the code contains curly brackets "{}" (other course blocks are
>> ok). For instance, the following snippet
>>
>> #+BEGIN_SRC python :results drawer
>> import matplotlib.pyplot as plt
>> plt.plot([1,2,3])
>> plt.show()
>> a=1
>> print("a={}".format{a})
>> #+END_SRC
>>
>> does not fontify correctly in either python or ipython source blocks.
>>
>> As a consequence, when exporting the org file to HTML (C-c C-e h h), it
>> fails with the following message:
>>
>> font-lock-fontify-keywords-region: Invalid function: #<subr progn>
>>
>> If changing the code snippet to:
>>
>> #+BEGIN_SRC python :results drawer
>> import matplotlib.pyplot as plt
>> plt.plot([1,2,3])
>> plt.show()
>> a=1
>> print("a=%s" % a)
>> #+END_SRC
>>
>> the block fontifies and the file is exported to HTML correctly.
>>
>> Can you reproduce this behaviour?
>>
>> Many thanks in advance!
>>
>

[-- Attachment #2: Type: text/html, Size: 4257 bytes --]

Re: Org mode fontification error in #<buffer myfile.org> in python and ipython source blocks

[John Kitchin]

[-- Attachment #1: Type: text/plain, Size: 3145 bytes --]

Does anyone see anything wrong with that snippet? All it should do is
improve font-lock on Python f-strings, and it used to work fine, but now it
doesn't. It isn't real critical for me, so I don't mind commenting it out,
but I neither understand why it stopped working!

John

-----------------------------------
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu



On Tue, Nov 3, 2020 at 4:55 AM Sebastian Gimeno <changimeno@gmail.com>
wrote:

> Dear Stardiviner,
>
> I used the "scimax" configuration. Scimax's maintainer, John Kitchin,
> helped me out to find the root cause of the problem, i.e. the following
> addition to " python-font-lock-keywords":
>
> (setq python-font-lock-keywords
>       (append python-font-lock-keywords
>      '(;; this is the full string.
> ;; group 1 is the quote type and a closing quote is matched
> ;; group 2 is the string part
> (
> ;; "f\\(['\"]\\{1,3\\}\\)\\([^\\1]+?\\)\\1"
> "f\\(['\"]\\{1,3\\}\\)\\(.+?\\)\\1"
> ;; these are the {keywords}
> ("{[^}]*?}"
>  ;; Pre-match form
>  (progn (goto-char (match-beginning 0)) (match-end 0))
>  ;; Post-match form
>  (goto-char (match-end 0))
>  ;; face for this match
>  (0 font-lock-variable-name-face t))))))
>
> Commenting out this code snippet, the fontfying / HTML-export problem goes
> away.
>
> Cheers,
> sebastian
>
>
>
>
> On Tue, Nov 3, 2020 at 6:50 AM stardiviner <numbchild@gmail.com> wrote:
>
>> I have same problem sometimes in different babel languages.
>> I would like to know what caused this problem too.
>> Can you send me a message after you solved problem? Thanks in advance. :)
>> smile
>>
>> [stardiviner]           <Hack this world!>      GPG key ID: 47C32433
>> IRC(freeenode): stardiviner                     Twitter:  @numbchild
>> Key fingerprint = 9BAA 92BC CDDD B9EF 3B36  CB99 B8C4 B8E5 47C3 2433
>> Blog: http://stardiviner.github.io/
>>
>>
>> On Mon, Nov 2, 2020 at 2:26 AM Sebastian Gimeno <changimeno@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I am using emacs 27.1 and org-plus-contrib 20201026.
>>>
>>> I am having problems with the fontification of python and ipython
>>> source blocks when the code contains curly brackets "{}" (other course
>>> blocks are ok). For instance, the following snippet
>>>
>>> #+BEGIN_SRC python :results drawer
>>> import matplotlib.pyplot as plt
>>> plt.plot([1,2,3])
>>> plt.show()
>>> a=1
>>> print("a={}".format{a})
>>> #+END_SRC
>>>
>>> does not fontify correctly in either python or ipython source blocks.
>>>
>>> As a consequence, when exporting the org file to HTML (C-c C-e h h), it
>>> fails with the following message:
>>>
>>> font-lock-fontify-keywords-region: Invalid function: #<subr progn>
>>>
>>> If changing the code snippet to:
>>>
>>> #+BEGIN_SRC python :results drawer
>>> import matplotlib.pyplot as plt
>>> plt.plot([1,2,3])
>>> plt.show()
>>> a=1
>>> print("a=%s" % a)
>>> #+END_SRC
>>>
>>> the block fontifies and the file is exported to HTML correctly.
>>>
>>> Can you reproduce this behaviour?
>>>
>>> Many thanks in advance!
>>>
>>

[-- Attachment #2: Type: text/html, Size: 5483 bytes --]

Re: Org mode fontification error in #<buffer myfile.org> in python and ipython source blocks

[Kyle Meyer]

John Kitchin writes:

>> (setq python-font-lock-keywords
>>       (append python-font-lock-keywords
>>      '(;; this is the full string.
>> ;; group 1 is the quote type and a closing quote is matched
>> ;; group 2 is the string part
>> (
>> ;; "f\\(['\"]\\{1,3\\}\\)\\([^\\1]+?\\)\\1"
>> "f\\(['\"]\\{1,3\\}\\)\\(.+?\\)\\1"
>> ;; these are the {keywords}
>> ("{[^}]*?}"
>>  ;; Pre-match form
>>  (progn (goto-char (match-beginning 0)) (match-end 0))
>>  ;; Post-match form
>>  (goto-char (match-end 0))
>>  ;; face for this match
>>  (0 font-lock-variable-name-face t))))))

> Does anyone see anything wrong with that snippet? All it should do is
> improve font-lock on Python f-strings, and it used to work fine, but now it
> doesn't. It isn't real critical for me, so I don't mind commenting it out,
> but I neither understand why it stopped working!

I haven't tested to confirm, but I'd guess that is due to the
restructuring of python-font-lock-keywords in 36b64e087e (Add
'font-lock-maximum-decoration' levels for Python, 2018-07-19), which was
included in Emacs 27.

In case it's helpful, here's the compatibility kludge I used in
snakemake-mode:
https://git.kyleam.com/snakemake-mode/commit/snakemake-mode.el?id=5d5bb0109f6a84c0c191559ab71670fae4650e6a

Re: Org mode fontification error in #<buffer myfile.org> in python and ipython source blocks

[stardiviner]

[-- Attachment #1: Type: text/plain, Size: 2949 bytes --]

Thanks for noting. :) I added that in my Emacs config now.

[stardiviner]           <Hack this world!>      GPG key ID: 47C32433
IRC(freeenode): stardiviner                     Twitter:  @numbchild
Key fingerprint = 9BAA 92BC CDDD B9EF 3B36  CB99 B8C4 B8E5 47C3 2433
Blog: http://stardiviner.github.io/


On Tue, Nov 3, 2020 at 5:53 PM Sebastian Gimeno <changimeno@gmail.com>
wrote:

> Dear Stardiviner,
>
> I used the "scimax" configuration. Scimax's maintainer, John Kitchin,
> helped me out to find the root cause of the problem, i.e. the following
> addition to " python-font-lock-keywords":
>
> (setq python-font-lock-keywords
>       (append python-font-lock-keywords
>      '(;; this is the full string.
> ;; group 1 is the quote type and a closing quote is matched
> ;; group 2 is the string part
> (
> ;; "f\\(['\"]\\{1,3\\}\\)\\([^\\1]+?\\)\\1"
> "f\\(['\"]\\{1,3\\}\\)\\(.+?\\)\\1"
> ;; these are the {keywords}
> ("{[^}]*?}"
>  ;; Pre-match form
>  (progn (goto-char (match-beginning 0)) (match-end 0))
>  ;; Post-match form
>  (goto-char (match-end 0))
>  ;; face for this match
>  (0 font-lock-variable-name-face t))))))
>
> Commenting out this code snippet, the fontfying / HTML-export problem goes
> away.
>
> Cheers,
> sebastian
>
>
>
>
> On Tue, Nov 3, 2020 at 6:50 AM stardiviner <numbchild@gmail.com> wrote:
>
>> I have same problem sometimes in different babel languages.
>> I would like to know what caused this problem too.
>> Can you send me a message after you solved problem? Thanks in advance. :)
>> smile
>>
>> [stardiviner]           <Hack this world!>      GPG key ID: 47C32433
>> IRC(freeenode): stardiviner                     Twitter:  @numbchild
>> Key fingerprint = 9BAA 92BC CDDD B9EF 3B36  CB99 B8C4 B8E5 47C3 2433
>> Blog: http://stardiviner.github.io/
>>
>>
>> On Mon, Nov 2, 2020 at 2:26 AM Sebastian Gimeno <changimeno@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I am using emacs 27.1 and org-plus-contrib 20201026.
>>>
>>> I am having problems with the fontification of python and ipython
>>> source blocks when the code contains curly brackets "{}" (other course
>>> blocks are ok). For instance, the following snippet
>>>
>>> #+BEGIN_SRC python :results drawer
>>> import matplotlib.pyplot as plt
>>> plt.plot([1,2,3])
>>> plt.show()
>>> a=1
>>> print("a={}".format{a})
>>> #+END_SRC
>>>
>>> does not fontify correctly in either python or ipython source blocks.
>>>
>>> As a consequence, when exporting the org file to HTML (C-c C-e h h), it
>>> fails with the following message:
>>>
>>> font-lock-fontify-keywords-region: Invalid function: #<subr progn>
>>>
>>> If changing the code snippet to:
>>>
>>> #+BEGIN_SRC python :results drawer
>>> import matplotlib.pyplot as plt
>>> plt.plot([1,2,3])
>>> plt.show()
>>> a=1
>>> print("a=%s" % a)
>>> #+END_SRC
>>>
>>> the block fontifies and the file is exported to HTML correctly.
>>>
>>> Can you reproduce this behaviour?
>>>
>>> Many thanks in advance!
>>>
>>

[-- Attachment #2: Type: text/html, Size: 5369 bytes --]