Issues w/ hacking Org font-lock for variable pitch prose

Issues w/ hacking Org font-lock for variable pitch prose

[Göktuğ Kayaalp]

Hello,

/I’ll first explain the situation and append the code to the message./
Go to code: [ M-x re-search-forward RET ^CODE RET ]

I read  in Org  mode a  lot, and  I dislike  reading prose  in monospace
fonts.  So  I have  turned on ‘variable-pitch-mode’  for Org  mode.  But
because some stuff has to stay in  stoichedon, so I set some faces to be
monospace:

I have also add a keyword to font-lock keywords as part of the Org hook,
so that I can  have a face attached to initial  whitespace and lists (-,
and 1., 2. etc).  I set this face  to monospace too, so that I have nice
aligned left line, while the actual prose stays in variable pitch.

I just  updated today, from  the default Org mode  of Emacs 24.5  to the
latest release (8.3.2, from git).

The problems are as follows:

1. My code is buggy, and I can’t understand why.  With previous Org some
font-lock actions were happening only after I  hit enter at the end of a
line,  and if  I  have less  than  3 newlines  at the  end  of the  file
sometimes font-locking  of headings  didn’t work.   For example,  if I’m
writing a  paragraph, I write  it in a single  physical line, and  I use
word  wrapping.   If  that  paragraph happended  to  contain  an  inline
footnote, which I  use very often, that note is  not highlighted until I
insert a line-feed.

And with the new version, while  the problem persists, when I hit enter,
thus inserting  a newline  at the  end of  a line,  Emacs hangs  up, and
sometimes I see this error:

org-element--current-element: Wrong type argument: integer-or-marker-p, nil
;; See end of message for the backtrace…
;; [ M-x re-search-forward RET ^BACK RET ]

But if I go to the next  line and ‘open-line’, everything is O.K.  Maybe
my keyword regex has a mistake?

2. In the new version of  org mode, ‘org-block-background’ face is gone,
and I can’t  set the face for  block to use a monospace  font.  There is
the org-block  face, which  doesn’t affect the  contents of  org blocks,
e.g. src  block.  Basically, the  text between the  org-block delimiters
don’t get a face, so I can’t set the font for them.  Any workarounds?

3. I do not know how to hook in properly to the org font-lock mechanism.
I read the code a bit,  and apparently org builds the font-lock keywords
list local  to the buffer when  it is booting  up as its major  mode.  I
know barely anything about font-lock, but from this fact I deduce that I
have to push in  my keywords in a hook after  it’s initialisation, but I
do not know if I do it properly.

I’ve written a wall  of text, I’m sorry.  I hope  there’ll be someone to
help me out  though, because the solution is beyond  my comprehension of
Org and Emacs Lisp.  Thanks in advance.

CODE

;; ‘gk-font’ merely returns a font from a plist with the given keyword.

(defvar gk-org-hide-leading-stuff-in-headers nil)

(defface gk-org-initial-space-and-list-stuff-face
  `((t . (:family ,(gk-font :mono))))
  "Face for initial space and list item bullets.
This face is used to keep them in monospace even when
variable-pitch mode is used in the org buffer.")

(defvar gk-org-font-lock-keywords
  `(;; Initial space is monospace
    ("^\s*\\([0-9-]+[) ] ?\\(\\[[X ]\\]\\)?\\)?"
     (0 (put-text-property
         (match-beginning 0)
         (match-end 0)
         'face 'gk-org-initial-space-and-list-stuff-face)))
    ;; Hide leading stars and space in headings.
    ("^\\*+ "
     (0 (when gk-org-hide-leading-stuff-in-headers
          (put-text-property
           (match-beginning 0)
           (match-end 0)
           'display '(space . (width 0))))))))

(defvar gk-org-mono-faces
  '(org-table
    org-code
    org-special-keyword
    org-verbatim
    org-meta-line
    org-block
    org-block-begin-line
    org-block-end-line))

(defun gk-org-visuals-hook ()
  "Set up how an Org buffer look."
  (when (window-system)
    ;; Variable pitch in general, monospace in tables.
    (variable-pitch-mode 1)
    (set-face-attribute 'variable-pitch nil
                        :font (gk-font :serif)
                        :height gk-font-org-variable-pitch-height)
    (set-face-attribute 'org-todo nil :font (gk-font :sans))
    (mapc (lambda (x)
            (set-face-attribute
             x nil :font (gk-font :mono)))
          gk-org-mono-faces)
    (set-face-attribute 'org-footnote nil :underline nil)
    (dolist (face org-level-faces)
      (set-face-attribute
       face nil
       :family (gk-font :sans)))
    (font-lock-add-keywords nil gk-org-font-lock-keywords)))

(add-hook 'org-mode-hook 'gk-org-visuals-hook)

;;; Code block ends here

BACKTRACE

;;; Some non-printables are replaced with  dots, because Gnus wanted so.
;;; If necessary, I can send in the full traceback.

Debugger entered--Lisp error: (wrong-type-argument integer-or-marker-p nil)
  org-element-item-parser(41809 ((41306 0 "- " nil nil nil 41341) (41341 0 "- " nil nil nil 41486) (41367 2 "- " nil nil nil 41486) (41403 4 "- " nil nil nil 41446) (41446 4 "- " nil nil nil 41486) (41486 0 "- " nil nil nil 41655) (41511 2 "- " nil nil nil 41633) (41556 4 "- " nil nil nil 41633) (41611 6 "- " nil nil nil 41633) (41633 2 "- " nil nil nil 41655) (41655 0 "- " nil nil nil 41809) (41705 2 "- " nil nil nil 41809) (41767 4 "- " nil nil nil 41809)) t)
  org-element--current-element(41809 element item ((41306 0 "- " nil nil nil 41341) (41341 0 "- " nil nil nil 41486) (41367 2 "- " nil nil nil 41486) (41403 4 "- " nil nil nil 41446) (41446 4 "- " nil nil nil 41486) (41486 0 "- " nil nil nil 41655) (41511 2 "- " nil nil nil 41633) (41556 4 "- " nil nil nil 41633) (41611 6 "- " nil nil nil 41633) (41633 2 "- " nil nil nil 41655) (41655 0 "- " nil nil nil 41809) (41705 2 "- " nil nil nil 41809) (41767 4 "- " nil nil nil 41809)))
  byte-code("\212\214~\210\bb\210	\205.\n\205.\306\307!\204.\v\205.\310\b\311\".\312\f.1.;\2031.\313\314.1.#\2028.\315.A@.1\"*.2\311\211.3\311.4\f\204m.\316.5\317 \211.6.7\320.6P.8\321 ,\203c.\322.4\311y\210\323\311w\210\324 \210\202[.\b.2U\203\237.\325\326.9\203\231.\327\f.1.;\203\216.\313\314.1.#\202\225.\315.A@.1\"*\202\232.\f\"\210\202[.\330\316.5\317 \211.6.7\320.6P+.2\316#\203\306.\311y\210\323\311w\210\324 \210\322.4\202[.\fd\bU\203\322.\bS\202\323.\b.:\331\f.1.;\203\351.\313\314.1.#\202\360.\315.A@.1\"*\206\366.2b\210\332.:.1.;\203\f.\313\314.1.#\202.\315.A@.1\"*\211.;\bX\205A.;b\205A.\327.:.1.;\2036.\313\314.1.#\202=.\315.A@.1\"*\211.:)\204\370.:\203Z.m\203T.:.\202Z.:.`.3*\332..1.;\203n.\313\314.1.#\202u.\315.A@.1\"*\206\220.\212\316.5\317 \211.6.7\320.6P.8\333 \210,`).<.;.9\203\305.`\bU\203\251.\325\326.<\"\210\202\305.=\211.=\205\274.\334 \206\274.\335.=\336 \")\203\305.\325\337\311\"\210.\204.\340.;\305.4\341.<.1.;\203\343.\313\314.1.#\202\352.\315.A@.1\"*$\211.\327.<.>.1\211.;\203.\342.\311.1.>$\202..A\343.A@.1.>#\240\210.+\210\344.!\210\332..1.;\203-.\313\314.1.#\2024.\315.A@.1\"*.\211.:\204E.;\205M.\345\202M.@9\205M.@).?\211.@\bX\203\336.d.@U\204\336.@b\210.?\311.A.?.A\203\253.\346.?\347\"\203{.\350\202\330.\346.?\351\"\203\207.\352\202\330.\346.?\353\"\203\223.\354\202\330.\346.?\350\"\203\237.\322\202\330.\346.?\355\"\205\330.\356\202\330.\346.?\352\"\203\267.\352\202\330.\346.?\354\"\203\303.\354\202\330.\346.?\322\"\203\317.\353\202\330.\346.?\356\"\205\330.\356*.4\202\366.?.B>\204\356.\325\326.\"\210\202\366.\331..1.;\203.\313\314.1.#\202\b.\315.A@.1\"*\357..1.;\203.\313\314.1.#\202#.\315.A@.1\"*.C.D.9\204a.D\205\355.C\205\355.D\bW\204M.D\bU\205\355.?\360>?\205\355.C\bV\204a.C\bU\205\355.d\bU\205\355.3\206h.Db\210\311.3.?\316.A.?.A\203\265.\346.?\347\"\203\205.\350\202\342.\346.?\351\"\203\221.\352\202\342.\346.?\353\"\203\235.\354\202\342.\346.?\350\"\203\251.\322\202\342.\346.?\355\"\205\342.\356\202\342.\346.?\352\"\203\301.\352\202\342.\346.?\354\"\203\315.\354\202\342.\346.?\322\"\203\331.\353\202\342.\346.?\356\"\205\342.\356*.4.<.C\211.;*\204\366.\325\326.\"\210*\311.\202\225." [pos org-element-use-cache org-element--cache orgstruct-mode cached element derived-mode-p org-mode org-element--cache-find nil :begin get-text-property 0 plist-get t org-get-limited-outline-regexp "^" outline-previous-heading planning " .	\n" beginning-of-line throw exit :parent re-search-backward :contents-begin :end outline-next-heading input-pending-p time-less-p current-time interrupt org-element--current-element :structure org-add-props plist-put org-element--cache-put plain-text eql headline section plain-list item property-drawer node-property table table-row :contents-end (plain-list table) property ...] 9)
  org-element--parse-to(41648)
  org-element-at-point()
  org-return()
  call-interactively(org-return nil nil)
  command-execute(org-return)

-- 
İ. Göktuğ Kayaalp.
http://gkayaalp.com/

Re: Issues w/ hacking Org font-lock for variable pitch prose

[Nicolas Goaziou]

Hello,

Göktuğ Kayaalp <self@gkayaalp.com> writes:

> /I’ll first explain the situation and append the code to the message./
> Go to code: [ M-x re-search-forward RET ^CODE RET ]
>
> I read  in Org  mode a  lot, and  I dislike  reading prose  in monospace
> fonts.  So  I have  turned on ‘variable-pitch-mode’  for Org  mode.  But
> because some stuff has to stay in  stoichedon, so I set some faces to be
> monospace:
>
> I have also add a keyword to font-lock keywords as part of the Org hook,
> so that I can  have a face attached to initial  whitespace and lists (-,
> and 1., 2. etc).  I set this face  to monospace too, so that I have nice
> aligned left line, while the actual prose stays in variable pitch.
>
> I just  updated today, from  the default Org mode  of Emacs 24.5  to the
> latest release (8.3.2, from git).
>
> The problems are as follows:
>
> 1. My code is buggy, and I can’t understand why.  With previous Org some
> font-lock actions were happening only after I  hit enter at the end of a
> line,  and if  I  have less  than  3 newlines  at the  end  of the  file
> sometimes font-locking  of headings  didn’t work.   For example,  if I’m
> writing a  paragraph, I write  it in a single  physical line, and  I use
> word  wrapping.   If  that  paragraph happended  to  contain  an  inline
> footnote, which I  use very often, that note is  not highlighted until I
> insert a line-feed.
>
> And with the new version, while  the problem persists, when I hit enter,
> thus inserting  a newline  at the  end of  a line,  Emacs hangs  up, and
> sometimes I see this error:
>
> org-element--current-element: Wrong type argument: integer-or-marker-p, nil
> ;; See end of message for the backtrace…
> ;; [ M-x re-search-forward RET ^BACK RET ]

Your code is probably not buggy. You are encountering a cache error.
Does it happen on a fresh buffer (e.g., open a new buffer, and copy
contents there, then let your code apply appropriate fontification)?

Regards,

-- 
Nicolas Goaziou

Re: Issues w/ hacking Org font-lock for variable pitch prose

[Göktuğ Kayaalp]

Hello,

On Sat, Dec 05 2015 at 03:05:38 PM, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
> Hello,
>
> Göktuğ Kayaalp <self@gkayaalp.com> writes:
>
>> […]
>
> Your code is probably not buggy. You are encountering a cache error.
> Does it happen on a fresh buffer (e.g., open a new buffer, and copy
> contents there, then let your code apply appropriate fontification)?
>

When I  copied the contents of  a file to  a fresh buffer, then  run M-x
org-mode in it, the last heading  and its contents didn’t get fontified,
and I got this error (the last entry was a level-1 entry):

Error during redisplay: (jit-lock-function 9029) signaled (end-of-buffer)

Another  time with  the  last  entry a  level-3,  immediate  child of  a
level-2, last two did not render:

Error during redisplay: (jit-lock-function 9490) signaled (end-of-buffer)
Error during redisplay: (jit-lock-function 9418) signaled (end-of-buffer)
font-lock-default-fontify-region: End of buffer

I had  ‘debug-on-error’ as ‘t’  during these  tests.  In both,  the file
ended with a single final newline.  If  there are more, I do not have an
error.

Best,
-- 
İ. Göktuğ Kayaalp.
http://gkayaalp.com/