Re: footnote fontify causing massive slowdown

[Nicolas Goaziou <mail@nicolasgoaziou.fr>]

Hello,

Derek Feichtinger <derek.feichtinger@psi.ch> writes:

> While diagnosing a server condition, I was listing parts of a system log
> via a babel expression. The 130 lines in the babel output are wrapped in
> an example block. This block caused massive slowdown of scrolling and
> other operations.
>
> Using the emacs profiler I see:
>
> - redisplay_internal (C function) 8232  88%
>  - jit-lock-function 8226  88%
>   - jit-lock-fontify-now 8226  88%
>    - funcall 8226  88%
>     - #<compiled 0x1cc274f>                                      8226  88%
>      - run-hook-with-args 8226  88%
>       - font-lock-fontify-region 8226  88%
>        - font-lock-default-fontify-region 8226  88%
>         - font-lock-fontify-keywords-region 8226  88%
>          - org-activate-footnote-links 8158  87%
>           - org-footnote-next-reference-or-definition 8158  87%
>            - byte-code 8158  87%
>             - org-footnote-at-reference-p 4114  44%
>              - org-footnote-in-valid-context-p 4106  44%
>               + org-inside-LaTeX-fragment-p 2380  25%
>               + org-in-block-p 1563  16%
>               + org-in-verbatim-emphasis 159   1%
>                 org-at-comment-p 4   0%
>
> Checking for footnote pattern matches (org-footnote-re) in the wrapped
> block, I see that
> every line matches based on the very simple and trivial pattern of
> number enclosed in angular brackets, so all the process numbers
> following the "sshd" in these lines like "sshd[1234]" do match and cause load.
>
> #################
>     /var/log/secure-20151129:Nov 23 02:25:36 some-host sshd[20089]: Rhosts
> authentication refused for userXYZ: bad ownership or modes for home directory.
>     /var/log/secure-20151129:Nov 23 02:25:36 some-host sshd[20089]: Rhosts
> authentication refused for userXYZ: bad ownership or modes for home directory.
>     /var/log/secure-20151129:Nov 23 02:25:41 some-host sshd[20089]:
> pam_ldap: error trying to bind as user "xxxxx" (Invalid credentials)
> #################
>
> Since this kind of pattern is so common in logs and 130 lines are really not
> a large number, it makes it hard to use
> org for this purpose. Can this be turned off selectively, or can it be
> prevented in example blocks?

This is a limitation of our current way to fontify a buffer. Changing it
implies some serious work, which I'd rather spend on switching to
syntax-based (instead of regexp-based) fontification.

However, this report raises an interesting question about footnotes:
should we still support plain (e.g., "[1]") footnotes in Org documents?

The pattern is very common an regularly introduces false positives.
Also, IIRC, it was introduced for non-Org buffers (e.g., in Message mode
buffers), to provide some common features with "footnote.el" library.

I think we could remove this kind of footnotes, and yet preserve
`org-footnote-normalize' to change Org footnotes into these ones, for
foreign documents.

WDYT?


Regards,

-- 
Nicolas Goaziou