emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* Bug: performance issue in org-font-lock-add-priority-faces related to org-priority-regexp [9.4 (9.4-7-g3eccc5-elpaplus @ /home/standard/.emacs.d/elpa/26.3/develop/org-plus-contrib-20200921/)]
@ 2020-09-24 12:59 Tim Frana
  0 siblings, 0 replies; 2+ messages in thread
From: Tim Frana @ 2020-09-24 12:59 UTC (permalink / raw)
  To: emacs-orgmode

Hello,

After updating from org-plus-contrib-20200810 to 
org-plus-contrib-20200921, opening org files has become extremely slow 
for me. As a rough estimate, a 400 KB org file of mine used to take 5 
seconds on first load in a session, and after the update needed more 
than 2 minutes. This is the case both in Emacs 27.1 in Windows, and 
Emacs 26.3 in Windows WSL1 Ubuntu 20.04 LTS, using spacemacs develop.

I can narrow it down to this change:

org-font-lock-add-priority-faces used to search for this regexp: "^\\*+ 
.*?\\(\\[#\\(.\\)\\]\\)"
Now it searches for org-priority-regexp, which by default resolves to 
".*?\\(\\[#\\([A-Z0-9]+\\)\\] ?\\)".

This is much slower, and also matches more than I expected. In the 
following example, I expect only [#A], but it will match [#B] and [#C] 
too. Is this intended?

** TODO [#A] Headline [#B]
this is matched despite not being a heading: [#C] more text

It seems plausible that the slowdown comes from the regex engine now 
doing elaborate matching on all lines, instead of fast skipping over 
non-heading lines.

This version seems to resolve my problem, where the beginning "^\\*+ " 
should narrow matches to heading lines only:
(setq org-priority-regexp "^\\*+ .*?\\(\\[#\\([A-Z0-9]+\\)\\] ?\\)")

Profiler excerpt (the other 51% were mostly eaten by spell checking 
during idling, as I got distracted waiting for the loading to finish):

- set-auto-mode                                     5498  49%
  - set-auto-mode-0                                  5498  49%
   - org-mode                                        5498  49%
    - run-mode-hooks                                 5498  49%
     - apply                                         5497  49%
      - run-hooks                                    5497  49%
       - org-superstar-mode                          5493  49%
        - org-superstar--fontify-buffer              5493  49%
         - font-lock-ensure                          5493  49%
          - jit-lock-fontify-now                     5493  49%
           - jit-lock--run-functions                 5493  49%
            - run-hook-wrapped                       5493  49%
             - #<compiled 0x28ec06d>                 5391 48%
              - font-lock-fontify-region             5391  48%
               - font-lock-default-fontify-region    5391  48%
                - font-lock-fontify-keywords-region  5391  48%
                   org-font-lock-add-priority-faces  5391  48%

Best regards,
Tim


------------------------------------------------------------------------



Emacs  : GNU Emacs 26.3 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.22.30)
  of 2019-09-16
Package: Org mode version 9.4 (9.4-7-g3eccc5-elpaplus @ 
/home/standard/.emacs.d/elpa/26.3/develop/org-plus-contrib-20200921/)

current state:
==============
(setq
  org-src-mode-hook '(org-src-babel-configure-edit-buffer 
org-src-mode-configure-edit-buffer)
  org-after-todo-state-change-hook '(org-expiry-insert-created)
  org-link-shell-confirm-function 'yes-or-no-p
  org-blank-before-new-entry '((heading . t) (plain-list-item))
  org-babel-after-execute-hook '(spacemacs/ob-fix-inline-images)
  org-metadown-hook '(org-babel-pop-to-session-maybe)
  org-agenda-tags-todo-honor-ignore-options t
  org-export-with-sub-superscripts '{}
  org-clock-out-hook '(org-clock-remove-empty-clock-drawer)
  org-refile-targets '((nil :maxlevel . 3) (org-agenda-files :maxlevel . 3))
  org-html-format-inlinetask-function 
'org-html-format-inlinetask-default-function
  org-enforce-todo-dependencies t
  org-edit-src-content-indentation 0
  org-odt-format-headline-function 'org-odt-format-headline-default-function
  org-imenu-depth 8
  org-download-annotate-function 'org-download-annotate-default
  org-ascii-format-inlinetask-function 'org-ascii-format-inlinetask-default
  org-clock-report-include-clocking-task t
  org-modules '(org-habit ol-w3m ol-bbdb ol-bibtex ol-docview ol-gnus 
ol-info ol-irc ol-mhe ol-rmail ol-eww)
  org-blocker-hook '(org-block-todo-from-children-or-siblings-or-parent 
org-depend-block-todo)
  org-archive-location "%s_archive.org::"
  org-id-link-to-org-use-id 'create-if-interactive
  org-mode-hook '((lambda nil (setq paragraph-start "\f\\|[ ]*$" 
paragraph-separate "[     \f]*$")) turn-on-auto-fill org-tempo-setup 
org-clock-load
                  #[0 "\300\301\302\303\304$\207" [add-hook 
change-major-mode-hook org-show-all append local] 5] flyspell-mode 
spacemacs/org-setup-evil-surround spacemacs/load-yasnippet
                  #[0 "\301\211\x10\207" [imenu-create-index-function 
org-imenu-get-tree] 2] toc-org-enable org-superstar-mode 
org-download-enable dotspacemacs//prettify-spacemacs-docs 
spacemacs//org-babel-do-load-languages
                  #[0 "\300\301\302\303\304$\207" [add-hook 
change-major-mode-hook org-babel-show-result-all append local] 5] 
org-babel-result-hide-spec org-babel-hide-all-hashes 
spacemacs//evil-org-mode org-eldoc-load
                  spacemacs//init-company-org-mode company-mode)
  org-clock-persist t
  org-odt-format-drawer-function #[514 "\207" [] 3 "\n\n(fn NAME CONTENTS)"]
  org-outline-path-complete-in-steps nil
  org-archive-hook '(org-attach-archive-delete-maybe)
  org-agenda-restore-windows-after-quit t
  org-superstar-headline-bullets-list '("■" "◆" "▲" "▶")
  org-confirm-elisp-link-function 'yes-or-no-p
  org-startup-indented t
  org-startup-with-inline-images t
  org-clock-history-length 20
  org-download-file-format-function 'org-download-file-format-default
  org-agenda-before-write-hook '(org-agenda-add-entry-text)
  org-hide-block-startup t
  org-metaup-hook '(org-babel-load-in-session-maybe)
  org-bibtex-headline-format-function #[257 "\300\x01\236A\207" [:title] 3 
"\n\n(fn ENTRY)"]
  org-latex-format-drawer-function #[514 "\207" [] 3 "\n\n(fn _ CONTENTS)"]
  org-expiry-handler-function 'org-toggle-archive-tag
  org-babel-pre-tangle-hook '(save-buffer)
  org-file-apps '((auto-mode . emacs) 
("\\.\\(xlsx?\\|docx?\\|pptx?\\|pdf\\|ods\\|png\\)\\'" . default))
  org-tab-first-hook '(org-babel-hide-result-toggle-maybe 
org-babel-header-arg-expand)
  org-babel-load-languages '((dot . t) (calc . t) (plantuml . t) (sql . 
t) (js . t) (shell . t) (emacs-lisp . t))
  org-log-done 'time
  org-export-backends '(confluence md ascii html icalendar latex odt)
  org-ascii-format-drawer-function #[771 "\x01\207" [] 4 "\n\n(fn NAME 
CONTENTS WIDTH)"]
  org-trigger-hook '(org-depend-trigger-todo)
  org-priority-lowest 68
  org-agenda-loop-over-headlines-in-active-region nil
  org-src-lang-modes '(("arduino" . arduino) ("redis" . redis) ("php" . 
php) ("C" . c) ("C++" . c++) ("asymptote" . asy) ("bash" . sh) ("beamer" 
. latex) ("calc" . fundamental) ("cpp" . c++) ("ditaa" . artist) ("dot" 
. fundamental)
                       ("elisp" . emacs-lisp) ("ocaml" . tuareg) 
("screen" . shell-script) ("shell" . sh) ("sqlite" . sql))
  org-agenda-prefix-format '((agenda . " %i %-12:c%?-12t% s") (timeline 
. "  % s") (todo . " %i %b") (tags . " %i %-12:c") (search . " %i %-12:c"))
  org-occur-hook '(org-first-headline-recenter)
  org-log-into-drawer t
  org-protocol-protocol-alist '(("capture-eww-readable" :protocol 
"capture-eww-readable" :function 
org-protocol-capture-html--capture-eww-readable :kill-client t)
                                ("capture-html" :protocol "capture-html" 
:function org-protocol-capture-html--with-pandoc :kill-client t))
  org-after-tags-change-hook '(org-expiry-insert-created)
  org-cycle-hook '(org-cycle-hide-archived-subtrees 
org-cycle-hide-drawers org-cycle-show-empty-lines 
org-optimize-window-after-visibility-change)
  org-todo-keywords '((sequence "TODO(t)" "NEXT(n)" "|" "DONE(d!/!)") 
(type "WAIT(w@/!)" "DELEGATED(e@/!)" "IDEA(i!/!)" "|" "CANCELED(c@/!)" 
"PASSED-ON(p@/!)"))
  org-speed-command-hook '(org-speed-command-activate 
org-babel-speed-command-activate)
  org-clock-persist-file "/home/standard/.emacs.d/.cache/org-clock-save.el"
  org-attach-id-dir "org-attach/"
  org-odt-format-inlinetask-function 
'org-odt-format-inlinetask-default-function
  org-latex-prefer-user-labels t
  org-publish-timestamp-directory 
"/home/standard/.emacs.d/.cache/.org-timestamps/"
  org-export-async-init-file 
"/home/standard/.emacs.d/layers/+emacs/org/local/org-async-init.el"
  org-refile-use-outline-path t
  org-export-before-parsing-hook '(org-attach-expand-links)
  org-id-locations-file "/home/standard/.emacs.d/.cache/.org-id-locations"
  org-confirm-shell-link-function 'yes-or-no-p
  org-link-parameters '(("eww" :follow org-eww-open :store 
org-eww-store-link) ("rmail" :follow org-rmail-open :store 
org-rmail-store-link) ("mhe" :follow org-mhe-open :store org-mhe-store-link)
                        ("irc" :follow org-irc-visit :store 
org-irc-store-link :export org-irc-export) ("info" :follow org-info-open 
:export org-info-export :store org-info-store-link)
                        ("gnus" :follow org-gnus-open :store 
org-gnus-store-link) ("docview" :follow org-docview-open :export 
org-docview-export :store org-docview-store-link)
                        ("bibtex" :follow org-bibtex-open :store 
org-bibtex-store-link) ("bbdb" :follow org-bbdb-open :export 
org-bbdb-export :complete org-bbdb-complete-link :store 
org-bbdb-store-link) ("w3m" :store org-w3m-store-link)
                        ("attachment" :follow org-attach-follow 
:complete org-attach-complete-link) ("id" :follow org-id-open)
                        ("orgit-rev" :store orgit-rev-store :follow 
orgit-rev-open :export orgit-rev-export :complete orgit-rev-complete-link)
                        ("orgit-log" :store orgit-log-store :follow 
orgit-log-open :export orgit-log-export :complete orgit-log-complete-link)
                        ("orgit" :store orgit-status-store :follow 
orgit-status-open :export orgit-status-export :complete 
orgit-status-complete-link) ("file+sys") ("file+emacs") ("shell" :follow 
org-link--open-shell)
                        ("news" :follow #[514 "\301\300\302\x04Q\x02\"\207" 
["news" browse-url ":"] 6 "\n\n(fn URL ARG)"]) ("mailto" :follow #[514 
"\301\300\302\x04Q\x02\"\207" ["mailto" browse-url ":"] 6 "\n\n(fn URL ARG)"])
                        ("https" :follow #[514 "\301\300\302\x04Q\x02\"\207" 
["https" browse-url ":"] 6 "\n\n(fn URL ARG)"]) ("http" :follow #[514 
"\301\300\302\x04Q\x02\"\207" ["http" browse-url ":"] 6 "\n\n(fn URL ARG)"])
                        ("ftp" :follow #[514 "\301\300\302\x04Q\x02\"\207" 
["ftp" browse-url ":"] 6 "\n\n(fn URL ARG)"]) ("help" :follow 
org-link--open-help) ("file" :complete org-link-complete-file) ("elisp" 
:follow org-link--open-elisp)
                        ("doi" :follow org-link--open-doi))
  org-html-postamble nil
  org-latex-format-headline-function 
'org-latex-format-headline-default-function
  org-expiry-inactive-timestamps t
  org-present-mode-hook '(spacemacs//org-present-start)
  org-link-translation-function 'toc-org-unhrefify
  org-link-elisp-confirm-function 'yes-or-no-p
  org-todo-keyword-faces '(("NEXT" . "yellow") ("WAIT" . "steel blue") 
("DELEGATED" . "steel blue") ("IDEA" . "khaki") ("DONE" . "PaleGreen3") 
("CANCELED" . "PaleGreen3") ("PASSED-ON" . "DimGrey"))
  org-latex-format-inlinetask-function 
'org-latex-format-inlinetask-default-function
  org-tab-before-tab-emulation-hook '(org-tempo-complete-tag)
  org-html-format-drawer-function #[514 "\207" [] 3 "\n\n(fn NAME 
CONTENTS)"]
  outline-mode-hook '(windows-scripts/dos-outline-hook)
  org-clock-into-drawer "CLOCK"
  org-image-actual-width nil
  org-clocktable-defaults '(:maxlevel 2 :lang "en" :scope file :block 
nil :wstart 1 :mstart 1 :tstart nil :tend nil :step nil :stepskip0 nil 
:fileskip0 nil :tags nil :match nil :emphasize nil :link nil :narrow nil 
:indent t :hidefiles
                            nil :formula nil :timestamp nil :level nil 
:tcolumns nil :formatter nil)
  org-present-mode-quit-hook '(spacemacs//org-present-end)
  org-html-format-headline-function 
'org-html-format-headline-default-function
  org-attach-dir-relative t
  org-use-sub-superscripts '{}
  org-agenda-custom-commands '(("d" "Clock report for today" agenda "" 
((org-agenda-span 1) (org-agenda-start-with-clockreport-mode t) 
(org-agenda-start-with-log-mode (quote clockcheck)))))
  org-download-link-format-function 
'org-download-link-format-function-default
  org-agenda-clockreport-parameter-plist '(:link t :maxlevel 3)
  org-ascii-text-width 999999
  org-attach-use-inheritance t
  org-download-abbreviate-filename-function 'file-relative-name
  org-agenda-todo-ignore-scheduled 'future
  org-default-notes-file "/home/standard/org/notes.org"
  org-insert-heading-hook '(org-expiry-insert-created)
  )



^ permalink raw reply	[flat|nested] 2+ messages in thread

* Bug: performance issue in org-font-lock-add-priority-faces related to org-priority-regexp [9.4 (9.4-7-g3eccc5-elpaplus @ /home/standard/.emacs.d/elpa/26.3/develop/org-plus-contrib-20200921/)]
       [not found] <87bbd664-2da6-0551-25f1-9ff287fed9f2%40mailbox.org>
@ 2020-10-07 21:09 ` Tim Frana
  0 siblings, 0 replies; 2+ messages in thread
From: Tim Frana @ 2020-10-07 21:09 UTC (permalink / raw)
  To: tfr; +Cc: emacs-orgmode

Apologies, the extreme slowdown for the regex in my setup is mostly 
credited to a very long line in my org file (~100k characters). I 
inserted a snippet from a log file to a quote block, without realizing 
that a large base64 string was hiding at the end of one line.
The old regex could exclude that line early on, because it wasn't a 
header line, so I didn't notice a performance hit back then.

With that line deleted (org file shrinks to ~300KB), I measured 
performance with the new and old regex (new on 1st line vs old on 2nd line):

(elp-instrument-function 'org-font-lock-add-priority-faces)
(elp-reset-all)
;; open org file
(elp-results)


      Function Name                     Call Count  Elapsed Time Average 
Time
new: org-font-lock-add-priority-faces  28          0.0228787000 0.0008170964
old: org-font-lock-add-priority-faces  28          0.0007937999 2.834...e-05

So the new one is still considerably slower, but probably not noticeable 
unless the file is really large and/or it contains long lines.


One last remark, my workaround from the last mail is flawed, e.g. it 
broke priority sorting in my agendas. This one worked better:

(defun adv--org-font-lock-add-priority-faces (wrapped-func &rest args)
   (let ((org-priority-regexp "^\\*+ .*?\\(\\[#\\([A-Z0-9]+\\)\\] ?\\)"))
     (apply wrapped-func args)))
(advice-add 'org-font-lock-add-priority-faces :around 
#'adv--org-font-lock-add-priority-faces)



^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2020-10-07 21:10 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-24 12:59 Bug: performance issue in org-font-lock-add-priority-faces related to org-priority-regexp [9.4 (9.4-7-g3eccc5-elpaplus @ /home/standard/.emacs.d/elpa/26.3/develop/org-plus-contrib-20200921/)] Tim Frana
     [not found] <87bbd664-2da6-0551-25f1-9ff287fed9f2%40mailbox.org>
2020-10-07 21:09 ` Tim Frana

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).