From: psychosis <psychosis@disroot.org>
To: Ihor Radchenko <yantar92@gmail.com>
Cc: Jamie Matthews <jdm204@cam.ac.uk>,
emacs-orgmode@gnu.org, Nicolas Goaziou <mail@nicolasgoaziou.fr>
Subject: Re: [PATCH] Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library
Date: Mon, 21 Mar 2022 17:51:14 +0100 [thread overview]
Message-ID: <EHT39R.I9M2S0G79UCC3@disroot.org> (raw)
In-Reply-To: <87zglldzl1.fsf@localhost>
[-- Attachment #1: Type: text/plain, Size: 1891 bytes --]
Hello,
On So, Mär 20 2022 at 12:20:42 +0800, Ihor Radchenko
<yantar92@gmail.com> wrote:
> I have seen your email. Persisting the cache between sessions is a
> more
> tricky. It will require a dedicated effort - significantly more than
> the
> few-liner patch here.
Ah, I am not programmer and made an uneducated assumption because of
the names “org-persist”, “org-cite-basic--bibliography-cache”
and “org-cite-basic--completion-cache”.
> I assume that your bibliography is much larger than 10Mb. Am I
> correct?
Actually, it is only 2Mb (7000 entries). Probably there is a problem
with the data structure of my .bib-file? I create it from Zotero and
the BibLaTeX-export of the BetterBibTeX-Plugin.
> This is bad. Would you mind constructing a reproducer? (using the
> patched Org version)
I made a mistake, I inserted the changed cite-key on a new line, which
does not work. Inserting the new key on the same line as the old one
works. I don’t know if this is the intended behavior.
> 2. You may provide profiling information using M-x profiler-start M-x
> profiler-report and using ELP as I described in the previous messages.
I attached the profiler-reports and the ELP-reports for “emacs -Q”
with and without the patch. I use the latest Emacs version, built with
“native-compilation” and “pgtk” (GNU Emacs 29.0.50,
x86_64-pc-linux-gnu, GTK+ Version 3.24.31, cairo version 1.17.4; Org
mode version 9.5.2, release_9.5.2-24-g668205) with an .org-file that
only contains one citation and the location of the bibliography. The
CPU is an Intel Core i5-3320M @ 2.60GHz × 4.
Thank you very much for your help! I should add, I am content with the
improvement of the parsing time that your patch provides. So if you
assume that the problem is only on my end, please attend to more
pressing issues.
Kind regards
Paul
[-- Attachment #2.1: reports_with_patch.org --]
[-- Type: text/plain, Size: 11418 bytes --]
* CPU-Profiler-Report
8720 53% + ...
7498 45% - command-execute
7060 42% - funcall-interactively
6681 40% - execute-extended-command
6681 40% - command-execute
6681 40% - funcall-interactively
6666 40% - org-cite-insert
6665 40% + #<compiled -0x1f00c9f117456103>
1 0% - org-element-context
1 0% - org-element--object-lex
1 0% org-element-citation-parser
373 2% - find-file
373 2% - find-file-noselect
372 2% - find-file-noselect-1
371 2% - after-find-file
370 2% - normal-mode
370 2% - set-auto-mode
369 2% - set-auto-mode--apply-alist
369 2% - set-auto-mode-0
330 2% - org-mode
236 1% - org-load-modules-maybe
224 1% - require
224 1% - byte-code
203 1% - require
202 1% - byte-code
164 0% - require
163 0% - byte-code
128 0% - require
124 0% - byte-code
101 0% - require
101 0% - byte-code
75 0% - require
73 0% - byte-code
47 0% - require
46 0% - byte-code
29 0% - require
29 0% - byte-code
29 0% - require
18 0% byte-code
8 0% - require
8 0% - byte-code
2 0% - require
1 0% - load-with-code-conversion
1 0% - hack-read-symbol-shorthands
1 0% hack-local-variables--find-variables
3 0% - eieio-defclass-internal
3 0% eieio--add-new-slot
1 0% eval-after-load
1 0% defvar
1 0% - custom-declare-group
1 0% custom-handle-keyword
1 0% - defvar
1 0% byte-code
1 0% - easy-menu-do-define
1 0% - easy-menu-create-menu
1 0% easy-menu-convert-item
2 0% - defvar
2 0% - byte-code
1 0% - sort-coding-systems
1 0% + #<compiled -0x6cdb28b2d51a5ea>
1 0% - mapcar
1 0% #<compiled -0x1344a1181027709e>
1 0% load-with-code-conversion
2 0% - custom-declare-variable
1 0% custom-handle-keyword
1 0% - custom-initialize-reset
1 0% - eval
1 0% - funcall
1 0% + #<compiled 0x160ff216c5504571>
1 0% custom-declare-face
1 0% - define-keymap
1 0% keymap-set
1 0% easy-menu-do-define
1 0% - defvar
1 0% - byte-code
1 0% - define-keymap
1 0% - keymap-set
1 0% - keymap--check
1 0% key-valid-p
2 0% - define-keymap
2 0% - keymap-set
1 0% - keymap--check
1 0% key-valid-p
1 0% - custom-declare-group
1 0% custom-handle-keyword
1 0% - custom-declare-face
1 0% - custom-handle-all-keywords
1 0% custom-handle-keyword
1 0% - easy-menu-do-define
1 0% - easy-menu-create-menu
1 0% - easy-menu-convert-item
1 0% - easy-menu-convert-item-1
1 0% easy-menu-create-menu
79 0% - byte-code
32 0% - require
24 0% - byte-code
9 0% - require
7 0% - byte-code
3 0% - require
3 0% - byte-code
1 0% - require
1 0% - byte-code
1 0% - custom-declare-variable
1 0% - custom-initialize-reset
1 0% - eval
1 0% funcall
3 0% - custom-declare-variable
3 0% - custom-initialize-reset
3 0% - eval
3 0% - funcall
3 0% + #<compiled -0x607663b6614a677>
3 0% - org-link-set-parameters
3 0% - org-link-make-regexps
1 0% - rx-to-string
1 0% - rx--translate
1 0% - rx--translate-form
1 0% rx--translate-seq
3 0% - load
2 0% load-with-code-conversion
2 0% - defvar
2 0% byte-code
1 0% - do-after-load-evaluation
1 0% elisp--font-lock-flush-elisp-buffers
5 0% - custom-declare-variable
5 0% - custom-initialize-reset
5 0% - org-babel-do-load-languages
1 0% - require
1 0% byte-code
4 0% - load
3 0% load-with-code-conversion
4 0% - org-set-regexps-and-options
2 0% - cl-mapcan
2 0% - byte-code
1 0% - require
1 0% - defvar
1 0% byte-code
4 0% - org-macro-initialize-templates
4 0% - byte-code
2 0% require
2 0% - defconst
2 0% - org-version
1 0% locate-library
1 0% hack-dir-local--get-variables
1 0% - run-hooks
1 0% - vc-refresh-state
1 0% - vc-backend
1 0% - vc-registered
1 0% - mapc
1 0% + #<compiled 0x176cf3f2aee2e339>
1 0% - insert-file-contents
1 0% make-lock-file-name
1 0% - find-buffer-visiting
1 0% - file-truename
1 0% - file-truename
1 0% - file-truename
1 0% file-truename
4 0% - next-line
1 0% - line-move
1 0% - line-move-visual
1 0% - eval
1 0% if
438 2% - byte-code
339 2% - read-extended-command
126 0% - completing-read-default
3 0% + redisplay_internal (C function)
1 0% - undo-auto--add-boundary
1 0% undo-auto--boundaries
1 0% - command-execute
1 0% - funcall-interactively
1 0% - minibuffer-complete
1 0% - completion-in-region
1 0% - completion--in-region
1 0% + #<compiled -0x73104f7a7357822>
99 0% - find-file-read-args
99 0% - read-file-name
99 0% - read-file-name-default
34 0% - completing-read-default
2 0% rfn-eshadow-update-overlay
1 0% - command-execute
1 0% - funcall-interactively
1 0% - minibuffer-complete
1 0% - completion-in-region
1 0% - completion--in-region
1 0% + #<compiled -0x73104f7a7357822>
1 0% - frame-windows-min-size
1 0% - window-min-size
1 0% - window--min-size-1
1 0% window-size-fixed-p
132 0% - timer-event-handler
132 0% - apply
130 0% + #<compiled -0x1426c3f9fec74ca4>
2 0% + #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_12>
81 0% + redisplay_internal (C function)
1 0% clear-minibuffer-message
* Memory-Profiler-Report
1,294,962,301 99% + command-execute
6,842,988 0% + redisplay_internal (C function)
257,232 0% - timer-event-handler
247,296 0% - apply
246,096 0% + #<compiled -0x1426c3f9fec74ca4>
1,056 0% show-paren-function
144 0% - blink-cursor-start
144 0% - blink-cursor--start-timer
144 0% - run-with-timer
144 0% - run-at-time
96 0% timer-set-time
48 0% - timer-activate
48 0% timer--activate
6,240 0% timer-inc-time
3,168 0% - timer-activate-when-idle
3,168 0% - timer--activate
3,168 0% timer--time-less-p
216 0% - timer-activate
216 0% timer--activate
4,165 0% - help-command-error-confusable-suggestions
4,165 0% substitute-command-keys
1,056 0% - undo-auto--add-boundary
1,056 0% undo-auto--boundaries
24 0% - eldoc-schedule-timer
24 0% - run-with-idle-timer
24 0% timer-set-idle-time
0 0% ...
* ELP Profiling Results
org-cite-basic--parse-bibliography 37183 14.256997996 0.0003834278
org-cite-basic--get-entry 37180 10.932912038 0.0002940535
org-cite-basic-activate 1 3.796198629 3.796198629
org-cite-basic--all-keys 2 3.65558611 1.827793055
[-- Attachment #2.2: reports_without_patch.org --]
[-- Type: text/plain, Size: 7553 bytes --]
* CPU-Profiler-Report
555055 98% - command-execute
554536 98% - funcall-interactively
554148 98% - execute-extended-command
554147 98% - command-execute
554147 98% - funcall-interactively
554136 98% - org-cite-insert
554136 98% + #<compiled -0x1c5f6fba0011a443>
1 0% - run-at-time
1 0% timer-set-time
382 0% - find-file
382 0% - find-file-noselect
380 0% - find-file-noselect-1
380 0% - after-find-file
379 0% - normal-mode
379 0% - set-auto-mode
378 0% - set-auto-mode--apply-alist
378 0% - set-auto-mode-0
339 0% - org-mode
227 0% - org-load-modules-maybe
215 0% - require
215 0% - byte-code
195 0% - require
192 0% - byte-code
153 0% - require
152 0% - byte-code
122 0% - require
118 0% - byte-code
96 0% - require
95 0% - byte-code
69 0% - require
68 0% - byte-code
39 0% - require
39 0% - byte-code
26 0% - require
26 0% - byte-code
25 0% - require
23 0% - byte-code
3 0% require
2 0% - require
2 0% byte-code
1 0% cl-generic-define
2 0% - custom-declare-variable
2 0% - custom-initialize-reset
2 0% - eval
2 0% - funcall
1 0% + #<compiled 0xe378fc5bb636585>
1 0% + #<compiled 0x4db6beb5f185dde>
1 0% - defvar
1 0% - byte-code
1 0% - define-keymap
1 0% keymap-set
1 0% do-after-load-evaluation
3 0% - defvar
3 0% - byte-code
2 0% - sort-coding-systems
2 0% + #<compiled -0x6cdb28b2d51a5ea>
1 0% - mapcar
1 0% #<compiled -0x16dce84f3afb709d>
1 0% load-with-code-conversion
1 0% - custom-declare-face
1 0% - face-spec-set
1 0% - face-spec-recalc
1 0% face-spec-reset-face
1 0% - define-keymap
1 0% keymap-set
1 0% - custom-declare-variable
1 0% - custom-initialize-reset
1 0% - eval
1 0% - funcall
1 0% + #<compiled -0x1c24747dda5cfbb4>
1 0% - defvar
1 0% - byte-code
1 0% - define-keymap
1 0% keymap-set
3 0% - define-keymap
3 0% - keymap-set
1 0% key-parse
1 0% custom-declare-variable
2 0% + defvar
1 0% + do-after-load-evaluation
100 0% + byte-code
5 0% + org-macro-initialize-templates
2 0% + defconst
1 0% + run-mode-hooks
1 0% + hack-local-variables
1 0% + file-truename
1 0% create-file-buffer
3 0% + save-buffer
1 0% next-line
519 0% - byte-code
414 0% - read-extended-command
141 0% - completing-read-default
59 0% - command-execute
59 0% - funcall-interactively
35 0% - minibuffer-complete
35 0% - completion-in-region
35 0% - completion--in-region
35 0% + #<compiled -0x73d91f7d4123822>
22 0% - next-line-or-history-element
22 0% - next-history-element
22 0% - goto-history-element
21 0% + #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_42>
19 0% + #<compiled -0xdc802aa55ec78c6>
3 0% - timer-event-handler
3 0% - apply
2 0% - show-paren-function
2 0% - show-paren--default
2 0% show-paren--locate-near-paren
1 0% - blink-cursor-start
1 0% - blink-cursor--start-timer
1 0% - run-with-timer
1 0% - run-at-time
1 0% timer-set-time
2 0% + redisplay_internal (C function)
104 0% - find-file-read-args
104 0% - read-file-name
104 0% - read-file-name-default
25 0% - completing-read-default
2 0% - timer-event-handler
2 0% - apply
2 0% #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_10>
9833 1% + ...
331 0% + redisplay_internal (C function)
131 0% - timer-event-handler
131 0% - apply
131 0% + #<compiled -0x81af4a86cf8d641>
* Memory-Profiler-Report
82,333,670,467 99% + command-execute
8,492,807 0% + redisplay_internal (C function)
200,120 0% - timer-event-handler
197,640 0% - apply
197,448 0% + #<compiled -0x81af4a86cf8d641>
192 0% - blink-cursor-start
192 0% - blink-cursor--start-timer
192 0% - run-with-timer
192 0% - run-at-time
120 0% timer-set-time
72 0% - timer-activate
72 0% timer--activate
1,760 0% timer-inc-time
192 0% - timer-activate
192 0% timer--activate
24 0% - eldoc-schedule-timer
24 0% - run-with-idle-timer
24 0% timer-set-idle-time
0 0% ...
* ELP Profiling Results
org-cite-basic--parse-bibliography 37185 556.48202320 0.0149652285
org-cite-basic--get-entry 37181 553.60200644 0.0148893791
org-cite-basic-activate 2 3.9012992189 1.9506496094
org-cite-basic--all-keys 3 3.74166148 1.2472204933
next prev parent reply other threads:[~2022-03-21 16:52 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-14 14:45 [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)] Jamie Matthews
2022-03-16 13:01 ` Bruce D'Arcus
2022-03-19 8:28 ` Ihor Radchenko
2022-03-19 8:57 ` Jamie Matthews
2022-03-19 9:23 ` Ihor Radchenko
2022-03-19 9:25 ` Jamie Matthews
2022-03-19 9:57 ` Ihor Radchenko
2022-03-19 10:12 ` Jamie Matthews
2022-03-19 10:28 ` Ihor Radchenko
2022-03-19 11:17 ` Jamie Matthews
2022-03-19 11:47 ` [PATCH] Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library Ihor Radchenko
2022-03-19 12:01 ` Jamie Matthews
2022-03-19 12:12 ` Ihor Radchenko
2022-03-19 20:13 ` psychosis
2022-03-20 4:20 ` Ihor Radchenko
2022-03-21 16:51 ` psychosis [this message]
2022-03-22 12:27 ` Ihor Radchenko
2022-03-22 16:42 ` psychosis
2022-03-23 11:07 ` Ihor Radchenko
2022-04-16 10:11 ` Ihor Radchenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=EHT39R.I9M2S0G79UCC3@disroot.org \
--to=psychosis@disroot.org \
--cc=emacs-orgmode@gnu.org \
--cc=jdm204@cam.ac.uk \
--cc=mail@nicolasgoaziou.fr \
--cc=yantar92@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).