Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]

["Bruce D'Arcus" <bdarcus@gmail.com>]

Thanks for posting this!

For Nicolas et al, my more general question about performance with
oc-basic was related to this, so you can ignore that.

On Tue, Mar 15, 2022 at 1:01 PM Jamie Matthews <jdm204@cam.ac.uk> wrote:
>
> # Issue
> Starting emacs with emacs -Q, then navigating to a minimal example org file with C-x C-f yields a ~10 second hang on an 8-core/16GB RAM machine with nothing else running. Also, scrolling commands like C-v are often laggy after the initial hang.
>
> The minimal org file begins:
> """
> #+bibliography: ~/cloud/library/lib.bib
>
> [cite:@tillyPrimaryAnaplasticLargeCell1997]
> """
> with another 80 citations afterwards, one-by-line, but nothing else. As mentioned in the title, the lib.bib file is ~10MB - if I swap this out for a non-existent or tiny bibtex file the problem goes away, and the in-buffer citations are rendered in a red face. Clearly from this and the below profile the issue is something to do with checking etc the citations for fontification purposes.
>
> # profile
>
> ## cpu
>   1,542,884,496  99% - redisplay_internal (C function)
>   1,542,860,504  99%  - jit-lock-function
>   1,542,860,504  99%   - jit-lock-fontify-now
>   1,542,860,504  99%    - jit-lock--run-functions
>   1,542,860,504  99%     - run-hook-wrapped
>   1,542,860,504  99%      - #<compiled 0x19ba3bad288bcf3d>
>   1,542,860,504  99%       - font-lock-fontify-region
>   1,542,860,504  99%        - font-lock-default-fontify-region
>   1,542,792,728  99%         - font-lock-fontify-keywords-region
>   1,542,661,211  99%          - org-cite-activate
>   1,542,542,267  99%           - org-cite-basic-activate
>     787,037,416  50%            - org-cite-basic--get-entry
>       4,065,194   0%             + org-cite-basic--parse-bibliography
>     754,769,872  48%            - org-cite-basic--all-keys
>       6,151,200   0%             + seq-mapcat
>       3,850,126   0%             + org-cite-basic--parse-bibliography
>         481,332   0%            + org-element-interpret-data
>         129,376   0%            + org-cite-basic--print-entry
>         109,615   0%            + org-cite-get-references
>          87,264   0%             org-element-citation-parser
>           5,988   0%            org-activate-links
>           5,677   0%            org-do-emphasis-faces
>           3,524   0%            org-fontify-meta-lines-and-blocks
>           3,072   0%          + org-activate-footnote-links
>           3,072   0%            org-do-latex-and-related
>           7,392   0%  - eval
>           7,392   0%     if
>           3,072   0%    kill-this-buffer-enabled-p
>           2,112   0%  - tool-bar-make-keymap
>           2,112   0%   - tool-bar-make-keymap-1
>           2,112   0%    - mapcar
>           2,112   0%       #<compiled 0x1dd4863a123beee7>
>           1,176   0%    menu-bar-update-buffers
>       6,219,935   0% + command-execute
>           1,280   0% + timer-event-handler
>              24   0% + eldoc-schedule-timer
>              21   0% + #<compiled -0xa96583997f59b99>
>               0   0%   ...
>
> ## memory
>
>         5778  95% - redisplay_internal (C function)
>         5778  95%  - jit-lock-function
>         5778  95%   - jit-lock-fontify-now
>         5778  95%    - jit-lock--run-functions
>         5778  95%     - run-hook-wrapped
>         5778  95%      - #<compiled 0x19ba3bad288bcf3d>
>         5778  95%       - font-lock-fontify-region
>         5778  95%        - font-lock-default-fontify-region
>         5778  95%         - font-lock-fontify-keywords-region
>         5774  95%          - org-cite-activate
>         5762  95%           - org-cite-basic-activate
>         2939  48%            - org-cite-basic--get-entry
>         1631  26%             - org-cite-basic--parse-bibliography
>          400   6%              - set-auto-coding
>          400   6%                 find-auto-coding
>           20   0%              + org-cite-list-bibliography-files
>            4   0%              + #<compiled -0x1c8cad8ab65dbcdf>
>            4   0%                after-insert-file-set-coding
>         2819  46%            - org-cite-basic--all-keys
>         1515  25%             - org-cite-basic--parse-bibliography
>          364   6%              - set-auto-coding
>          364   6%                 find-auto-coding
>           16   0%              - org-cite-list-bibliography-files
>            8   0%               + org-collect-keywords
>            4   0%                 #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_23>
>           32   0%             + seq-mapcat
>            4   0%            + org-cite-basic--print-entry
>            4   0%             org-element-citation-parser
>            4   0%            org-activate-links
>          233   3% - command-execute
>          218   3%  - byte-code
>          149   2%   + read-extended-command
>           69   1%   - find-file-read-args
>           69   1%    - read-file-name
>           69   1%     - read-file-name-default
>           15   0%      - completing-read-default
>            1   0%       - command-execute
>            1   0%        - funcall-interactively
>            1   0%         - minibuffer-complete
>            1   0%          - completion-in-region
>            1   0%           - completion--in-region
>            1   0%            + #<compiled -0xf4b91251a4f5a2>
>           15   0%  - funcall-interactively
>            6   0%   + find-file
>            5   0%   + execute-extended-command
>            4   0%     scroll-up-command
>           26   0% + ...
>            4   0% + timer-event-handler
>            4   0% + #<compiled -0xa96583997f59b99>
>
>
>
> # Version info:
> Emacs  : GNU Emacs 29.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version 3.24.30, cairo version 1.16.0)
>  of 2022-03-11
> Package: Org mode version 9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)