[BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]

[Jamie Matthews <jdm204@cam.ac.uk>]

[-- Attachment #1: Type: text/plain, Size: 5598 bytes --]

# Issue
Starting emacs with emacs -Q​, then navigating to a minimal example org file with C-x C-f​ yields a ~10 second hang on an 8-core/16GB RAM machine with nothing else running. Also, scrolling commands like C-v​ are often laggy after the initial hang.

The minimal org file begins:
"""
#+bibliography: ~/cloud/library/lib.bib

[cite:@tillyPrimaryAnaplasticLargeCell1997]
"""
with another 80 citations afterwards, one-by-line, but nothing else. As mentioned in the title, the lib.bib file is ~10MB - if I swap this out for a non-existent or tiny bibtex file the problem goes away, and the in-buffer citations are rendered in a red face. Clearly from this and the below profile the issue is something to do with checking etc the citations for fontification purposes.

# profile

## cpu
  1,542,884,496  99% - redisplay_internal (C function)
  1,542,860,504  99%  - jit-lock-function
  1,542,860,504  99%   - jit-lock-fontify-now
  1,542,860,504  99%    - jit-lock--run-functions
  1,542,860,504  99%     - run-hook-wrapped
  1,542,860,504  99%      - #<compiled 0x19ba3bad288bcf3d>
  1,542,860,504  99%       - font-lock-fontify-region
  1,542,860,504  99%        - font-lock-default-fontify-region
  1,542,792,728  99%         - font-lock-fontify-keywords-region
  1,542,661,211  99%          - org-cite-activate
  1,542,542,267  99%           - org-cite-basic-activate
    787,037,416  50%            - org-cite-basic--get-entry
      4,065,194   0%             + org-cite-basic--parse-bibliography
    754,769,872  48%            - org-cite-basic--all-keys
      6,151,200   0%             + seq-mapcat
      3,850,126   0%             + org-cite-basic--parse-bibliography
        481,332   0%            + org-element-interpret-data
        129,376   0%            + org-cite-basic--print-entry
        109,615   0%            + org-cite-get-references
         87,264   0%             org-element-citation-parser
          5,988   0%            org-activate-links
          5,677   0%            org-do-emphasis-faces
          3,524   0%            org-fontify-meta-lines-and-blocks
          3,072   0%          + org-activate-footnote-links
          3,072   0%            org-do-latex-and-related
          7,392   0%  - eval
          7,392   0%     if
          3,072   0%    kill-this-buffer-enabled-p
          2,112   0%  - tool-bar-make-keymap
          2,112   0%   - tool-bar-make-keymap-1
          2,112   0%    - mapcar
          2,112   0%       #<compiled 0x1dd4863a123beee7>
          1,176   0%    menu-bar-update-buffers
      6,219,935   0% + command-execute
          1,280   0% + timer-event-handler
             24   0% + eldoc-schedule-timer
             21   0% + #<compiled -0xa96583997f59b99>
              0   0%   ...

## memory

        5778  95% - redisplay_internal (C function)
        5778  95%  - jit-lock-function
        5778  95%   - jit-lock-fontify-now
        5778  95%    - jit-lock--run-functions
        5778  95%     - run-hook-wrapped
        5778  95%      - #<compiled 0x19ba3bad288bcf3d>
        5778  95%       - font-lock-fontify-region
        5778  95%        - font-lock-default-fontify-region
        5778  95%         - font-lock-fontify-keywords-region
        5774  95%          - org-cite-activate
        5762  95%           - org-cite-basic-activate
        2939  48%            - org-cite-basic--get-entry
        1631  26%             - org-cite-basic--parse-bibliography
         400   6%              - set-auto-coding
         400   6%                 find-auto-coding
          20   0%              + org-cite-list-bibliography-files
           4   0%              + #<compiled -0x1c8cad8ab65dbcdf>
           4   0%                after-insert-file-set-coding
        2819  46%            - org-cite-basic--all-keys
        1515  25%             - org-cite-basic--parse-bibliography
         364   6%              - set-auto-coding
         364   6%                 find-auto-coding
          16   0%              - org-cite-list-bibliography-files
           8   0%               + org-collect-keywords
           4   0%                 #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_23>
          32   0%             + seq-mapcat
           4   0%            + org-cite-basic--print-entry
           4   0%             org-element-citation-parser
           4   0%            org-activate-links
         233   3% - command-execute
         218   3%  - byte-code
         149   2%   + read-extended-command
          69   1%   - find-file-read-args
          69   1%    - read-file-name
          69   1%     - read-file-name-default
          15   0%      - completing-read-default
           1   0%       - command-execute
           1   0%        - funcall-interactively
           1   0%         - minibuffer-complete
           1   0%          - completion-in-region
           1   0%           - completion--in-region
           1   0%            + #<compiled -0xf4b91251a4f5a2>
          15   0%  - funcall-interactively
           6   0%   + find-file
           5   0%   + execute-extended-command
           4   0%     scroll-up-command
          26   0% + ...
           4   0% + timer-event-handler
           4   0% + #<compiled -0xa96583997f59b99>


# Version info:
Emacs  : GNU Emacs 29.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version 3.24.30, cairo version 1.16.0)
 of 2022-03-11
Package: Org mode version 9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)

[-- Attachment #2: Type: text/html, Size: 11541 bytes --]