Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]

[Jamie Matthews <jdm204@cam.ac.uk>]

[-- Attachment #1: Type: text/plain, Size: 3838 bytes --]

It seems a very significant improvement - the lag on scrolling after the buffer is loaded is not noticeable, and the initial loading hang essentially disappears!

I'm slightly surprised that elp-results​ doesn't seem to show as dramatic an improvement as it feels for me:

```
org-cite-basic-activate             123         6.5652978389  0.0533764051
org-cite-basic--all-keys            123         6.2679872230  0.0509592457
org-cite-basic--parse-bibliography  270         6.2513217860  0.0231530436
org-cite-basic--get-entry           147         0.1550045990  0.0010544530
```

________________________________
From: Ihor Radchenko <yantar92@gmail.com>
Sent: Saturday, March 19, 2022 9:57 AM
To: Jamie Matthews <jdm204@cam.ac.uk>
Cc: emacs-orgmode@gnu.org <emacs-orgmode@gnu.org>
Subject: Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]

Jamie Matthews <jdm204@cam.ac.uk> writes:

> Thanks:
>
> ```
> org-cite-basic-activate             59          10.724349447  0.1817686346
> org-cite-basic--parse-bibliography  129         10.559936049  0.0818599693
> org-cite-basic--all-keys            59          7.830202561   0.1327152976
> org-cite-basic--get-entry           70          2.7772344940  0.0396747784
> ```

org-cite-basic--parse-bibliography appears to be the main bottleneck.

I tried to write a quick fix (untested).
Can you try to redefine org-cite-basic--parse-bibliography to the
version below (note an extra defvar) and let me know how it goes:

(defvar org-cite-basic--file-id-cache nil
  "Hash table linking files to their hash.")
(defun org-cite-basic--parse-bibliography (&optional info)
  "List all entries available in the buffer.

Each association follows the pattern

  (FILE . ENTRIES)

where FILE is the absolute file name of the BibTeX file, and ENTRIES is a hash
table where keys are references and values are association lists between fields,
as symbols, and values as strings or nil.

Optional argument INFO is the export state, as a property list."
  (unless (hash-table-p org-cite-basic--file-id-cache)
    (setq org-cite-basic--file-id-cache (make-hash-table :test #'equal)))
  (if (plist-member info :cite-basic/bibliography)
      (plist-get info :cite-basic/bibliography)
    (let ((results nil))
      (dolist (file (org-cite-list-bibliography-files))
        (when (file-readable-p file)
          (with-temp-buffer
            (when (or (file-has-changed-p file)
                      (not (gethash file org-cite-basic--file-id-cache)))
              (insert-file-contents file))
            (unless (gethash file org-cite-basic--file-id-cache)
              (puthash file (org-buffer-hash) org-cite-basic--file-id-cache))
            (let* ((file-id (cons file (gethash file org-cite-basic--file-id-cache)))
                   (entries
                    (or (cdr (assoc file-id org-cite-basic--bibliography-cache))
                        (let ((table
                               (pcase (file-name-extension file)
                                 ("json" (org-cite-basic--parse-json))
                                 ("bib" (org-cite-basic--parse-bibtex 'biblatex))
                                 ("bibtex" (org-cite-basic--parse-bibtex 'BibTeX))
                                 (ext
                                  (user-error "Unknown bibliography extension: %S"
                                              ext)))))
                          (push (cons file-id table) org-cite-basic--bibliography-cache)
                          table))))
              (push (cons file entries) results)))))
      (when info (plist-put info :cite-basic/bibliography results))
      results)))

Best,
Ihor

[-- Attachment #2: Type: text/html, Size: 8737 bytes --]