Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]

[Ihor Radchenko <yantar92@gmail.com>]

Jamie Matthews <jdm204@cam.ac.uk> writes:

> It seems a very significant improvement - the lag on scrolling after the buffer is loaded is not noticeable, and the initial loading hang essentially disappears!

Great! Do you mean that there is no apparent slowness at all?

> I'm slightly surprised that elp-results​ doesn't seem to show as dramatic an improvement as it feels for me:
>
> ```
> org-cite-basic-activate             123         6.5652978389  0.0533764051
> org-cite-basic--all-keys            123         6.2679872230  0.0509592457
> org-cite-basic--parse-bibliography  270         6.2513217860  0.0231530436
> org-cite-basic--get-entry           147         0.1550045990  0.0010544530
> ```

It does. You need to look at the last column. The total time in the
second column is large probably because you interacted (scrolled?) with
the buffer a bit more than first time and the functions got called more
(see second column).

Also, can you try the following version of
org-cite-basic--parse-bibliography again and try to change cite key of
one of the cited entries of the bibliography outside emacs? The key
should be marked red if you try to insert the changed entry in Org.

(defvar org-cite-basic--file-id-cache nil
  "Hash table linking files to their hash.")
(defun org-cite-basic--parse-bibliography (&optional info)
  "List all entries available in the buffer.

Each association follows the pattern

  (FILE . ENTRIES)

where FILE is the absolute file name of the BibTeX file, and ENTRIES is a hash
table where keys are references and values are association lists between fields,
as symbols, and values as strings or nil.

Optional argument INFO is the export state, as a property list."
  (unless (hash-table-p org-cite-basic--file-id-cache)
    (setq org-cite-basic--file-id-cache (make-hash-table :test #'equal)))
  (if (plist-member info :cite-basic/bibliography)
      (plist-get info :cite-basic/bibliography)
    (let ((results nil))
      (dolist (file (org-cite-list-bibliography-files))
        (when (file-readable-p file)
          (with-temp-buffer
            (when (or (file-has-changed-p file)
                      (not (gethash file org-cite-basic--file-id-cache)))
              (insert-file-contents file)
              (puthash file (org-buffer-hash) org-cite-basic--file-id-cache))
	    (let* ((file-id (cons file (gethash file org-cite-basic--file-id-cache)))
                   (entries
                    (or (cdr (assoc file-id org-cite-basic--bibliography-cache))
                        (let ((table
                               (pcase (file-name-extension file)
                                 ("json" (org-cite-basic--parse-json))
                                 ("bib" (org-cite-basic--parse-bibtex 'biblatex))
                                 ("bibtex" (org-cite-basic--parse-bibtex 'BibTeX))
                                 (ext
                                  (user-error "Unknown bibliography extension: %S"
                                              ext)))))
                          (push (cons file-id table) org-cite-basic--bibliography-cache)
                          table))))
              (push (cons file entries) results)))))
      (when info (plist-put info :cite-basic/bibliography results))
      results)))

Best,
Ihor