Re: [PATCH] Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library

[Ihor Radchenko <yantar92@gmail.com>]

psychosis <psychosis@disroot.org> writes:

>> I assume that your bibliography is much larger than 10Mb. Am I 
>> correct?
>
> Actually, it is only 2Mb (7000 entries). Probably there is a problem 
> with the data structure of my .bib-file? I create it from Zotero and 
> the BibLaTeX-export of the BetterBibTeX-Plugin.

> * ELP Profiling Results
>
> org-cite-basic--parse-bibliography  37183       14.256997996  0.0003834278
> org-cite-basic--get-entry           37180       10.932912038  0.0002940535
> org-cite-basic-activate             1           3.796198629   3.796198629
> org-cite-basic--all-keys            2           3.65558611    1.827793055

> I attached the profiler-reports and the ELP-reports for “emacs -Q” 
> with and without the patch. I use the latest Emacs version, built  with 
> “native-compilation” and “pgtk” (GNU Emacs 29.0.50, 
> x86_64-pc-linux-gnu, GTK+ Version 3.24.31, cairo version 1.17.4; Org 
> mode version 9.5.2, release_9.5.2-24-g668205) with an .org-file that 
> only contains one citation and the location of the bibliography. The 
> CPU is an Intel Core i5-3320M @ 2.60GHz × 4.

Judging from ELP data, most of the time is spent fontifying individual
citations, not reading the .bib file. And the fontification gets called
37k times! This is fishy. It is not supposed to happen.

Could you provide your example file and the relevant .bib record?

>> This is bad. Would you mind constructing a reproducer? (using the
>> patched Org version)
>
> I made a mistake, I inserted the changed cite-key on a new line, which 
> does not work. Inserting the new key on the same line as the old one 
> works. I don’t know if this is the intended behavior.

To clarify, org-cite must detect changes in your .bib file and if a
citation record key in the .bib files was changed, trying to insert it
should trigger org-cite to mark the new citation in red. Regardless if
the citation gets inserted at the same of other line.

Or maybe I misunderstand what you did.

>> 2. You may provide profiling information using M-x profiler-start M-x
>> profiler-report and using ELP as I described in the previous messages.
>
> Thank you very much for your help! I should add, I am content with the 
> improvement of the parsing time that your patch provides. So if you 
> assume that the problem is only on my end, please attend to more 
> pressing issues.

It's fine. I plan to work on org-cite in future, so I use such bug
reports as an opportunity to familiarise myself with the source code.

Best,
Ihor