From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id sL10HUvFOWIG4gAAgWs5BA (envelope-from ) for ; Tue, 22 Mar 2022 13:47:07 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id CPjpGkvFOWL9bgEA9RJhRA (envelope-from ) for ; Tue, 22 Mar 2022 13:47:07 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E945417A48 for ; Tue, 22 Mar 2022 13:47:06 +0100 (CET) Received: from localhost ([::1]:46546 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nWdui-0004TL-Rq for larch@yhetil.org; Tue, 22 Mar 2022 08:47:04 -0400 Received: from eggs.gnu.org ([209.51.188.92]:40236) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nWdbL-0005Tx-Ha for emacs-orgmode@gnu.org; Tue, 22 Mar 2022 08:27:05 -0400 Received: from [2607:f8b0:4864:20::72f] (port=36816 helo=mail-qk1-x72f.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nWdbK-0002h5-3W for emacs-orgmode@gnu.org; Tue, 22 Mar 2022 08:27:03 -0400 Received: by mail-qk1-x72f.google.com with SMTP id v13so13809084qkv.3 for ; Tue, 22 Mar 2022 05:27:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version:content-transfer-encoding; bh=8s2dDHIuDTdPl5V9DgPKDmnOdlUCXm0AmBuANc7UUXg=; b=YD9vWmUT0oueyo3qN3IJaGXxzQZpBVOB2pCGuum7weIIRJhf0HzbCX/4Q+pIqGSb02 zqEjB1V9IAViEits6a3RHxqT3cFTUpKnD3/dc/X3ZMnvgkrMOXrPDk02GXHQn4YAvG5F nVEMy2lLNwZctADvkRZqQsGz5zjxO3rEJmTWcIutHrNK/zKpDG6Q2y4H+G2rfm4jbokA 5L/8NZp6AHRU2a0XGlX6Hd1ZcBhTId6YtjqAwRlBQ/pJlfXM3N/xm7smL+DP34DaGxbc Aue0xcshikn9Wed/CfPdGhRhzuQZe+kjYqhEMocYnt/Ow/yj6O83vx/rp7pkOORMZOHM U8Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=8s2dDHIuDTdPl5V9DgPKDmnOdlUCXm0AmBuANc7UUXg=; b=0mpeJNsvbC6f5t6xw10ktp+qxl8DkImmqk1jHvoGxax3F62+JTIG0+DyIUKVib6iDa KI3sJTT+UVcC5LVwf/jnf3+guif3eCGhVFZmCj4HOQ85Tqb//hhC+Gmem80I1ZenOs+4 VWU5LXl4zRpXDjvp4VVgvafBpVALXBkaB43EclqwqVnf+SaiZt6mU13P4bPKCqZfwxVW eH4J120dHsWL+b6n2EtkiJc9v4fqiZK2xdsoyNrmUHgKZgnzmhbdupuvcVuKxFEsx07t v17J/3TjwvEhv1wFS/rhcGFusgF/aysxpxLof4/SMyDWzy/UeP9NGYptQYkKt1/cuzQY ZCOQ== X-Gm-Message-State: AOAM532YiKsp2/PevEj6gCDJ7OHfVivEQFguWIOqH+xduQevBdQ4+1bv P4HRbei5e4JNG38f/armjVVsg2RdCjJyng== X-Google-Smtp-Source: ABdhPJwL08hzR+pU2R6ZldlCUGmgInvQGPUror9PfmtTYsJTju4YcIS6StwbMA3bLSKoYX6c//gkCQ== X-Received: by 2002:a05:620a:2488:b0:67b:a6a6:b201 with SMTP id i8-20020a05620a248800b0067ba6a6b201mr14710987qkn.580.1647952021124; Tue, 22 Mar 2022 05:27:01 -0700 (PDT) Received: from localhost (22.b2.a86c.ip4.static.sl-reverse.com. [108.168.178.34]) by smtp.gmail.com with ESMTPSA id l126-20020a37bb84000000b0067b3c2bcc0dsm9284868qkf.1.2022.03.22.05.26.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Mar 2022 05:27:00 -0700 (PDT) From: Ihor Radchenko To: psychosis Subject: Re: [PATCH] Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library In-Reply-To: References: <87pmmifirh.fsf@localhost> <87fsnefg8b.fsf@localhost> <87cziifeo4.fsf@localhost> <87a6dmfd89.fsf@localhost> <877d8qf9k8.fsf@localhost> <8735jef8eg.fsf@localhost> <87zglldzl1.fsf@localhost> Date: Tue, 22 Mar 2022 20:27:27 +0800 Message-ID: <87zglicguo.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Host-Lookup-Failed: Reverse DNS lookup failed for 2607:f8b0:4864:20::72f (failed) Received-SPF: pass client-ip=2607:f8b0:4864:20::72f; envelope-from=yantar92@gmail.com; helo=mail-qk1-x72f.google.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, PDS_HP_HELO_NORDNS=0.659, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: emacs-orgmode@gnu.org, Nicolas Goaziou , Jamie Matthews Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1647953227; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=8s2dDHIuDTdPl5V9DgPKDmnOdlUCXm0AmBuANc7UUXg=; b=ta6APP8/DZHSuxD9Wagwl8EIHA/oxMWG4bepPOzZhGGh0k3DqP/KV9tT59RWBmeSwlrmyw sokckYnK4Vt4JFWviOufW1YvItuePjlFRJ/YBSeXOV872xc+fDEo5JMYm38nXUk0SpKQSz 3IDAFWtBb42rdsiy10EvNhDRHzYhTB1B0/1LSmOaLK6ckiGngSU9/e1EXYbAWrw6tNSW7z tlY34/PHG9o+c3CKQc1z4L6M8y85yXBrIIXxtstJP5o5ENYvROHE8boLpo3CXPGqCMigP+ ms9mqzv6Lok9ArbwuVtu4CY2uE8yCzItBZkTZUMnxj7z8NzNhl6T0tbeZ/e4jA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1647953227; a=rsa-sha256; cv=none; b=l8Tr15+BTQovFMTGbhyEsi01IJqUsJYHrreYo+KqcfbSuXWYqxjGydonjRaT/McwLjPwNH H0DYnXE3NYHNUnq5xOkt3lsD7sRCcHEXQA6XmtBMs7U5tDyV2MN4ftyqhyqcv5m+pwxA3z l4JhK6ZPvSUH7kP+r94ZO9ma3e4Ovx2XcjRS9rubYddejiXu8Kx50qDMgmT2kLTYwM2G6x JUxE4XMXJpABRfz2puWuFT2ZadMxxYiSsTxdOpI0GxGXgtCTrLdbtASePmqIUsMjcezka0 id4UFhEDjAiDr0PZC3jhis4yk4qFmJGd9t/t+raKwf9TjFmXtxXcGcocyMVC6Q== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=YD9vWmUT; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: 6.59 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=YD9vWmUT; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: E945417A48 X-Spam-Score: 6.59 X-Migadu-Scanner: scn1.migadu.com X-TUID: N6+KBuWM4YLT psychosis writes: >> I assume that your bibliography is much larger than 10Mb. Am I=20 >> correct? > > Actually, it is only 2Mb (7000 entries). Probably there is a problem=20 > with the data structure of my .bib-file? I create it from Zotero and=20 > the BibLaTeX-export of the BetterBibTeX-Plugin. > * ELP Profiling Results > > org-cite-basic--parse-bibliography 37183 14.256997996 0.0003834278 > org-cite-basic--get-entry 37180 10.932912038 0.0002940535 > org-cite-basic-activate 1 3.796198629 3.796198629 > org-cite-basic--all-keys 2 3.65558611 1.827793055 > I attached the profiler-reports and the ELP-reports for =E2=80=9Cemacs -Q= =E2=80=9D=20 > with and without the patch. I use the latest Emacs version, built with=20 > =E2=80=9Cnative-compilation=E2=80=9D and =E2=80=9Cpgtk=E2=80=9D (GNU Emac= s 29.0.50,=20 > x86_64-pc-linux-gnu, GTK+ Version 3.24.31, cairo version 1.17.4; Org=20 > mode version 9.5.2, release_9.5.2-24-g668205) with an .org-file that=20 > only contains one citation and the location of the bibliography. The=20 > CPU is an Intel Core i5-3320M @ 2.60GHz =C3=97 4. Judging from ELP data, most of the time is spent fontifying individual citations, not reading the .bib file. And the fontification gets called 37k times! This is fishy. It is not supposed to happen. Could you provide your example file and the relevant .bib record? >> This is bad. Would you mind constructing a reproducer? (using the >> patched Org version) > > I made a mistake, I inserted the changed cite-key on a new line, which=20 > does not work. Inserting the new key on the same line as the old one=20 > works. I don=E2=80=99t know if this is the intended behavior. To clarify, org-cite must detect changes in your .bib file and if a citation record key in the .bib files was changed, trying to insert it should trigger org-cite to mark the new citation in red. Regardless if the citation gets inserted at the same of other line. Or maybe I misunderstand what you did. >> 2. You may provide profiling information using M-x profiler-start M-x >> profiler-report and using ELP as I described in the previous messages. > > Thank you very much for your help! I should add, I am content with the=20 > improvement of the parsing time that your patch provides. So if you=20 > assume that the problem is only on my end, please attend to more=20 > pressing issues. It's fine. I plan to work on org-cite in future, so I use such bug reports as an opportunity to familiarise myself with the source code. Best, Ihor