From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id qKCZBFWwNWLxbgAAgWs5BA (envelope-from ) for ; Sat, 19 Mar 2022 11:28:37 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id uPP3AVWwNWJTTQEA9RJhRA (envelope-from ) for ; Sat, 19 Mar 2022 11:28:37 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 634431591D for ; Sat, 19 Mar 2022 11:28:36 +0100 (CET) Received: from localhost ([::1]:37246 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nVWK3-0001PJ-JA for larch@yhetil.org; Sat, 19 Mar 2022 06:28:35 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37618) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nVWJR-0001NI-UD for emacs-orgmode@gnu.org; Sat, 19 Mar 2022 06:27:58 -0400 Received: from [2607:f8b0:4864:20::42f] (port=38826 helo=mail-pf1-x42f.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nVWJQ-0007fW-1t for emacs-orgmode@gnu.org; Sat, 19 Mar 2022 06:27:57 -0400 Received: by mail-pf1-x42f.google.com with SMTP id p5so5481326pfo.5 for ; Sat, 19 Mar 2022 03:27:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version:content-transfer-encoding; bh=9Cdjz4+alurHb/hKg+ZfX9Wqw5mOgS3O2utbOrI/ZD8=; b=DgXEngWebT2S7tisT+S0LVjrqmt2ND50QJZKFlU5GSW6lCORblHcLgNfirlY6tttgU jr3N8kubYnGzF3gcjwYeimbLawzG0kj3WaiUabTXRj4c42TK7mJZ8G4Gy8Svopzjc0v0 AJ+qCS/18l6NtQsQ8wdF2YBuWtMoAIdkXtNT94v8tpjv9K6vZk24a0e8+cRm1KOeIe17 GBzC2w/eJieP1dpD5WwxV8I3FUScpI4agSgHv8BiISFYwbC9t1eLJ0iT69UvaPNc6HzL sYPvVIl1PJh+B/lLy0od6Ki7jmuOH2hTOOsFNu3ISAXJ4LgP92q5QzToRT1+He545KnB LhHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=9Cdjz4+alurHb/hKg+ZfX9Wqw5mOgS3O2utbOrI/ZD8=; b=gaUPygZnvgGhuy1Vsr521bU7Jh25T+TPaWzNGwr48KddQ7jRncdRsevIK3o3gX2tf6 eBq3L5xnMXgv7r1slCKbU350zMw102/SkBLA0rrdYqkSGcX+g3yazXuaN9PtBOjlONXz n+Bn8FoVSfOHPsPALPQeluVPSb2ruoG43qQk+GXeEEYtg19A1ezUqmzVHQ3/b+x/tntb q9nYSclLpLFe47qiKx18+pqTg7vGixjPcV4nukWT739CEXxRmPKZWEthUMxDbP/yGUnz FijhB17O2TX3ShMr/p0REf7rtuNLl0t4wwljPLqrUcwL3U99Z9MCned6P9Yz1dwNLC6k /yZg== X-Gm-Message-State: AOAM532w7SKbYH9Z5bBkKtbTKBQntBHjHUZcNgcYwVp3OeNe77zNuRjB ZmvfbAgA6hDI/t5ImcvGVglztcahTQnIzg== X-Google-Smtp-Source: ABdhPJxIDiPge2ltFkVABWrH2Z4S1XP4o2pS/AZn0cm71NNw2jM+z1pV5cauQE8URBcCTRR8CQffrw== X-Received: by 2002:a63:d342:0:b0:381:fd6f:4792 with SMTP id u2-20020a63d342000000b00381fd6f4792mr10951502pgi.101.1647685674390; Sat, 19 Mar 2022 03:27:54 -0700 (PDT) Received: from localhost ([45.128.72.3]) by smtp.gmail.com with ESMTPSA id j6-20020a63b606000000b003808b0ea96fsm9082103pgf.66.2022.03.19.03.27.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Mar 2022 03:27:53 -0700 (PDT) From: Ihor Radchenko To: Jamie Matthews Subject: Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)] In-Reply-To: References: <87pmmifirh.fsf@localhost> <87fsnefg8b.fsf@localhost> <87cziifeo4.fsf@localhost> Date: Sat, 19 Mar 2022 18:28:22 +0800 Message-ID: <87a6dmfd89.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Host-Lookup-Failed: Reverse DNS lookup failed for 2607:f8b0:4864:20::42f (failed) Received-SPF: pass client-ip=2607:f8b0:4864:20::42f; envelope-from=yantar92@gmail.com; helo=mail-pf1-x42f.google.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, PDS_HP_HELO_NORDNS=0.659, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "emacs-orgmode@gnu.org" Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1647685716; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=9Cdjz4+alurHb/hKg+ZfX9Wqw5mOgS3O2utbOrI/ZD8=; b=sG0H+Mpr3XCrarSyoZeFvUwifCIiQhc3KBDkbgRoBiR58z42sViHcMQ7GmZS5a1OD5X5J0 yuWXjNDzsofH+tSRhdVf5ikFr37+KXoetXmHlJH4+v4w6ehr2DH6+rLkzXkEqkLRsThjAy JJkQlYF6ARLLTpLiRh0Qt/8Ee+F/dK/co6OLzzfdaOzFyUX8/PaW++F2wqQkkpLyB5WnWs OSw60koBpGVRoWlsH0AhIFQFbXD8XPYQR5wTnuTD1rmVwIPL47gLIwL5NKuOoc7W92UrOy xc8Fztu3OaygGgT5QOV759Bd1woFCvmWTMOIszeb4a19c23+A5Z5Rx1RB8iq9g== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1647685716; a=rsa-sha256; cv=none; b=PAEvrZ6S5PvaJs3fHxfeaI3Q7tIDiqOfEcKhZnWAdSEWn2WggJh0JzLoIkc58HpH6EqJtX uqUv3onNDAP3paRKnefb95vJ5e2xgqPRM0+h6JkhuN6rheYl8DAHB5LOfwPBQcc/5uXjKj fypX0bZyakWfblmz2gcJ3gKVJU92OtCYirMQk+v8t7n4dFT+9bPDmbSKXtrGFpWJdggbQr 14LmPbVKyGLhyRJCD0HcrExTsi8X1KBUgdzDdbDbF2dAUDztyg9uISrPET7PqcY8kJ6zTe yhWoG1NVd3O/3ubnw/XjixHCRzaP//NSdbqXdFluVuIbZGP7M0KMYq5bfIeHVQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=DgXEngWe; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -3.64 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=DgXEngWe; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 634431591D X-Spam-Score: -3.64 X-Migadu-Scanner: scn1.migadu.com X-TUID: f5wEvCBpiMe8 Jamie Matthews writes: > It seems a very significant improvement - the lag on scrolling after the = buffer is loaded is not noticeable, and the initial loading hang essentiall= y disappears! Great! Do you mean that there is no apparent slowness at all? > I'm slightly surprised that elp-results=E2=80=8B doesn't seem to show as = dramatic an improvement as it feels for me: > > ``` > org-cite-basic-activate 123 6.5652978389 0.0533764051 > org-cite-basic--all-keys 123 6.2679872230 0.0509592457 > org-cite-basic--parse-bibliography 270 6.2513217860 0.0231530436 > org-cite-basic--get-entry 147 0.1550045990 0.0010544530 > ``` It does. You need to look at the last column. The total time in the second column is large probably because you interacted (scrolled?) with the buffer a bit more than first time and the functions got called more (see second column). Also, can you try the following version of org-cite-basic--parse-bibliography again and try to change cite key of one of the cited entries of the bibliography outside emacs? The key should be marked red if you try to insert the changed entry in Org. (defvar org-cite-basic--file-id-cache nil "Hash table linking files to their hash.") (defun org-cite-basic--parse-bibliography (&optional info) "List all entries available in the buffer. Each association follows the pattern (FILE . ENTRIES) where FILE is the absolute file name of the BibTeX file, and ENTRIES is a h= ash table where keys are references and values are association lists between fi= elds, as symbols, and values as strings or nil. Optional argument INFO is the export state, as a property list." (unless (hash-table-p org-cite-basic--file-id-cache) (setq org-cite-basic--file-id-cache (make-hash-table :test #'equal))) (if (plist-member info :cite-basic/bibliography) (plist-get info :cite-basic/bibliography) (let ((results nil)) (dolist (file (org-cite-list-bibliography-files)) (when (file-readable-p file) (with-temp-buffer (when (or (file-has-changed-p file) (not (gethash file org-cite-basic--file-id-cache))) (insert-file-contents file) (puthash file (org-buffer-hash) org-cite-basic--file-id-cache= )) (let* ((file-id (cons file (gethash file org-cite-basic--file-id-cache= ))) (entries (or (cdr (assoc file-id org-cite-basic--bibliography-ca= che)) (let ((table (pcase (file-name-extension file) ("json" (org-cite-basic--parse-json)) ("bib" (org-cite-basic--parse-bibtex 'bibl= atex)) ("bibtex" (org-cite-basic--parse-bibtex 'B= ibTeX)) (ext (user-error "Unknown bibliography extensi= on: %S" ext))))) (push (cons file-id table) org-cite-basic--biblio= graphy-cache) table)))) (push (cons file entries) results))))) (when info (plist-put info :cite-basic/bibliography results)) results))) Best, Ihor