From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id QG6EDmbiMWK4bAEAgWs5BA (envelope-from ) for ; Wed, 16 Mar 2022 14:13:10 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id sOhHC2biMWL9/QAAauVa8A (envelope-from ) for ; Wed, 16 Mar 2022 14:13:10 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 8C6DA373CB for ; Wed, 16 Mar 2022 14:13:09 +0100 (CET) Received: from localhost ([::1]:46456 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nUTSe-0001vl-9W for larch@yhetil.org; Wed, 16 Mar 2022 09:13:08 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35918) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nUTHO-0001Hl-G0 for emacs-orgmode@gnu.org; Wed, 16 Mar 2022 09:01:35 -0400 Received: from [2a00:1450:4864:20::32c] (port=41519 helo=mail-wm1-x32c.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nUTHI-0008K6-QZ for emacs-orgmode@gnu.org; Wed, 16 Mar 2022 09:01:30 -0400 Received: by mail-wm1-x32c.google.com with SMTP id 7-20020a05600c228700b00385fd860f49so1390763wmf.0 for ; Wed, 16 Mar 2022 06:01:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=lRAWaxRSt1masVF2XzeB+6hcNyhVeySv6CRmzN24H5A=; b=KvqenZhsSmnYw8QmxXMOLPfkx+SFXVvv4l1q4rpDVWCnnsNxAoOaKSQOaFU7QsLlTX 9vBnb4BkudphiPMoDE8QP1wLeW7ZRRG3MMWxbMNk1Ps5ax3QC+O+AmpMj4SAh4US/roq axaQes+aZDzSBTHbRlxhnAC6UDp2UMgdr9u/GPkoujVL9tW70xf6CbpVglYQBjc/rJrc xFenOjaLCmjMpz00Tt1pcVscPe+9CR0qB90gMcOVl7MTj3GZUCOqFquum7wEnGtwADqN l7DeUMNfIIuy6QChWCLGyA0aZrfsN8LXVCQcCi2oy9smL7sS26zOXQ3uVOz7zTAMxaG4 EVQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=lRAWaxRSt1masVF2XzeB+6hcNyhVeySv6CRmzN24H5A=; b=UCYTQuy/CyDc38LOl/OVrHnuT1yEfZWfzeoABsmVMlt1Bf27aDG1MR5RhdoV1Oe0RI 4cBZ8fit/PHQiZN62A8aBsBflgcQ+oqcHJpXwFGHvoC0Ykn7RcVcf0tMMfzKedlcyFPU c7mQMrDUqYfrgwsbnqBKr6OjNoNmlZn38C7SaY7/IGR8qsdyCKzPOaF3PJLn1en4SlLR PPjuWhVO/fNZrvkRsiLe/gauQ2jS7UZZ42mttPqkIuMa5g6xnN9q83MsBC8cxEcJmrq9 CqaDpws2QvFoMh+W8YnWRlyr6goco2hYcKCFQtnQwl9AMdGzWfp/6dt0ii5Pzxfa/RQW cjkg== X-Gm-Message-State: AOAM533BcmAATAZbK0DEXnZkOYwii2zf3ZDJqZEp7Hv94JwH7YJ/b1Pl K3COMnc5wghfDJO4da/KBKPbHxYDHLww+QsEDRY= X-Google-Smtp-Source: ABdhPJwDBkL7PefXR8eVAbCUWVaUt4HCi6pL40qqT/f8cR+RYpWWUTCOr28K0HodqdPQ6qGP4XfWY02OEEiCibo3xlI= X-Received: by 2002:a1c:720c:0:b0:38c:7201:1055 with SMTP id n12-20020a1c720c000000b0038c72011055mr1326349wmc.145.1647435681960; Wed, 16 Mar 2022 06:01:21 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: "Bruce D'Arcus" Date: Wed, 16 Mar 2022 09:01:34 -0400 Message-ID: Subject: Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)] To: Jamie Matthews Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::32c (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::32c; envelope-from=bdarcus@gmail.com; helo=mail-wm1-x32c.google.com X-Spam_score_int: -6 X-Spam_score: -0.7 X-Spam_bar: / X-Spam_report: (-0.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, PDS_HP_HELO_NORDNS=0.659, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "emacs-orgmode@gnu.org" Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1647436389; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=lRAWaxRSt1masVF2XzeB+6hcNyhVeySv6CRmzN24H5A=; b=WCWymE87I67Eg/drHZHlJ0vnWlguwYdj3JuSRVZQ0Hnif3Lx1dvmHGEVM2jV8kIYj7s+b6 sLtUt/QVi/9NuiIBbNoC0h1meiCyc8P/LSCkSQIUD3dyzvRLStgK02zjt+FUIhXDSc0Std KTfXX1Tgx1t2r283jLCVEw6I1i3pVe2fiIOa56x5u5xcz9qfz7fxSXQt8FQs8NpYTY7N8c GAhYX7Cm5FJzMCUwC7VGEAFR8ac8JiBW5nA9yB94Nd3MvnOTvtz7ENR7ruiWK/wT8JJN2v RhqPafIsKc6ZxFI78kW5Vsprv0cCZb5Xe9e1uXPjl+N3LKP4+6A0/1cH0bx7Wg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1647436389; a=rsa-sha256; cv=none; b=m4w3AIyc4HVPBQbSqNiKxmQaD+cNtGQKtmTpqDi6td9pyk5QxGcf0sS76isz1Ip8nVAKyA V/X8smU7MCXnSHg3cStsx2+Od2tX+2QCX3V7pwSDuleQ1X4OcJOrbSw6XVfex2PNskJJbg h0ZTCTDbw1btC3h5B9RE4FQ3+RaoJbETCuzLuokSb/6FK0QtUqNAzkfegeZ02z69h0UgTb grxRhAUlvFCJpqx6MbKtaG+fbiWaN+1v04FfhuMEctxBqWgPDtNJbE63M3oBBSGg5yQeg4 n8j1C9T9wU1SLrOltwQpQSqwBzS0MibaNS8powvYbj/l3i+iM1zygeKwmSch+w== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=KvqenZhs; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.16 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=KvqenZhs; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 8C6DA373CB X-Spam-Score: -4.16 X-Migadu-Scanner: scn1.migadu.com X-TUID: fxE/C3nU+li7 Thanks for posting this! For Nicolas et al, my more general question about performance with oc-basic was related to this, so you can ignore that. On Tue, Mar 15, 2022 at 1:01 PM Jamie Matthews wrote: > > # Issue > Starting emacs with emacs -Q, then navigating to a minimal example org fi= le with C-x C-f yields a ~10 second hang on an 8-core/16GB RAM machine with= nothing else running. Also, scrolling commands like C-v are often laggy af= ter the initial hang. > > The minimal org file begins: > """ > #+bibliography: ~/cloud/library/lib.bib > > [cite:@tillyPrimaryAnaplasticLargeCell1997] > """ > with another 80 citations afterwards, one-by-line, but nothing else. As m= entioned in the title, the lib.bib file is ~10MB - if I swap this out for a= non-existent or tiny bibtex file the problem goes away, and the in-buffer = citations are rendered in a red face. Clearly from this and the below profi= le the issue is something to do with checking etc the citations for fontifi= cation purposes. > > # profile > > ## cpu > 1,542,884,496 99% - redisplay_internal (C function) > 1,542,860,504 99% - jit-lock-function > 1,542,860,504 99% - jit-lock-fontify-now > 1,542,860,504 99% - jit-lock--run-functions > 1,542,860,504 99% - run-hook-wrapped > 1,542,860,504 99% - # > 1,542,860,504 99% - font-lock-fontify-region > 1,542,860,504 99% - font-lock-default-fontify-region > 1,542,792,728 99% - font-lock-fontify-keywords-region > 1,542,661,211 99% - org-cite-activate > 1,542,542,267 99% - org-cite-basic-activate > 787,037,416 50% - org-cite-basic--get-entry > 4,065,194 0% + org-cite-basic--parse-bibliography > 754,769,872 48% - org-cite-basic--all-keys > 6,151,200 0% + seq-mapcat > 3,850,126 0% + org-cite-basic--parse-bibliography > 481,332 0% + org-element-interpret-data > 129,376 0% + org-cite-basic--print-entry > 109,615 0% + org-cite-get-references > 87,264 0% org-element-citation-parser > 5,988 0% org-activate-links > 5,677 0% org-do-emphasis-faces > 3,524 0% org-fontify-meta-lines-and-blocks > 3,072 0% + org-activate-footnote-links > 3,072 0% org-do-latex-and-related > 7,392 0% - eval > 7,392 0% if > 3,072 0% kill-this-buffer-enabled-p > 2,112 0% - tool-bar-make-keymap > 2,112 0% - tool-bar-make-keymap-1 > 2,112 0% - mapcar > 2,112 0% # > 1,176 0% menu-bar-update-buffers > 6,219,935 0% + command-execute > 1,280 0% + timer-event-handler > 24 0% + eldoc-schedule-timer > 21 0% + # > 0 0% ... > > ## memory > > 5778 95% - redisplay_internal (C function) > 5778 95% - jit-lock-function > 5778 95% - jit-lock-fontify-now > 5778 95% - jit-lock--run-functions > 5778 95% - run-hook-wrapped > 5778 95% - # > 5778 95% - font-lock-fontify-region > 5778 95% - font-lock-default-fontify-region > 5778 95% - font-lock-fontify-keywords-region > 5774 95% - org-cite-activate > 5762 95% - org-cite-basic-activate > 2939 48% - org-cite-basic--get-entry > 1631 26% - org-cite-basic--parse-bibliography > 400 6% - set-auto-coding > 400 6% find-auto-coding > 20 0% + org-cite-list-bibliography-files > 4 0% + # > 4 0% after-insert-file-set-coding > 2819 46% - org-cite-basic--all-keys > 1515 25% - org-cite-basic--parse-bibliography > 364 6% - set-auto-coding > 364 6% find-auto-coding > 16 0% - org-cite-list-bibliography-files > 8 0% + org-collect-keywords > 4 0% # > 32 0% + seq-mapcat > 4 0% + org-cite-basic--print-entry > 4 0% org-element-citation-parser > 4 0% org-activate-links > 233 3% - command-execute > 218 3% - byte-code > 149 2% + read-extended-command > 69 1% - find-file-read-args > 69 1% - read-file-name > 69 1% - read-file-name-default > 15 0% - completing-read-default > 1 0% - command-execute > 1 0% - funcall-interactively > 1 0% - minibuffer-complete > 1 0% - completion-in-region > 1 0% - completion--in-region > 1 0% + # > 15 0% - funcall-interactively > 6 0% + find-file > 5 0% + execute-extended-command > 4 0% scroll-up-command > 26 0% + ... > 4 0% + timer-event-handler > 4 0% + # > > > > # Version info: > Emacs : GNU Emacs 29.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version 3.= 24.30, cairo version 1.16.0) > of 2022-03-11 > Package: Org mode version 9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/ema= cs/straight/build/org/)