From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id gKKILJ8AFGJWyQAAgWs5BA (envelope-from ) for ; Mon, 21 Feb 2022 22:14:07 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id mBJeKZ8AFGIIDQAAauVa8A (envelope-from ) for ; Mon, 21 Feb 2022 22:14:07 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3CE591C22B for ; Mon, 21 Feb 2022 22:14:07 +0100 (CET) Received: from localhost ([::1]:49742 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nMG0U-00009j-6F for larch@yhetil.org; Mon, 21 Feb 2022 16:14:06 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54052) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nMFzg-00009D-0D for emacs-orgmode@gnu.org; Mon, 21 Feb 2022 16:13:16 -0500 Received: from [2607:f8b0:4864:20::1035] (port=51096 helo=mail-pj1-x1035.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nMFzd-0003Oa-2m for emacs-orgmode@gnu.org; Mon, 21 Feb 2022 16:13:15 -0500 Received: by mail-pj1-x1035.google.com with SMTP id m22so2470382pja.0 for ; Mon, 21 Feb 2022 13:13:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:from:date:message-id:subject:to; bh=v2/77Uu55jYOJinXlyrB/QD+ghX80RQJ3hN0MFDr6ig=; b=Ri6sHnRgowSGUioGeeXU0KPwTovze8T1tZ2MYfl6U3gzJR+VYaen8rAvJkyKm6I/VF YoQRhZzgeIijVRqegcjGKY3BFUCXgUaW+jt9AqOQjNWzdnbvXC/uXiZ5Z+H1l82LZXYh ek4GIVHlaVhGWfkEXsQLhRSlYgYrqb9ycQctOJqb8Bu6ElcMOZzRLnWSZF0MBvzXEHDd XRpltx+cmywJVokM9anEepfFx+Tu6OaU+EZXU7pGf9AQQQBpQGzzOm7lMxnwfu6R6t1w p+sP1KtB0di7niabKbpF9uQz707Y4N/U8yf4pGsqmadJhhtBXK571ncOjwBh18+QLvYL zojA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=v2/77Uu55jYOJinXlyrB/QD+ghX80RQJ3hN0MFDr6ig=; b=NoGr+hZnMcSI2WbbewNcgpZlLSJYYUfkAAh3H6qZVEc+1ObmErXAi0mZbBtaMF7GDG NbLReU2G9rGASi7vpnzT6WubtNa4XJKtQJsvjtuEYe77jxd5v0TWrzrELkCxN9BI5//b 4J1YMBKN8LciaggDb5jiK0XC4wUGo6A8NfkQqBg7kw36AQ4G2XnlihtH16LNziBYe1yo rGdGcjwGAX5ezCuNImqCT3QsKDwdNAvbY7Z+YTyrKzLRFjoox1YF9pnWFJIwgsNFPU94 Da+OkomJH/Rjox61xeGkGfp6ngDLnYCvuakQK7Ft8cpWYJknLsBuxsJYHKJaxv8U2gEY h66A== X-Gm-Message-State: AOAM532k26V++yusS28DZ0Twb5vi1NRA3YLFQBiDXYhe+JqNqaRDk3we mAJEN6t9ihFNPvYlkaKDU7fDLR9OeGyoR2/PwmvAd9WpEOU= X-Google-Smtp-Source: ABdhPJzra+meN9FX4rZEehWTEi3HiBkQI2kRN20cHWodGb2XCfj2cp4TOI0TxujC468pn6b9dNX9HW+xIsmkDMf7X/0= X-Received: by 2002:a67:e087:0:b0:31b:57fe:7ac1 with SMTP id f7-20020a67e087000000b0031b57fe7ac1mr8481512vsl.81.1645477574369; Mon, 21 Feb 2022 13:06:14 -0800 (PST) MIME-Version: 1.0 From: Matt Price Date: Mon, 21 Feb 2022 16:06:03 -0500 Message-ID: Subject: profiling latency in large org-mode buffers (under both main & org-fold feature) To: Org Mode Content-Type: multipart/alternative; boundary="0000000000004e663405d88d9a68" X-Host-Lookup-Failed: Reverse DNS lookup failed for 2607:f8b0:4864:20::1035 (failed) Received-SPF: pass client-ip=2607:f8b0:4864:20::1035; envelope-from=moptop99@gmail.com; helo=mail-pj1-x1035.google.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, PDS_HP_HELO_NORDNS=0.659, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1645478047; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=v2/77Uu55jYOJinXlyrB/QD+ghX80RQJ3hN0MFDr6ig=; b=F/wSxao/EMQ5wxMyoz3SEgmE/eO4Yc5nbsc69Z7mXVxchku0JPvP3ypyB7qOJT9zYJjn+V FoewvrutIpnJNT0uGXyh474gjUs4A3s+r5lzM3MGPtuKwerKgcHjRD6ZQdgKwB45/1BhSp +yrAIV1dz5IKuhVRKIwmOnJGDFwrJUiDKB0zLrnpl6zPftZUE7Q10vkziJ1XMToOkMswHY nAb3YCxZyn89DQvSpEAygybZ4ShevVjiOK8tRSc/QiyekzpVpe7FpOj8YwQm9xcmi6FbrI ul36CqVo2fwUG6UVGzLGmzJeiVxDRJS0H+wv+Ev5TF3Qn9o4xNRQaWkHIHH3Xw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1645478047; a=rsa-sha256; cv=none; b=rOHBFZYjwltQZuF5gmkQIjfWrHhuUqUYwTlKTcJnI0hCTW+WYO3/2ENzNQVY6far1XokAX YkdpzMYCn5jt9uSFHLyuYcwk4wr7pdSc7MZKaywgTZ6O9UUUius06+rpISGGjiWAYIZqVd Wn31THuKwadzk24FcXB9my+/LdxsaE3w2/Q86P8LTi7y+XukhKnMNQbEGjPKZq3NSjn90o bl/5gzreRY2ZslRxpvmVLVAkyWA/1597bTr1vwyMoOKy/T8ZuwytsT4HMjimt0C4gMwL91 XqE4jSBQpyUqP683RB+w96ROmcKNw5eicrTVYVKftbRtWD6MNLw+mFx7/zrrrQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Ri6sHnRg; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.33 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Ri6sHnRg; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 3CE591C22B X-Spam-Score: -4.33 X-Migadu-Scanner: scn1.migadu.com X-TUID: Q7PTdtXb5UXC --0000000000004e663405d88d9a68 Content-Type: text/plain; charset="UTF-8" I'm trying to figure out what causes high latency while typing in large org-mode files. The issue is very clearly a result of my large config file, but I'm not sure how to track it down with any precision. My main literate config file is ~/.emacs.d/emacs-init.org, currently 15000 lines, 260 src blocks. If I create a ~minimal.el~ config like this: (let* ((all-paths '("/home/matt/src/org-mode/emacs/site-lisp/org"))) (dolist (p all-paths) (add-to-list 'load-path p))) (require 'org) (find-file "~/.emacs.d/emacs-init.org") then I do not notice any latency while typing. If I run the profiler while using the minimal config, the profile looks about like this at a high level: 1397 71% - command-execute 740 37% - funcall-interactively 718 36% - org-self-insert-command 686 34% + org-element--cache-after-change 10 0% + org-fold-core--fix-folded-region 3 0% + blink-paren-post-self-insert-function 2 0% + jit-lock-after-change 1 0% org-fold-check-before-invisible-edit--text-properties 9 0% + previous-line 6 0% + minibuffer-complete 3 0% + org-return 3 0% + execute-extended-command 657 33% - byte-code 657 33% - read-extended-command 64 3% - completing-read-default 14 0% + redisplay_internal (C function) 1 0% + timer-event-handler 371 18% - redisplay_internal (C function) 251 12% + jit-lock-function 90 4% + assq 7 0% + substitute-command-keys 3 0% + eval 125 6% + timer-event-handler 69 3% + ... -------------------------- However, if I instead use my fairly extensive main config, latency is high enough that there's a noticeable delay while typing ordinary words. I see this regardless of whether I build from main or from Ihor's org-fold feature branch on github. The profiler overview here is pretty different -- redisplay_internal takes a much higher percentage of the CPU requirement: 3170 56% - redisplay_internal (C function) 693 12% - substitute-command-keys 417 7% + # 59 1% + assq 49 0% + org-in-subtree-not-table-p 36 0% + tab-bar-make-keymap 35 0% and 24 0% + not 16 0% org-at-table-p 13 0% + jit-lock-function 8 0% keymap-canonicalize 7 0% + # 4 0% + funcall 4 0% display-graphic-p 3 0% + # 3 0% file-readable-p 3 0% + table--probe-cell 3 0% table--row-column-insertion-point-p 1486 26% - command-execute 1200 21% - byte-code 1200 21% - read-extended-command 1200 21% - completing-read-default 1200 21% - apply 1200 21% - vertico--advice 475 8% + # ---------------------- I've almost never used the profiler and am not quite sure how I should proceed to debug this. I realize I can comment out parts of the config one at a time, but that is not so easy for me to do in my current setup, and I suppose there are likely to be multiple contributing causes, which I may not really notice except in the aggregate. If anyone has suggestions, I would love to hear them! Thanks, Matt --0000000000004e663405d88d9a68 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I'm trying to figure out what causes high latency= while typing in large org-mode files.=C2=A0 The issue is very clearly a re= sult of my large config file, but I'm not sure how to track it down wit= h any precision.=C2=A0

My main literate confi= g file is ~/.emacs.d/emacs-init.org, = currently 15000 lines, 260 src blocks.=C2=A0
If I create a ~= minimal.el~ config like this:

(let* ((all-paths=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 '("/home/matt/src/org-mode/ema= cs/site-lisp/org")))
=C2=A0 =C2=A0 (dolist (p all-paths)
=C2=A0 = =C2=A0 =C2=A0 (add-to-list 'load-path p)))

=C2=A0 (require '= org)
=C2=A0 (find-file "~/.emacs.d/emacs-init.org")

then I do not notice a= ny latency while typing.=C2=A0 If I run the profiler while using the minima= l config, the profile looks about like this at a high level:

=C2=A0 =C2=A0 =C2=A0 =C2=A0 1397 =C2=A071% - command-execut= e
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0740 =C2=A037% =C2=A0- funcall-intera= ctively
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0718 =C2=A036% =C2=A0 - org-sel= f-insert-command
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0686 =C2=A034% =C2=A0 = =C2=A0+ org-element--cache-after-change
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 10 =C2=A0 0% =C2=A0 =C2=A0+ org-fold-core--fix-folded-region
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A03 =C2=A0 0% =C2=A0 =C2=A0+ blink-paren-po= st-self-insert-function
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A02 =C2= =A0 0% =C2=A0 =C2=A0+ jit-lock-after-change
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A01 =C2=A0 0% =C2=A0 =C2=A0 =C2=A0org-fold-check-before-invisibl= e-edit--text-properties
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A09 =C2= =A0 0% =C2=A0 + previous-line
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A06= =C2=A0 0% =C2=A0 + minibuffer-complete
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A03 =C2=A0 0% =C2=A0 + org-return
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A03 =C2=A0 0% =C2=A0 + execute-extended-command
=C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0657 =C2=A033% =C2=A0- byte-code
=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0657 =C2=A033% =C2=A0 - read-extended-command
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 64 =C2=A0 3% =C2=A0 =C2=A0- completing-read-defaul= t
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 14 =C2=A0 0% =C2=A0 =C2=A0 + redisp= lay_internal (C function)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A01 =C2= =A0 0% =C2=A0 =C2=A0 + timer-event-handler
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0371 =C2=A018% - redisplay_internal (C function)
=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0251 =C2=A012% =C2=A0+ jit-lock-function
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 90 =C2=A0 4% =C2=A0+ assq
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A07 =C2=A0 0% =C2=A0+ substitute-command-keys
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A03 =C2=A0 0% =C2=A0+ eval
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0125 =C2=A0 6% + timer-event-handler
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 69 =C2=A0 3% + ...

----------= ----------------
However, if I instead use my fairly extensive ma= in config, latency is high enough that there's a noticeable delay while= typing ordinary words. I see this=C2=A0 regardless of whether I build from= main or from Ihor's org-fold feature branch on github. The profiler ov= erview here is pretty different -- redisplay_internal takes a much higher p= ercentage of the CPU requirement:

=C2=A0 =C2=A0 = =C2=A0 =C2=A0=C2=A0 3170 =C2=A056% - redisplay_internal (C function)
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0693 =C2=A012% =C2=A0- substitute-command-key= s
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0417 =C2=A0 7% =C2=A0 + #<compiled= -0x1c8b98a4b03336f3>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 59 =C2=A0 1%= =C2=A0+ assq
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 49 =C2=A0 0% =C2=A0+ or= g-in-subtree-not-table-p
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 36 =C2=A0 0%= =C2=A0+ tab-bar-make-keymap
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 35 =C2= =A0 0% =C2=A0 =C2=A0and
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 24 =C2=A0 0% = =C2=A0+ not
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 16 =C2=A0 0% =C2=A0 =C2= =A0org-at-table-p
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 13 =C2=A0 0% =C2=A0= + jit-lock-function
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A08 =C2=A0 0%= =C2=A0 =C2=A0keymap-canonicalize
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A07 =C2=A0 0% =C2=A0+ #<compiled 0x74a551771c7fdf1>
=C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A04 =C2=A0 0% =C2=A0+ funcall
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A04 =C2=A0 0% =C2=A0 =C2=A0display-graphic-p
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A03 =C2=A0 0% =C2=A0+ #<compiled = 0xe5940664f7881ee>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A03 =C2=A0 = 0% =C2=A0 =C2=A0file-readable-p
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A03 =C2=A0 0% =C2=A0+ table--probe-cell
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A03 =C2=A0 0% =C2=A0 =C2=A0table--row-column-insertion-point-p
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 1486 =C2=A026% - command-execute
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 1200 =C2=A021% =C2=A0- byte-code
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 1200 =C2=A021% =C2=A0 - read-extended-command
=C2=A0 =C2=A0 =C2= =A0 =C2=A0 1200 =C2=A021% =C2=A0 =C2=A0- completing-read-default
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 1200 =C2=A021% =C2=A0 =C2=A0 - apply
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 1200 =C2=A021% =C2=A0 =C2=A0 =C2=A0- vertico--advice
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0475 =C2=A0 8% =C2=A0 =C2=A0 =C2=A0 + #<su= br completing-read-default>

---------------= -------
I've almost never used the profiler and am not quite = sure how I should proceed to debug this.=C2=A0 I realize I can comment out = parts of the config one at a time, but that is not so easy for me to do in = my current setup, and I suppose there are likely to be multiple contributin= g causes, which I may not really notice except in the aggregate.=C2=A0
=

If anyone has suggestions, I would love to hear t= hem!

Thanks,

Matt
--0000000000004e663405d88d9a68--