From mboxrd@z Thu Jan 1 00:00:00 1970 From: Matt Lundin Subject: org-element checks make flyspell prohibitively slow Date: Mon, 17 Mar 2014 09:32:17 -0500 Message-ID: <87zjkolwum.fsf@fastmail.fm> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:57282) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WPYaY-0000w3-Td for emacs-orgmode@gnu.org; Mon, 17 Mar 2014 10:32:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WPYaU-0008WO-Bh for emacs-orgmode@gnu.org; Mon, 17 Mar 2014 10:32:26 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:48824) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WPYaU-0008W0-3y for emacs-orgmode@gnu.org; Mon, 17 Mar 2014 10:32:22 -0400 Received: from archdesk (unknown [209.147.100.248]) by mail.messagingengine.com (Postfix) with ESMTPA id C6938C0000C for ; Mon, 17 Mar 2014 10:32:17 -0400 (EDT) List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Emacs-orgmode list --=-=-= Content-Type: text/plain The rewrite of org-mode-flyspell-verify in commit 4a27c2b4b67201e0b23f431bdaeb6460b31e1394 (Nov 21, 2013) makes navigating org-mode files with large chunks of text very slow. For instance, I started up a minimal emacs: /usr/bin/emacs -Q -l ~/config/minimal.el ...where minimal.el is... --=-=-= Content-Type: application/emacs-lisp Content-Disposition: inline; filename=minimal.el Content-Transfer-Encoding: quoted-printable (add-to-list 'load-path "~/org-mode/lisp") (add-hook 'text-mode-hook (lambda () (flyspell-mode 1))) --=-=-= Content-Type: text/plain and (insert "\n\n=> " (emacs-version)) => GNU Emacs 24.3.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.10.7) of 2014-01-28 on var-lib-archbuild-extra-x86_64-juergen and (insert "\n\n=> " (org-version nil t)) => Org-mode version 8.2.5h (release_8.2.5h-757-gc444e4 @ /home/matt/org-mode/lisp/) I open a test.org file containing the following. --8<---------------cut here---------------start------------->8--- * A headline * Arch packages * Another headline --8<---------------cut here---------------end--------------->8--- After opening a line under "Arch Packages" I call... C-u M-! pacman -Ss [RET] (Of course, this only works with archlinux.) This inserts a long list of packages that look like this: --8<---------------cut here---------------start------------->8--- core/acl 2.2.52-2 [installed] Access control list utilities, libraries and headers core/archlinux-keyring 20140220-1 [installed] Arch Linux PGP keyring core/attr 2.4.47-1 [installed] Extended attribute support library for ACL support core/autoconf 2.69-1 (base-devel) [installed] A GNU tool for automatically configuring source code core/automake 1.14.1-1 (base-devel) [installed] A GNU tool for automatically creating Makefiles --8<---------------cut here---------------end--------------->8--- All in all, it's 12680 lines.... I navigate to the bottom of the file. I type... M-x elp-instrument-package [RET] org [RET] M-x elp-instrument-package [RET] flyspell [RET] M-x elp-instrument-function [RET] scroll-down-command [RET] Then I hit M-v three times. This takes a while. Here are the top elp offenders: --8<---------------cut here---------------start------------->8--- flyspell-post-command-hook 6 10.753828775 1.7923047959 flyspell-word 6 10.752069764 1.7920116273 org-mode-flyspell-verify 5 8.973166134 1.7946332267 org-element-context 5 8.364142505 1.6728285010 org-element--get-next-object-candidates 699 8.307898326 0.0118854053 org-element-latex-or-entity-successor 5 3.7592736849 0.7518547369 org-element-link-successor 40 1.1079495280 0.0276987382 org-element-sub/superscript-successor 659 1.0986591029 0.0016671610 org-element-line-break-successor 5 0.9729438699 0.194588774 org-element-at-point 5 0.607910786 0.1215821572 org-element--parse-to 5 0.606992172 0.1213984344 org-element--current-element 10 0.4201667370 0.0420166737 org-element-paragraph-parser 10 0.416739094 0.0416739094 org-element-inline-src-block-successor 5 0.3740871620 0.0748174324 org-element-text-markup-successor 10 0.309006309 0.0309006308 org-element-timestamp-successor 5 0.087275674 0.0174551348 org-element-statistics-cookie-successor 5 0.086838821 0.0173677642 org-element-footnote-reference-successor 5 0.0866179840 0.0173235968 org-element-target-successor 5 0.086057234 0.0172114468 org-element-radio-target-successor 5 0.083322691 0.0166645382 org-element-export-snippet-successor 5 0.083078665 0.016615733 org-element-macro-successor 5 0.0828692849 0.0165738569 scroll-down-command 3 0.059660938 0.0198869793 --8<---------------cut here---------------end--------------->8--- Interestingly, after calling elp-results, just trying to navigate to the org buffer with other-window takes some time. Here's the top of the new elp list: --8<---------------cut here---------------start------------->8--- flyspell-post-command-hook 1 1.780324266 1.780324266 flyspell-word 1 1.780091208 1.780091208 org-mode-flyspell-verify 1 1.779600437 1.779600437 org-element-context 1 1.6563819400 1.6563819400 org-element--get-next-object-candidates 137 1.6448783359 0.0120064112 org-element-latex-or-entity-successor 1 0.753972365 0.753972365 org-element-link-successor 8 0.2225488189 0.0278186023 org-element-sub/superscript-successor 129 0.206221951 0.0015986197 org-element-line-break-successor 1 0.19533769 0.19533769 org-element-at-point 1 0.12299361 0.12299361 org-element--parse-to 1 0.122834024 0.122834024 org-element--current-element 2 0.085711871 0.0428559355 org-element-paragraph-parser 2 0.084995934 0.042497967 org-element-inline-src-block-successor 1 0.074795629 0.074795629 org-element-text-markup-successor 2 0.0633956 0.0316978 org-element-export-snippet-successor 1 0.020310027 0.020310027 org-element-timestamp-successor 1 0.017533021 0.017533021 org-element-footnote-reference-successor 1 0.017371898 0.017371898 org-element-statistics-cookie-successor 1 0.017308972 0.017308972 org-element-radio-target-successor 1 0.016797529 0.016797529 org-element-macro-successor 1 0.016773084 0.016773084 org-element-target-successor 1 0.016545404 0.016545404 org-element-subscript-parser 128 0.0039024930 3.048...e-05 org-element-inline-babel-call-successor 1 0.000752604 0.000752604 org-element-link-parser 7 0.000386358 5.5194e-05 flyspell-get-word 1 0.000357585 0.000357585 --8<---------------cut here---------------end--------------->8--- My apologies for the Arch specific example. The indentation of the output seems to cause particular problems. I'd be happy to provide a full test file off-list if required. But this works (more or less) with other very large chunks of text. E.g., C-u M-! w3m -dump http://www.gnu.org/software/emacs/manual/html_mono/emacs.html Is it possible to speed up org-element-context here? For something called as often as org-mode-flyspell-verify, do we need all the overhead of the org-element parser? Or would a hack optimized for speed (which is what the older version of org-mode-flyspell-verify represented) be enough? I recall (though my memory may be faulty) discussions on the list quite some time back in which we decided to prioritize speed/efficiency over thoroughness/completeness in the checks run by org-mode-flyspell-verify. Thanks, Matt --=-=-=--