Re: Dramatic slowdown in org mode

[Nick Dokos <nicholas.dokos@hp.com>]

Bastien <bzg@altern.org> wrote:

> Hi Eric,
> 
> "Fraga, Eric" <e.fraga@ucl.ac.uk> writes:
> 
> > Unfortunately, I haven't had the chance to go through the changes over the
> > past week to see if I could identify which one causes the dramatic change
> > in performance.
> 
> My guess is commit 980e522 -- can you confirm this?
> 


I doubt it - this commit is not in the range that Eric identified (indirectly
in the filenames, so I'm guessing):

,----
| $ git log --oneline 47ea1666..76cf2538 
| 76cf253 Merge branch 'maint'
| 7739e8b Small refactoring
| 0f3264f org-agenda.el: Use `org-todo-keyword-faces' to highlight selected TODO keywords.
| 0db5f28 org-html.el and org-e-html.el: Add CSS classes to list tags.
| bbb451a Merge branch 'maint'
| 374efdd org.el (org-in-fixed-width-region-p): Rewrite using org-element.el.
| 4996e5b Merge branch 'maint'
| 9d4dd69 org.el (org-fill-paragraph): Fill correctly in source code block.
| 1af0b39 Allow C-S-<up/down> to sync update clock timestamps by several units.
| 709bf92 Backwards compatibility for `user-error´
| 676180a org-e-texinfo.el: Fix link export to account for alternate TOC names
| 83b5679 org-export: Fix docstring
| 442b2ad Merge branch 'maint'
| 20839ce org-element: Fix parsing of a list in a block in a list
| 4588244 Revert "tangled files should not be writable"
| 457c27d org-e-confluence: Add asynchronous export support
| fa0e8fe org-test.el (org-test-with-temp-text-in-file): Wrap ,@body into (progn ...).
| c8452bf Merge branch 'maint'
| c0675b4 test-org-src.el: Fix a test.
| 8f3b8ee testing/org-test.el: Fix `org-test-with-temp-text'.
| e77f26e contrib/lisp/org-e-confluence.el: New file by Sébastien Delafond
| 20940bb contrib/lisp/org-bullets.el: New file by Evgeni Sabof
| db5881f Merge branch 'master' of orgmode.org:org-mode
| beea46f Merge branch 'maint'
| 0b44eb6 Add doclicense.texi to include the GNU FDL license and fix the copyright notice.
| e556812 org-export: Fix docstring
| 68d4de2 org-koma-letter: Add asynchronous export support
| a223d1a * contrib/lisp/org-e-texinfo.el: Remove messages used for debugging (org-e-texinfo--generate-menu-items): Remove messages used for debugging.
| 2dbd84e Merge branch 'maint'
| 6f5b4a8 orgguide.texi: Fix the copyright notice.
| 6af6e95 Merge branch 'maint'
| cd24962 org.texi: Update copyright notice.
| db51b80 Merge branch 'maint'
| df09917 org.el (org-edit-special): Fix bug about editing special environments.
| 81d5ebc org-src.el (org-edit-src-code): Check if we are in a source code block with `org-in-src-block-p'.
| a7afe7d org.el (org-in-src-block-p): Return t when on #+BEGIN|END_SRC lines.
| b866204 Merge branch 'maint'
| c4bf1ee Remove gplmanual.pl as we are not allowed to dual-license the Org manual.
| 6202df5 Merge branch 'master' of orgmode.org:org-mode
| 2c164be Merge branch 'maint'
| 4524a02 Edit source: fix 'end' initialisation
| 22e4a4e Revert "org-src.el (org-edit-src-code): Fix bug on Emacs <24.1."
| 286fd7f org-e-beamer/html: Asynchronous export ignores `org-export-show-temporary-export-buffer'
| 26d2490 Merge branch 'maint'
| 0584dff ob-exp.el (obe-marker): Delete useless var.
| ac1b117 Merge branch 'maint'
| 70b0f42 Fix copyright notices for test files.
| a0cb4a3 add tests for org-src-edit
| f72bb57 Merge branch 'master' of orgmode.org:org-mode
| 22ee29b Merge branch 'maint'
| 7a770c1 org-src.el (org-edit-src-code): Fix bug on Emacs <24.1.
| 06872a4 export back-ends: Activate asynchronous export
| 5319cec org-e-publish: Activate asynchronous publishing
| ffb630b org-export: Add asynchronous process wrapper for export
| 5c1eab5 * contrib/lisp/org-e-texinfo.el: Provided export of info: links
| 31d24aa Merge branch 'maint'
| 82db0aa org.texi: Fix typos.
| 8c04fb7 org-exp.el (org-export-copy-to-kill-ring): Add a new choice 'if-interactive and use it as the default.
| 627d573 org-e-html: Small refactoring
| 2c39a4a export back-ends: Apply line break changes
| 6e37a4d org-element: Fix line break parsing
| 8dd3fa3 Use the org-mime-default-header customized variable
| 2c34715 option to byte-compile elisp loaded from Org files
| 5e3b825 tangled files should not be writable
| ff00818 requiring ob now pulls in all of Babel
| 0694965 renaming lisp/ob.el to lisp/ob-core.el
| ba59eeb ob-python: specify defcustom parameters
| 01bf706 org-babel: fix bug introduced in commit 78cdf149
| cf9838f org.el: Display the blocking heading when a TODO state change is blocked.
| 1bfea39 Support `C-1' prefix for `org-agenda-capture' and `org-capture'.
| 275f07c Merge branch 'maint'
| 488eea6 org-latex.el: Escape raw path when exporting links to files.
`----

a7afe7d was my initial guess because of the profiles:

org-in-src-block-p                                            813         0.0134103450  1.649...e-05

org-in-src-block-p                                            813         28.441751922  0.0349837047

That's a whopping difference for such a simple function.

Abdo Roig-Maranges apparently did a bisect and came to the same conclusion.

A simple profiling exercise with the two versions of the function indicates a big difference but
as Abdo indicated as well, not as big as Eric finds:

--8<---------------cut here---------------start------------->8---
(defun org-in-src-block1-p (&optional inside)
  "Whether point is in a code source block.
When INSIDE is non-nil, don't consider we are within a src block
when point is at #+BEGIN_SRC or #+END_SRC."
  (let (ov)
    (or (when (setq ov (overlays-at (point)))
	  (memq 'org-block-background
		(overlay-properties
		 (car ov))))
	(and (not inside)
	     (save-match-data
	       (save-excursion
		 (move-beginning-of-line 1)
		 (looking-at ".*#\\+\\(BEGIN\\|END\\)_SRC")))))))

(defun org-in-src-block2-p ()
  "Whether point is in a code source block."
  (let (ov)
    (when (setq ov (overlays-at (point)))
	  (memq 'org-block-background
		(overlay-properties
		 (car ov))))))

(defun my-test1 ()
  (let ((n 1000))
    (while (> n 0)
      (org-in-src-block1-p)
      (setq n (1- n)))))

(defun my-test2 ()
  (let ((n 1000))
    (while (> n 0)
      (org-in-src-block2-p)
      (setq n (1- n)))))
--8<---------------cut here---------------end--------------->8---

When executed on a begin_src line, my-test1 is 5x as slow as my-test2
in my setup. Otherwise the factor is about 2.

But Eric gets a factor of 2500 which is a bit of a mystery to me.
There might be additional factors entering in his case.

It's clear however that the regexp slows things down substantially.
BTW, do we really need the regexp to match anything before the
#+BEGIN_SRC, or just white space? If the latter is ok, it should speed
up the regexp matching.

Nick