emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* Performance issues after upgrading from Emacs 23.3 to 24.3+prelude
@ 2014-01-06 16:54 Karl Voit
  2014-01-07  9:19 ` Bastien
  0 siblings, 1 reply; 8+ messages in thread
From: Karl Voit @ 2014-01-06 16:54 UTC (permalink / raw)
  To: emacs-orgmode

Hi!

I have got very severe performance issues after I upgraded my Emacs.

Before upgrading:
    - Debian stable x64
    - Emacs 23.3 (Debian stable deb-package)
    - Org-mode from git: version 8.2.3c (release_8.2.3c-225-g668ba5)
    - no prelude

Now:
    - Debian stable x64
      - no change but I replaced Debian Emacs packages with
        Emacs-snapshot and MELPA/Marmelade
    - Emacs-snapshot 24.3.50.1 from [1]
    - Org-mode from git: version 8.2.3c (release_8.2.3c-225-g668ba5)
      - no change
    - Emacs prelude from [2]


I mainly use Emacs for Org-mode and therefore I got issues only in
Org-mode. Some tasks are very slow and take 100% of a CPU core. This
is the rare occasion where I am happy that Emacs is single-threaded
:-)

Further down, there are three examples with their profiling results.
I don't know how to interpret the profiling results properly. 


Most annoying: sometimes, Emacs starts using up its CPU core at 100%
without any particular reason or event I am aware of. Probably it is
while auto-saving (not every time!) but I am not sure. 

I have no idea on how to debug this behavior because when Emacs is
blocking completely, it is not even updating its buffer display nor
reacting on C-g or similar. However, I am able to kill (with default
kill signal) the Emacs process without leaving unsaved buffers.


I guess that with Emacs prelude, I got some functionality which is
causing these issues. So this might be of interest of other
Emacs/prelude users as well.


* Example 1: moving an item in a list up/down

I just switch two lines of a simple list by invoking M-<up>:
- foo
- bar
- baz (is moved one line up)

This took only a fraction of a second before. Now I ave to wait
approx. 2-3 seconds.

I learned how to use the Emacs profiler and generated a CPU-report
when invoking the command: [3]

Interestingly, when I am using M-x org-move-item-up, it does not
show the slow-down effect.


* Example 2: M-<up> on a heading containing 20000 lines of
appointments and notes

This task took almost two minutes. Before the update, this was
probably done in a couple of seconds.

Here is the CPU-report of the profiler: [4]


* Example 3: Re-calculating and formatting of a 10x10 table

The table has 10x10 cells and four very simple formulas. The
operation took approx. 1-3s before the update. Now, it takes over
four minutes.

The command invoked was: C-u C-u C-c C-c

CPU-profiling report: [5]


  1. http://emacs.naquadah.org/
  2. git://github.com/bbatsov/prelude.git
  3. http://pastebin.com/mQRDYbq7 -> M-<up> on a list item
  4. http://pastebin.com/P2sJPsHy -> moving large heading
  5. http://pastebin.com/3Kw6knYU -> re-calculating table
-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Performance issues after upgrading from Emacs 23.3 to 24.3+prelude
  2014-01-06 16:54 Performance issues after upgrading from Emacs 23.3 to 24.3+prelude Karl Voit
@ 2014-01-07  9:19 ` Bastien
  2014-01-07 13:50   ` Karl Voit
  2014-01-09 19:17   ` Karl Voit
  0 siblings, 2 replies; 8+ messages in thread
From: Bastien @ 2014-01-07  9:19 UTC (permalink / raw)
  To: Karl Voit; +Cc: news1142, emacs-orgmode

Karl Voit <devnull@Karl-Voit.at> writes:

> I guess that with Emacs prelude, I got some functionality which is
> causing these issues. So this might be of interest of other
> Emacs/prelude users as well.

Yes, probably, since the Org version is the same :)
Better ping the Emacs prelude devs directly IMHO.

-- 
 Bastien

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Performance issues after upgrading from Emacs 23.3 to 24.3+prelude
  2014-01-07  9:19 ` Bastien
@ 2014-01-07 13:50   ` Karl Voit
  2014-01-09 16:24     ` Eric S Fraga
  2014-01-09 19:17   ` Karl Voit
  1 sibling, 1 reply; 8+ messages in thread
From: Karl Voit @ 2014-01-07 13:50 UTC (permalink / raw)
  To: emacs-orgmode

* Bastien <bzg@gnu.org> wrote:
> Karl Voit <devnull@Karl-Voit.at> writes:
>
>> I guess that with Emacs prelude, I got some functionality which is
>> causing these issues. So this might be of interest of other
>> Emacs/prelude users as well.
>
> Yes, probably, since the Org version is the same :)
> Better ping the Emacs prelude devs directly IMHO.

OK, I'll file a bug on GitHub. Thanks!

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Performance issues after upgrading from Emacs 23.3 to 24.3+prelude
  2014-01-07 13:50   ` Karl Voit
@ 2014-01-09 16:24     ` Eric S Fraga
  0 siblings, 0 replies; 8+ messages in thread
From: Eric S Fraga @ 2014-01-09 16:24 UTC (permalink / raw)
  To: Karl Voit; +Cc: news1142, emacs-orgmode

Hi,

Just to add that I have experienced some severe performance hits in a
recent snapshot, particularly noticeable when yanking from the X
clipboard.  I haven't tracked it down yet nor looked on the emacs dev
groups and lists.

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 24.3.1, Org release_8.2.4-322-gece429

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Performance issues after upgrading from Emacs 23.3 to 24.3+prelude
  2014-01-07  9:19 ` Bastien
  2014-01-07 13:50   ` Karl Voit
@ 2014-01-09 19:17   ` Karl Voit
  2014-01-09 19:39     ` Nicolas Goaziou
  1 sibling, 1 reply; 8+ messages in thread
From: Karl Voit @ 2014-01-09 19:17 UTC (permalink / raw)
  To: emacs-orgmode

* Bastien <bzg@gnu.org> wrote:
> Karl Voit <devnull@Karl-Voit.at> writes:
>
>> I guess that with Emacs prelude, I got some functionality which is
>> causing these issues. So this might be of interest of other
>> Emacs/prelude users as well.
>
> Yes, probably, since the Org version is the same :)
> Better ping the Emacs prelude devs directly IMHO.

Today, I did another test. I converted my config to Emacs24 with the
most current Org-mode from git and without prelude at all.

Unfortunately, the performance issue stayed :-(

IMHO, the profiler reports showed a common pattern: a reasonable
amount of CPU got into line-number-at-pos if I read the profiler
report correctly. (see below)

I also turned off all visible minor modes from the mode line. No
change.

Since prelude is out of this game, what can I check to find the
source of this poor performance?


profiler report (CPU) of ~M-<up>~ of a simple list item without prelude:
#+BEGIN_EXAMPLE
- command-execute                                                1280  98%
 - call-interactively                                            1280  98%
  - org-metadown                                                 1004  77%
   - run-hook-with-args-until-success                            1004  77%
    - org-babel-pop-to-session-maybe                             1004  77%
     - org-babel-get-inline-src-block-matches                    1000  76%
        line-number-at-pos                                       1000  76%
  - byte-code                                                     265  20%
   - read-extended-command                                        265  20%
    - completing-read                                             265  20%
     - completing-read-default                                    265  20%
      - read-from-minibuffer                                      253  19%
       + command-execute                                           43   3%
       + redisplay_internal (C function)                            3   0%
  + execute-extended-command                                       11   0%
+ ...                                                              15   1%
+ redisplay_internal (C function)                                   4   0%
#+END_EXAMPLE

profiler-report (CPU) of ~C-c C-c~ in a simple table without prelude:
#+BEGIN_EXAMPLE
 command-execute                                               17492  94%
 - call-interactively                                           17492  94%
  - org-ctrl-c-ctrl-c                                           16137  86%
   - org-table-align                                             5844  31%
    + org-indent-refresh-maybe                                      8   0%
      org-table-goto-column                                         4   0%
   - run-hook-with-args-until-success                            4469  24%
    - org-babel-execute-safely-maybe                             4469  24%
     - org-babel-execute-maybe                                   4469  24%
      - org-babel-execute-src-block-maybe                        4469  24%
       - org-babel-get-inline-src-block-matches                  4449  23%
          line-number-at-pos                                     4449  23%
         org-babel-where-is-src-block-head                         16   0%
  + byte-code                                                    1311   7%
  + minibuffer-complete                                            33   0%
  + execute-extended-command                                        7   0%
+ timer-event-handler                                            1076   5%
+ ...                                                              24   0%
+ redisplay_internal (C function)                                  14   0%
#+END_EXAMPLE

profiler-report (CPU) of ~M-<down>~ of a heading containing
approx. 20000 lines of appointments and notes without prelude:
#+BEGIN_EXAMPLE
- command-execute                                              123676  83%
 - call-interactively                                          123676  83%
  - org-metaup                                                 115952  77%
   - call-interactively                                         87337  58%
    - org-move-subtree-up                                       87337  58%
     - org-move-subtree-down                                    87225  58%
      - insert-before-markers                                   86598  58%
       - org-indent-refresh-maybe                               86594  58%
        - org-indent-add-properties                             82086  55%
         - org-at-item-p                                        72884  48%
          - org-list-in-valid-context-p                         69497  46%
           - org-in-block-p                                     69493  46%
            - byte-code                                         69421  46%
             - mapc                                             63094  42%
              - #<compiled 0xf1d27b>                            63022  42%
               - org-between-regexps-p                          62878  42%
                - org-at-regexp-p                               32772  22%
                   byte-code                                    32716  21%
                  byte-code                                        16   0%
               outline-previous-heading                          3188   2%
               outline-next-heading                              2751   1%
           org-list-item-body-column                              691   0%
      + org-clean-visibility-after-subtree-move                   587   0%
      + outline-end-of-subtree                                     12   0%
      + org-get-last-sibling                                       12   0%
      + hide-subtree                                                8   0%
      + org-first-sibling-p                                         4   0%
   - run-hook-with-args-until-success                           28615  19%
    - org-babel-load-in-session-maybe                           28615  19%
     - org-babel-get-inline-src-block-matches                   28611  19%
        line-number-at-pos                                      28603  19%
      + thing-at-point                                              4   0%
       org-babel-where-is-src-block-head                            4   0%
  + byte-code                                                    7423   4%
  + list                                                          147   0%
  + previous-line                                                  44   0%
  + minibuffer-complete                                            33   0%
  + next-line                                                      32   0%
  + execute-extended-command                                       11   0%
  + org-return                                                      8   0%
  + org-yank                                                        8   0%
  + kill-ring-save                                                  6   0%
  + org-cycle                                                       6   0%
    scroll-up-command                                               3   0%
  + org-self-insert-command                                         3   0%
- timer-event-handler                                           24395  16%
 - byte-code                                                    24388  16%
  - apply                                                       24388  16%
   - org-indent-initialize-agent                                24338  16%
    - org-indent-initialize-buffer                              24335  16%
     - byte-code                                                24335  16%
      - org-indent-add-properties                               18006  12%
       - org-at-item-p                                           8487   5%
        - org-list-in-valid-context-p                            6184   4%
         - org-in-block-p                                        6180   4%
          - byte-code                                            6076   4%
           - mapc                                                4666   3%
            - #<compiled 0xf1d27b>                               4630   3%
             - org-between-regexps-p                             4546   3%
                org-at-regexp-p                                    40   0%
                byte-code                                           8   0%
             outline-next-heading                                 741   0%
             outline-previous-heading                             585   0%
         org-get-indentation                                     1326   0%
         org-list-item-body-column                                387   0%
         org-inlinetask-in-task-p                                  87   0%
       + byte-code                                                 85   0%
   + org-element--cache-sync                                       32   0%
     appt-check                                                     4   0%
   + blink-cursor-start                                             4   0%
   timer-activate-when-idle                                         3   0%
+ ...                                                             574   0%
+ redisplay_internal (C function)                                  83   0%
+ xselect-convert-to-string                                        11   0%
  internal-timer-start-idle                                         8   0%
#+END_EXAMPLE

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Performance issues after upgrading from Emacs 23.3 to 24.3+prelude
  2014-01-09 19:17   ` Karl Voit
@ 2014-01-09 19:39     ` Nicolas Goaziou
  2014-01-09 20:24       ` Performance issues after upgrading from Emacs 23.3 to 24.3 Karl Voit
  0 siblings, 1 reply; 8+ messages in thread
From: Nicolas Goaziou @ 2014-01-09 19:39 UTC (permalink / raw)
  To: news1142; +Cc: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 297 bytes --]

Hello,

Karl Voit <devnull@Karl-Voit.at> writes:

> IMHO, the profiler reports showed a common pattern: a reasonable
> amount of CPU got into line-number-at-pos if I read the profiler
> report correctly. (see below)

Does the following patch improve the situation?


Regards,

-- 
Nicolas Goaziou

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-core-Speed-improvement.patch --]
[-- Type: text/x-diff, Size: 960 bytes --]

From 54b8e7466d4689be3c34f7041244563771e2178a Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou <n.goaziou@gmail.com>
Date: Thu, 9 Jan 2014 20:36:23 +0100
Subject: [PATCH] ob-core: Speed improvement

* lisp/ob-core.el (org-babel-get-inline-src-block-matches): Do not
  compute line number if all is needed is to know if we're on the
  first one.
---
 lisp/ob-core.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index f06cdaf..e8943c6 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -217,7 +217,7 @@ Returns non-nil if match-data set"
   (let ((src-at-0-p (save-excursion
 		      (beginning-of-line 1)
 		      (string= "src" (thing-at-point 'word))))
-	(first-line-p (= 1 (line-number-at-pos)))
+	(first-line-p (= (line-beginning-position) (point-min)))
 	(orig (point)))
     (let ((search-for (cond ((and src-at-0-p first-line-p  "src_"))
 			    (first-line-p "[[:punct:] \t]src_")
-- 
1.8.5.2


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: Performance issues after upgrading from Emacs 23.3 to 24.3
  2014-01-09 19:39     ` Nicolas Goaziou
@ 2014-01-09 20:24       ` Karl Voit
  2014-01-09 21:10         ` Nicolas Goaziou
  0 siblings, 1 reply; 8+ messages in thread
From: Karl Voit @ 2014-01-09 20:24 UTC (permalink / raw)
  To: emacs-orgmode

Hi Nicolas!

* Nicolas Goaziou <n.goaziou@gmail.com> wrote:
>
> Hello,
>
> Karl Voit <devnull@Karl-Voit.at> writes:
>
>> IMHO, the profiler reports showed a common pattern: a reasonable
>> amount of CPU got into line-number-at-pos if I read the profiler
>> report correctly. (see below)
>
> Does the following patch improve the situation?

Yes!

M-<up>/<down> of list item is fast again.

Re-calculate table is fast again.

Wohoo! :-) Thanks!


However, M-<up>/<down> of a big heading is still slow (see profile
below). Probably, I am able to find other examples of slow behavior
on the weekend, where I am using the Linux-machine that has the
issue.

profiler-report (CPU) of ~M-<down>~ of a heading containing
approx. 20000 lines of appointments and notes without prelude and
with patch:
#+BEGIN_EXAMPLE
- command-execute                                               84760  70%
 - call-interactively                                           84760  70%
  - org-metaup                                                  83960  69%
   - call-interactively                                         83960  69%
    - org-move-subtree-up                                       83960  69%
     - org-move-subtree-down                                    83924  69%
      - insert-before-markers                                   83308  69%
       - org-indent-refresh-maybe                               83300  69%
        - org-indent-add-properties                             82729  68%
         - org-at-item-p                                        73452  61%
          - org-list-in-valid-context-p                         69849  58%
           - org-in-block-p                                     69841  58%
            - byte-code                                         69781  58%
             - mapc                                             63655  53%
              - #<compiled 0x8ce3c3>                            63595  52%
               - org-between-regexps-p                          63467  52%
                - org-at-regexp-p                               32900  27%
                   byte-code                                    32817  27%
                  byte-code                                        12   0%
               outline-previous-heading                          2885   2%
               outline-next-heading                              2785   2%
           org-list-item-body-column                              857   0%
           org-get-indentation                                      8   0%
         org-element--cache-before-change                           4   0%
         jit-lock-after-change                                      4   0%
      + org-clean-visibility-after-subtree-move                   572   0%
      + outline-end-of-subtree                                     12   0%
      + hide-subtree                                               12   0%
  + byte-code                                                     784   0%
  + minibuffer-complete                                             8   0%
  + execute-extended-command                                        8   0%
- timer-event-handler                                           34748  28%
 - byte-code                                                    34748  28%
  - apply                                                       34748  28%
   - org-indent-initialize-agent                                34748  28%
    - org-indent-initialize-buffer                              34748  28%
     - byte-code                                                34748  28%
      - org-indent-add-properties                               30627  25%
       - org-at-item-p                                          22699  18%
        - org-list-in-valid-context-p                           20136  16%
         - org-in-block-p                                       20136  16%
          - byte-code                                           20040  16%
           - mapc                                               17820  14%
            - #<compiled 0x8ce3c3>                              17772  14%
             - org-between-regexps-p                            17705  14%
              - org-at-regexp-p                                  9070   7%
                 byte-code                                       9040   7%
                byte-code                                           4   0%
             outline-previous-heading                            1031   0%
             outline-next-heading                                 969   0%
         org-get-indentation                                      524   0%
         org-list-item-body-column                                476   0%
         org-inlinetask-in-task-p                                  23   0%
       + byte-code                                                 12   0%
+ ...                                                             475   0%
+ redisplay_internal (C function)                                  12   0%
#+END_EXAMPLE

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Performance issues after upgrading from Emacs 23.3 to 24.3
  2014-01-09 20:24       ` Performance issues after upgrading from Emacs 23.3 to 24.3 Karl Voit
@ 2014-01-09 21:10         ` Nicolas Goaziou
  0 siblings, 0 replies; 8+ messages in thread
From: Nicolas Goaziou @ 2014-01-09 21:10 UTC (permalink / raw)
  To: news1142; +Cc: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 741 bytes --]

Karl Voit <devnull@Karl-Voit.at> writes:

> M-<up>/<down> of list item is fast again.
>
> Re-calculate table is fast again.
>
> Wohoo! :-) Thanks!

Applied. Thank you for the report.

> However, M-<up>/<down> of a big heading is still slow (see profile
> below). Probably, I am able to find other examples of slow behavior
> on the weekend, where I am using the Linux-machine that has the
> issue.

This operation is bound to be slow anyway. On top of it, you use
`org-indent-mode', which can be costly on large buffer changes.

Is it really worse than in 23.3?

Also, would you mind testing the following patch on top of master?
I don't expect much out of it since you're moving a behemoth, but it may
help.


Regards,

-- 
Nicolas Goaziou

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Try-something-out.patch --]
[-- Type: text/x-diff, Size: 1211 bytes --]

From 6681efd30c34e86ede597b5b9ec219c9358fc7eb Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou <n.goaziou@gmail.com>
Date: Thu, 9 Jan 2014 21:59:59 +0100
Subject: [PATCH] Try something out

---
 lisp/org-list.el | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/lisp/org-list.el b/lisp/org-list.el
index c0cf77e..f85c19b 100644
--- a/lisp/org-list.el
+++ b/lisp/org-list.el
@@ -134,6 +134,8 @@
 (declare-function org-export-string-as "ox"
 		  (string backend &optional body-only ext-plist))
 
+(declare-function org-element-type "org-element" (element))
+
 
 
 \f
@@ -489,10 +491,13 @@ group 4: description tag")
 	       (t (forward-line -1)))))))))))
 
 (defun org-at-item-p ()
-  "Is point in a line starting a hand-formatted item?"
+  "Non-nil when point is in a line starting a list item."
   (save-excursion
     (beginning-of-line)
-    (and (looking-at (org-item-re)) (org-list-in-valid-context-p))))
+    (and (looking-at (org-item-re))
+	 (or (not (derived-mode-p 'org-mode))
+	     (memq (org-element-type (save-match-data (org-element-at-point)))
+		   '(item plain-list))))))
 
 (defun org-at-item-bullet-p ()
   "Is point at the bullet of a plain list item?"
-- 
1.8.5.2


^ permalink raw reply related	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2014-01-09 21:10 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-01-06 16:54 Performance issues after upgrading from Emacs 23.3 to 24.3+prelude Karl Voit
2014-01-07  9:19 ` Bastien
2014-01-07 13:50   ` Karl Voit
2014-01-09 16:24     ` Eric S Fraga
2014-01-09 19:17   ` Karl Voit
2014-01-09 19:39     ` Nicolas Goaziou
2014-01-09 20:24       ` Performance issues after upgrading from Emacs 23.3 to 24.3 Karl Voit
2014-01-09 21:10         ` Nicolas Goaziou

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).