Why navigating in Org mode is so slow in overview mode?

Why navigating in Org mode is so slow in overview mode?

[Q]

Hello everyone,

I am view this file
http://doc.norang.ca/org-mode.org

It's a prettey long document, when it all folded up, navigating to previous
line and next line takes about 2 seconds. When the sub heading is shown,
it's more responsive. Is this a bug?

Thank you!

Re: Why navigating in Org mode is so slow in overview mode?

[Q]

It slows everything, even change buffer takes about 2 seconds. And all this
slowness disappear when the buffer is unfolded.

Re: Why navigating in Org mode is so slow in overview mode?

[Nicolas Goaziou]

Hello,

Q <godblessfq@gmail.com> writes:

> I am view this file
> http://doc.norang.ca/org-mode.org
>
> It's a prettey long document, when it all folded up, navigating to previous
> line and next line takes about 2 seconds. When the sub heading is shown,
> it's more responsive. Is this a bug?

I don't know. What Org version are you using? Could you use a profiler
and report results?


Regards,

-- 
Nicolas Goaziou

Re: Why navigating in Org mode is so slow in overview mode?

[Gregor Zattler]

Hi Nicolas,
* Nicolas Goaziou <mail@nicolasgoaziou.fr> [17. Jun. 2015]:
> Hello,
> 
> Q <godblessfq@gmail.com> writes:
> 
>> I am view this file
>> http://doc.norang.ca/org-mode.org
>>
>> It's a prettey long document, when it all folded up, navigating to previous
>> line and next line takes about 2 seconds. When the sub heading is shown,
>> it's more responsive. Is this a bug?
> 
> I don't know. What Org version are you using? Could you use a profiler
> and report results?

I thought this is a known problem which gets worse with the
amount of hidden text.  The workaround is to use a
„#+STARTUP: showeverything“ directive.  I use this in order to
make a longish org file with many clocking lines in drawers
usable.

It would be great if this could be fixed.

I would like to help but need help myself on how to produce
profiler reports.  My org mode version is

8.3beta (release_8.3beta-1228-gd1f9aa @ /home/grfz/src/org-mode/lisp/)

but I have this problem since months or years.

Regards, Gregor

Re: Why navigating in Org mode is so slow in overview mode?

[Andreas Leha]

Hi Gregor,

Gregor Zattler <telegraph@gmx.net> writes:
> Hi Nicolas,
> * Nicolas Goaziou <mail@nicolasgoaziou.fr> [17. Jun. 2015]:
>> Hello,
>> 
>> Q <godblessfq@gmail.com> writes:
>> 
>>> I am view this file
>>> http://doc.norang.ca/org-mode.org
>>>
>>> It's a prettey long document, when it all folded up, navigating to previous
>>> line and next line takes about 2 seconds. When the sub heading is shown,
>>> it's more responsive. Is this a bug?
>> 
>> I don't know. What Org version are you using? Could you use a profiler
>> and report results?
>
> I thought this is a known problem which gets worse with the
> amount of hidden text.  The workaround is to use a
> „#+STARTUP: showeverything“ directive.  I use this in order to
> make a longish org file with many clocking lines in drawers
> usable.
>
> It would be great if this could be fixed.
>
> I would like to help but need help myself on how to produce
> profiler reports.  My org mode version is
>
> 8.3beta (release_8.3beta-1228-gd1f9aa @ /home/grfz/src/org-mode/lisp/)
>
> but I have this problem since months or years.
>

Shot in the dark: Do have linum-mode enabled?  That is always my first
candidate when I experience slowdowns.

Best,
Andreas

Re: Why navigating in Org mode is so slow in overview mode?

[Nicolas Goaziou]

Gregor Zattler <telegraph@gmx.net> writes:

> I would like to help but need help myself on how to produce
> profiler reports.

M-x profiler-start

Do something slow

M-x profiler-report

Regards,

Re: Why navigating in Org mode is so slow in overview mode?

[Gregor Zattler]

Hi Andreas,
* Andreas Leha <andreas.leha@med.uni-goettingen.de> [17. Jun. 2015]:
> Shot in the dark: Do have linum-mode enabled?  That is always my first
> candidate when I experience slowdowns.

No I don’t use linum-mode.  Thanks, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

Re: Why navigating in Org mode is so slow in overview mode?

[Qiang Fang]

[-- Attachment #1: Type: text/plain, Size: 616 bytes --]

I found it linum mode that slows all things down.
Thank you for the help!
Qiang

On Wed, Jun 17, 2015 at 3:51 AM, Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:

> Hello,
>
> Q <godblessfq@gmail.com> writes:
>
> > I am view this file
> > http://doc.norang.ca/org-mode.org
> >
> > It's a prettey long document, when it all folded up, navigating to
> previous
> > line and next line takes about 2 seconds. When the sub heading is shown,
> > it's more responsive. Is this a bug?
>
> I don't know. What Org version are you using? Could you use a profiler
> and report results?
>
>
> Regards,
>
> --
> Nicolas Goaziou
>

[-- Attachment #2: Type: text/html, Size: 1220 bytes --]

Re: Why navigating in Org mode is so slow in overview mode?

[Gregor Zattler]

Hi Nicolas,
* Nicolas Goaziou <mail@nicolasgoaziou.fr> [17. Jun. 2015]:
> Gregor Zattler <telegraph@gmx.net> writes:
>> I would like to help but need help myself on how to produce
>> profiler reports.
> 
> M-x profiler-start
> 
> Do something slow
> 
> M-x profiler-report

Thanks, this seemed rather easy and I open the org file in
question in overview now:

- most of the time it’s not slow! That’s great and a big
  difference to months ago, when it was slow very often.

- but there some hangs now and then.  At that moment I cannot
  start the profiler.  If I start the profiler at the
  beginning of using this org file and stop it maybe hours later
  after something was slow: Would such report be helpful? 


Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

Re: Why navigating in Org mode is so slow in overview mode?

[Nicolas Goaziou]

Gregor Zattler <telegraph@gmx.net> writes:

> - but there some hangs now and then.

Is it a complete freeze (i.e., you need to use C-g to get out of the
loop) or a slowdown of a couple of seconds? 

The former is a bug. However, some changes in the buffer imply a massive
loss of the cache (e.g., inserting a block clears cache in the whole
section), so the next action can be slow since you need to fill it
again, hence the slowdown.

> At that moment I cannot
>   start the profiler.  If I start the profiler at the
>   beginning of using this org file and stop it maybe hours later
>   after something was slow: Would such report be helpful?

It might not give any useful information. However, the best way to know
is to look at it.

Regards,

Re: Why navigating in Org mode is so slow in overview mode?

[Gregor Zattler]

Hi Nicolas,
* Nicolas Goaziou <mail@nicolasgoaziou.fr> [21. Jun. 2015]:
> Gregor Zattler <telegraph@gmx.net> writes:
>> - but there some hangs now and then.
> 
> Is it a complete freeze (i.e., you need to use C-g to get out of the
> loop) or a slowdown of a couple of seconds? 

It’s a slow down.

> The former is a bug. However, some changes in the buffer imply a massive
> loss of the cache (e.g., inserting a block clears cache in the whole
> section), so the next action can be slow since you need to fill it
> again, hence the slowdown.

Hmh.  Last time I was annoyed was when I wanted the cursor to
move down a few lines in the agenda: pressing „arrow down“ took
two seconds or some such before the cursor actually moved.

>> At that moment I cannot
>>   start the profiler.  If I start the profiler at the
>>   beginning of using this org file and stop it maybe hours later
>>   after something was slow: Would such report be helpful?
> 
> It might not give any useful information. However, the best way to know
> is to look at it.

OK I’ll start with profiler then.  Thanks, Gregor

Re: Why navigating in Org mode is so slow in overview mode?

[Sebastien Vauban]

Gregor Zattler <telegraph-hi6Y0CQ0nG0@public.gmane.org> writes:
> * Nicolas Goaziou <mail-Gpy5sJQTEQHwkn9pgDnJRVAUjnlXr6A1@public.gmane.org> [17. Jun. 2015]:
>> Gregor Zattler <telegraph-hi6Y0CQ0nG0@public.gmane.org> writes:
>>> I would like to help but need help myself on how to produce
>>> profiler reports.
>> 
>> M-x profiler-start
>> 
>> Do something slow
>> 
>> M-x profiler-report
>
> Thanks, this seemed rather easy and I open the org file in
> question in overview now:
>
> - most of the time it’s not slow! That’s great and a big
>   difference to months ago, when it was slow very often.
>
> - but there some hangs now and then.  At that moment I cannot
>   start the profiler.  If I start the profiler at the
>   beginning of using this org file and stop it maybe hours later
>   after something was slow: Would such report be helpful? 

FWIW, I've always used:

--8<---------------cut here---------------start------------->8---
  ;; Do not switch to OVERVIEW at startup.
  (setq org-startup-folded nil)
--8<---------------cut here---------------end--------------->8---

and I don't see any particular slowdown for my Org files:

--8<---------------cut here---------------start------------->8---
(Info) Found file ~/Public/Repositories/org-mode/lisp/../doc/library-of-babel.org in 1.28 s
(Info) Found file ~/org/personal/travaux-listing.org in 2.32 s
(Info) Found file ~/org/personal/travaux.org in 0.28 s
(Info) Found file ~/org/personal/conso.txt in 1.71 s
(Info) Found file ~/org/notes/Notes-on-Lisp.txt in 0.61 s
(Info) Found file ~/org/personal/Personal.org in 0.97 s
--8<---------------cut here---------------end--------------->8---

PS- Code to get the above timings:

--8<---------------cut here---------------start------------->8---
  (defadvice find-file (around my/find-file activate)
    "Open the file named FILENAME and report time spent."
    (let ((filename (ad-get-arg 0))
          (find-file-time-start (float-time)))
      (message "(Info) Finding file %s..." filename)
      ad-do-it
      (message "(Info) Found file %s in %.2f s" filename
               (- (float-time) find-file-time-start))))
--8<---------------cut here---------------end--------------->8---

Best regards,
  Seb

-- 
Sebastien Vauban

Re: Why navigating in Org mode is so slow in overview mode?

[Gregor Zattler]

[-- Attachment #1: Type: text/plain, Size: 1159 bytes --]

Hi Nicolas,
* Nicolas Goaziou <mail@nicolasgoaziou.fr> [21. Jun. 2015]:

[Profiler Reports]

> It might not give any useful information. However, the best way to know
> is to look at it.

The first profiler report is after 105 minutes of Emacs uptime.
I had captured infos via org-protocol, refiled them to another
file and then wanted to navigate (arrow keys) to the TODO item in
order to mark it done.  It took seconds till the cursor moved.
Sadly I saved only the memory profiler report.

Then I wrote an reply to your message and while examining the
situation, there was another slow down, when I wanted to switch
buffers via helm, this did nothing for a few secs.  This are the
second reports, only minutes later.

Then I typed ahead and there was a slow down while typing.
System began to swap, emnacs did not respond, I killed Emacs,
firefox, was not able to do a swapoff -a because of memory
restrictions, rebooted the computer, started Emacs with org-mode,
tried to go to the last clocked-in item: and there was a slow
down after say 2 minutes (third profiler reports).

These reports are attachet.

HTH, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

[-- Attachment #2: profiler-report-cpu-2.txt.xz --]
[-- Type: application/x-xz, Size: 6468 bytes --]

[-- Attachment #3: profiler-report-cpu-3.txt.xz --]
[-- Type: application/x-xz, Size: 5476 bytes --]

[-- Attachment #4: profiler-report-mem-3.txt.xz --]
[-- Type: application/x-xz, Size: 15916 bytes --]

[-- Attachment #5: profiler-report.txt.xz --]
[-- Type: application/x-xz, Size: 43536 bytes --]

[-- Attachment #6: profiler-report-mem-2.txt.xz --]
[-- Type: application/x-xz, Size: 10708 bytes --]

Re: Why navigating in Org mode is so slow in overview mode?

[Nicolas Goaziou]

Gregor Zattler <telegraph@gmx.net> writes:

> These reports are attachet.

Thank you. However, I don't see anything incriminating Org in these
reports.

Regards,

Re: Why navigating in Org mode is so slow in overview mode?

[Stefan-W. Hahn]

Mail von Q, Wed, 17 Jun 2015 at 00:51:02 +0000:

Hello,

according to emacs bug#16803 I'm using
toggle-cache-long-scans in my org-files.

Best (first line in org-file):

-*- eva: (toggle-cache-long-scans); -*-

Greetings
Stefan

-- 
Stefan-W. Hahn                          It is easy to make things.
                                        It is hard to make things simple.

Re: Why navigating in Org mode is so slow in overview mode?

[Gregor Zattler]

[-- Attachment #1: Type: text/plain, Size: 1360 bytes --]

Hi Nicolas,
* Nicolas Goaziou <mail@nicolasgoaziou.fr> [23. Jun. 2015]:
> Gregor Zattler <telegraph@gmx.net> writes:
> 
>> These reports are attachet.
> 
> Thank you. However, I don't see anything incriminating Org in these
> reports.

Now I think I found something specific.  I do open the org-mode
file in question with #+STARTUP: showeverything because otherwise
it’s slow.  Today I discovered org-goto.  This presents the
buffer in overview mode.  So I started the profiler, started
org-goto and typed 11 letters in order to jump to a heading.
This takes quite some time while my notebooks fan is making noise
and Emacs uses 100% cpu according to htop.

I did this two times with the same org-mode file:

- with Org-mode version 8.3beta (release_8.3beta-1291-gd6e601 @
  /home/grfz/src/org-mode/lisp/) on GNU Emacs 25.0.50.4
  (i686-pc-linux-gnu, GTK+ Version 3.14.5) of 2015-07-17 on boo

- with Org-mode version 8.2.10 (release_8.2.10 @
  /usr/local/stow/emacs-snapshot/share/Emacs/25.0.50/lisp/org/)
  on the same version of Emacs.

I do not know how to read these reports, but to me it seems, that
most of the cpu cycles are used by org-context, org-end-of-item,
org-list-struct and org-list-context.  The memory reports do not
seem to be of interest but I attach them for the sake of
completeness.

HTH, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

[-- Attachment #2: org-goto-profiler-reports.tar.gz --]
[-- Type: application/gzip, Size: 30415 bytes --]

Re: Why navigating in Org mode is so slow in overview mode?

[Nicolas Goaziou]

Gregor Zattler <telegraph@gmx.net> writes:

> Now I think I found something specific.  I do open the org-mode
> file in question with #+STARTUP: showeverything because otherwise
> it’s slow.  Today I discovered org-goto.  This presents the
> buffer in overview mode.  So I started the profiler, started
> org-goto and typed 11 letters in order to jump to a heading.
> This takes quite some time while my notebooks fan is making noise
> and Emacs uses 100% cpu according to htop.
>
> I did this two times with the same org-mode file:
>
> - with Org-mode version 8.3beta (release_8.3beta-1291-gd6e601 @
>   /home/grfz/src/org-mode/lisp/) on GNU Emacs 25.0.50.4
>   (i686-pc-linux-gnu, GTK+ Version 3.14.5) of 2015-07-17 on boo
>
> - with Org-mode version 8.2.10 (release_8.2.10 @
>   /usr/local/stow/emacs-snapshot/share/Emacs/25.0.50/lisp/org/)
>   on the same version of Emacs.
>
> I do not know how to read these reports, but to me it seems, that
> most of the cpu cycles are used by org-context, org-end-of-item,
> org-list-struct and org-list-context.  The memory reports do not
> seem to be of interest but I attach them for the sake of
> completeness.

I removed a call to `org-context' in
985420eceb3661aae8b1caafc95e959ab12ad1cb. Does it fix your problem?

For the record, `org-context' predates Org Elements and should
ultimately be removed from the code base.

Regards,

Re: Why navigating in Org mode is so slow in overview mode?

[Gregor Zattler]

Hi Nicolas,

thanks for this change it makes a tremendous difference.

* Nicolas Goaziou <mail@nicolasgoaziou.fr> [20. Jul. 2015]:
> Gregor Zattler <telegraph@gmx.net> writes:
> 
>> Now I think I found something specific.  I do open the org-mode
>> file in question with #+STARTUP: showeverything because otherwise
>> it’s slow.  Today I discovered org-goto.  This presents the
>> buffer in overview mode.  So I started the profiler, started
>> org-goto and typed 11 letters in order to jump to a heading.
>> This takes quite some time while my notebooks fan is making noise
>> and Emacs uses 100% cpu according to htop.
>>
>> I did this two times with the same org-mode file:
>>
>> - with Org-mode version 8.3beta (release_8.3beta-1291-gd6e601 @
>>   /home/grfz/src/org-mode/lisp/) on GNU Emacs 25.0.50.4
>>   (i686-pc-linux-gnu, GTK+ Version 3.14.5) of 2015-07-17 on boo
>>
>> - with Org-mode version 8.2.10 (release_8.2.10 @
>>   /usr/local/stow/emacs-snapshot/share/Emacs/25.0.50/lisp/org/)
>>   on the same version of Emacs.
>>
>> I do not know how to read these reports, but to me it seems, that
>> most of the cpu cycles are used by org-context, org-end-of-item,
>> org-list-struct and org-list-context.  The memory reports do not
>> seem to be of interest but I attach them for the sake of
>> completeness.
> 
> I removed a call to `org-context' in
> 985420eceb3661aae8b1caafc95e959ab12ad1cb. Does it fix your problem?

Yes, it’s *much* faster now, it’s so responsive it’s almost
instantaneous.  Thanks.

> For the record, `org-context' predates Org Elements and should
> ultimately be removed from the code base.


Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-