My org-agenda is incredibly slow. Please help me understand what is happening


My computer is not so slow. It is a dual core

 Intel(R) Pentium(R) D CPU 2.80GHz
with 2GB of RAM

My organizer.org has 21000 lines

I can split it in a couple of files if this is the solution.

My org-version is 6.21b

GNU Emacs 23.1.50.1 (i486-pc-linux-gnu, GTK+ Version 2.12.9)
 of 2009-08-01 on radon, modified by Debian

Daniel

-----

org-agenda                                                    1           29.408703     29.408703
org-agenda-list                                               1           28.648282000  28.648282000
org-agenda-get-day-entries                                    27          26.459491999  0.9799811851
org-get-tags-at                                               737         21.298774999  0.0288992876
org-up-heading-all                                            2950        21.033886999  0.0071301311
org-agenda-get-scheduled                                      24          9.637174      0.4015489166
org-finalize-agenda                                           1           7.912788      7.912788
org-agenda-to-appt                                            1           7.68922       7.68922
org-agenda-get-todos                                          3           7.542505      2.5141683333
org-agenda-get-deadlines                                      24          6.2434250000  0.2601427083
org-outline-level                                             200310      3.8892260000  1.941...e-05
org-agenda-get-timestamps                                     24          2.4559390000  0.1023307916
org-end-of-subtree                                            3021        1.9833440000  0.0006565190
org-back-to-heading                                           5187        1.0270179999  0.0001979984
org-prepare-agenda-buffers                                    2           0.7970789999  0.3985394999
org-format-agenda-item                                        676         0.7946449999  0.0011755103
org-agenda-get-restriction-and-command                        1           0.760327      0.760327
org-finalize-agenda-entries                                   8           0.641847      0.080230875
org-prepare-agenda                                            1           0.471942      0.471942
org-agenda-highlight-todo                                     510         0.4001490000  0.0007846058
org-get-effort                                                665         0.3708070000  0.0005576045
org-entry-get                                                 665         0.3672690000  0.0005522842
org-get-property-block                                        665         0.3582529999  0.0005387263
org-agenda-get-blocks                                         24          0.3424489999  0.0142687083
org-agenda-get-sexps                                          21          0.2189269999  0.0104250952
org-parse-time-string                                         2599        0.2118380000  8.150...e-05
org-on-heading-p                                              502         0.1997950000  0.0003979980
org-agenda-skip                                               5744        0.1968189999  3.426...e-05
org-get-priority                                              663         0.1852529999  0.0002794162
org-get-todo-state                                            764         0.1823689999  0.0002387028
org-time-string-to-time                                       144         0.1796119999  0.0012473055
org-activate-bracket-links                                    26          0.1460520000  0.0056173846
org-time-string-to-absolute                                   2455        0.097197      3.959...e-05
org-at-date-range-p                                           488         0.0821609999  0.0001683627
org-agenda-align-tags                                         1           0.073629      0.073629
org-get-entries-from-diary                                    7           0.069128      0.0098754285
org-refresh-category-properties                               6           0.052706      0.0087843333
org-fit-window-to-buffer                                      2           0.050455      0.0252275
org-fit-agenda-window                                         1           0.049025      0.049025
org-add-props                                                 1920        0.0480150000  2.500...e-05
org-entries-lessp                                             2348        0.0298160000  1.269...e-05
org-closest-date                                              519         0.0278379999  5.363...e-05
org-get-agenda-file-buffer                                    33          0.0172350000  0.0005222727
org-find-base-buffer-visiting                                 33          0.0170039999  0.0005152727
org-agenda-add-inherited-tags                                 676         0.0158009999  2.337...e-05
org-date-to-gregorian                                         1039        0.0139500000  1.342...e-05
org-split-string                                              2001        0.0123330000  6.163...e-06
org-agenda-new-marker                                         1096        0.0122370000  1.116...e-05
org-diary-sexp-entry                                          413         0.0116180000  2.813...e-05
org-downcase-keep-props                                       2734        0.0074770000  2.734...e-06
org-add-prop-inherited                                        2606        0.0053880000  2.067...e-06
org-calendar-holiday                                          7           0.005129      0.0007327142
org-get-time-of-day                                           154         0.0036909999  2.396...e-05
org-fontify-priorities                                        1           0.003512      0.003512
org-remove-uniherited-tags                                    2541        0.0032780000  1.290...e-06
org-trim                                                      578         0.0032199999  5.570...e-06
org-check-agenda-file                                         30          0.0029500000  9.833...e-05
org-get-wdays                                                 688         0.0029190000  4.242...e-06
org-get-category                                              766         0.0018540000  2.420...e-06
org-agenda-mode                                               1           0.001529      0.001529
org-agenda-add-time-grid-maybe                                7           0.001149      0.0001641428
org-agenda-todayp                                             48          0.0011        2.291...e-05
org-before-change-function                                    530         0.0008880000  1.675...e-06
org-get-todo-face                                             443         0.0008620000  1.945...e-06
org-agenda-files                                              4           0.0007669999  0.0001917499
org-modify-diary-entry-string                                 11          0.0006640000  6.036...e-05
org-agenda-cleanup-fancy-diary                                7           0.000655      9.357...e-05
org-diary-default-entry                                       7           0.0006129999  8.757...e-05
org-agenda-check-for-timestamp-as-reason-to-ignore-todo-item  342         0.0005920000  1.730...e-06
org-add-to-diary-list                                         7           0.000554      7.914...e-05
org-agenda-deadline-face                                      113         0.0005210000  4.610...e-06
org-switch-to-buffer-other-window                             2           0.000457      0.0002285
org-agenda-format-date-aligned                                7           0.000405      5.785...e-05
org-agenda-reset-markers                                      1           0.000398      0.000398
org-uniquify                                                  6           0.0003730000  6.216...e-05
org-days-to-iso-week                                          9           0.0002069999  2.299...e-05
org-overlay-put                                               84          0.0001279999  1.523...e-06
org-make-overlay                                              42          0.0001240000  2.952...e-06
org-compile-prefix-format                                     1           6.2e-05       6.2e-05
org-delete-overlay                                            42          5.599...e-05  1.333...e-06
org-overlay-get                                               43          5.199...e-05  1.209...e-06
org-agenda-set-mode-name                                      1           1.6e-05       1.6e-05
org-add-hook                                                  2           1.400...e-05  7.000...e-06
org-file-menu-entry                                           3           1.3e-05       4.333...e-06
org-overlays-in                                               1           1.3e-05       1.3e-05
org-unhighlight                                               1           4e-06         4e-06
org-agenda-post-command-hook                                  1           3e-06         3e-06
org-region-active-p                                           1           3e-06         3e-06
org-agenda-ndays-to-span                                      2           3e-06         1.5e-06
org-set-sorting-strategy                                      1           2e-06         2e-06
org-release-buffers                                           1           2e-06         2e-06
org-font-lock-add-tag-faces                                   1           2e-06         2e-06


2009/4/30 Carsten Dominik <carsten.dominik@gmail.com>

On Apr 28, 2009, at 12:33 PM, Rainer Stengele wrote:

Carsten Dominik schrieb:
I have just pushed one more optimization which may improve things a bit
for you, Rainer.

- Carsten
...


Hi Carsten,

results see here:


before your last optimsation:

org-run-agenda-series                                         1           1.797         1.797

after your optimisation (see below):

org-run-agenda-series                                         1           1.703         1.703
org-agenda                                                    1           1.703         1.703
org-let2                                                      2           1.3900000000  0.6950000000
org-agenda-list                                               1           1.2810000000  1.2810000000
org-agenda-get-day-entries                                    12          1.202         0.1001666666
org-agenda-get-scheduled                                      10          0.7969999999  0.0797
org-end-of-subtree                                            1533        0.4400000000  0.0002870189
org-time-string-to-absolute                                   1635        0.2360000000  0.0001443425
org-prepare-agenda                                            3           0.2189999999  0.073
org-prepare-agenda-buffers                                    1           0.2029999999  0.2029999999
org-parse-time-string                                         1805        0.188         0.0001041551
org-agenda-get-timestamps                                     10          0.14          0.0140000000
org-todo-list                                                 1           0.109         0.109
org-agenda-get-deadlines                                      10          0.094         0.0094
org-closest-date                                              240         0.094         0.0003916666
org-back-to-heading                                           2206        0.094         4.26...e-005
org-finalize-agenda                                           3           0.094         0.0313333333
org-let                                                       1           0.094         0.094
org-outline-level                                             1533        0.094         6.13...e-005
org-agenda-get-todos                                          2           0.093         0.0465
org-date-to-gregorian                                         480         0.078         0.0001625
org-agenda-fontify-priorities                                 1           0.078         0.078
org-get-entries-from-diary                                    5           0.063         0.0126
org-agenda-skip                                               3220        0.063         1.95...e-005
org-agenda-skip-if                                            167         0.063         0.0003772455
org-agenda-skip-entry-if                                      167         0.063         0.0003772455
org-refresh-category-properties                               2           0.0470000000  0.0235000000
org-get-todo-state                                            335         0.047         0.0001402985
org-agenda-get-blocks                                         10          0.046         0.0046
org-agenda-files                                              4           0.032         0.008
org-get-effort                                                72          0.031         0.0004305555
org-entry-get                                                 72          0.031         0.0004305555
org-format-agenda-item                                        74          0.031         0.0004189189
org-at-date-range-p                                           219         0.031         0.0001415525
org-get-wdays                                                 150         0.016         0.0001066666
org-entries-lessp                                             219         0.016         7.30...e-005
org-agenda-get-sexps                                          10          0.016         0.0016
org-agenda-align-tags                                         1           0.016         0.016
org-finalize-agenda-entries                                   5           0.016         0.0032
org-check-agenda-file                                         14          0.015         0.0010714285
org-get-property-block                                        72          0.015         0.0002083333



So it looks like I have come from 1.797 down to 1.703.
And by the way - the times are even now varying between
org-run-agenda-series                                         1           1.563         1.563
and little over 2 seconds.


Will I see much improvement when archiving a lot of subtrees inside the org file?

I don't think so, archiving to a separate file is the best way to keep things fast.

- Carsten




Thanks so much for this special "Rainer" optimisation ;)
I also dream of always carrying around a small netbook running mainly emacs with org-mode ...


Rainer


On Apr 24, 2009, at 3:37 PM, Rainer Stengele wrote:

Hi Carsten,

strange enough the latest git version does not improve time for my
agenda at all.
It still is as slow or fast - however you see it.

org-version is 6.26trans

,----
| org-run-agenda-series
1           1.797         1.797
| org-agenda
1           1.797         1.797
| org-let2
2           1.422         0.711
| org-agenda-get-day-entries
12          1.248         0.104
| org-agenda-list
1           1.2349999999  1.2349999999
| org-agenda-get-scheduled
10          0.7979999999  0.0798
| org-end-of-subtree
1533        0.6430000000  0.0004194390
| org-prepare-agenda
3           0.359         0.1196666666
| org-prepare-agenda-buffers
1           0.25          0.25
| org-back-to-heading
2198        0.1880000000  8.55...e-005
| org-todo-list
1           0.187         0.187
| org-agenda-get-todos
2           0.171         0.0855
| org-refresh-category-properties
2           0.141         0.0705
| org-get-entries-from-diary
5           0.126         0.0252
| org-agenda-get-deadlines
10          0.107         0.0107
| org-agenda-get-timestamps
10          0.078         0.0078
| org-outline-level
5376        0.064         1.19...e-005
| org-agenda-skip-if
167         0.061         0.0003652694
| org-agenda-skip-entry-if
167         0.061         0.0003652694
| org-agenda-get-blocks
10          0.046         0.0046
| org-check-agenda-file
14          0.032         0.0022857142
| org-get-todo-state
333         0.031         9.30...e-005
| org-finalize-agenda
3           0.016         0.0053333333
| org-time-string-to-absolute
1627        0.016         9.83...e-006
| org-agenda-get-sexps
10          0.016         0.0016
| org-at-date-range-p
217         0.016         7.37...e-005
| org-agenda-align-tags
1           0.016         0.016
| org-let
1           0.016         0.016
| org-agenda-files
4           0.015         0.00375
| org-agenda-skip
3213        0.015         4.66...e-006
| org-on-heading-p
164         0.015         9.14...e-005
| org-agenda-set-mode-name
1           0.0           0.0
| org-date-to-gregorian
474         0.0           0.0
| org-delete-overlay
65          0.0           0.0
| org-agenda-reset-markers
1           0.0           0.0
| org-find-base-buffer-visiting
14          0.0           0.0
| org-set-sorting-strategy
2           0.0           0.0
| org-get-effort
69          0.0           0.0
| org-downcase-keep-props
41          0.0           0.0
| org-get-agenda-file-buffer
14          0.0           0.0
| org-fit-agenda-window
1           0.0           0.0
| org-agenda-check-for-timestamp-as-reason-to-ignore-todo-item
189         0.0           0.0
| org-entry-get
69          0.0           0.0
| org-agenda-post-command-hook
2           0.0           0.0
| org-overlay-get
241         0.0           0.0
| org-agenda-cleanup-fancy-diary
5           0.0           0.0
| org-get-wdays
150         0.0           0.0
| org-parse-time-string
1797        0.0           0.0
| org-entries-lessp
215         0.0           0.0
| org-trim
29          0.0           0.0
| org-closest-date
237         0.0           0.0
| org-agenda-add-inherited-tags
72          0.0           0.0
| org-modify-diary-entry-string
5           0.0           0.0
| org-make-overlay
65          0.0           0.0
| org-file-menu-entry
2           0.0           0.0
| org-add-to-diary-list
5           0.0           0.0
| org-split-string
39          0.0           0.0
| org-activate-bracket-links
1           0.0           0.0
| org-add-props
186         0.0           0.0
| org-agenda-add-time-grid-maybe
3           0.0           0.0
| org-get-category
69          0.0           0.0
| org-days-to-iso-week
3           0.0           0.0
| org-agenda-mode
1           0.0           0.0
| org-uniquify
4           0.0           0.0
| org-agenda-new-marker
107         0.0           0.0
| org-agenda-todayp
20          0.0           0.0
| org-format-agenda-item
72          0.0           0.0
| org-overlays-in
1           0.0           0.0
| org-get-property-block
69          0.0           0.0
| org-fontify-priorities
1           0.0           0.0
| org-get-tags-at
69          0.0           0.0
| org-overlay-put
130         0.0           0.0
| org-get-todo-face
68          0.0           0.0
| org-compile-prefix-format
2           0.0           0.0
| org-agenda-highlight-todo
72          0.0           0.0
| org-agenda-ndays-to-span
1           0.0           0.0
| org-agenda-deadline-face
4           0.0           0.0
| org-add-hook
2           0.0           0.0
| org-time-string-to-time
170         0.0           0.0
| org-agenda-fontify-priorities
1           0.0           0.0
| org-agenda-format-date-aligned
3           0.0           0.0
| org-diary-default-entry
5           0.0           0.0
| org-font-lock-add-tag-faces
1           0.0           0.0
| org-get-priority
69          0.0           0.0
| org-remove-uniherited-tags
69          0.0           0.0
| org-get-time-of-day
37          0.0           0.0
| org-finalize-agenda-entries
4           0.0           0.0
| org-unhighlight
2           0.0           0.0
| org-before-change-function
199         0.0           0.0
`----

my org file is about 15000 lines large.


Rainer


Carsten Dominik schrieb:

On Apr 24, 2009, at 10:24 AM, Eric S Fraga wrote:

Carsten Dominik writes:
On Apr 24, 2009, at 12:10 AM, Eric S Fraga wrote:
Which part can you not reproduce?

When I have the agenda on an arbitrary date, pressing "j" does *not*
first switch the agenda to today.

<blush>

Okay, the problem was mine: I had included a very useful method,
originally from Tassilo (cf. th-calendar-open-agenda), which would
have the agenda view updated when the calendar view changed.  Removing
this gives the behaviour I was now looking for, and which is indeed
the default behaviour.  Apologies for the noise.

</blush>

Actually, this experience is a testament to the insidious nature (in a
positive way) of org-mode: it is taking over my life!  I used to have
the Emacs diary + calendar as the focus of my attention for task and
time management and org-mode was initially an addition to this.
org-mode is now almost completely the focus and I really need to clean
up my customisations to remove the old vestiges of how I used to work.

Again, apologies and continued thanks for an excellent system which I
could no longer live without!

I still would like to to see profiling information as I described in my
message from yesterday, so see if specific operations are particularly
slow on netbooks or a zaurus.

- Carsten




_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode





_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode





_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode