another example of org being slow, with some analysis

[Eric S Fraga <e.fraga@ucl.ac.uk>]

Hello,

there have been a few threads recently mentioning poor
performance.  Although some of these have been resolved (e.g. the use of
=linum=), I wonder if I could add a data point.

I have a medium (for me) sized (385 kB) org file with all of my "tasks"
in a date-tree structure.  One of the tasks requires me to edit a small
table every day for a couple of weeks (see below for the table).  Asking
for a recalculation of the table (C-u C-c C-c) takes 17 seconds on a new
laptop (16 GB RAM, SSD, i7 cpu, Debian linux testing) which is quite
fast at most other tasks.

The output of the ELP profiler is here:

--8<---------------cut here---------------start------------->8---
org-ctrl-c-ctrl-c                   1    17.655796272  17.655796272
org-table-recalculate               1    17.652738791  17.652738791
org-table-get-range                 14   13.166332242  0.940452303
org-table-eval-formula              19   13.101183716  0.6895359851
org-goto-line                       104  10.761145733  0.1034725551
org-table-copy-region               13   7.362366204   0.5663358618
org-current-line                    66   6.8422078910  0.1036698165
org-table-get-descriptor-line       28   1.659497877   0.0592677813
org-table-align                     1    0.415633428   0.415633428
org-table-get-specials              1    0.206756703   0.206756703
org-table-expand-lhs-ranges         1    0.103551707   0.103551707
org-element--parse-to               6    0.0031655370  0.0005275895
org-babel-execute-safely-maybe      1    0.002872891   0.002872891
org-babel-execute-maybe             1    0.002870729   0.002870729
org-babel-execute-src-block-maybe   1    0.002846968   0.002846968
org-element-at-point                3    0.0028012790  0.0009337596
org-babel-where-is-src-block-head   1    0.002664822   0.002664822
org-element--cache-compare          303  0.0025883040  8.542...e-06
org-table-current-column            119  0.002235665   1.878...e-05
--8<---------------cut here---------------end--------------->8---

The table, for completeness, is here:

#+begin_src org
  | date             | TBD | Victoria | London Bridge | Blackfriars | City | Farrindgon | St Pancras | Totals | Miles |
  |                  | 2.5 |        5 |           4.8 |         3.2 |  2.9 |          5 |        1.6 |        |       |
  |------------------+-----+----------+---------------+-------------+------+------------+------------+--------+-------|
  | [2015-06-08 Mon] |   2 |          |               |             |    1 |            |          1 |    9.5 |   5.9 |
  | [2015-06-12 Fri] |   2 |          |               |           1 |      |            |          1 |    9.8 |   6.0 |
  | [2015-06-15 Mon] |   2 |          |             1 |             |      |            |          1 |   11.4 |   7.0 |
  | [2015-06-17 Wed] |   2 |          |               |           1 |      |            |          1 |    9.8 |   6.0 |
  | [2015-06-18 Thu] |   1 |          |               |           1 |      |            |            |    5.7 |   3.5 |
  |------------------+-----+----------+---------------+-------------+------+------------+------------+--------+-------|
  | Totals           |   9 |        0 |             1 |           3 |    1 |          0 |          4 |   46.2 |  28.5 |
  ,#+TBLFM: @>$2..@>$>>=vsum(@I..@II)::$9=($2..$>>>)*(@<<$2..@<<$>>>);EN::$10=$9/1.62;%.1f
#+end_src

If I change, for instance, the entry in the TBD column for today from 1
to 2, in the original file, it takes 17 seconds.  If I do it in this
email buffer (changing mode to org etc.), it is essentially
instantaneous.

Large buffers seem to be causing problems with org.  I've noticed this
elsewhere as I do tend to work with org files that are often 0.25-1 MB
in size.  Having looked at the profiler output, my gut feeling was that
the buffer is being repeatedly parsed (or some similar activity) given
where the time is being spent.  So... as the table is currently 99% of
the way into the buffer, I have copied the table to the start of the
same large document and, lo and behold, the table recalculation on the
copy is instantaneous!

(the moral of the story, for me, may be to use a reverse date-tree
structure for my tasks, if such were possible... maybe it is, given the
hidden gems in org! ;-)

I would be happy to instrument more than just org, if it would help.  I
should add that the above profile was done using emacs -Q so no
customisations.

I have no real issue with the performance of org overall but I thought
I'd add a data point in case it helps.

thanks,
eric

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 24.4.1, Org release_8.3beta-1216-gb856f6