Very slow performance in Org-mode on 10k line file?

Very slow performance in Org-mode on 10k line file?

[John Hendy]

Greetings,


I just started experiencing major lag in Org-mode on my main work
notes file, which is at about 10k lines. Is that getting up to the
point where files get unwieldy? In googling around, I found a few
suggestions:

- Fiddle with linum settings
http://stackoverflow.com/questions/5229705/emacs-org-mode-turn-off-line-numbers

I set linum-eager to off and linum-delay to on for the current setting
via the customize interface and didn't perceive an effect. Any
keystroke in my org file takes 1-2 seconds to appear.

- Fontification?
http://comments.gmane.org/gmane.emacs.orgmode/45197

Comments there have files in the 5-137k line range and many say
there's no/little lag unless running agenda commands.

Any other suggestions? I use this file almost daily, mostly for
reference, not adding... that's to say it hasn't grown majorly in the
past even 3months (maybe a few hundred lines), but performance
*definitely* wasn't anything like this until the last week or so.

Thanks for any suggestions on improving or tracking down the source.
In the mean time, I'm going to revert to a few git commits ago and see
if that does anything for me.


Best regards,
John

Re: Very slow performance in Org-mode on 10k line file?

[Rainer Stengele]

Am 07.08.2013 22:25, schrieb John Hendy:
> Greetings,
> 
> 
> I just started experiencing major lag in Org-mode on my main work
> notes file, which is at about 10k lines. Is that getting up to the
> point where files get unwieldy? In googling around, I found a few
> suggestions:
> 
> - Fiddle with linum settings
> http://stackoverflow.com/questions/5229705/emacs-org-mode-turn-off-line-numbers
> 
> I set linum-eager to off and linum-delay to on for the current setting
> via the customize interface and didn't perceive an effect. Any
> keystroke in my org file takes 1-2 seconds to appear.
> 
> - Fontification?
> http://comments.gmane.org/gmane.emacs.orgmode/45197
> 
> Comments there have files in the 5-137k line range and many say
> there's no/little lag unless running agenda commands.
> 
> Any other suggestions? I use this file almost daily, mostly for
> reference, not adding... that's to say it hasn't grown majorly in the
> past even 3months (maybe a few hundred lines), but performance
> *definitely* wasn't anything like this until the last week or so.
> 
> Thanks for any suggestions on improving or tracking down the source.
> In the mean time, I'm going to revert to a few git commits ago and see
> if that does anything for me.
> 
> 
> Best regards,
> John
> 
> 
Hi,

just jumping on the bandwagon.
My one and only "biggest" issue with wonderful Orgmode is slow Emacs.
I run Emacs on i7 hardware with lots of memory and still have an Emacs Orgmode that answers rather slowly.
Slow meaning it is just not snappy at all. Creating any aganda takes several seconds which is a long time to wait
for the result. I already spent a lot of thought into how to optimise the performance of my system, archiving and
splitting Org files, using sticky agenda etc.
I know there is no quick solution to the slowness because of the limitation of threading in Emacs - I just wanted
to mention that very "unmodern" behaviour of Emacs running Orgmode. I have to use Windows 7 so this makes it even
slower. I assume my environment would run faster on Linux.

So yes, this is nothing more than something like a rant.
For me Orgmode is still the killer app in Emacs, it is just sad to have a slow environment on quite modern
hardware with some bigger Org files.
My files are of size:

$ wc *org
    124    1690   31670 file1.org
   1555   11829   97805 file2.org
  35022  262820 2314234 file3.org
    999    4968  105854 file4.org
    557    4029   30586 file5.org
   2523   20324  162165 file6.org
   2447   19974  139768 file7.org
    689    4703   36495 file8.org
   6789   58782  461211 file9.org
  53078  403126 3531142 total

Rainer

Re: Very slow performance in Org-mode on 10k line file?

[John Hendy]

On Wed, Aug 7, 2013 at 4:39 PM, Rainer Stengele
<rainer.stengele@online.de> wrote:
> Am 07.08.2013 22:25, schrieb John Hendy:
>> Greetings,
>>
>>
>> I just started experiencing major lag in Org-mode on my main work
>> notes file, which is at about 10k lines. Is that getting up to the
>> point where files get unwieldy? In googling around, I found a few
>> suggestions:
>>
>> - Fiddle with linum settings
>> http://stackoverflow.com/questions/5229705/emacs-org-mode-turn-off-line-numbers
>>
>> I set linum-eager to off and linum-delay to on for the current setting
>> via the customize interface and didn't perceive an effect. Any
>> keystroke in my org file takes 1-2 seconds to appear.
>>
>> - Fontification?
>> http://comments.gmane.org/gmane.emacs.orgmode/45197
>>
>> Comments there have files in the 5-137k line range and many say
>> there's no/little lag unless running agenda commands.
>>
>> Any other suggestions? I use this file almost daily, mostly for
>> reference, not adding... that's to say it hasn't grown majorly in the
>> past even 3months (maybe a few hundred lines), but performance
>> *definitely* wasn't anything like this until the last week or so.
>>
>> Thanks for any suggestions on improving or tracking down the source.
>> In the mean time, I'm going to revert to a few git commits ago and see
>> if that does anything for me.
>>
>>
>> Best regards,
>> John
>>
>>
> Hi,
>
> just jumping on the bandwagon.
> My one and only "biggest" issue with wonderful Orgmode is slow Emacs.
> I run Emacs on i7 hardware with lots of memory and still have an Emacs Orgmode that answers rather slowly.
> Slow meaning it is just not snappy at all. Creating any aganda takes several seconds which is a long time to wait
> for the result. I already spent a lot of thought into how to optimise the performance of my system, archiving and
> splitting Org files, using sticky agenda etc.
> I know there is no quick solution to the slowness because of the limitation of threading in Emacs - I just wanted
> to mention that very "unmodern" behaviour of Emacs running Orgmode. I have to use Windows 7 so this makes it even
> slower. I assume my environment would run faster on Linux.
>

I'm running Arch Linux 64bit on an HP EliteBook 8540w with an i7 and
8G of RAM. HD is 54% full at present. I agree that this shouldn't be a
big burden, and whatever happened recently really made this unusable
for me. Oddly, generating an agenda only takes a couple of seconds,
which is plenty fast for me, even using search.

> So yes, this is nothing more than something like a rant.
> For me Orgmode is still the killer app in Emacs, it is just sad to have a slow environment on quite modern
> hardware with some bigger Org files.
> My files are of size:
>
> $ wc *org
>     124    1690   31670 file1.org
>    1555   11829   97805 file2.org
>   35022  262820 2314234 file3.org
>     999    4968  105854 file4.org
>     557    4029   30586 file5.org
>    2523   20324  162165 file6.org
>    2447   19974  139768 file7.org
>     689    4703   36495 file8.org
>    6789   58782  461211 file9.org
>   53078  403126 3531142 total
>

$ wc *.org
    23     90    867 bibliography.org
    42    192   1756 clocking.org
  2137  18286 122303 devel.org
  9837  74994 494234 projects.org
  1536   9692  77261 reference.org
  1057   6673  48309 tf.org
 14632 109927 744730 total

projects.org is my most used file by far, and the biggest, but nothing
compared to some of the folks posting on the link I showed who have
30-130k line files!


John

> Rainer
>

Re: Very slow performance in Org-mode on 10k line file?

[John Hendy]

Just an update:
- I reverted back to commit release_8.0.2-73-g9998f2 (early May) with
no change in behavior, so perhaps it wasn't anything other than
growing file size
- I just created archive files for work journal entries in 2011 and
2012, storing them in separate archive files
- I'm now down to ~6500 lines, and lag is unnoticeable

Perhaps there's some magical cutoff between 6,000 and 10,000 lines
that starts to really bog things down?



John

On Wed, Aug 7, 2013 at 4:47 PM, John Hendy <jw.hendy@gmail.com> wrote:
> On Wed, Aug 7, 2013 at 4:39 PM, Rainer Stengele
> <rainer.stengele@online.de> wrote:
>> Am 07.08.2013 22:25, schrieb John Hendy:
>>> Greetings,
>>>
>>>
>>> I just started experiencing major lag in Org-mode on my main work
>>> notes file, which is at about 10k lines. Is that getting up to the
>>> point where files get unwieldy? In googling around, I found a few
>>> suggestions:
>>>
>>> - Fiddle with linum settings
>>> http://stackoverflow.com/questions/5229705/emacs-org-mode-turn-off-line-numbers
>>>
>>> I set linum-eager to off and linum-delay to on for the current setting
>>> via the customize interface and didn't perceive an effect. Any
>>> keystroke in my org file takes 1-2 seconds to appear.
>>>
>>> - Fontification?
>>> http://comments.gmane.org/gmane.emacs.orgmode/45197
>>>
>>> Comments there have files in the 5-137k line range and many say
>>> there's no/little lag unless running agenda commands.
>>>
>>> Any other suggestions? I use this file almost daily, mostly for
>>> reference, not adding... that's to say it hasn't grown majorly in the
>>> past even 3months (maybe a few hundred lines), but performance
>>> *definitely* wasn't anything like this until the last week or so.
>>>
>>> Thanks for any suggestions on improving or tracking down the source.
>>> In the mean time, I'm going to revert to a few git commits ago and see
>>> if that does anything for me.
>>>
>>>
>>> Best regards,
>>> John
>>>
>>>
>> Hi,
>>
>> just jumping on the bandwagon.
>> My one and only "biggest" issue with wonderful Orgmode is slow Emacs.
>> I run Emacs on i7 hardware with lots of memory and still have an Emacs Orgmode that answers rather slowly.
>> Slow meaning it is just not snappy at all. Creating any aganda takes several seconds which is a long time to wait
>> for the result. I already spent a lot of thought into how to optimise the performance of my system, archiving and
>> splitting Org files, using sticky agenda etc.
>> I know there is no quick solution to the slowness because of the limitation of threading in Emacs - I just wanted
>> to mention that very "unmodern" behaviour of Emacs running Orgmode. I have to use Windows 7 so this makes it even
>> slower. I assume my environment would run faster on Linux.
>>
>
> I'm running Arch Linux 64bit on an HP EliteBook 8540w with an i7 and
> 8G of RAM. HD is 54% full at present. I agree that this shouldn't be a
> big burden, and whatever happened recently really made this unusable
> for me. Oddly, generating an agenda only takes a couple of seconds,
> which is plenty fast for me, even using search.
>
>> So yes, this is nothing more than something like a rant.
>> For me Orgmode is still the killer app in Emacs, it is just sad to have a slow environment on quite modern
>> hardware with some bigger Org files.
>> My files are of size:
>>
>> $ wc *org
>>     124    1690   31670 file1.org
>>    1555   11829   97805 file2.org
>>   35022  262820 2314234 file3.org
>>     999    4968  105854 file4.org
>>     557    4029   30586 file5.org
>>    2523   20324  162165 file6.org
>>    2447   19974  139768 file7.org
>>     689    4703   36495 file8.org
>>    6789   58782  461211 file9.org
>>   53078  403126 3531142 total
>>
>
> $ wc *.org
>     23     90    867 bibliography.org
>     42    192   1756 clocking.org
>   2137  18286 122303 devel.org
>   9837  74994 494234 projects.org
>   1536   9692  77261 reference.org
>   1057   6673  48309 tf.org
>  14632 109927 744730 total
>
> projects.org is my most used file by far, and the biggest, but nothing
> compared to some of the folks posting on the link I showed who have
> 30-130k line files!
>
>
> John
>
>> Rainer
>>

Re: Very slow performance in Org-mode on 10k line file?

[Russell Adams]

John,

I have a 17,000 file I work out of constantly in Org 7.8.10 with very
little lag. Just another data point.

Thanks.

On Wed, Aug 07, 2013 at 05:06:51PM -0500, John Hendy wrote:
> Just an update:
> - I reverted back to commit release_8.0.2-73-g9998f2 (early May) with
> no change in behavior, so perhaps it wasn't anything other than
> growing file size
> - I just created archive files for work journal entries in 2011 and
> 2012, storing them in separate archive files
> - I'm now down to ~6500 lines, and lag is unnoticeable
>
> Perhaps there's some magical cutoff between 6,000 and 10,000 lines
> that starts to really bog things down?
>
>
>
> John
>
> On Wed, Aug 7, 2013 at 4:47 PM, John Hendy <jw.hendy@gmail.com> wrote:
> > On Wed, Aug 7, 2013 at 4:39 PM, Rainer Stengele
> > <rainer.stengele@online.de> wrote:
> >> Am 07.08.2013 22:25, schrieb John Hendy:
> >>> Greetings,
> >>>
> >>>
> >>> I just started experiencing major lag in Org-mode on my main work
> >>> notes file, which is at about 10k lines. Is that getting up to the
> >>> point where files get unwieldy? In googling around, I found a few
> >>> suggestions:
> >>>
> >>> - Fiddle with linum settings
> >>> http://stackoverflow.com/questions/5229705/emacs-org-mode-turn-off-line-numbers
> >>>
> >>> I set linum-eager to off and linum-delay to on for the current setting
> >>> via the customize interface and didn't perceive an effect. Any
> >>> keystroke in my org file takes 1-2 seconds to appear.
> >>>
> >>> - Fontification?
> >>> http://comments.gmane.org/gmane.emacs.orgmode/45197
> >>>
> >>> Comments there have files in the 5-137k line range and many say
> >>> there's no/little lag unless running agenda commands.
> >>>
> >>> Any other suggestions? I use this file almost daily, mostly for
> >>> reference, not adding... that's to say it hasn't grown majorly in the
> >>> past even 3months (maybe a few hundred lines), but performance
> >>> *definitely* wasn't anything like this until the last week or so.
> >>>
> >>> Thanks for any suggestions on improving or tracking down the source.
> >>> In the mean time, I'm going to revert to a few git commits ago and see
> >>> if that does anything for me.
> >>>
> >>>
> >>> Best regards,
> >>> John
> >>>
> >>>
> >> Hi,
> >>
> >> just jumping on the bandwagon.
> >> My one and only "biggest" issue with wonderful Orgmode is slow Emacs.
> >> I run Emacs on i7 hardware with lots of memory and still have an Emacs Orgmode that answers rather slowly.
> >> Slow meaning it is just not snappy at all. Creating any aganda takes several seconds which is a long time to wait
> >> for the result. I already spent a lot of thought into how to optimise the performance of my system, archiving and
> >> splitting Org files, using sticky agenda etc.
> >> I know there is no quick solution to the slowness because of the limitation of threading in Emacs - I just wanted
> >> to mention that very "unmodern" behaviour of Emacs running Orgmode. I have to use Windows 7 so this makes it even
> >> slower. I assume my environment would run faster on Linux.
> >>
> >
> > I'm running Arch Linux 64bit on an HP EliteBook 8540w with an i7 and
> > 8G of RAM. HD is 54% full at present. I agree that this shouldn't be a
> > big burden, and whatever happened recently really made this unusable
> > for me. Oddly, generating an agenda only takes a couple of seconds,
> > which is plenty fast for me, even using search.
> >
> >> So yes, this is nothing more than something like a rant.
> >> For me Orgmode is still the killer app in Emacs, it is just sad to have a slow environment on quite modern
> >> hardware with some bigger Org files.
> >> My files are of size:
> >>
> >> $ wc *org
> >>     124    1690   31670 file1.org
> >>    1555   11829   97805 file2.org
> >>   35022  262820 2314234 file3.org
> >>     999    4968  105854 file4.org
> >>     557    4029   30586 file5.org
> >>    2523   20324  162165 file6.org
> >>    2447   19974  139768 file7.org
> >>     689    4703   36495 file8.org
> >>    6789   58782  461211 file9.org
> >>   53078  403126 3531142 total
> >>
> >
> > $ wc *.org
> >     23     90    867 bibliography.org
> >     42    192   1756 clocking.org
> >   2137  18286 122303 devel.org
> >   9837  74994 494234 projects.org
> >   1536   9692  77261 reference.org
> >   1057   6673  48309 tf.org
> >  14632 109927 744730 total
> >
> > projects.org is my most used file by far, and the biggest, but nothing
> > compared to some of the folks posting on the link I showed who have
> > 30-130k line files!
> >
> >
> > John
> >
> >> Rainer
> >>
>


------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com

PGP Key ID:     0x1160DCB3           http://www.adamsinfoserv.com/

Fingerprint:    1723 D8CA 4280 1EC9 557F  66E8 1154 E018 1160 DCB3

Re: Very slow performance in Org-mode on 10k line file?

[John Hendy]

On Wed, Aug 7, 2013 at 5:12 PM, Russell Adams <RLAdams@adamsinfoserv.com> wrote:
> John,
>
> I have a 17,000 file I work out of constantly in Org 7.8.10 with very
> little lag. Just another data point.

Is it inconvenient for you to git pull and try on Org 8.0 to see if
there's any difference?

John

>
> Thanks.
>
> On Wed, Aug 07, 2013 at 05:06:51PM -0500, John Hendy wrote:
>> Just an update:
>> - I reverted back to commit release_8.0.2-73-g9998f2 (early May) with
>> no change in behavior, so perhaps it wasn't anything other than
>> growing file size
>> - I just created archive files for work journal entries in 2011 and
>> 2012, storing them in separate archive files
>> - I'm now down to ~6500 lines, and lag is unnoticeable
>>
>> Perhaps there's some magical cutoff between 6,000 and 10,000 lines
>> that starts to really bog things down?
>>
>>
>>
>> John
>>
>> On Wed, Aug 7, 2013 at 4:47 PM, John Hendy <jw.hendy@gmail.com> wrote:
>> > On Wed, Aug 7, 2013 at 4:39 PM, Rainer Stengele
>> > <rainer.stengele@online.de> wrote:
>> >> Am 07.08.2013 22:25, schrieb John Hendy:
>> >>> Greetings,
>> >>>
>> >>>
>> >>> I just started experiencing major lag in Org-mode on my main work
>> >>> notes file, which is at about 10k lines. Is that getting up to the
>> >>> point where files get unwieldy? In googling around, I found a few
>> >>> suggestions:
>> >>>
>> >>> - Fiddle with linum settings
>> >>> http://stackoverflow.com/questions/5229705/emacs-org-mode-turn-off-line-numbers
>> >>>
>> >>> I set linum-eager to off and linum-delay to on for the current setting
>> >>> via the customize interface and didn't perceive an effect. Any
>> >>> keystroke in my org file takes 1-2 seconds to appear.
>> >>>
>> >>> - Fontification?
>> >>> http://comments.gmane.org/gmane.emacs.orgmode/45197
>> >>>
>> >>> Comments there have files in the 5-137k line range and many say
>> >>> there's no/little lag unless running agenda commands.
>> >>>
>> >>> Any other suggestions? I use this file almost daily, mostly for
>> >>> reference, not adding... that's to say it hasn't grown majorly in the
>> >>> past even 3months (maybe a few hundred lines), but performance
>> >>> *definitely* wasn't anything like this until the last week or so.
>> >>>
>> >>> Thanks for any suggestions on improving or tracking down the source.
>> >>> In the mean time, I'm going to revert to a few git commits ago and see
>> >>> if that does anything for me.
>> >>>
>> >>>
>> >>> Best regards,
>> >>> John
>> >>>
>> >>>
>> >> Hi,
>> >>
>> >> just jumping on the bandwagon.
>> >> My one and only "biggest" issue with wonderful Orgmode is slow Emacs.
>> >> I run Emacs on i7 hardware with lots of memory and still have an Emacs Orgmode that answers rather slowly.
>> >> Slow meaning it is just not snappy at all. Creating any aganda takes several seconds which is a long time to wait
>> >> for the result. I already spent a lot of thought into how to optimise the performance of my system, archiving and
>> >> splitting Org files, using sticky agenda etc.
>> >> I know there is no quick solution to the slowness because of the limitation of threading in Emacs - I just wanted
>> >> to mention that very "unmodern" behaviour of Emacs running Orgmode. I have to use Windows 7 so this makes it even
>> >> slower. I assume my environment would run faster on Linux.
>> >>
>> >
>> > I'm running Arch Linux 64bit on an HP EliteBook 8540w with an i7 and
>> > 8G of RAM. HD is 54% full at present. I agree that this shouldn't be a
>> > big burden, and whatever happened recently really made this unusable
>> > for me. Oddly, generating an agenda only takes a couple of seconds,
>> > which is plenty fast for me, even using search.
>> >
>> >> So yes, this is nothing more than something like a rant.
>> >> For me Orgmode is still the killer app in Emacs, it is just sad to have a slow environment on quite modern
>> >> hardware with some bigger Org files.
>> >> My files are of size:
>> >>
>> >> $ wc *org
>> >>     124    1690   31670 file1.org
>> >>    1555   11829   97805 file2.org
>> >>   35022  262820 2314234 file3.org
>> >>     999    4968  105854 file4.org
>> >>     557    4029   30586 file5.org
>> >>    2523   20324  162165 file6.org
>> >>    2447   19974  139768 file7.org
>> >>     689    4703   36495 file8.org
>> >>    6789   58782  461211 file9.org
>> >>   53078  403126 3531142 total
>> >>
>> >
>> > $ wc *.org
>> >     23     90    867 bibliography.org
>> >     42    192   1756 clocking.org
>> >   2137  18286 122303 devel.org
>> >   9837  74994 494234 projects.org
>> >   1536   9692  77261 reference.org
>> >   1057   6673  48309 tf.org
>> >  14632 109927 744730 total
>> >
>> > projects.org is my most used file by far, and the biggest, but nothing
>> > compared to some of the folks posting on the link I showed who have
>> > 30-130k line files!
>> >
>> >
>> > John
>> >
>> >> Rainer
>> >>
>>
>
>
> ------------------------------------------------------------------
> Russell Adams                            RLAdams@AdamsInfoServ.com
>
> PGP Key ID:     0x1160DCB3           http://www.adamsinfoserv.com/
>
> Fingerprint:    1723 D8CA 4280 1EC9 557F  66E8 1154 E018 1160 DCB3
>

Re: Very slow performance in Org-mode on 10k line file?

[Nick Dokos]

John Hendy <jw.hendy@gmail.com> writes:

> ...
> Thanks for any suggestions on improving or tracking down the source.
> In the mean time, I'm going to revert to a few git commits ago and see
> if that does anything for me.
>

That's definitely a good idea: it sounds as if something got worse
suddenly so it may have been a "minor" change.

Before you try going back, can you also try to get a profile of a slow
operation and then repeat the profile on the same operation with the
earlier org?

M-x elp-instrument-package org
M-x elp-reset-all
<run your workload>
M-x elp-results

should be enough as a first approximation.

-- 
Nick

Re: Very slow performance in Org-mode on 10k line file?

[John Hendy]

On Wed, Aug 7, 2013 at 5:22 PM, Nick Dokos <ndokos@gmail.com> wrote:
> John Hendy <jw.hendy@gmail.com> writes:
>
>> ...
>> Thanks for any suggestions on improving or tracking down the source.
>> In the mean time, I'm going to revert to a few git commits ago and see
>> if that does anything for me.
>>
>
> That's definitely a good idea: it sounds as if something got worse
> suddenly so it may have been a "minor" change.
>

Well, see my follow-up email. Reverting back to early May did not
change the situation.

> Before you try going back, can you also try to get a profile of a slow
> operation and then repeat the profile on the same operation with the
> earlier org?
>
> M-x elp-instrument-package org
> M-x elp-reset-all
> <run your workload>
> M-x elp-results

Would it help to do this on a 6k file vs. a 10k file? Reducing my file
size made a huge difference, so if those results would be of
interest/help, I can definitely do that?


John

>
> should be enough as a first approximation.
>
> --
> Nick
>
>

Re: Very slow performance in Org-mode on 10k line file?

[Nick Dokos]

John Hendy <jw.hendy@gmail.com> writes:

>> M-x elp-instrument-package org
>> M-x elp-reset-all
>> <run your workload>
>> M-x elp-results
>
> Would it help to do this on a 6k file vs. a 10k file? Reducing my file
> size made a huge difference, so if those results would be of
> interest/help, I can definitely do that?
>

The more data the better, so yes, I think it's a useful exercise.

-- 
Nick

Re: Very slow performance in Org-mode on 10k line file?

[Russell Adams]

On Wed, Aug 07, 2013 at 05:17:53PM -0500, John Hendy wrote:
> Is it inconvenient for you to git pull and try on Org 8.0 to see if
> there's any difference?

I'm not prepared to upgrade at this time. I just thought it odd the
line length you were experiencing issues at compared to mine.

Good luck!

------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com

PGP Key ID:     0x1160DCB3           http://www.adamsinfoserv.com/

Fingerprint:    1723 D8CA 4280 1EC9 557F  66E8 1154 E018 1160 DCB3

Re: Very slow performance in Org-mode on 10k line file?

[John Hendy]

Alrighty. Here you are:

#+begin_src minimal emacs (which I've moved to ~/.emacs) and started a
fresh session
(add-to-list 'load-path "~/.elisp/org.git/contrib/lisp")
(add-to-list 'load-path "~/.elisp/org.git/lisp/")
#+end_src


Really simple operation, but that's that had huge lag on the 10k file.
All I did was delete a word and re-write it, and then insert a line
above a table ("#+attr_latex: :align llp{7cm}"). Here's the process:

- Start emacs (/usr/bin/emacs)
- Navigate to file (6k ~/org/projects.org file)
- Run: M-x elp-instrument-package [RET] org
- Run: M-x elp-reset-all
- Navigate to the headline I was trying to work on, delete a word a
character at a time, re-type it, add the above latex line above a
table
- Run: M-x elp-results
- Copied 2011 and 2012 journal entries back into projects.org, saved,
quit emacs, and repeated

Results with 6k line file
==========
org-self-insert-command                             41
2.026747151   0.0494328573
org-activate-dates                                  54
0.0175029940  0.0003241295
org-fontify-meta-lines-and-blocks                   161
0.006726777   4.178...e-05
org-fontify-meta-lines-and-blocks-1                 161
0.0047893350  2.974...e-05
org-delete-backward-char                            12
0.002401629   0.0002001357
org-activate-footnote-links                         54
0.002203194   4.079...e-05
org-activate-plain-links                            54
0.0020726379  3.838...e-05
org-unfontify-region                                54
0.0020696900  3.832...e-05
org-at-table-p                                      54
0.0016839339  3.118...e-05
org-footnote-next-reference-or-definition           54
0.0015487719  2.868...e-05
org-do-latex-and-related                            54
0.0014039899  2.599...e-05
org-do-emphasis-faces                               54
0.001028353   1.904...e-05
org-string-nw-p                                     54
0.0008700800  1.611...e-05
org-activate-angle-links                            54
0.0008193899  1.517...e-05
org-return                                          1
0.000758448   0.000758448
org-activate-tags                                   54
0.0005812419  1.076...e-05
org-in-item-p                                       1
0.000436519   0.000436519
org-activate-bracket-links                          54
0.0004324309  8.007...e-06
org-activate-code                                   54
0.000416118   7.705...e-06
org-fix-tags-on-the-fly                             53
0.000332257   6.269e-06
org-font-lock-add-priority-faces                    54
0.0003180110  5.889...e-06
org-string-match-p                                  54
0.000269684   4.994...e-06
org-remove-font-lock-display-properties             54
0.0002692999  4.987...e-06
org-list-context                                    1
0.000268703   0.000268703
org-hide-wide-columns                               54
0.000228154   4.225...e-06
org-before-change-function                          54
0.0001231199  2.279...e-06
org-remove-flyspell-overlays-in                     21
0.0001044190  4.972...e-06
org-font-lock-hook                                  54
8.081...e-05  1.496...e-06
org-check-before-invisible-edit                     53
7.8635e-05    1.483...e-06
org-activate-target-links                           54
7.542...e-05  1.396...e-06
org-fontify-entities                                54
6.752e-05     1.250...e-06
org-raise-scripts                                   54
6.415...e-05  1.188...e-06
org-font-lock-add-tag-faces                         54
5.994...e-05  1.110...e-06
org-in-src-block-p                                  1
5.1549e-05    5.1549e-05
org-in-regexp                                       1
3.4068e-05    3.4068e-05
org-back-to-heading                                 1
3.3785e-05    3.3785e-05
org-get-limited-outline-regexp                      1
2.1714e-05    2.1714e-05
org-at-heading-p                                    1
1.3884e-05    1.3884e-05
org-get-indentation                                 2
1.337...e-05  6.689...e-06
org-item-re                                         1
4.116e-06     4.116e-06
==========



Results with ~10k line file
==========
org-self-insert-command                             39
0.85596741    0.0219478823
org-cycle                                           6
0.107894177   0.0179823628
org-cycle-internal-local                            6
0.1058569380  0.0176428230
org-optimize-window-after-visibility-change         6
0.047916078   0.007986013
org-subtree-end-visible-p                           5
0.046233295   0.009246659
org-end-of-subtree                                  27
0.029686826   0.0010995120
org-outline-level                                   394
0.0162884779  4.134...e-05
org-cycle-show-empty-lines                          6
0.008189213   0.0013648688
org-fontify-meta-lines-and-blocks                   182
0.0078306620  4.302...e-05
org-fontify-meta-lines-and-blocks-1                 182
0.0057315880  3.149...e-05
org-cycle-hide-archived-subtrees                    6
0.005463361   0.0009105601
org-delete-backward-char                            10
0.004027681   0.0004027681
org-activate-plain-links                            79
0.0035584700  4.504...e-05
org-activate-footnote-links                         78
0.0028586259  3.664...e-05
org-unfontify-region                                78
0.0024188670  3.101...e-05
org-do-emphasis-faces                               89
0.002085356   2.343...e-05
org-footnote-next-reference-or-definition           78
0.0020824370  2.669...e-05
org-do-latex-and-related                            78
0.0017855160  2.289...e-05
org-at-table-p                                      56
0.0016877709  3.013...e-05
org-activate-dates                                  106
0.0016536720  1.560...e-05
org-cycle-hide-drawers                              9
0.0016145939  0.0001793993
org-back-to-heading                                 432
0.0015116170  3.499...e-06
org-activate-angle-links                            78
0.001141598   1.463...e-05
org-string-nw-p                                     78
0.0011044000  1.415...e-05
org-activate-tags                                   81
0.0009928790  1.225...e-05
org-return                                          1
0.000793986   0.000793986
org-activate-code                                   78
0.0006506530  8.341...e-06
org-activate-bracket-links                          78
0.00060669    7.778...e-06
org-font-lock-add-priority-faces                    78
0.0004675280  5.993...e-06
org-in-item-p                                       1
0.00044091    0.00044091
org-at-item-p                                       21
0.0004120960  1.962...e-05
org-hide-archived-subtrees                          5
0.0004082699  8.165...e-05
org-string-match-p                                  78
0.0003752040  4.810...e-06
org-remove-font-lock-display-properties             78
0.0003220950  4.129...e-06
org-fix-tags-on-the-fly                             49
0.0003208529  6.548...e-06
org-flag-drawer                                     7
0.0003203140  4.575...e-05
org-hide-wide-columns                               78
0.0002948089  3.779...e-06
org-show-entry                                      3
0.000274877   9.162...e-05
org-list-context                                    1
0.0002685     0.0002685
org-before-first-heading-p                          11
0.0002303880  2.094...e-05
org-hide-block-toggle-maybe                         6
0.0002303300  3.838...e-05
org-cycle-item-indentation                          6
0.0002270370  3.78395e-05
org-remove-flyspell-overlays-in                     53
0.0002091870  3.946...e-06
org-babel-hide-result-toggle-maybe                  6
0.000174206   2.903...e-05
org-get-level-face                                  162
0.0001498009  9.246...e-07
org-before-change-function                          50
0.0001198120  2.396...e-06
org-fontify-entities                                78
0.0001098809  1.408...e-06
org-cycle-level                                     6
0.0001042630  1.737...e-05
org-activate-target-links                           78
0.000101329   1.299...e-06
org-font-lock-hook                                  78
0.0001005800  1.289...e-06
org-flag-subtree                                    1
9.9225e-05    9.9225e-05
org-in-src-block-p                                  2
8.5134e-05    4.2567e-05
org-font-lock-add-tag-faces                         78
8.087...e-05  1.036...e-06
org-item-re                                         23
8.068...e-05  3.508...e-06
org-raise-scripts                                   78
7.867...e-05  1.008...e-06
org-list-search-forward                             1
7.5083e-05    7.5083e-05
org-check-before-invisible-edit                     49
7.2543e-05    1.480...e-06
org-list-search-generic                             1
6.0865e-05    6.0865e-05
org-at-heading-p                                    8
5.9246e-05    7.40575e-06
org-get-limited-outline-regexp                      4
4.654...e-05  1.163...e-05
org-in-regexp                                       1
3.485e-05     3.485e-05
org-point-at-end-of-empty-headline                  6
3.2403e-05    5.4005e-06
org-babel-header-arg-expand                         6
2.753...e-05  4.589...e-06
org-get-indentation                                 2
1.3004e-05    6.502e-06
org-load-modules-maybe                              6
1.2065e-05    2.010...e-06
org-src-native-tab-command-maybe                    6
1.1753e-05    1.958...e-06
org-try-cdlatex-tab                                 6
1.1116e-05    1.852...e-06
org-cycle-hide-inline-tasks                         6
6.695e-06     1.115...e-06
==========

Yeah... and of course when you try to measure something, the problem
goes away. There was no lag at all in that second experiment, and I'm
baffled. RAM is still at about the same useage (25%) as when I was
experiencing this before, as is CPU usage. All I've done since then:
- Copy ~4k lines of headlines from projects.org into other files
- Git pull to change from my reverted early May commit to the current
master branch
- make clean && make && make doc, which is my standard update process
- Copy the 4k lines of files back into the file

No clue.


John




On Wed, Aug 7, 2013 at 5:39 PM, Nick Dokos <ndokos@gmail.com> wrote:
> John Hendy <jw.hendy@gmail.com> writes:
>
>>> M-x elp-instrument-package org
>>> M-x elp-reset-all
>>> <run your workload>
>>> M-x elp-results
>>
>> Would it help to do this on a 6k file vs. a 10k file? Reducing my file
>> size made a huge difference, so if those results would be of
>> interest/help, I can definitely do that?
>>
>
> The more data the better, so yes, I think it's a useful exercise.
>
> --
> Nick
>
>

Re: Very slow performance in Org-mode on 10k line file?

[Achim Gratz]

John Hendy writes:
> - make clean && make && make doc, which is my standard update process

I should maybe point out again that this has been redundant for a long
time and a simple "make" would suffice if you removed the line "oldorg:"
from your local.mk file.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

faster agenda with properties support disabled (no org-refresh-properties)

[Daniel Clemente]

> 
> M-x elp-instrument-package org
> M-x elp-reset-all
> <run your workload>
> M-x elp-results

  Incidentally I ran that and I saw:

org-agenda                                                    1           15.709354028  15.709354028
org-agenda-list                                               1           15.495628529  15.495628529
org-agenda-prepare                                            1           8.388162561   8.388162561
org-agenda-prepare-buffers                                    1           8.349513929   8.349513929
org-agenda-get-day-entries                                    477         5.7457141640  0.0120455223
org-agenda-get-scheduled                                      477         2.0763930930  0.0043530253
org-agenda-get-timestamps                                     477         2.046089454   0.0042894957
org-set-regexps-and-options-for-tags                          164         1.8202055340  0.0110988142
org-refresh-properties                                        318         1.3865960840  0.0043603650
org-refresh-category-properties                               159         1.1616332550  0.0073058695
org-agenda-get-deadlines                                      477         0.5512838650  0.0011557313
org-agenda-get-blocks                                         477         0.5356222019  0.0011228976
org-get-todo-state                                            3581        0.4114158859  0.0001148885
org-agenda-get-sexps                                          477         0.4037585499  0.0008464539
…

  I looked at org-refresh-properties.

  In org.el there is:

(defun org-agenda-prepare-buffers (files)
…
	    (org-refresh-category-properties)
	    (org-refresh-properties org-effort-property 'org-effort)
	    (org-refresh-properties "APPT_WARNTIME" 'org-appt-warntime)
…

  Since I am not using effort/category/appointment properties in my agenda, I would like to disable it. I commented it in the code and I get the same agenda but 2'4 seconds faster (even more than 1'4 from org-refresh-properties). The new instrumentation report is:

org-agenda                                                    1           13.345656663  13.345656663
org-agenda-list                                               1           13.113396681  13.113396681
org-agenda-prepare                                            1           7.086576653   7.086576653
org-agenda-prepare-buffers                                    1           7.054057855   7.054057855
org-agenda-get-day-entries                                    477         5.7340928759  0.0120211590
org-agenda-get-scheduled                                      477         3.3844209709  0.0070952221
org-set-regexps-and-options-for-tags                          164         1.8059163709  0.0110116851
org-refresh-properties                                        318         1.3982702620  0.0043970762
org-refresh-category-properties                               159         1.1513761240  0.0072413592
org-agenda-get-timestamps                                     477         0.6975214329  0.0014623090
org-agenda-get-deadlines                                      477         0.557952655   0.0011697120
org-agenda-get-blocks                                         477         0.533165758   0.0011177479
org-agenda-skip                                               3977        0.4244523499  0.0001067267
…

  So I would like to ask: is there a clean way to disable calls to org-refresh-properties?

Re: faster agenda with properties support disabled (no org-refresh-properties)

[Carsten Dominik]

On 12.8.2013, at 05:43, Daniel Clemente <n142857@gmail.com> wrote:

> 
>> 
>> M-x elp-instrument-package org
>> M-x elp-reset-all
>> <run your workload>
>> M-x elp-results
> 
>  Incidentally I ran that and I saw:
> 
> org-agenda                                                    1           15.709354028  15.709354028
> org-agenda-list                                               1           15.495628529  15.495628529
> org-agenda-prepare                                            1           8.388162561   8.388162561
> org-agenda-prepare-buffers                                    1           8.349513929   8.349513929
> org-agenda-get-day-entries                                    477         5.7457141640  0.0120455223
> org-agenda-get-scheduled                                      477         2.0763930930  0.0043530253
> org-agenda-get-timestamps                                     477         2.046089454   0.0042894957
> org-set-regexps-and-options-for-tags                          164         1.8202055340  0.0110988142
> org-refresh-properties                                        318         1.3865960840  0.0043603650
> org-refresh-category-properties                               159         1.1616332550  0.0073058695
> org-agenda-get-deadlines                                      477         0.5512838650  0.0011557313
> org-agenda-get-blocks                                         477         0.5356222019  0.0011228976
> org-get-todo-state                                            3581        0.4114158859  0.0001148885
> org-agenda-get-sexps                                          477         0.4037585499  0.0008464539
> …
> 
>  I looked at org-refresh-properties.
> 
>  In org.el there is:
> 
> (defun org-agenda-prepare-buffers (files)
> …
> 	    (org-refresh-category-properties)
> 	    (org-refresh-properties org-effort-property 'org-effort)
> 	    (org-refresh-properties "APPT_WARNTIME" 'org-appt-warntime)
> …
> 
>  Since I am not using effort/category/appointment properties in my agenda, I would like to disable it. I commented it in the code and I get the same agenda but 2'4 seconds faster (even more than 1'4 from org-refresh-properties). The new instrumentation report is:
> 
> org-agenda                                                    1           13.345656663  13.345656663
> org-agenda-list                                               1           13.113396681  13.113396681
> org-agenda-prepare                                            1           7.086576653   7.086576653
> org-agenda-prepare-buffers                                    1           7.054057855   7.054057855
> org-agenda-get-day-entries                                    477         5.7340928759  0.0120211590
> org-agenda-get-scheduled                                      477         3.3844209709  0.0070952221
> org-set-regexps-and-options-for-tags                          164         1.8059163709  0.0110116851
> org-refresh-properties                                        318         1.3982702620  0.0043970762
> org-refresh-category-properties                               159         1.1513761240  0.0072413592
> org-agenda-get-timestamps                                     477         0.6975214329  0.0014623090
> org-agenda-get-deadlines                                      477         0.557952655   0.0011697120
> org-agenda-get-blocks                                         477         0.533165758   0.0011177479
> org-agenda-skip                                               3977        0.4244523499  0.0001067267
> …
> 
>  So I would like to ask: is there a clean way to disable calls to org-refresh-properties?

No, that would require a patch and a config variable.

- Carsten

> 
> 

Re: faster agenda with properties support disabled (no org-refresh-properties)

[Daniel Clemente]

>>  So I would like to ask: is there a clean way to disable calls to
>> org-refresh-properties?
>
> No, that would require a patch and a config variable.
>
> - Carsten
>

  I send a patch to do this. Setting this new variable to t reduced 10
seconds my agenda export time (down from 1 minute 6 seconds) as well
as the update.
  You may add a comment about what to expect if your agenda depends on
property data.


diff --git a/lisp/org.el b/lisp/org.el
index 572b797..167e7a8 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -17656,6 +17656,14 @@ is not set, the tables are not re-aligned, etc."
   :version "24.3"
   :group 'org-agenda)

+(defcustom org-agenda-ignore-properties nil
+  "Avoid updating text properties when building the agenda.
+Properties are used for effort estimation, appointments, categories.
+If you don't use these in the agenda, set it to t and it will be faster."
+  :type 'boolean
+  :version "24.3"
+  :group 'org-agenda)
+
 (defun org-duration-string-to-minutes (s &optional output-to-string)
   "Convert a duration string S to minutes.

@@ -18017,9 +18025,11 @@ When a buffer is unmodified, it is just
killed.  When modified, it is saved
 		;; this is only run for setting agenda tags from setup
 		;; file
 		(org-set-regexps-and-options)))
-	    (org-refresh-category-properties)
-	    (org-refresh-properties org-effort-property 'org-effort)
-	    (org-refresh-properties "APPT_WARNTIME" 'org-appt-warntime)
+	    (unless org-agenda-ignore-properties
+	      (org-refresh-category-properties)
+	      (org-refresh-properties org-effort-property 'org-effort)
+	      (org-refresh-properties "APPT_WARNTIME" 'org-appt-warntime)
+	    )
 	    (setq org-todo-keywords-for-agenda
 		  (append org-todo-keywords-for-agenda org-todo-keywords-1))
 	    (setq org-done-keywords-for-agenda

Re: faster agenda with properties support disabled (no org-refresh-properties)

[Samuel Wales]

Probably the only property most people use most of the time in the
agenda is category.  Would this get rid of showing category too?  66s
to 10s is impressive.  I'd give up sorting and selecting category for
that.  But I wouldn't give up showing category.

Samuel

-- 
The Kafka Pandemic: http://thekafkapandemic.blogspot.com

The disease DOES progress.  MANY people have died from it.  ANYBODY can get it.

Denmark: free Karina Hansen NOW.

Re: faster agenda with properties support disabled (no org-refresh-properties)

[Daniel Clemente]

The speedup was 66 → 56 seconds (not to 10, but by 10) when exporting the agenda. It's small but noticeable.
I don't use :CATEGORY: but I'm using category icons (org-agenda-category-icon-alist) and I keep seeing them, so I assume categories are still shown. You can try it in your files to see what happens.

El Tue, 27 Aug 2013 21:28:27 -0700 Samuel Wales va escriure:
> 
> Probably the only property most people use most of the time in the
> agenda is category.  Would this get rid of showing category too?  66s
> to 10s is impressive.  I'd give up sorting and selecting category for
> that.  But I wouldn't give up showing category.
> 
> Samuel

Re: faster agenda with properties support disabled (no org-refresh-properties)

[Carsten Dominik]

Hi Daniel,

I have implemented a different version of the patch.  Please take a look at the new variable
org-agenda-ignore-drawer-properties.

Regards, and thanks!

- Carsten

On 23.8.2013, at 11:24, Daniel Clemente <n142857@gmail.com> wrote:

>>> So I would like to ask: is there a clean way to disable calls to
>>> org-refresh-properties?
>> 
>> No, that would require a patch and a config variable.
>> 
>> - Carsten
>> 
> 
>  I send a patch to do this. Setting this new variable to t reduced 10
> seconds my agenda export time (down from 1 minute 6 seconds) as well
> as the update.
>  You may add a comment about what to expect if your agenda depends on
> property data.
> 
> 
> diff --git a/lisp/org.el b/lisp/org.el
> index 572b797..167e7a8 100644
> --- a/lisp/org.el
> +++ b/lisp/org.el
> @@ -17656,6 +17656,14 @@ is not set, the tables are not re-aligned, etc."
>   :version "24.3"
>   :group 'org-agenda)
> 
> +(defcustom org-agenda-ignore-properties nil
> +  "Avoid updating text properties when building the agenda.
> +Properties are used for effort estimation, appointments, categories.
> +If you don't use these in the agenda, set it to t and it will be faster."
> +  :type 'boolean
> +  :version "24.3"
> +  :group 'org-agenda)
> +
> (defun org-duration-string-to-minutes (s &optional output-to-string)
>   "Convert a duration string S to minutes.
> 
> @@ -18017,9 +18025,11 @@ When a buffer is unmodified, it is just
> killed.  When modified, it is saved
> 		;; this is only run for setting agenda tags from setup
> 		;; file
> 		(org-set-regexps-and-options)))
> -	    (org-refresh-category-properties)
> -	    (org-refresh-properties org-effort-property 'org-effort)
> -	    (org-refresh-properties "APPT_WARNTIME" 'org-appt-warntime)
> +	    (unless org-agenda-ignore-properties
> +	      (org-refresh-category-properties)
> +	      (org-refresh-properties org-effort-property 'org-effort)
> +	      (org-refresh-properties "APPT_WARNTIME" 'org-appt-warntime)
> +	    )
> 	    (setq org-todo-keywords-for-agenda
> 		  (append org-todo-keywords-for-agenda org-todo-keywords-1))
> 	    (setq org-done-keywords-for-agenda
> 

Re: faster agenda with properties support disabled (no org-refresh-properties)

[Bastien]

Hi all,

Carsten Dominik <carsten.dominik@gmail.com> writes:

> I have implemented a different version of the patch.  Please take a look at the new variable
> org-agenda-ignore-drawer-properties.

Great -- could someone document this on this page?
http://orgmode.org/worg/agenda-optimization.html

Thanks!

-- 
 Bastien

Re: faster agenda with properties support disabled (no org-refresh-properties)

[Carsten Dominik]

On 31.8.2013, at 08:22, Bastien <bzg@gnu.org> wrote:

> Hi all,
> 
> Carsten Dominik <carsten.dominik@gmail.com> writes:
> 
>> I have implemented a different version of the patch.  Please take a look at the new variable
>> org-agenda-ignore-drawer-properties.
> 
> Great -- could someone document this on this page?
> http://orgmode.org/worg/agenda-optimization.html

Done.

- Carsten

> 
> Thanks!
> 
> -- 
> Bastien

Re: faster agenda with properties support disabled (no org-refresh-properties)

[Bastien]

Carsten Dominik <carsten.dominik@gmail.com> writes:

>> Great -- could someone document this on this page?
>> http://orgmode.org/worg/agenda-optimization.html
>
> Done.

Thanks!

-- 
 Bastien

Re: faster agenda with properties support disabled (no org-refresh-properties)

[Daniel Clemente]

Thank you.
  With this on, I reduced 1'7 seconds my normal agenda time (C-a a), from 13'5 to 11'8. Numbers are from elp but I checked them with an external stopwatch because sometimes I have the impression that elp makes things slower.
  The strange thing is, I don't see the difference I saw days before in (org-batch-agenda). I could reproduceably run a slow export (with no patch) and a fast export (with the patch). Now both are fast. I suppose that the contents of my agenda might have changed in a way that is fast to handle. Anyway, this is only good.

El Sat, 31 Aug 2013 07:58:00 +0200 Carsten Dominik va escriure:
> 
> Hi Daniel,
> 
> I have implemented a different version of the patch.  Please take a look at the new variable
> org-agenda-ignore-drawer-properties.
> 
> Regards, and thanks!
> 
> - Carsten
> 
> On 23.8.2013, at 11:24, Daniel Clemente <n142857@gmail.com> wrote:
> 
> >>> So I would like to ask: is there a clean way to disable calls to
> >>> org-refresh-properties?
> >> 
> >> No, that would require a patch and a config variable.
> >> 
> >> - Carsten
> >> 
> > 
> >  I send a patch to do this. Setting this new variable to t reduced 10
> > seconds my agenda export time (down from 1 minute 6 seconds) as well
> > as the update.
> >  You may add a comment about what to expect if your agenda depends on
> > property data.
> > 
> > 
> > diff --git a/lisp/org.el b/lisp/org.el
> > index 572b797..167e7a8 100644
> > --- a/lisp/org.el
> > +++ b/lisp/org.el
> > @@ -17656,6 +17656,14 @@ is not set, the tables are not re-aligned, etc."
> >   :version "24.3"
> >   :group 'org-agenda)
> > 
> > +(defcustom org-agenda-ignore-properties nil
> > +  "Avoid updating text properties when building the agenda.
> > +Properties are used for effort estimation, appointments, categories.
> > +If you don't use these in the agenda, set it to t and it will be faster."
> > +  :type 'boolean
> > +  :version "24.3"
> > +  :group 'org-agenda)
> > +
> > (defun org-duration-string-to-minutes (s &optional output-to-string)
> >   "Convert a duration string S to minutes.
> > 
> > @@ -18017,9 +18025,11 @@ When a buffer is unmodified, it is just
> > killed.  When modified, it is saved
> > 		;; this is only run for setting agenda tags from setup
> > 		;; file
> > 		(org-set-regexps-and-options)))
> > -	    (org-refresh-category-properties)
> > -	    (org-refresh-properties org-effort-property 'org-effort)
> > -	    (org-refresh-properties "APPT_WARNTIME" 'org-appt-warntime)
> > +	    (unless org-agenda-ignore-properties
> > +	      (org-refresh-category-properties)
> > +	      (org-refresh-properties org-effort-property 'org-effort)
> > +	      (org-refresh-properties "APPT_WARNTIME" 'org-appt-warntime)
> > +	    )
> > 	    (setq org-todo-keywords-for-agenda
> > 		  (append org-todo-keywords-for-agenda org-todo-keywords-1))
> > 	    (setq org-done-keywords-for-agenda
> > 
> 

Re: faster agenda with properties support disabled (no org-refresh-properties)

[Carsten Dominik]

On Sep 3, 2013, at 2:02 PM, Daniel Clemente <n142857@gmail.com> wrote:

> 
> Thank you.
>  With this on, I reduced 1'7 seconds my normal agenda time (C-a a), from 13'5 to 11'8. Numbers are from elp but I checked them with an external stopwatch because sometimes I have the impression that elp makes things slower.
>  The strange thing is, I don't see the difference I saw days before in (org-batch-agenda). I could reproduceably run a slow export (with no patch) and a fast export (with the patch). Now both are fast. I suppose that the contents of my agenda might have changed in a way that is fast to handle. Anyway, this is only good.

OK, I also have no idea why that happens.

Thanks for the feedback.

- Carsten

> 
> El Sat, 31 Aug 2013 07:58:00 +0200 Carsten Dominik va escriure:
>> 
>> Hi Daniel,
>> 
>> I have implemented a different version of the patch.  Please take a look at the new variable
>> org-agenda-ignore-drawer-properties.
>> 
>> Regards, and thanks!
>> 
>> - Carsten
>> 
>> On 23.8.2013, at 11:24, Daniel Clemente <n142857@gmail.com> wrote:
>> 
>>>>> So I would like to ask: is there a clean way to disable calls to
>>>>> org-refresh-properties?
>>>> 
>>>> No, that would require a patch and a config variable.
>>>> 
>>>> - Carsten
>>>> 
>>> 
>>> I send a patch to do this. Setting this new variable to t reduced 10
>>> seconds my agenda export time (down from 1 minute 6 seconds) as well
>>> as the update.
>>> You may add a comment about what to expect if your agenda depends on
>>> property data.
>>> 
>>> 
>>> diff --git a/lisp/org.el b/lisp/org.el
>>> index 572b797..167e7a8 100644
>>> --- a/lisp/org.el
>>> +++ b/lisp/org.el
>>> @@ -17656,6 +17656,14 @@ is not set, the tables are not re-aligned, etc."
>>>  :version "24.3"
>>>  :group 'org-agenda)
>>> 
>>> +(defcustom org-agenda-ignore-properties nil
>>> +  "Avoid updating text properties when building the agenda.
>>> +Properties are used for effort estimation, appointments, categories.
>>> +If you don't use these in the agenda, set it to t and it will be faster."
>>> +  :type 'boolean
>>> +  :version "24.3"
>>> +  :group 'org-agenda)
>>> +
>>> (defun org-duration-string-to-minutes (s &optional output-to-string)
>>>  "Convert a duration string S to minutes.
>>> 
>>> @@ -18017,9 +18025,11 @@ When a buffer is unmodified, it is just
>>> killed.  When modified, it is saved
>>> 		;; this is only run for setting agenda tags from setup
>>> 		;; file
>>> 		(org-set-regexps-and-options)))
>>> -	    (org-refresh-category-properties)
>>> -	    (org-refresh-properties org-effort-property 'org-effort)
>>> -	    (org-refresh-properties "APPT_WARNTIME" 'org-appt-warntime)
>>> +	    (unless org-agenda-ignore-properties
>>> +	      (org-refresh-category-properties)
>>> +	      (org-refresh-properties org-effort-property 'org-effort)
>>> +	      (org-refresh-properties "APPT_WARNTIME" 'org-appt-warntime)
>>> +	    )
>>> 	    (setq org-todo-keywords-for-agenda
>>> 		  (append org-todo-keywords-for-agenda org-todo-keywords-1))
>>> 	    (setq org-done-keywords-for-agenda
>>> 
>>