* Re: Largest org file you have + performance
2011-08-01 16:35 Largest org file you have + performance Marcelo de Moraes Serpa
@ 2011-08-01 16:54 ` suvayu ali
2011-08-01 17:06 ` Marcelo de Moraes Serpa
2011-08-01 18:13 ` Florian Beck
` (3 subsequent siblings)
4 siblings, 1 reply; 16+ messages in thread
From: suvayu ali @ 2011-08-01 16:54 UTC (permalink / raw)
To: Marcelo de Moraes Serpa; +Cc: Org Mode
Hi Marcelo,
On Mon, Aug 1, 2011 at 6:35 PM, Marcelo de Moraes Serpa
<celoserpa@gmail.com> wrote:
> What's the largest orgmode file you have, and what's the performance you get
> while manipulating/navigating it? Mine is a 5k lines file called
> reference.org, and I basically keep all kind of notes and attachments there.
My largest agenda file used to be ~2k lines. Now I have archived many
things, so my 2nd largest agenda file is ~1.5k lines. I don't see any
performance problems. All agenda commands are almost instantaneous.
Hope this helps.
--
Suvayu
Open source is the future. It sets us free.
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: Largest org file you have + performance
2011-08-01 16:54 ` suvayu ali
@ 2011-08-01 17:06 ` Marcelo de Moraes Serpa
2011-08-01 20:27 ` suvayu ali
0 siblings, 1 reply; 16+ messages in thread
From: Marcelo de Moraes Serpa @ 2011-08-01 17:06 UTC (permalink / raw)
To: suvayu ali; +Cc: Org Mode
[-- Attachment #1: Type: text/plain, Size: 745 bytes --]
What platform? OSX?
On Mon, Aug 1, 2011 at 11:54 AM, suvayu ali <fatkasuvayu+linux@gmail.com>wrote:
> Hi Marcelo,
>
> On Mon, Aug 1, 2011 at 6:35 PM, Marcelo de Moraes Serpa
> <celoserpa@gmail.com> wrote:
> > What's the largest orgmode file you have, and what's the performance you
> get
> > while manipulating/navigating it? Mine is a 5k lines file called
> > reference.org, and I basically keep all kind of notes and attachments
> there.
>
> My largest agenda file used to be ~2k lines. Now I have archived many
> things, so my 2nd largest agenda file is ~1.5k lines. I don't see any
> performance problems. All agenda commands are almost instantaneous.
>
> Hope this helps.
>
> --
> Suvayu
>
> Open source is the future. It sets us free.
>
[-- Attachment #2: Type: text/html, Size: 1212 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Largest org file you have + performance
2011-08-01 16:35 Largest org file you have + performance Marcelo de Moraes Serpa
2011-08-01 16:54 ` suvayu ali
@ 2011-08-01 18:13 ` Florian Beck
2011-08-02 7:14 ` Performance of org-mode with BIDI (was: Largest org file you have + performance) Reiner Steib
2011-08-01 18:17 ` Largest org file you have + performance Eric Abrahamsen
` (2 subsequent siblings)
4 siblings, 1 reply; 16+ messages in thread
From: Florian Beck @ 2011-08-01 18:13 UTC (permalink / raw)
To: Marcelo de Moraes Serpa; +Cc: Org Mode
Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:
> Hey list,
>
>
> What's the largest orgmode file you have, and what's the performance you get while manipulating/navigating it? Mine is a 5k lines
> file called reference.org, and I basically keep all kind of notes and attachments there. It's a bit slow to navigate on my Emacs
> 240.50.1, orgmode 7.4, although still usable. I'm just afraid that it will keep getting slower with time at the point of being
> unusable inside emacs+orgmode. I'm curious if there are users with larger orgmode files and what their experience is, and open to
> suggestions on how to improve performance.
emacs-bzr, org-7.7
The largest file I have is 7.2M. Some 6300 entries, 137k lines. Agenda
commands are slow, of course, but otherwise the performance is
acceptable. Anything that involves completion is quite slow. Opening ID
links is also slower than I would like. Anything that works on the whole
buffer can take a long time (cycling from OVERVIEW to CONTENTS takes more
than a minute), so try to avoid that. (I keep old notes in that file, so
mostly I search or link to entries.)
If you are using the current bzr version of emacs, make sure to set
`bidi-display-reordering' to nil.
>
>
>
> Cheers,
>
>
>
> Marcelo.
--
Florian Beck
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: Largest org file you have + performance
2011-08-01 16:35 Largest org file you have + performance Marcelo de Moraes Serpa
2011-08-01 16:54 ` suvayu ali
2011-08-01 18:13 ` Florian Beck
@ 2011-08-01 18:17 ` Eric Abrahamsen
2011-08-02 0:56 ` Bernt Hansen
2011-08-04 20:18 ` Milan Zamazal
4 siblings, 0 replies; 16+ messages in thread
From: Eric Abrahamsen @ 2011-08-01 18:17 UTC (permalink / raw)
To: emacs-orgmode
On Mon, Aug 01 2011, Marcelo de Moraes Serpa wrote:
> Hey list,
>
> What's the largest orgmode file you have, and what's the performance
> you get while manipulating/navigating it? Mine is a 5k lines file
> called reference.org, and I basically keep all kind of notes and
> attachments there. It's a bit slow to navigate on my Emacs 240.50.1,
> orgmode 7.4, although still usable. I'm just afraid that it will keep
> getting slower with time at the point of being unusable inside
> emacs+orgmode. I'm curious if there are users with larger orgmode
> files and what their experience is, and open to suggestions on how to
> improve performance.
My largest file is 32k lines. It's a novel translation (so essentially
two very long novels in one file), and there's no perceptible lag in
navigation/folding. BUT! It's almost all plain text, very little
fontification besides the headlines, and I understand that fontification
is one of the major causes of lag. So, dunno if that's helpful, but it's
another data point.
Eric
(Using the Ubuntu version of Emacs, 23.3 I believe, and yesterday's org-mode)
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Largest org file you have + performance
2011-08-01 16:35 Largest org file you have + performance Marcelo de Moraes Serpa
` (2 preceding siblings ...)
2011-08-01 18:17 ` Largest org file you have + performance Eric Abrahamsen
@ 2011-08-02 0:56 ` Bernt Hansen
2011-08-04 20:18 ` Milan Zamazal
4 siblings, 0 replies; 16+ messages in thread
From: Bernt Hansen @ 2011-08-02 0:56 UTC (permalink / raw)
To: Marcelo de Moraes Serpa; +Cc: Org Mode
Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:
> What's the largest orgmode file you have, and what's the performance
> you get while manipulating/navigating it? Mine is a 5k lines file
> called reference.org, and I basically keep all kind of notes and
> attachments there. It's a bit slow to navigate on my Emacs 240.50.1,
> orgmode 7.4, although still usable. I'm just afraid that it will keep
> getting slower with time at the point of being unusable inside
> emacs+orgmode. I'm curious if there are users with larger orgmode
> files and what their experience is, and open to suggestions on how to
> improve performance.
My largest .org file is 6120 lines, 306741 bytes but I have 51 files
contributing to my agenda for a total of 29488 lines, 1,263,120 bytes
total.
My archive files are much bigger but not normally accessed. This is on
a 32 bit linux system and the only slow thing I have is 5 seconds to
generate my block agenda which includes all of my org files.
Archiving things you don't need is the best way to keep things fast I
think. You can still search the archive from the agenda if necessary.
Regards,
Bernt
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: Largest org file you have + performance
2011-08-01 16:35 Largest org file you have + performance Marcelo de Moraes Serpa
` (3 preceding siblings ...)
2011-08-02 0:56 ` Bernt Hansen
@ 2011-08-04 20:18 ` Milan Zamazal
2011-08-05 2:20 ` jiangzuoyan
4 siblings, 1 reply; 16+ messages in thread
From: Milan Zamazal @ 2011-08-04 20:18 UTC (permalink / raw)
To: emacs-orgmode
My largest org file is an org-drill file of almost 1 MB size, containing
about 32000 lines (most of them are entry properties) and 2000 org-drill
entries. It's well useable but tag and property editing is better to be
done by hand instead of using org commands and I have to use some
folding trick to prevent org-drill performance problems.
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: Largest org file you have + performance
2011-08-04 20:18 ` Milan Zamazal
@ 2011-08-05 2:20 ` jiangzuoyan
2011-08-05 15:31 ` Matt Lundin
0 siblings, 1 reply; 16+ messages in thread
From: jiangzuoyan @ 2011-08-05 2:20 UTC (permalink / raw)
To: Milan Zamazal; +Cc: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 784 bytes --]
My largest org file is QuanSongCi.org, contains ci poetry of Song dynasty,
about 6MB size.
Editing in emacs is very slow, save needs to wait about minutes. convert to
html needs to wait about minutes, too. So, sometimes, sed like tools is
preferred to do editing and converting.
Changsheng Jiang
On Fri, Aug 5, 2011 at 04:18, Milan Zamazal <pdm@zamazal.org> wrote:
> My largest org file is an org-drill file of almost 1 MB size, containing
> about 32000 lines (most of them are entry properties) and 2000 org-drill
> entries. It's well useable but tag and property editing is better to be
> done by hand instead of using org commands and I have to use some
> folding trick to prevent org-drill performance problems.
>
>
>
>
[-- Attachment #2: Type: text/html, Size: 1141 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: Largest org file you have + performance
2011-08-05 2:20 ` jiangzuoyan
@ 2011-08-05 15:31 ` Matt Lundin
2011-08-14 16:50 ` Bastien
0 siblings, 1 reply; 16+ messages in thread
From: Matt Lundin @ 2011-08-05 15:31 UTC (permalink / raw)
To: jiangzuoyan@gmail.com; +Cc: Org Mode
"jiangzuoyan@gmail.com" <jiangzuoyan@gmail.com> writes:
> My largest org file is QuanSongCi.org, contains ci poetry of Song
> dynasty, about 6MB size.
>
> Editing in emacs is very slow, save needs to wait about
> minutes. convert to html needs to wait about minutes, too. So,
> sometimes, sed like tools is preferred to do editing and converting.
Thanks for these details. I can report similar experiences with large files.
As far as I understand, large org-mode buffers are slow because there
are many overlays, which (unlike text properties), don't scale well:
(info "(elisp) Overlays")
This is right now an inescapable limitation of org-mode, since
outline-mode relies on overlays.
There was brief talk on emacs-dev of rewriting text-properties so that
outline-mode could use them instead of overlays, but I do not believe
the proposal was pursued further:
http://permalink.gmane.org/gmane.emacs.devel/131304
Best,
Matt
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Largest org file you have + performance
2011-08-05 15:31 ` Matt Lundin
@ 2011-08-14 16:50 ` Bastien
2012-05-24 12:29 ` SW
0 siblings, 1 reply; 16+ messages in thread
From: Bastien @ 2011-08-14 16:50 UTC (permalink / raw)
To: Matt Lundin; +Cc: Org Mode
Hi Matt,
Matt Lundin <mdl@imapmail.org> writes:
> There was brief talk on emacs-dev of rewriting text-properties so that
> outline-mode could use them instead of overlays, but I do not believe
> the proposal was pursued further:
>
> http://permalink.gmane.org/gmane.emacs.devel/131304
Quite interesting, thanks for the link. I would love to see some
optimization in this area.
--
Bastien
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Largest org file you have + performance
2011-08-14 16:50 ` Bastien
@ 2012-05-24 12:29 ` SW
2012-05-25 22:42 ` Bastien
0 siblings, 1 reply; 16+ messages in thread
From: SW @ 2012-05-24 12:29 UTC (permalink / raw)
To: emacs-orgmode
I'm not familiar enough with Emacs to comment about the performance of overlays,
etc., but I'm surprised that processing *text* can be so CPU intensive.
These days we have games running with millions of pixels and shading or whatever
(some use the GPU of course), and browsers with fancy Flash animation and dozens
of tabs open and spreadsheets with thousands of cells and word processor
documents with hundreds of formatted pages on dual cores and quad cores with GB
of RAM.
Is this performance overhead just a result of poorly-scalable overlays, rather
than an inherent shortcoming of Emacs and/or org-mode?
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Largest org file you have + performance
2012-05-24 12:29 ` SW
@ 2012-05-25 22:42 ` Bastien
0 siblings, 0 replies; 16+ messages in thread
From: Bastien @ 2012-05-25 22:42 UTC (permalink / raw)
To: SW; +Cc: emacs-orgmode
Hi,
SW <sabrewolfy@gmail.com> writes:
> Is this performance overhead just a result of poorly-scalable overlays, rather
> than an inherent shortcoming of Emacs and/or org-mode?
I would say this is a combination of the implementation overlays
and the use of overlays in outline.el.
Perhaps we should dive into outline.el and see how we can improve
things from there. (We tend to forget that Org is just a derived
mode from outline.el!)
--
Bastien
^ permalink raw reply [flat|nested] 16+ messages in thread