emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* Org element cache in indirect buffers
@ 2023-06-11 12:13 Sebastian Wålinder
  2023-06-11 12:51 ` Ihor Radchenko
  0 siblings, 1 reply; 4+ messages in thread
From: Sebastian Wålinder @ 2023-06-11 12:13 UTC (permalink / raw)
  To: emacs-orgmode

Hello!

It appears that indirect buffers duplicate the entire org element cache from the base buffer whenever created.

Running `memory-report` indicates that each indirect buffer has the same memory blueprint.

This becomes a massive issue with large org files.

My in my 160 000 line org file, the org element cache takes 100 mb, and so each new indirect buffer consumes an additional 100 mbs of memory.

I also wonder if this means the caching is performed by each indirect buffer independently, and doesn't carry over.

Thoughts?

Thanks,
Sebastian


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Org element cache in indirect buffers
  2023-06-11 12:13 Org element cache in indirect buffers Sebastian Wålinder
@ 2023-06-11 12:51 ` Ihor Radchenko
  2023-06-13 10:33   ` Ihor Radchenko
  0 siblings, 1 reply; 4+ messages in thread
From: Ihor Radchenko @ 2023-06-11 12:51 UTC (permalink / raw)
  To: Sebastian Wålinder; +Cc: emacs-orgmode

Sebastian Wålinder <s.walinder@gmail.com> writes:

> It appears that indirect buffers duplicate the entire org element cache from the base buffer whenever created.

Not exactly. Only link to the cache root is copied and some cache
elements may not be garbage-collected.

> Running `memory-report` indicates that each indirect buffer has the same memory blueprint.

> This becomes a massive issue with large org files.
>
> My in my 160 000 line org file, the org element cache takes 100 mb, and so each new indirect buffer consumes an additional 100 mbs of memory.

This has to be a bug in `memory-report'. Most of the cache elements will
be referenced by link and thus there should be double-counting going on somewhere.

> I also wonder if this means the caching is performed by each indirect buffer independently, and doesn't carry over.

Caching is using base buffer actually.
I guess we can explicitly set reference to the cache root to nil in
indirect buffers.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Org element cache in indirect buffers
  2023-06-11 12:51 ` Ihor Radchenko
@ 2023-06-13 10:33   ` Ihor Radchenko
       [not found]     ` <CAMBKESWeHCneRwHPWkW_4-xym7OTwPEc65Z-C2WJdjcDFYwocg@mail.gmail.com>
  0 siblings, 1 reply; 4+ messages in thread
From: Ihor Radchenko @ 2023-06-13 10:33 UTC (permalink / raw)
  To: Sebastian Wålinder; +Cc: emacs-orgmode

Ihor Radchenko <yantar92@posteo.net> writes:

>> I also wonder if this means the caching is performed by each indirect buffer independently, and doesn't carry over.
>
> Caching is using base buffer actually.
> I guess we can explicitly set reference to the cache root to nil in
> indirect buffers.

Done, on main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=2f7b35ac8
But I would not expect much actual gain from this. Maybe in some edge cases.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Org element cache in indirect buffers
       [not found]     ` <CAMBKESWeHCneRwHPWkW_4-xym7OTwPEc65Z-C2WJdjcDFYwocg@mail.gmail.com>
@ 2023-06-13 18:02       ` Ihor Radchenko
  0 siblings, 0 replies; 4+ messages in thread
From: Ihor Radchenko @ 2023-06-13 18:02 UTC (permalink / raw)
  To: Sebastian; +Cc: emacs-orgmode

[Adding Org ML back to CC]

Sebastian <s.walinder@gmail.com> writes:

> Thank you so much for the detailed explanation!
>
> Maybe memory-report is counting what's at the end of the reference, and
> your change fixes it.

memory-report is counting buffer data separately, even when buffer-local
variables reference to the same data objects.
I just reported this upstream: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=64050

My change simply clears the references to AVL-tree roots that are copied
over when Emacs clones the buffer. The edge cases I was talking about
are when the main AVL-tree changes its root. That would make indirect
buffer actually keep reference to obsolete root and the rest of the
obsolete AVL-tree, preventing it from being garbage-collected.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2023-06-13 20:15 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-06-11 12:13 Org element cache in indirect buffers Sebastian Wålinder
2023-06-11 12:51 ` Ihor Radchenko
2023-06-13 10:33   ` Ihor Radchenko
     [not found]     ` <CAMBKESWeHCneRwHPWkW_4-xym7OTwPEc65Z-C2WJdjcDFYwocg@mail.gmail.com>
2023-06-13 18:02       ` Ihor Radchenko

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).