* Is there a size limit to org files?
@ 2023-09-08 16:26 Carlo Tambuatco
2023-09-08 16:52 ` Ihor Radchenko
2023-09-08 17:29 ` Marcin Borkowski
0 siblings, 2 replies; 9+ messages in thread
From: Carlo Tambuatco @ 2023-09-08 16:26 UTC (permalink / raw)
To: Org-Mode Mailing List
[-- Attachment #1: Type: text/plain, Size: 1016 bytes --]
I noticed that an org file I was using for my capture templates was getting
quite large
and that my capture templates that were pasting TODO entries in that file
were starting to exhibit
strange behavior. Mostly just getting stuck at a mysterious "clipboard
pasted to level 2 subtree"
message, then it would just hang there until I hit ctrl g.
Since this file was getting quite big and unwieldy, > 5000 lines, I decided
to split it up, so that
the various entries under their headers would be split up into different
files. I basically just
put each header into its own separate file, and updated my
org-capture-templates accordingly.
Since then, no strange behavior, no stalling and hanging for no apparent
reason.
So, I was wondering if there is hardcoded somewhere in org some kind of
size limit to the
files it has to deal with, or is this some limitation arising naturally
from the size of
the clipboard? I'm not sure how org handles large files when considering
how to paste
entries from capture templates.
[-- Attachment #2: Type: text/html, Size: 1224 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Is there a size limit to org files?
2023-09-08 16:26 Is there a size limit to org files? Carlo Tambuatco
@ 2023-09-08 16:52 ` Ihor Radchenko
2023-09-08 17:10 ` Carlo Tambuatco
2023-09-09 3:35 ` Max Nikulin
2023-09-08 17:29 ` Marcin Borkowski
1 sibling, 2 replies; 9+ messages in thread
From: Ihor Radchenko @ 2023-09-08 16:52 UTC (permalink / raw)
To: Carlo Tambuatco; +Cc: Org-Mode Mailing List
Carlo Tambuatco <oraclmaster@gmail.com> writes:
> I noticed that an org file I was using for my capture templates was getting
> quite large
> and that my capture templates that were pasting TODO entries in that file
> were starting to exhibit
> strange behavior. Mostly just getting stuck at a mysterious "clipboard
> pasted to level 2 subtree"
> message, then it would just hang there until I hit ctrl g.
That could be a bug, but it is impossible to fix it without knowing more
details.
> Since this file was getting quite big and unwieldy, > 5000 lines, I
> decided ...
I am using 100x (yes, 100 times) larger org file right now. With 500k lines.
> So, I was wondering if there is hardcoded somewhere in org some kind of
> size limit to the
> files it has to deal with, or is this some limitation arising naturally
> from the size of
> the clipboard? I'm not sure how org handles large files when considering
> how to paste
> entries from capture templates.
There is no size limit on Org files. Although certain operations may
become slower when the file size reaches tens of megabytes.
Clipboard should normally not be a problem either.
If you can interrupt with C-g, there might be some infinite loop
somewhere in the code.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Is there a size limit to org files?
2023-09-08 16:52 ` Ihor Radchenko
@ 2023-09-08 17:10 ` Carlo Tambuatco
2023-09-08 17:14 ` Ihor Radchenko
2023-09-09 3:35 ` Max Nikulin
1 sibling, 1 reply; 9+ messages in thread
From: Carlo Tambuatco @ 2023-09-08 17:10 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: Org-Mode Mailing List
[-- Attachment #1: Type: text/plain, Size: 2047 bytes --]
You might want to look into what changed between org 9.5.5 and 9.6, because
that is when this strange behavior started (for me anyway). I've been
hanging onto
9.5.5 for as long as I could until I upgraded to emacs 29. Then the above
mentioned
strange behavior resumed. My fix for it was to split up my large org files
into
smaller files, and so far no strange behavior.
On Fri, Sep 8, 2023 at 12:51 PM Ihor Radchenko <yantar92@posteo.net> wrote:
> Carlo Tambuatco <oraclmaster@gmail.com> writes:
>
> > I noticed that an org file I was using for my capture templates was
> getting
> > quite large
> > and that my capture templates that were pasting TODO entries in that file
> > were starting to exhibit
> > strange behavior. Mostly just getting stuck at a mysterious "clipboard
> > pasted to level 2 subtree"
> > message, then it would just hang there until I hit ctrl g.
>
> That could be a bug, but it is impossible to fix it without knowing more
> details.
>
> > Since this file was getting quite big and unwieldy, > 5000 lines, I
> > decided ...
>
> I am using 100x (yes, 100 times) larger org file right now. With 500k
> lines.
>
>
> > So, I was wondering if there is hardcoded somewhere in org some kind of
> > size limit to the
> > files it has to deal with, or is this some limitation arising naturally
> > from the size of
> > the clipboard? I'm not sure how org handles large files when considering
> > how to paste
> > entries from capture templates.
>
> There is no size limit on Org files. Although certain operations may
> become slower when the file size reaches tens of megabytes.
>
> Clipboard should normally not be a problem either.
>
> If you can interrupt with C-g, there might be some infinite loop
> somewhere in the code.
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>
>
[-- Attachment #2: Type: text/html, Size: 2851 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Is there a size limit to org files?
2023-09-08 17:10 ` Carlo Tambuatco
@ 2023-09-08 17:14 ` Ihor Radchenko
0 siblings, 0 replies; 9+ messages in thread
From: Ihor Radchenko @ 2023-09-08 17:14 UTC (permalink / raw)
To: Carlo Tambuatco; +Cc: Org-Mode Mailing List
Carlo Tambuatco <oraclmaster@gmail.com> writes:
> You might want to look into what changed between org 9.5.5 and 9.6, because
> that is when this strange behavior started (for me anyway)...
Unfortunately, such information is not helpful. A lot have changed.
If you observe the problem in future, I encourage you to follow the
steps described in https://orgmode.org/manual/Feedback.html#Feedback, so
that we can identify and fix the bug.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Is there a size limit to org files?
2023-09-08 16:26 Is there a size limit to org files? Carlo Tambuatco
2023-09-08 16:52 ` Ihor Radchenko
@ 2023-09-08 17:29 ` Marcin Borkowski
2023-09-08 17:37 ` Ihor Radchenko
1 sibling, 1 reply; 9+ messages in thread
From: Marcin Borkowski @ 2023-09-08 17:29 UTC (permalink / raw)
To: Carlo Tambuatco; +Cc: Org-Mode Mailing List
On 2023-09-08, at 18:26, Carlo Tambuatco <oraclmaster@gmail.com> wrote:
> I noticed that an org file I was using for my capture templates was getting
> quite large
> and that my capture templates that were pasting TODO entries in that file
> were starting to exhibit
> strange behavior. Mostly just getting stuck at a mysterious "clipboard
> pasted to level 2 subtree"
> message, then it would just hang there until I hit ctrl g.
>
> Since this file was getting quite big and unwieldy, > 5000 lines, I decided
> to split it up, so that
> the various entries under their headers would be split up into different
> files. I basically just
> put each header into its own separate file, and updated my
> org-capture-templates accordingly.
>
> Since then, no strange behavior, no stalling and hanging for no apparent
> reason.
>
> So, I was wondering if there is hardcoded somewhere in org some kind of
> size limit to the
> files it has to deal with, or is this some limitation arising naturally
> from the size of
> the clipboard? I'm not sure how org handles large files when considering
> how to paste
> entries from capture templates.
It's a /very/ long shot, but I noticed that editing large tables (1000+
lines) can become pretty slow. Apart from that, I have Org files with
tens of thousands lines and they work pretty well.
Best,
--
Marcin Borkowski
http://mbork.pl
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Is there a size limit to org files?
2023-09-08 17:29 ` Marcin Borkowski
@ 2023-09-08 17:37 ` Ihor Radchenko
2023-09-08 18:37 ` Marcin Borkowski
0 siblings, 1 reply; 9+ messages in thread
From: Ihor Radchenko @ 2023-09-08 17:37 UTC (permalink / raw)
To: Marcin Borkowski; +Cc: Carlo Tambuatco, Org-Mode Mailing List
Marcin Borkowski <mbork@mbork.pl> writes:
> It's a /very/ long shot, but I noticed that editing large tables (1000+
> lines) can become pretty slow. Apart from that, I have Org files with
> tens of thousands lines and they work pretty well.
Expected. The current code re-inserts the whole table every time you
need to re-align it. You can guess how such approach scales.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Is there a size limit to org files?
2023-09-08 17:37 ` Ihor Radchenko
@ 2023-09-08 18:37 ` Marcin Borkowski
2023-09-08 18:53 ` Ihor Radchenko
0 siblings, 1 reply; 9+ messages in thread
From: Marcin Borkowski @ 2023-09-08 18:37 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: Carlo Tambuatco, Org-Mode Mailing List
On 2023-09-08, at 19:37, Ihor Radchenko <yantar92@posteo.net> wrote:
> Marcin Borkowski <mbork@mbork.pl> writes:
>
>> It's a /very/ long shot, but I noticed that editing large tables (1000+
>> lines) can become pretty slow. Apart from that, I have Org files with
>> tens of thousands lines and they work pretty well.
>
> Expected. The current code re-inserts the whole table every time you
> need to re-align it. You can guess how such approach scales.
Agreed, just wanted to point out one possible reason for slowness.
Another one, it seems, is clocking in when the entry already has a lot
of clock entries. I have a headline with 8000+ clock entries and
starting another one takes about half a second.
Best,
--
Marcin Borkowski
http://mbork.pl
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Is there a size limit to org files?
2023-09-08 18:37 ` Marcin Borkowski
@ 2023-09-08 18:53 ` Ihor Radchenko
0 siblings, 0 replies; 9+ messages in thread
From: Ihor Radchenko @ 2023-09-08 18:53 UTC (permalink / raw)
To: Marcin Borkowski; +Cc: Carlo Tambuatco, Org-Mode Mailing List
Marcin Borkowski <mbork@mbork.pl> writes:
> Another one, it seems, is clocking in when the entry already has a lot
> of clock entries. I have a headline with 8000+ clock entries and
> starting another one takes about half a second.
Yup. The list parser is not incremental (for historic reasons), so every
time you change a list, it is re-parsed from scratch.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Is there a size limit to org files?
2023-09-08 16:52 ` Ihor Radchenko
2023-09-08 17:10 ` Carlo Tambuatco
@ 2023-09-09 3:35 ` Max Nikulin
1 sibling, 0 replies; 9+ messages in thread
From: Max Nikulin @ 2023-09-09 3:35 UTC (permalink / raw)
To: Ihor Radchenko, Carlo Tambuatco; +Cc: Org-Mode Mailing List
On 08/09/2023 23:52, Ihor Radchenko wrote:
> Carlo Tambuatco writes:
>
>> I noticed that an org file I was using for my capture templates was getting
>> quite large
>> and that my capture templates that were pasting TODO entries in that file
>> were starting to exhibit
>> strange behavior. Mostly just getting stuck at a mysterious "clipboard
>> pasted to level 2 subtree"
>> message, then it would just hang there until I hit ctrl g.
>
> That could be a bug, but it is impossible to fix it without knowing more
> details.
Looks like a follow-up of
https://list.orgmode.org/CAJb92ExdvB7FMkSnmZAFTs4AuSZHBp5Rj4bK-4EWAGAbAGgC=g@mail.gmail.com/
Carlo Tambuatco. [BUG] Org-9.6.x org templates hang on 'clipboard pasted
as level 2 subtree'... Mon, 3 Apr 2023 15:53:22 -0400
Certainly a way to reproduce the issue or detailed enough stack traces
are required.
Have you tried to run `org-lint'?
I have seen a couple of times emacs consuming 100% CPU and not reacting
to C-g, but in that cases I was not ready to debug these issues. Almost
certainly it was unrelated to capture a clipboard, more likely
incremental search or a wrong key while calling it.
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2023-09-09 4:01 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-09-08 16:26 Is there a size limit to org files? Carlo Tambuatco
2023-09-08 16:52 ` Ihor Radchenko
2023-09-08 17:10 ` Carlo Tambuatco
2023-09-08 17:14 ` Ihor Radchenko
2023-09-09 3:35 ` Max Nikulin
2023-09-08 17:29 ` Marcin Borkowski
2023-09-08 17:37 ` Ihor Radchenko
2023-09-08 18:37 ` Marcin Borkowski
2023-09-08 18:53 ` Ihor Radchenko
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).