From: Nicolas Goaziou <mail@nicolasgoaziou.fr>
To: Adam Porter <adam@alphapapa.net>
Cc: emacs-orgmode@gnu.org
Subject: Re: How to make agenda generation faster
Date: Sat, 20 Oct 2018 10:12:41 +0200 [thread overview]
Message-ID: <87bm7phw92.fsf@nicolasgoaziou.fr> (raw)
In-Reply-To: <CAO_b3FUxu8bm7ChoZ49-NX2rZYDr7TwfQN5NQNp8mb=i_B_hZA@mail.gmail.com> (Adam Porter's message of "Fri, 19 Oct 2018 21:12:01 -0500")
Hello,
Adam Porter <adam@alphapapa.net> writes:
> Yes, because this is the fastest way to search for matching entries in a
> buffer, when it's possible to use a regexp search.
You would still do regexp searches, but not at the time of queries.
> That would be ideal. The problem I foresee is that, when a buffer's cache
> is not up-to-date, and the user runs an agenda query, the user will have to
> wait for the buffer to be parsed and cached, which is much slower than a
> regexp search through the buffer.
No, because filling cache is still a regexp search.
> That was what I first tried with org-agenda-ng: I parsed the whole buffer
> with org-element and ran predicates against the element tree.
Org Element is not needed, and even shouldn't be used, to retrieve most
agenda related data.
There are exceptions of course, mainly plain timestamps and clocks. This
is where the current agenda is hard to beat, because 1. it cheats and
includes timestamps without checking context, 2. it only searches for
timestamps related to the day being displayed in the agenda view. The
last point makes it particularly fast for single day views.
> Another idea I've had, similar to yours, would be to pre-process buffers,
> adding metadata as text-properties on heading lines. However, I haven't
> tested it, and I don't know what the performance would be like. And it
> would still suffer from the caching problem I mentioned.
It is still a way to cache stuff. The difficulty here is to keep data
up-to-date with changes. Storing per-node cache could be nice,
nevertheless.
> I think the fundamental problems are 1) keeping the cache in sync with the
> raw buffer,
Yes, whole buffer caching is simpler here: drop all cached data if
buffer contents differ from the cached one. That's what I did in may
last attempt to speed up agenda, comparing md5sums. It works reasonably
well.
I also cached per agenda data type (schedules, deadlines, clocks…) but
that means you know something about the query. I think querying and
searching should be separated should it shouldn't be done.
> and 2) the slow speed of parsing an entire buffer's metadata at
> once (depending on the size of the files, of course, but mine are big
> enough to be slow, and I'm sure many users have larger ones).
I think this could be solved by fetching data preemptively during idle
time. I would also work well with per-node caching, since you can
interrupt fetching easily.
Regards,
--
Nicolas Goaziou
next prev parent reply other threads:[~2018-10-20 8:12 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-10-07 4:53 How to make agenda generation faster Marcin Borkowski
2018-10-08 7:20 ` Michael Welle
2018-10-10 20:03 ` Marcin Borkowski
2018-10-10 21:01 ` Samuel Wales
2018-10-11 6:48 ` Michael Welle
2018-10-11 8:48 ` Marcin Borkowski
2018-10-11 19:59 ` Samuel Wales
2018-10-14 8:51 ` Marcin Borkowski
2018-10-09 6:37 ` Adam Porter
2018-10-09 16:11 ` Nicolas Goaziou
2018-10-10 20:01 ` Marcin Borkowski
2018-10-16 20:35 ` Adam Porter
2018-10-17 7:04 ` Ihor Radchenko
2018-10-17 13:01 ` Nicolas Goaziou
2018-10-17 19:12 ` Adam Porter
2018-10-18 22:48 ` Nicolas Goaziou
2018-10-19 0:04 ` stardiviner
2018-10-20 2:12 ` Adam Porter
2018-10-20 8:12 ` Nicolas Goaziou [this message]
2018-10-10 19:59 ` Marcin Borkowski
2018-10-09 11:47 ` Julius Dittmar
2018-10-10 20:03 ` Marcin Borkowski
2018-10-11 6:40 ` Michael Welle
2018-10-14 7:42 ` Marcin Borkowski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87bm7phw92.fsf@nicolasgoaziou.fr \
--to=mail@nicolasgoaziou.fr \
--cc=adam@alphapapa.net \
--cc=emacs-orgmode@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).