From: Adam Porter <adam@alphapapa.net>
Cc: emacs-orgmode@gnu.org
Subject: Re: How to make agenda generation faster
Date: Fri, 19 Oct 2018 21:12:01 -0500 [thread overview]
Message-ID: <CAO_b3FUxu8bm7ChoZ49-NX2rZYDr7TwfQN5NQNp8mb=i_B_hZA@mail.gmail.com> (raw)
In-Reply-To: <87r2gm6fdp.fsf@nicolasgoaziou.fr>
[-- Attachment #1: Type: text/plain, Size: 2076 bytes --]
On Oct 18, 2018 5:48 PM, "Nicolas Goaziou" <mail@nicolasgoaziou.fr> wrote:
> Are you saying that queries are turned into regexp searches within Org
files? If so, I don't think they should.
Yes, because this is the fastest way to search for matching entries in a
buffer, when it's possible to use a regexp search.
> Queries should only operate on the output of the data extraction,
possibly a list of defstructs. I.e., you first extract all meaningful data
from the document (during idle time, with cache, or whatever optimization
would be chosen), store it in an appropriate format, then query it.
>
> WDYT?
That would be ideal. The problem I foresee is that, when a buffer's cache
is not up-to-date, and the user runs an agenda query, the user will have to
wait for the buffer to be parsed and cached, which is much slower than a
regexp search through the buffer.
That was what I first tried with org-agenda-ng: I parsed the whole buffer
with org-element and ran predicates against the element tree. It was much
too slow to be practical, so I switched to the current approach, which runs
predicates against each node, only checking the necessary metadata. It's
fast enough to be useful, but can still be slow in some cases, and I don't
think it would be fast enough as a replacement for the current agenda
code. But with further optimization, like using whole-buffer regexp
searches when possible, it might be.
Another idea I've had, similar to yours, would be to pre-process buffers,
adding metadata as text-properties on heading lines. However, I haven't
tested it, and I don't know what the performance would be like. And it
would still suffer from the caching problem I mentioned.
I think the fundamental problems are 1) keeping the cache in sync with the
raw buffer, and 2) the slow speed of parsing an entire buffer's metadata at
once (depending on the size of the files, of course, but mine are big
enough to be slow, and I'm sure many users have larger ones).
Of course, maybe someone cleverer than me can figure out a clever solution
to these problems. :)
[-- Attachment #2: Type: text/html, Size: 2358 bytes --]
next prev parent reply other threads:[~2018-10-20 2:12 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-10-07 4:53 How to make agenda generation faster Marcin Borkowski
2018-10-08 7:20 ` Michael Welle
2018-10-10 20:03 ` Marcin Borkowski
2018-10-10 21:01 ` Samuel Wales
2018-10-11 6:48 ` Michael Welle
2018-10-11 8:48 ` Marcin Borkowski
2018-10-11 19:59 ` Samuel Wales
2018-10-14 8:51 ` Marcin Borkowski
2018-10-09 6:37 ` Adam Porter
2018-10-09 16:11 ` Nicolas Goaziou
2018-10-10 20:01 ` Marcin Borkowski
2018-10-16 20:35 ` Adam Porter
2018-10-17 7:04 ` Ihor Radchenko
2018-10-17 13:01 ` Nicolas Goaziou
2018-10-17 19:12 ` Adam Porter
2018-10-18 22:48 ` Nicolas Goaziou
2018-10-19 0:04 ` stardiviner
2018-10-20 2:12 ` Adam Porter [this message]
2018-10-20 8:12 ` Nicolas Goaziou
2018-10-10 19:59 ` Marcin Borkowski
2018-10-09 11:47 ` Julius Dittmar
2018-10-10 20:03 ` Marcin Borkowski
2018-10-11 6:40 ` Michael Welle
2018-10-14 7:42 ` Marcin Borkowski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAO_b3FUxu8bm7ChoZ49-NX2rZYDr7TwfQN5NQNp8mb=i_B_hZA@mail.gmail.com' \
--to=adam@alphapapa.net \
--cc=emacs-orgmode@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).