From: Adam Porter <adam@alphapapa.net>
To: emacs-orgmode@gnu.org
Subject: Re: How to make agenda generation faster
Date: Tue, 16 Oct 2018 15:35:56 -0500 [thread overview]
Message-ID: <87ftx5fx3n.fsf@alphapapa.net> (raw)
In-Reply-To: 877eirdrqt.fsf@nicolasgoaziou.fr
Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>> my understanding is that code that runs with lexical-binding enabled
>> is generally faster.
>
> Not really. But it's certainly easier to understand since it removes one
> class of problems.
From what I've read, the byte-compiler can optimize better when
lexical-binding is used.
> Instead of re-inventing the wheel, or putting efforts into a
> wheel-like invention, wouldn't it make sense to actually work on Org
> Agenda itself?
>
> So again, wouldn't it be nice to think about Org Agenda-ng?
As a matter of fact, what's now called org-ql-agenda was originally
called org-agenda-ng. I factored org-ql out of it and realized that it
should probably be its own, standalone package. Then I renamed
org-agenda-ng to org-ql-agenda, so I could reasonably keep them in the
same repo, and because I don't know if I will ever develop it far enough
to be worthy of the name org-agenda-ng. It started as an experiment to
build a foundation for a new, modular agenda implementation, and maybe
it could be.
> I didn't look closely at org-ql, but I had the idea of splitting the
> Agenda in two distinct parts. One would be responsible for collecting,
> possibly asynchronously, and caching data from Org documents. The other
> one would provide a DSL to query and display the results extracted from
> the output of the first part. The second part could even be made generic
> enough to be extracted from Org and become some part of Emacs.
> Displaying filtered data, maybe in a timeline, could be useful for other
> packages. Unfortunately, I don't have time to work on this. Ah well.
I've thought about this for a while. It seems to me that the issue is
that Org buffers are, of course, plain-text buffers. There is no
persistent, in-memory representation other than the buffer, so whenever
Org needs structured/semantic data, it must parse it out of the buffer,
which is necessarily rather slow. If there were a way to keep an
outline tree in memory, parallel to the buffer itself, that would allow
operations like search, agenda, etc. to be greatly sped up.
But how would that work in Emacs? Theoretically, we could write some
code, applied on self-insert-command, to update the "parallel tree
structure" as the user manipulates the plain-text in the buffer
(e.g. add a new node when the user types a "*" to create a new heading),
and also apply it to functions that manipulate the outline structurally
in the buffer. But, of course, that sounds very complicated. I would
not relish the idea of debugging code to keep a cached tree in sync with
a plain-text buffer outline. :)
Besides that, AFAIK there would be no way to do it asynchronously other
than calling out to a child Emacs process (because elisp is still
single-threaded), printing and reading the data back and forth (which
would tie up the parent process when reading). Maybe in the future
elisp will be multithreaded...
Anyway, org-ql tries to do some of what you mentioned. It does
rudimentary, per-buffer, per-query caching (as long as the buffer is not
modified, the cache remains valid), which helps when there are several
Org files open that are referred to often but not as often modified.
And the query and presentation code are separated (org-ql and
org-ql-agenda).
I don't know how widely it's used, but the repo is getting some regular
traffic, and I'm using it as the backend for my org-sidebar package.
I'd be happy if it could be made more generally useful, or if it could
be helpful to Org itself in some way. Contributions are welcome.
next prev parent reply other threads:[~2018-10-16 20:36 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-10-07 4:53 How to make agenda generation faster Marcin Borkowski
2018-10-08 7:20 ` Michael Welle
2018-10-10 20:03 ` Marcin Borkowski
2018-10-10 21:01 ` Samuel Wales
2018-10-11 6:48 ` Michael Welle
2018-10-11 8:48 ` Marcin Borkowski
2018-10-11 19:59 ` Samuel Wales
2018-10-14 8:51 ` Marcin Borkowski
2018-10-09 6:37 ` Adam Porter
2018-10-09 16:11 ` Nicolas Goaziou
2018-10-10 20:01 ` Marcin Borkowski
2018-10-16 20:35 ` Adam Porter [this message]
2018-10-17 7:04 ` Ihor Radchenko
2018-10-17 13:01 ` Nicolas Goaziou
2018-10-17 19:12 ` Adam Porter
2018-10-18 22:48 ` Nicolas Goaziou
2018-10-19 0:04 ` stardiviner
2018-10-20 2:12 ` Adam Porter
2018-10-20 8:12 ` Nicolas Goaziou
2018-10-10 19:59 ` Marcin Borkowski
2018-10-09 11:47 ` Julius Dittmar
2018-10-10 20:03 ` Marcin Borkowski
2018-10-11 6:40 ` Michael Welle
2018-10-14 7:42 ` Marcin Borkowski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87ftx5fx3n.fsf@alphapapa.net \
--to=adam@alphapapa.net \
--cc=emacs-orgmode@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).