From mboxrd@z Thu Jan 1 00:00:00 1970 From: Adam Porter Subject: Re: How to make agenda generation faster Date: Wed, 17 Oct 2018 14:12:08 -0500 Message-ID: <87h8hkqtfb.fsf@alphapapa.net> References: <87h8hy1ho5.fsf@mbork.pl> <87o9c34ocl.fsf@alphapapa.net> <877eirdrqt.fsf@nicolasgoaziou.fr> <87ftx5fx3n.fsf@alphapapa.net> <87pnw8engh.fsf@nicolasgoaziou.fr> Mime-Version: 1.0 Content-Type: text/plain Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:37082) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gCrFM-0004P3-CJ for emacs-orgmode@gnu.org; Wed, 17 Oct 2018 15:12:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gCrFH-0005fl-V3 for emacs-orgmode@gnu.org; Wed, 17 Oct 2018 15:12:44 -0400 Received: from [195.159.176.226] (port=52143 helo=blaine.gmane.org) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gCrFG-0005LP-UF for emacs-orgmode@gnu.org; Wed, 17 Oct 2018 15:12:39 -0400 Received: from list by blaine.gmane.org with local (Exim 4.84_2) (envelope-from ) id 1gCrCn-0004OD-8e for emacs-orgmode@gnu.org; Wed, 17 Oct 2018 21:10:05 +0200 List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: "Emacs-orgmode" To: emacs-orgmode@gnu.org Nicolas Goaziou writes: > It can, but AFAIK, it doesn't yet. It also means un-optimized lexical > binding may be slightly slower than dynamic scoping for the time > being. Well, I can't vouch for it myself, because I haven't studied the code. But here's one of the resources that suggests it is faster to use lexical binding: https://emacs.stackexchange.com/questions/2129/why-is-let-faster-with-lexical-scope > That's not exactly what I'm suggesting. I suggest to move the work in > Org tree, e.g., as an org-agenda-ng.el library, and, from there, > implement back most of the features of the current agenda. > > Org cannot really benefit from libraries living outside Emacs, as we > recently learnt with htmlize issue. Org is welcome to take any of the org-ql or org-ql-agenda code you think would be useful. However, before it could be suitable as a possible replacement, it will likely require more optimization. Some queries, especially more complex ones, are slower than the equivalent searches and agendas in the current Org Agenda code. This is because of the way the queries run predicates on each heading. Despite the current Org Agenda code's complexity, it is well optimized and hard to beat. I have a proof-of-concept branch that begins to implement a relatively simple optimization that converts one suitable predicate in a query to a buffer-global regexp search. It significantly improves speed in some cases, but a query with several predicates still has to run all but one of them as predicates. Another possible optimization would be to convert as many predicates in a query to buffer regexp searches as possible, collecting a list of heading positions in the buffer, and then do a final pass with the appropriate union/intersection/difference operations on the lists. Then the list of positions could be used to gather the heading data. I use a similar technique in helm-org-rifle, and it seems to work quickly. It would require some work on a sort of "query compiler" to do the transformation and optimization. I don't have much experience with that kind of programming; maybe someone else would be interested in helping with that. So before taking any of the code into Org itself, you might want to consider these issues and decide whether it could be a suitable approach. Let me know what you'd like to do and how I can help. Thanks, Adam