From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ihor Radchenko Subject: Re: Asynchronous org-agenda-redo Date: Thu, 12 Dec 2019 23:46:12 +0800 Message-ID: <87o8wda6nv.fsf@yantar92-laptop.i-did-not-set--mail-host-address--so-tickle-me> References: <87k172ot2m.fsf@yantar92-laptop.i-did-not-set--mail-host-address--so-tickle-me> <87pngtsppt.fsf@alphapapa.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:45803) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ifQhH-0001oM-B8 for emacs-orgmode@gnu.org; Thu, 12 Dec 2019 10:48:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ifQhF-0001mP-MV for emacs-orgmode@gnu.org; Thu, 12 Dec 2019 10:48:11 -0500 Received: from mail-wr1-x42f.google.com ([2a00:1450:4864:20::42f]:41236) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ifQhF-0001ix-Cw for emacs-orgmode@gnu.org; Thu, 12 Dec 2019 10:48:09 -0500 Received: by mail-wr1-x42f.google.com with SMTP id c9so3240567wrw.8 for ; Thu, 12 Dec 2019 07:48:09 -0800 (PST) In-Reply-To: <87pngtsppt.fsf@alphapapa.net> List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: "Emacs-orgmode" To: Adam Porter , emacs-orgmode@gnu.org > Be sure to read the Emacs Lisp manual regarding threads. They are > cooperative, so functions called as threads must yield back to the main > thread for Emacs to do anything else before the function returns. I tried to read the manual, but I clearly misunderstand something. The manual says: > Currently, thread switching will occur upon explicit request via > =E2=80=98thread-yield=E2=80=99, when waiting for keyboard input...=20 So, except directly calling thread-yield, it should be possible to trigger switching the current thread when keyboard input is expected. I tried the following demo code: (defun test () (let ((a 0)) (dotimes (_ 5) (setq a (1+ a)) (sleep-for 2) (message "%s" a)))) (progn ;This should return to command loop quickly (make-thread #'test) (message "Executed...")); `eval-last-sexp' here I can move around the buffer while the progn is running. However, it is not the case with `org-agenda-redo' for a reason I do not fully understand.=20 For the async.el, I agree that loading packages may take time, but I believe that the configuration might be transferred more easily (though it may depend on the org-agenda and user-defined org-agenda-skip-functions implementation). > 1. The process would have to load the same Org buffers, which takes > time, especially in large buffers. Depending on configuration, it > can take some time, indeed. > 3. Ensuring that configuration and state between the main Emacs process > and the separate, agenda-generating process is not necessarily > simple. Consider as well that if a buffer had unsaved changes, > those would not be readable by the other process, which would lead > to invalid results. One could force the buffers to be saved first, > but that may not always be desirable, as saving buffers can have > side effects. Why cannot org-buffer simply be copied into the subordinate process? If all be buffer-locals, text properties, and overlays are copied directly from the main emacs process, there may be no need to even initialise org-mode (the idea is to do something similar to clone-buffer). The question though is whether buffer-locals + overlays + propertized .org files text + org-agenda-buffer copy can be sufficient to make the org-agenda-redo run properly. Are there any other buffers, variables, or other environment settings used by org-agenda-redo? > If your agenda buffers are taking too long to refresh, you might > consider org-ql's views/saved-searches as an alternative. ... I know org-ql and I am pretty sure that it will improve performance. Actually, if one can make built-in org-agenda asynchronous, org-ql can probably use similar approach and become even faster :) I am trying on default org-agenda now mostly because my current config is heavily geared towards default agenda and I am not sure if refactoring everything to use org-ql will worth it at the end in terms of performance. I use too many slow custom skip-functions. > ... The built-in > caching in org-ql significantly improves performance, especially when > refreshing views. Yeah. I wish org-entry-get and other org-get* functions support caching as well... Or, at least, org-agenda functions might also support simple caching based on file modifications. Best, Ihor Adam Porter writes: > Be sure to read the Emacs Lisp manual regarding threads. They are > cooperative, so functions called as threads must yield back to the main > thread for Emacs to do anything else before the function returns. > > If you're feeling adventurous, you could experiment with adding yields > in relevant agenda functions. But that wouldn't be suitable for merging > into Org, because that yielding also decreases performance generally. > > As long as Elisp threads are cooperative, they are of very limited use. > > Generating agendas with async.el in a separate Emacs process is an > interesting idea, but probably generally impractical for a few reasons: > > 1. The process would have to load the same Org buffers, which takes > time, especially in large buffers. Depending on configuration, it > can take some time, indeed. > 2. The process would also have to load the same packages (or, at least, > all the necessary ones, which depends on configuration), which takes > time. > 3. Ensuring that configuration and state between the main Emacs process > and the separate, agenda-generating process is not necessarily > simple. Consider as well that if a buffer had unsaved changes, > those would not be readable by the other process, which would lead > to invalid results. One could force the buffers to be saved first, > but that may not always be desirable, as saving buffers can have > side effects. > > If your agenda buffers are taking too long to refresh, you might > consider org-ql's views/saved-searches as an alternative. The built-in > caching in org-ql significantly improves performance, especially when > refreshing views. > > --=20 Ihor Radchenko, PhD, Center for Advancing Materials Performance from the Nanoscale (CAMP-nano) State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong U= niversity, Xi'an, China Email: yantar92@gmail.com, ihor_radchenko@alumni.sutd.edu.sg