From mboxrd@z Thu Jan 1 00:00:00 1970 From: Adam Porter Subject: Re: Asynchronous org-agenda-redo Date: Fri, 13 Dec 2019 00:49:46 -0600 Message-ID: <87h824sos5.fsf@alphapapa.net> References: <87k172ot2m.fsf@yantar92-laptop.i-did-not-set--mail-host-address--so-tickle-me> <87pngtsppt.fsf@alphapapa.net> <87o8wda6nv.fsf@yantar92-laptop.i-did-not-set--mail-host-address--so-tickle-me> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:58047) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ifem8-0001gs-RQ for emacs-orgmode@gnu.org; Fri, 13 Dec 2019 01:50:10 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ifem7-0000Ei-Hn for emacs-orgmode@gnu.org; Fri, 13 Dec 2019 01:50:08 -0500 Received: from 195-159-176-226.customer.powertech.no ([195.159.176.226]:40354 helo=blaine.gmane.org) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ifem7-0008P9-AJ for emacs-orgmode@gnu.org; Fri, 13 Dec 2019 01:50:07 -0500 Received: from list by blaine.gmane.org with local (Exim 4.89) (envelope-from ) id 1ifely-000dWv-EF for emacs-orgmode@gnu.org; Fri, 13 Dec 2019 07:49:58 +0100 List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: "Emacs-orgmode" To: emacs-orgmode@gnu.org Ihor Radchenko writes: >> Be sure to read the Emacs Lisp manual regarding threads. They are >> cooperative, so functions called as threads must yield back to the main >> thread for Emacs to do anything else before the function returns. > > I tried to read the manual, but I clearly misunderstand something. > The manual says: > >> Currently, thread switching will occur upon explicit request via >> ‘thread-yield’, when waiting for keyboard input... > > So, except directly calling thread-yield, it should be possible to > trigger switching the current thread when keyboard input is expected. > I tried the following demo code: > > (defun test () > (let ((a 0)) > (dotimes (_ 5) > (setq a (1+ a)) > (sleep-for 2) > (message "%s" a)))) > > (progn ;This should return to command loop quickly > (make-thread #'test) > (message "Executed...")); `eval-last-sexp' here > > I can move around the buffer while the progn is running. > However, it is not the case with `org-agenda-redo' for a reason I do not > fully understand. Org Agenda code does not wait for keyboard input; it's busy building the agenda. This is the case with most code in Emacs: it's not written to be asynchronous, and it doesn't return to the main thread until done. So you can sprinkle yields here and there and maybe be able to move point around while some code is running, but that will decrease performance, as well as introducing another level of complexity and another class of bugs (e.g. what if the user modifies a buffer while the agenda code is scanning it?). >> 1. The process would have to load the same Org buffers, which takes >> time, especially in large buffers. Depending on configuration, it >> can take some time, indeed. > >> 3. Ensuring that configuration and state between the main Emacs process >> and the separate, agenda-generating process is not necessarily >> simple. Consider as well that if a buffer had unsaved changes, >> those would not be readable by the other process, which would lead >> to invalid results. One could force the buffers to be saved first, >> but that may not always be desirable, as saving buffers can have >> side effects. > > Why cannot org-buffer simply be copied into the subordinate process? If > all be buffer-locals, text properties, and overlays are copied directly > from the main emacs process, there may be no need to even initialise > org-mode (the idea is to do something similar to clone-buffer). AFAIK there exists no way to do such a thing. Buffers are not designed to be serialized/deserialized like that. You could try writing some Elisp code to do it, but the end result would probably be much slower than existing agenda code, as well as more difficult to debug. > The question though is whether buffer-locals + overlays + propertized > .org files text + org-agenda-buffer copy can be sufficient to make the > org-agenda-redo run properly. Are there any other buffers, variables, > or other environment settings used by org-agenda-redo? As you can see in org-agenda.el, it's complicated. Remember that an Emacs process is like a Lisp image, full of state. The more symbols and other structures you copy to the async Emacs process (by printing and reading them as text, remember), the slower it's going to be--and it will always be slower than not using async. >> If your agenda buffers are taking too long to refresh, you might >> consider org-ql's views/saved-searches as an alternative. ... > > I know org-ql and I am pretty sure that it will improve performance. > Actually, if one can make built-in org-agenda asynchronous, org-ql can > probably use similar approach and become even faster :) Asynchronous code is not faster; it's generally slower because of yielding and synchronization. > I am trying on default org-agenda now mostly because my current config > is heavily geared towards default agenda and I am not sure if > refactoring everything to use org-ql will worth it at the end in terms > of performance. I use too many slow custom skip-functions. org-ql doesn't use skip functions, just queries.