From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ihor Radchenko Subject: Re: Asynchronous org-babel-tangle (was Re: Asynchronous org-agenda-redo) Date: Sun, 15 Dec 2019 21:40:23 +0800 Message-ID: <87bls96720.fsf@yantar92-laptop.i-did-not-set--mail-host-address--so-tickle-me> References: <87k172ot2m.fsf@yantar92-laptop.i-did-not-set--mail-host-address--so-tickle-me> <87r219a8uu.fsf@yantar92-laptop.i-did-not-set--mail-host-address--so-tickle-me> Mime-Version: 1.0 Content-Type: text/plain Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:45320) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1igUAE-0008FV-Gi for emacs-orgmode@gnu.org; Sun, 15 Dec 2019 08:42:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1igUAC-0006E7-Hc for emacs-orgmode@gnu.org; Sun, 15 Dec 2019 08:42:26 -0500 Received: from mail-yb1-xb31.google.com ([2607:f8b0:4864:20::b31]:33306) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1igUAC-0006AF-Ai for emacs-orgmode@gnu.org; Sun, 15 Dec 2019 08:42:24 -0500 Received: by mail-yb1-xb31.google.com with SMTP id o63so1846209ybc.0 for ; Sun, 15 Dec 2019 05:42:23 -0800 (PST) In-Reply-To: List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: "Emacs-orgmode" To: Diego Zamboni Cc: emacs-orgmode Mailinglist Thanks for the feedback. > - For me, the tangle works if I only load the org library in the > sub-process, so I disabled the loading of my config. I guess this depends > heavily and what other configuration one has loaded, but I think for > tangling, not much else is needed :) Note that your version tangling may not work as expected if your init file changes any default headers args affecting the tangling. For example, I set (:comments . "link") in my init file. Not loading init file would disallow detangling in my case. > - In you code, the process-list variable was not actually being modified > since plist-put returns the new value but does not modify the > variable. (facepalm) > I > tried a few variations wrapping the plist-put in (setq ...), and adding > corresponding code to clear the element after the tangle was finished, but > could not get it to work. Well. Things did not work because plist-put compares keys by #'eq, which returns nil on equal strings. > - I wanted my code to report the time the tangle took. This works well by > passing the time as the result of the initial lambda. Interestingly, it > works if I return the time already formatted as a string (as in my code) > but not if I return the result of (float-time ..) directly. Thanks for this neat idea. Will also grab it to my config. float-time did not work because of bug in async.el. It uses backward-sexp to find the returned value of the lambda (in the process buffer). But backwards-sexp fails on float (it moves point just after the point). Best, Ihor Diego Zamboni writes: > Hi Ihor, > > On Thu, Dec 12, 2019 at 4:00 PM Ihor Radchenko wrote: > >> See the relevant code from my config below. Let me know if you have any >> questions or suggestions. >> > > Thank you, this was extremely helpful. I am now using a modified version of > your code (see below) with the following comments/questions: > > - For me, the tangle works if I only load the org library in the > sub-process, so I disabled the loading of my config. I guess this depends > heavily and what other configuration one has loaded, but I think for > tangling, not much else is needed :) > > - In you code, the process-list variable was not actually being modified > since plist-put returns the new value but does not modify the variable. I > tried a few variations wrapping the plist-put in (setq ...), and adding > corresponding code to clear the element after the tangle was finished, but > could not get it to work. As this is not really critical for me, I just > removed the whole process-checking code, significantly simplifying the rest. > > - I wanted my code to report the time the tangle took. This works well by > passing the time as the result of the initial lambda. Interestingly, it > works if I return the time already formatted as a string (as in my code) > but not if I return the result of (float-time ..) directly. > > Here's my code: > > #+begin_src emacs-lisp > (defun zz/org-babel-tangle-async (file) > "Invoke `org-babel-tangle-file' asynchronously." > (message "Tangling %s..." (buffer-file-name)) > (async-start > (let ((args (list file))) > `(lambda () > (require 'org) > ;;(load "~/.emacs.d/init.el") > (let ((start-time (current-time))) > (apply #'org-babel-tangle-file ',args) > (format "%.2f" (float-time (time-since start-time)))))) > (let ((message-string (format "Tangling %S completed after " file))) > `(lambda (tangle-time) > (message (concat ,message-string > (format "%s seconds" tangle-time))))))) > > (defun zz/org-babel-tangle-current-buffer-async () > "Tangle current buffer asynchronously." > (zz/org-babel-tangle-async (buffer-file-name))) > #+end_src > > Thanks again for your help, and for the inspiration to finally get this > done, it had been bugging me for a while :) > > --Diego > > > >> Also, my config is written in the way that everything related to user >> interaction can be disabled if I load emacs for tangling >> (with org-tangle-flag). This is to speed up the async process. >> >> #+begin_src emacs-lisp >> (defvar yant/org-babel-tangle-async-process-list nil >> "Plist of (file . process) for all the currently running async tangle >> processes.") >> >> >> (defun yant/org-babel-tangle-async (file &optional target-file lang) >> "Invoke `org-babel-tangle-file' asynchronously." >> (require 'async) >> (let ((oldproc (plist-get yant/org-babel-tangle-async-process-list >> file))) >> (when (or (not oldproc) >> (async-wait oldproc)) >> (message "Tangling %s..." (buffer-file-name)) >> (plist-put yant/org-babel-tangle-async-process-list >> file >> (async-start >> (let ((args (list file target-file lang))) >> `(lambda () >> (require 'org) >> (setq org-tangle-flag t) >> (load "~/.emacs.d/config.el") >> (apply #'org-babel-tangle-file ',args))) >> (let ((message-string (format "Tangling (%S %S %S) >> completed." file target-file lang))) >> `(lambda (result) (message ,message-string)))))))) >> >> (defvar yant/auto-tangle-list nil >> "List of files, which can be safely tangled on save. >> The list is saved between Emacs sessions.") >> >> (when init-flag >> (use-package savehist >> :config >> (add-to-list 'savehist-additional-variables 'yant/auto-tangle-list)) >> (savehist-mode +1) >> (defun yant/toggle-buffer-auto-tangle (arg) >> "Toggle auto tangling of a buffer." >> (interactive "P") >> (if (not (eq major-mode 'org-mode)) >> (message "Org-mode is not active in buffer \"%s\"" (buffer-name)) >> (cond ((not arg) >> (if (member (buffer-file-name) yant/auto-tangle-list) >> (progn (setq yant/auto-tangle-list (delete >> (buffer-file-name) yant/auto-tangle-list)) >> (message "Auto tangling disabled for %s" >> (buffer-file-name))) >> (add-to-list 'yant/auto-tangle-list (buffer-file-name)) >> (message "Auto tangling enabled for %s" >> (buffer-file-name)))) >> ((or (and (not (listp arg)) (> arg 0)) >> (equal arg '(4))) >> (add-to-list 'yant/auto-tangle-list (buffer-file-name)) >> (message "Auto tangling enabled for %s" (buffer-file-name))) >> (t >> (setq yant/auto-tangle-list (delete (buffer-file-name) >> yant/auto-tangle-list)) >> (message "Auto tangling disabled for %s" >> (buffer-file-name)))))) >> >> (bind-key "C-c C-*" #'yant/toggle-buffer-auto-tangle org-mode-map)) >> >> (defun yant/org-babel-tangle-current-buffer-async () >> "Tangle current buffer asynchronously." >> (when (and (eq major-mode 'org-mode) >> (member (buffer-file-name) yant/auto-tangle-list)) >> (yant/org-babel-tangle-async (buffer-file-name)))) >> >> (add-hook 'after-save-hook #'yant/org-babel-tangle-current-buffer-async) >> #+end_src >> >> >> Diego Zamboni writes: >> >> > Hi Ihor, >> > >> > I cannot answer your question, but I am curious about using async >> together >> > with tangling, since for some of my buffers, tangling takes some time and >> > freezes Emacs in the process. Do you have some examples of this that you >> > could share? >> > >> > Thanks, >> > --Diego >> > >> > >> > On Thu, Dec 12, 2019 at 9:21 AM Ihor Radchenko >> wrote: >> > >> >> I am thinking if it is possible to implement org-agenda-redo >> >> asynchronously. >> >> >> >> Rebuilding agenda should normally not affect any buffer except agenda >> >> buffer. So, it should be sufficient to block any agenda modifying >> >> commands in the agenda buffer, redo the agenda buffer in separate >> >> thread, and replace the old agenda with the calculated one. >> >> Then, emacs should remain responsive while updating agenda (except for >> >> modifying the agenda buffer). >> >> >> >> For example, this naive code kind of works (forgetting that buffer-local >> >> variables will not be passed to the thread): >> >> >> >> (define-advice org-agenda-redo (:around (oldfun &optional all) >> make-async) >> >> (make-thread oldfun "org-agenda-redo")) >> >> >> >> The problem is that emacs does not become responsive... >> >> >> >> Another approach would be using async.el package, which allows calling >> >> arbitrary function in subordinate emacs process. Then, the main emacs >> >> instance should not be "frozen" (I use same approach for tangling and it >> >> works fine). >> >> >> >> However, the question is how to pass the .org and agenda buffers to this >> >> subordinate process. Opening the .org files there is not a good option >> >> since it would give too much overhead to this asynchronous agenda. >> >> >> >> Any suggestions? Alternative ideas? >> >> >> >> Best, >> >> Ihor >> >> >> >> >> >> >> >> -- >> Ihor Radchenko, >> PhD, >> Center for Advancing Materials Performance from the Nanoscale (CAMP-nano) >> State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong >> University, Xi'an, China >> Email: yantar92@gmail.com, ihor_radchenko@alumni.sutd.edu.sg >>