From mboxrd@z Thu Jan 1 00:00:00 1970 From: John Kitchin Subject: Re: asynchronous python code blocks in org-mode Date: Mon, 30 Nov 2015 16:06:51 -0500 Message-ID: References: <2015-11-30T12-27-30@devnull.Karl-Voit.at> Mime-Version: 1.0 Content-Type: text/plain Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:36574) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a3Vez-0005dN-Fx for emacs-orgmode@gnu.org; Mon, 30 Nov 2015 16:07:01 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a3Vev-0002pE-6z for emacs-orgmode@gnu.org; Mon, 30 Nov 2015 16:06:57 -0500 Received: from mail-qk0-x22b.google.com ([2607:f8b0:400d:c09::22b]:34879) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a3Veu-0002p3-TM for emacs-orgmode@gnu.org; Mon, 30 Nov 2015 16:06:53 -0500 Received: by qkao63 with SMTP id o63so64248928qka.2 for ; Mon, 30 Nov 2015 13:06:52 -0800 (PST) In-reply-to: <2015-11-30T12-27-30@devnull.Karl-Voit.at> List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Karl Voit Cc: emacs-orgmode@gnu.org Karl Voit writes: > * John Kitchin wrote: >> Hi everyone, > > Hi John! > >> I worked out some ideas on running Python blocks asynchronously in >> org-mode, while seeing the output of the code, /and/ still capturing the >> output in org-mode! > > Found the link to your video + blog on Twitter. On the weekend, I > watched it with big eyes and tried your code on my side as well: > great work! > >> It seems to work pretty well as a proof of concept, but the present >> approach has some limitations, e.g. no sessions, no src-block :var, and >> maybe others. These might be fixable at some point. >> >> Anyway, I thought I would share it, to see if anyone has thoughts on >> it, and in case it looks useful to you. > > I'm loving it. > > I'd love to see this async method for shell script as well. Many > shell snippets I am using do take some time: backup, audio > conversion, bulk image conversion, ... It turns out to be pretty trivial to modify this for sh. Change change org-babel-variable-assignments:python to org-babel-variable-assignments:sh and python to bash, and rename the function to org-babel-async-execute:sh, and you should have it. If you don't mind redefining an org command, this will let you put :async in the header to trigger the asynchronous mode, otherwise, run the usual way: (defun org-babel-execute:sh (body params) "Execute a block of Shell commands with Babel. This function is called by `org-babel-execute-src-block'." (let* ((session (org-babel-sh-initiate-session (cdr (assoc :session params)))) (async (assoc :async params)) (stdin (let ((stdin (cdr (assoc :stdin params)))) (when stdin (org-babel-sh-var-to-string (org-babel-ref-resolve stdin))))) (full-body (org-babel-expand-body:generic body params (org-babel-variable-assignments:sh params)))) (if async ;; run asynchronously (org-babel-async-execute:sh) ;; else run regularly (org-babel-reassemble-table (org-babel-sh-evaluate session full-body params stdin) (org-babel-pick-name (cdr (assoc :colname-names params)) (cdr (assoc :colnames params))) (org-babel-pick-name (cdr (assoc :rowname-names params)) (cdr (assoc :rownames params))))))) I have not thought about sessions and asynchronous execution. It would mean a queue and a different way to pass code to the process, which I have not thought through yet. What to do when there are dependencies for example,... > > 1) How big is the chance that we're going to see async babel block > execution by default in a future org-mode release? > > 2) In the meantime: Wouldn't it be nice to have additional > source-block definitions like python + apython (for asynchronous > python) or sh + ash? -- Professor John Kitchin Doherty Hall A207F Department of Chemical Engineering Carnegie Mellon University Pittsburgh, PA 15213 412-268-7803 @johnkitchin http://kitchingroup.cheme.cmu.edu