From mboxrd@z Thu Jan 1 00:00:00 1970 From: John Kitchin Subject: Re: asynchronous python code blocks in org-mode Date: Tue, 01 Dec 2015 18:07:33 -0500 Message-ID: References: <2015-11-30T12-27-30@devnull.Karl-Voit.at> Mime-Version: 1.0 Content-Type: text/plain Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:55822) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a3u1P-0006fa-AD for emacs-orgmode@gnu.org; Tue, 01 Dec 2015 18:07:45 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a3u1J-0008Mv-Jz for emacs-orgmode@gnu.org; Tue, 01 Dec 2015 18:07:43 -0500 Received: from mail-qg0-x22f.google.com ([2607:f8b0:400d:c04::22f]:35384) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a3u1J-0008M8-FK for emacs-orgmode@gnu.org; Tue, 01 Dec 2015 18:07:37 -0500 Received: by qgec40 with SMTP id c40so19132613qge.2 for ; Tue, 01 Dec 2015 15:07:36 -0800 (PST) In-reply-to: List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Tom Cc: Karl Voit , emacs-orgmode@gnu.org Tom writes: > On Mon, Nov 30, 2015 at 1:06 PM, John Kitchin > wrote: > >> I have not thought about sessions and asynchronous execution. It would >> mean a queue and a different way to pass code to the process, which I >> have not thought through yet. What to do when there are dependencies for >> example,... >> > > A good way of converting synchronous into asynchronous code is to use > futures/promises. Usually, that's done via data structures, but within > Emacs buffers, it could be done via text strings. > > How might that work? org-babel-execute:python could wait for, say, 0.1 sec > for an immediate result. If the computation doesn't finish within that > time, it returns a "future", a magic string like > "org_mode_future_result(1234) ###MAGIC###". This would then get inserted as > output into the org-mode buffer. Later, when the actual result becomes > available from the subprocess, that invokes a callback on the org-python > mode buffer and replaces tihs magic string with the actual result, and > dequeues and executes the next command if necessary. This is basically what happens now, I insert a uuid that gets replaced by a callback function. A session is doable I think as you describe. Its not a high-priority for me though. I feel like it would eventually be necessary to have some queue management functions, e.g. what to do if you run a block twice? or want to cancel a run, etc... How to make sure dependencies are correct, e.g. you can start a second block before the first block finishes, but it is queued. etc... > (Picking a Python-parseable expression would even allow futures to be used > in some other Python computations as if it were an actual value.) > > I think that would allow much of the current API to remain in place. > Obviously, some things simply can't work. For example, > org-babel-reassemble-table expects an actual result, not a future; such > post-processing would have to move to a hook function, which probably would > be cleaner anyway. > > Tom -- Professor John Kitchin Doherty Hall A207F Department of Chemical Engineering Carnegie Mellon University Pittsburgh, PA 15213 412-268-7803 @johnkitchin http://kitchingroup.cheme.cmu.edu