From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tom Subject: Re: long running processes Date: Thu, 19 Nov 2015 09:10:23 -0800 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=001a1141b2c4c736990524e7d6ba Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:39892) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZzSjO-0008MJ-1j for emacs-orgmode@gnu.org; Thu, 19 Nov 2015 12:10:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZzSjL-0000LI-Q2 for emacs-orgmode@gnu.org; Thu, 19 Nov 2015 12:10:45 -0500 Received: from mail-io0-x233.google.com ([2607:f8b0:4001:c06::233]:34015) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZzSjL-0000Kv-Jj for emacs-orgmode@gnu.org; Thu, 19 Nov 2015 12:10:43 -0500 Received: by ioir85 with SMTP id r85so95994682ioi.1 for ; Thu, 19 Nov 2015 09:10:43 -0800 (PST) In-Reply-To: List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: John Kitchin Cc: emacs-orgmode@gnu.org --001a1141b2c4c736990524e7d6ba Content-Type: text/plain; charset=UTF-8 Unfortunately, changing from the current synchronous to asynchronous processing probably requires changes to the API that would require changes to every existing language mode. A better way forward may simply be to implement a new block type and then let people gradually convert their language bindings to that. I'll have a look at it for Python. Tom On Thu, Nov 19, 2015 at 5:38 AM, John Kitchin wrote: > Your suggestions sounds possible to me. If you are up for it, I suggest > trying to implement it, and offering it as a patch. > > Tom writes: > > > On Wed, Nov 18, 2015 at 2:55 PM, John Kitchin > > wrote: > > > >> I am pretty sure this is not directly possible right now. > >> > >> Some approaches that resemble it could be: > >> 1. write a src block that will be tangled to a script. > >> 2. tangle the block > >> 3. Run the script in a shell src block with an & so it runs > >> non-blocking. > >> > >> or, use an elisp block like: > >> > >> (org-babel-tangle) > >> (async-shell-command "your script" some-output-buffer) > >> > >> I don't know a way to get continuous updated output in an org-buffer > >> though. > >> > > > > Thanks for the response. I didn't necessarily expect continuous output > into > > the org-buffer itself to work, but I don't see why the Python subprocess > > can't display output as it occurs. After all, it uses comint, and comint > > certainly has facilities for collecting output incrementally while still > > displaying it (cf comint-output-filter-functions). > > > > It looks to me like the problem is that org-babel-comint-with-output > uses a > > "while" loop to collect process output (ob-comint.el:92). At least, it > > could insert the output into the subprocess buffer and make redisplay > > happen. > > > > But I'm not sure why the code is written that way anyway. Long running > > "while" loops in Emacs code don't seem like a good idea to begin with. > > Wouldn't the more natural way for this code to be written in Emacs be the > > following? > > > > - an output filter gets added to the subprocess that collects output > > - the code is sent to the subprocess for execution > > - the command returns > > - the output filter inserts any data it gets into the subprocess buffer, > > into its "results" data structure, perhaps even into the org-buffer > > - when the output filter gets the eoe-indicator, it removes itself from > the > > output filter list and sends a notification that execution has completed > > > > If the user schedules a second block for execution, the simplest thing to > > do is return an error if there is already a block executing for that > > subprocess; alternatively, it could be queued somewhere. > > > > Thanks, > > Tom > > -- > Professor John Kitchin > Doherty Hall A207F > Department of Chemical Engineering > Carnegie Mellon University > Pittsburgh, PA 15213 > 412-268-7803 > @johnkitchin > http://kitchingroup.cheme.cmu.edu > --001a1141b2c4c736990524e7d6ba Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Unfortunately, changing from the current synchronous = to asynchronous processing probably requires changes to the API that would = require changes to every existing language mode.

A better= way forward may simply be to implement a new block type and then let peopl= e gradually convert their language bindings to that. I'll have a look a= t it for Python.

Tom

On Thu, Nov 19, 2015 at 5:38 AM, John = Kitchin <jkitchin@andrew.cmu.edu> wrote:
Your suggestions sounds possible to me. If you are up = for it, I suggest
trying to implement it, and offering it as a patch.

Tom writes:

> On Wed, Nov 18, 2015 at 2:55 PM, John Kitchin <jkitchin@andrew.cmu.edu>
> wrote:
>
>> I am pretty sure this is not directly possible right now.
>>
>> Some approaches that resemble it could be:
>> 1. write a src block that will be tangled to a script.
>> 2. tangle the block
>> 3. Run the script in a shell src block with an & so it runs >> non-blocking.
>>
>> or, use an elisp block like:
>>
>> (org-babel-tangle)
>> (async-shell-command "your script" some-output-buffer) >>
>> I don't know a way to get continuous updated output in an org-= buffer
>> though.
>>
>
> Thanks for the response. I didn't necessarily expect continuous ou= tput into
> the org-buffer itself to work, but I don't see why the Python subp= rocess
> can't display output as it occurs. After all, it uses comint, and = comint
> certainly has facilities for collecting output incrementally while sti= ll
> displaying it (cf comint-output-filter-functions).
>
> It looks to me like the problem is that org-babel-comint-with-output u= ses a
> "while" loop to collect process output (ob-comint.el:92). At= least, it
> could insert the output into the subprocess buffer and make redisplay<= br> > happen.
>
> But I'm not sure why the code is written that way anyway. Long run= ning
> "while" loops in Emacs code don't seem like a good idea = to begin with.
> Wouldn't the more natural way for this code to be written in Emacs= be the
> following?
>
> - an output filter gets added to the subprocess that collects output > - the code is sent to the subprocess for execution
> - the command returns
> - the output filter inserts any data it gets into the subprocess buffe= r,
> into its "results" data structure, perhaps even into the org= -buffer
> - when the output filter gets the eoe-indicator, it removes itself fro= m the
> output filter list and sends a notification that execution has complet= ed
>
> If the user schedules a second block for execution, the simplest thing= to
> do is return an error if there is already a block executing for that > subprocess; alternatively, it could be queued somewhere.
>
> Thanks,
> Tom

--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu

--001a1141b2c4c736990524e7d6ba--