* long running processes
@ 2015-11-18 20:04 Tom
2015-11-18 22:55 ` John Kitchin
0 siblings, 1 reply; 5+ messages in thread
From: Tom @ 2015-11-18 20:04 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 397 bytes --]
When I run Python code in a session from Org mode and the execution takes a
while, the cursor changes to a wait cursor. In addition, the session window
doesn't display any intermediate output until the execution has finished.
Is there any way of changing this behavior? I would like to start
long-running functions from within org-mode and see the progress of the
computation as it happens.
Tom
[-- Attachment #2: Type: text/html, Size: 461 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: long running processes
2015-11-18 20:04 long running processes Tom
@ 2015-11-18 22:55 ` John Kitchin
2015-11-18 23:31 ` Tom
0 siblings, 1 reply; 5+ messages in thread
From: John Kitchin @ 2015-11-18 22:55 UTC (permalink / raw)
To: Tom; +Cc: emacs-orgmode
I am pretty sure this is not directly possible right now.
Some approaches that resemble it could be:
1. write a src block that will be tangled to a script.
2. tangle the block
3. Run the script in a shell src block with an & so it runs
non-blocking.
or, use an elisp block like:
(org-babel-tangle)
(async-shell-command "your script" some-output-buffer)
I don't know a way to get continuous updated output in an org-buffer
though.
I do things like this by making code blocks that submit scripts to a
queue system if needed, and then raising an exception. If not needed,
the scripts read data files and return the answer I want.
You might find a way to use the async module
(https://github.com/jwiegley/emacs-async) to achieve something like this
too but I suspect it still involves tangling, or calling the named block asynchronously.
Tom writes:
> When I run Python code in a session from Org mode and the execution takes a
> while, the cursor changes to a wait cursor. In addition, the session window
> doesn't display any intermediate output until the execution has finished.
>
> Is there any way of changing this behavior? I would like to start
> long-running functions from within org-mode and see the progress of the
> computation as it happens.
>
> Tom
--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: long running processes
2015-11-18 22:55 ` John Kitchin
@ 2015-11-18 23:31 ` Tom
2015-11-19 13:38 ` John Kitchin
0 siblings, 1 reply; 5+ messages in thread
From: Tom @ 2015-11-18 23:31 UTC (permalink / raw)
To: John Kitchin; +Cc: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 1990 bytes --]
On Wed, Nov 18, 2015 at 2:55 PM, John Kitchin <jkitchin@andrew.cmu.edu>
wrote:
> I am pretty sure this is not directly possible right now.
>
> Some approaches that resemble it could be:
> 1. write a src block that will be tangled to a script.
> 2. tangle the block
> 3. Run the script in a shell src block with an & so it runs
> non-blocking.
>
> or, use an elisp block like:
>
> (org-babel-tangle)
> (async-shell-command "your script" some-output-buffer)
>
> I don't know a way to get continuous updated output in an org-buffer
> though.
>
Thanks for the response. I didn't necessarily expect continuous output into
the org-buffer itself to work, but I don't see why the Python subprocess
can't display output as it occurs. After all, it uses comint, and comint
certainly has facilities for collecting output incrementally while still
displaying it (cf comint-output-filter-functions).
It looks to me like the problem is that org-babel-comint-with-output uses a
"while" loop to collect process output (ob-comint.el:92). At least, it
could insert the output into the subprocess buffer and make redisplay
happen.
But I'm not sure why the code is written that way anyway. Long running
"while" loops in Emacs code don't seem like a good idea to begin with.
Wouldn't the more natural way for this code to be written in Emacs be the
following?
- an output filter gets added to the subprocess that collects output
- the code is sent to the subprocess for execution
- the command returns
- the output filter inserts any data it gets into the subprocess buffer,
into its "results" data structure, perhaps even into the org-buffer
- when the output filter gets the eoe-indicator, it removes itself from the
output filter list and sends a notification that execution has completed
If the user schedules a second block for execution, the simplest thing to
do is return an error if there is already a block executing for that
subprocess; alternatively, it could be queued somewhere.
Thanks,
Tom
[-- Attachment #2: Type: text/html, Size: 2492 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: long running processes
2015-11-18 23:31 ` Tom
@ 2015-11-19 13:38 ` John Kitchin
2015-11-19 17:10 ` Tom
0 siblings, 1 reply; 5+ messages in thread
From: John Kitchin @ 2015-11-19 13:38 UTC (permalink / raw)
To: Tom; +Cc: emacs-orgmode
Your suggestions sounds possible to me. If you are up for it, I suggest
trying to implement it, and offering it as a patch.
Tom writes:
> On Wed, Nov 18, 2015 at 2:55 PM, John Kitchin <jkitchin@andrew.cmu.edu>
> wrote:
>
>> I am pretty sure this is not directly possible right now.
>>
>> Some approaches that resemble it could be:
>> 1. write a src block that will be tangled to a script.
>> 2. tangle the block
>> 3. Run the script in a shell src block with an & so it runs
>> non-blocking.
>>
>> or, use an elisp block like:
>>
>> (org-babel-tangle)
>> (async-shell-command "your script" some-output-buffer)
>>
>> I don't know a way to get continuous updated output in an org-buffer
>> though.
>>
>
> Thanks for the response. I didn't necessarily expect continuous output into
> the org-buffer itself to work, but I don't see why the Python subprocess
> can't display output as it occurs. After all, it uses comint, and comint
> certainly has facilities for collecting output incrementally while still
> displaying it (cf comint-output-filter-functions).
>
> It looks to me like the problem is that org-babel-comint-with-output uses a
> "while" loop to collect process output (ob-comint.el:92). At least, it
> could insert the output into the subprocess buffer and make redisplay
> happen.
>
> But I'm not sure why the code is written that way anyway. Long running
> "while" loops in Emacs code don't seem like a good idea to begin with.
> Wouldn't the more natural way for this code to be written in Emacs be the
> following?
>
> - an output filter gets added to the subprocess that collects output
> - the code is sent to the subprocess for execution
> - the command returns
> - the output filter inserts any data it gets into the subprocess buffer,
> into its "results" data structure, perhaps even into the org-buffer
> - when the output filter gets the eoe-indicator, it removes itself from the
> output filter list and sends a notification that execution has completed
>
> If the user schedules a second block for execution, the simplest thing to
> do is return an error if there is already a block executing for that
> subprocess; alternatively, it could be queued somewhere.
>
> Thanks,
> Tom
--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: long running processes
2015-11-19 13:38 ` John Kitchin
@ 2015-11-19 17:10 ` Tom
0 siblings, 0 replies; 5+ messages in thread
From: Tom @ 2015-11-19 17:10 UTC (permalink / raw)
To: John Kitchin; +Cc: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 2964 bytes --]
Unfortunately, changing from the current synchronous to asynchronous
processing probably requires changes to the API that would require changes
to every existing language mode.
A better way forward may simply be to implement a new block type and then
let people gradually convert their language bindings to that. I'll have a
look at it for Python.
Tom
On Thu, Nov 19, 2015 at 5:38 AM, John Kitchin <jkitchin@andrew.cmu.edu>
wrote:
> Your suggestions sounds possible to me. If you are up for it, I suggest
> trying to implement it, and offering it as a patch.
>
> Tom writes:
>
> > On Wed, Nov 18, 2015 at 2:55 PM, John Kitchin <jkitchin@andrew.cmu.edu>
> > wrote:
> >
> >> I am pretty sure this is not directly possible right now.
> >>
> >> Some approaches that resemble it could be:
> >> 1. write a src block that will be tangled to a script.
> >> 2. tangle the block
> >> 3. Run the script in a shell src block with an & so it runs
> >> non-blocking.
> >>
> >> or, use an elisp block like:
> >>
> >> (org-babel-tangle)
> >> (async-shell-command "your script" some-output-buffer)
> >>
> >> I don't know a way to get continuous updated output in an org-buffer
> >> though.
> >>
> >
> > Thanks for the response. I didn't necessarily expect continuous output
> into
> > the org-buffer itself to work, but I don't see why the Python subprocess
> > can't display output as it occurs. After all, it uses comint, and comint
> > certainly has facilities for collecting output incrementally while still
> > displaying it (cf comint-output-filter-functions).
> >
> > It looks to me like the problem is that org-babel-comint-with-output
> uses a
> > "while" loop to collect process output (ob-comint.el:92). At least, it
> > could insert the output into the subprocess buffer and make redisplay
> > happen.
> >
> > But I'm not sure why the code is written that way anyway. Long running
> > "while" loops in Emacs code don't seem like a good idea to begin with.
> > Wouldn't the more natural way for this code to be written in Emacs be the
> > following?
> >
> > - an output filter gets added to the subprocess that collects output
> > - the code is sent to the subprocess for execution
> > - the command returns
> > - the output filter inserts any data it gets into the subprocess buffer,
> > into its "results" data structure, perhaps even into the org-buffer
> > - when the output filter gets the eoe-indicator, it removes itself from
> the
> > output filter list and sends a notification that execution has completed
> >
> > If the user schedules a second block for execution, the simplest thing to
> > do is return an error if there is already a block executing for that
> > subprocess; alternatively, it could be queued somewhere.
> >
> > Thanks,
> > Tom
>
> --
> Professor John Kitchin
> Doherty Hall A207F
> Department of Chemical Engineering
> Carnegie Mellon University
> Pittsburgh, PA 15213
> 412-268-7803
> @johnkitchin
> http://kitchingroup.cheme.cmu.edu
>
[-- Attachment #2: Type: text/html, Size: 4055 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2015-11-19 17:10 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-11-18 20:04 long running processes Tom
2015-11-18 22:55 ` John Kitchin
2015-11-18 23:31 ` Tom
2015-11-19 13:38 ` John Kitchin
2015-11-19 17:10 ` Tom
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).