* long running processes @ 2015-11-18 20:04 Tom 2015-11-18 22:55 ` John Kitchin 0 siblings, 1 reply; 5+ messages in thread From: Tom @ 2015-11-18 20:04 UTC (permalink / raw) To: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 397 bytes --] When I run Python code in a session from Org mode and the execution takes a while, the cursor changes to a wait cursor. In addition, the session window doesn't display any intermediate output until the execution has finished. Is there any way of changing this behavior? I would like to start long-running functions from within org-mode and see the progress of the computation as it happens. Tom [-- Attachment #2: Type: text/html, Size: 461 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: long running processes 2015-11-18 20:04 long running processes Tom @ 2015-11-18 22:55 ` John Kitchin 2015-11-18 23:31 ` Tom 0 siblings, 1 reply; 5+ messages in thread From: John Kitchin @ 2015-11-18 22:55 UTC (permalink / raw) To: Tom; +Cc: emacs-orgmode I am pretty sure this is not directly possible right now. Some approaches that resemble it could be: 1. write a src block that will be tangled to a script. 2. tangle the block 3. Run the script in a shell src block with an & so it runs non-blocking. or, use an elisp block like: (org-babel-tangle) (async-shell-command "your script" some-output-buffer) I don't know a way to get continuous updated output in an org-buffer though. I do things like this by making code blocks that submit scripts to a queue system if needed, and then raising an exception. If not needed, the scripts read data files and return the answer I want. You might find a way to use the async module (https://github.com/jwiegley/emacs-async) to achieve something like this too but I suspect it still involves tangling, or calling the named block asynchronously. Tom writes: > When I run Python code in a session from Org mode and the execution takes a > while, the cursor changes to a wait cursor. In addition, the session window > doesn't display any intermediate output until the execution has finished. > > Is there any way of changing this behavior? I would like to start > long-running functions from within org-mode and see the progress of the > computation as it happens. > > Tom -- Professor John Kitchin Doherty Hall A207F Department of Chemical Engineering Carnegie Mellon University Pittsburgh, PA 15213 412-268-7803 @johnkitchin http://kitchingroup.cheme.cmu.edu ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: long running processes 2015-11-18 22:55 ` John Kitchin @ 2015-11-18 23:31 ` Tom 2015-11-19 13:38 ` John Kitchin 0 siblings, 1 reply; 5+ messages in thread From: Tom @ 2015-11-18 23:31 UTC (permalink / raw) To: John Kitchin; +Cc: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 1990 bytes --] On Wed, Nov 18, 2015 at 2:55 PM, John Kitchin <jkitchin@andrew.cmu.edu> wrote: > I am pretty sure this is not directly possible right now. > > Some approaches that resemble it could be: > 1. write a src block that will be tangled to a script. > 2. tangle the block > 3. Run the script in a shell src block with an & so it runs > non-blocking. > > or, use an elisp block like: > > (org-babel-tangle) > (async-shell-command "your script" some-output-buffer) > > I don't know a way to get continuous updated output in an org-buffer > though. > Thanks for the response. I didn't necessarily expect continuous output into the org-buffer itself to work, but I don't see why the Python subprocess can't display output as it occurs. After all, it uses comint, and comint certainly has facilities for collecting output incrementally while still displaying it (cf comint-output-filter-functions). It looks to me like the problem is that org-babel-comint-with-output uses a "while" loop to collect process output (ob-comint.el:92). At least, it could insert the output into the subprocess buffer and make redisplay happen. But I'm not sure why the code is written that way anyway. Long running "while" loops in Emacs code don't seem like a good idea to begin with. Wouldn't the more natural way for this code to be written in Emacs be the following? - an output filter gets added to the subprocess that collects output - the code is sent to the subprocess for execution - the command returns - the output filter inserts any data it gets into the subprocess buffer, into its "results" data structure, perhaps even into the org-buffer - when the output filter gets the eoe-indicator, it removes itself from the output filter list and sends a notification that execution has completed If the user schedules a second block for execution, the simplest thing to do is return an error if there is already a block executing for that subprocess; alternatively, it could be queued somewhere. Thanks, Tom [-- Attachment #2: Type: text/html, Size: 2492 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: long running processes 2015-11-18 23:31 ` Tom @ 2015-11-19 13:38 ` John Kitchin 2015-11-19 17:10 ` Tom 0 siblings, 1 reply; 5+ messages in thread From: John Kitchin @ 2015-11-19 13:38 UTC (permalink / raw) To: Tom; +Cc: emacs-orgmode Your suggestions sounds possible to me. If you are up for it, I suggest trying to implement it, and offering it as a patch. Tom writes: > On Wed, Nov 18, 2015 at 2:55 PM, John Kitchin <jkitchin@andrew.cmu.edu> > wrote: > >> I am pretty sure this is not directly possible right now. >> >> Some approaches that resemble it could be: >> 1. write a src block that will be tangled to a script. >> 2. tangle the block >> 3. Run the script in a shell src block with an & so it runs >> non-blocking. >> >> or, use an elisp block like: >> >> (org-babel-tangle) >> (async-shell-command "your script" some-output-buffer) >> >> I don't know a way to get continuous updated output in an org-buffer >> though. >> > > Thanks for the response. I didn't necessarily expect continuous output into > the org-buffer itself to work, but I don't see why the Python subprocess > can't display output as it occurs. After all, it uses comint, and comint > certainly has facilities for collecting output incrementally while still > displaying it (cf comint-output-filter-functions). > > It looks to me like the problem is that org-babel-comint-with-output uses a > "while" loop to collect process output (ob-comint.el:92). At least, it > could insert the output into the subprocess buffer and make redisplay > happen. > > But I'm not sure why the code is written that way anyway. Long running > "while" loops in Emacs code don't seem like a good idea to begin with. > Wouldn't the more natural way for this code to be written in Emacs be the > following? > > - an output filter gets added to the subprocess that collects output > - the code is sent to the subprocess for execution > - the command returns > - the output filter inserts any data it gets into the subprocess buffer, > into its "results" data structure, perhaps even into the org-buffer > - when the output filter gets the eoe-indicator, it removes itself from the > output filter list and sends a notification that execution has completed > > If the user schedules a second block for execution, the simplest thing to > do is return an error if there is already a block executing for that > subprocess; alternatively, it could be queued somewhere. > > Thanks, > Tom -- Professor John Kitchin Doherty Hall A207F Department of Chemical Engineering Carnegie Mellon University Pittsburgh, PA 15213 412-268-7803 @johnkitchin http://kitchingroup.cheme.cmu.edu ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: long running processes 2015-11-19 13:38 ` John Kitchin @ 2015-11-19 17:10 ` Tom 0 siblings, 0 replies; 5+ messages in thread From: Tom @ 2015-11-19 17:10 UTC (permalink / raw) To: John Kitchin; +Cc: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 2964 bytes --] Unfortunately, changing from the current synchronous to asynchronous processing probably requires changes to the API that would require changes to every existing language mode. A better way forward may simply be to implement a new block type and then let people gradually convert their language bindings to that. I'll have a look at it for Python. Tom On Thu, Nov 19, 2015 at 5:38 AM, John Kitchin <jkitchin@andrew.cmu.edu> wrote: > Your suggestions sounds possible to me. If you are up for it, I suggest > trying to implement it, and offering it as a patch. > > Tom writes: > > > On Wed, Nov 18, 2015 at 2:55 PM, John Kitchin <jkitchin@andrew.cmu.edu> > > wrote: > > > >> I am pretty sure this is not directly possible right now. > >> > >> Some approaches that resemble it could be: > >> 1. write a src block that will be tangled to a script. > >> 2. tangle the block > >> 3. Run the script in a shell src block with an & so it runs > >> non-blocking. > >> > >> or, use an elisp block like: > >> > >> (org-babel-tangle) > >> (async-shell-command "your script" some-output-buffer) > >> > >> I don't know a way to get continuous updated output in an org-buffer > >> though. > >> > > > > Thanks for the response. I didn't necessarily expect continuous output > into > > the org-buffer itself to work, but I don't see why the Python subprocess > > can't display output as it occurs. After all, it uses comint, and comint > > certainly has facilities for collecting output incrementally while still > > displaying it (cf comint-output-filter-functions). > > > > It looks to me like the problem is that org-babel-comint-with-output > uses a > > "while" loop to collect process output (ob-comint.el:92). At least, it > > could insert the output into the subprocess buffer and make redisplay > > happen. > > > > But I'm not sure why the code is written that way anyway. Long running > > "while" loops in Emacs code don't seem like a good idea to begin with. > > Wouldn't the more natural way for this code to be written in Emacs be the > > following? > > > > - an output filter gets added to the subprocess that collects output > > - the code is sent to the subprocess for execution > > - the command returns > > - the output filter inserts any data it gets into the subprocess buffer, > > into its "results" data structure, perhaps even into the org-buffer > > - when the output filter gets the eoe-indicator, it removes itself from > the > > output filter list and sends a notification that execution has completed > > > > If the user schedules a second block for execution, the simplest thing to > > do is return an error if there is already a block executing for that > > subprocess; alternatively, it could be queued somewhere. > > > > Thanks, > > Tom > > -- > Professor John Kitchin > Doherty Hall A207F > Department of Chemical Engineering > Carnegie Mellon University > Pittsburgh, PA 15213 > 412-268-7803 > @johnkitchin > http://kitchingroup.cheme.cmu.edu > [-- Attachment #2: Type: text/html, Size: 4055 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2015-11-19 17:10 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-11-18 20:04 long running processes Tom 2015-11-18 22:55 ` John Kitchin 2015-11-18 23:31 ` Tom 2015-11-19 13:38 ` John Kitchin 2015-11-19 17:10 ` Tom
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).