emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
From: David Dynerman <emperordali@block-party.net>
To: Nathaniel Nicandro <nathanielnicandro@gmail.com>
Cc: Org Mode List <emacs-orgmode@gnu.org>
Subject: Re: Feedback on Emacs-Jupyter
Date: Wed, 5 Jan 2022 09:39:11 -0800	[thread overview]
Message-ID: <15302A63-FD67-4754-A32E-46EE40AD4076@block-party.net> (raw)
In-Reply-To: <87o84rkru0.fsf@gmail.com>

Dear Nathaniel,

Let me echo others’ comments by saying how much I appreciate your work on emacs-jupyter. I use the package daily in my work and it’s been really fantastic. Thank you very, very much for your huge efforts and time investment in creating and maintaining this package.

emacs-jupyter has proved so useful that I’ve been collecting a list of suggested improvements - I was actually intending on trying to contribute some of these myself, but I think it would be good to discuss first with you and other users of emacs-jupyter (is there an emacs-jupyter mailing list or preferred venue?). For now I’ll just share what I think would be the biggest improvement for my use case.

I use emacs-jupyter to connect from my laptop to remote jupyter kernels and notebook servers running on powerful machines provided by my employer. I work in a notebook locally on my laptop, but execute code blocks remotely. I specify a single remote jupyter notebook server+kernel to connect to in a #+PROPERTY line at the top of my org file which globally sets header-args for jupyter-python to include the correct :session argument.

For me, the biggest improvements would be addressing some pain points with disconnecting/reconnecting to the jupyter notebook server or kernel. I don’t have technical suggestions yet on how these could be addressed, so I’ll just describe the user experience side of things.

1) Reconnecting to the notebook server after disconnecting. Commonly I’ll work in my notebook, then take a break and close my laptop or disconnect from my work VPN. When I re-open my laptop/reconnect to VPN it seems like the connections to the remote notebook server are in a bad state (this may be a tramp thing, also). Currently my work around is to kill the associated emacs jupyter REPL buffer, then navigate to one of the source blocks in the notebook and run org-babel-pop-to-session which has the side-effect of re-connecting to the jupyter notebook server and recreating the jupyter REPL buffer. 

The main downside to this workaround is that the scroll back history of the REPL buffer is lost, since the buffer was killed. It would be great to be able to re-establish the connection to the REPL without discarding the buffer.

2) Getting results from a running block after disconnecting/reconnecting. Currently it’s difficult to manage long-running code blocks using emacs-jupyter if I disconnect from the remote server while the code block is running. The work around in #1 above to re-connect to a remote kernel doesn’t work if the kernel is still executing a block. If the kernel has completed executing the block, then the results are not populated back into the notebook (the execution UUID populated when the code block began executing remains).

I don’t know much about jupyter, but I saw some jupyter logs about buffered messages. Perhaps jupyter buffers output when a remote client disconnects? If so, maybe this buffered output could be replayed in a notebook when the emacs jupyter REPL connection is re-established?

3) Similarly, improving feedback from a long-running block would be very helpful. Currently you can use print statements in a long running block to report progress, but these are lost after disconnecting (maybe they are buffered on the jupyter server side and could be replayed on reconnect?). Further, I don’t know if this is feasible, but it would be amazing to have support for progress bars for long-running tasks (e.g., have a tqdm progress bar render in the org notebook).

Thanks again for the wonderful package - I hope we can talk a bit more about some of these friction points. I’d be happy to give a crack at contributing a PR, if that would be welcome.

Thank you,
David

> On Jan 4, 2022, at 15:24, Nathaniel Nicandro <nathanielnicandro@gmail.com> wrote:
> 
> 
> Hello all,
> 
> I'm the maintainer of the emacs-jupyter project
> (https://github.com/nnicandro/emacs-jupyter) which essentially
> integrates Jupyter kernels (https://jupyter.org) with Org mode source
> blocks.
> 
> I wanted to make an introduction to the Org community.  So...hello!  And
> thanks for promoting the project on https://orgmode.org/features.html.
> 
> I believe a lot of users of the project use it mainly for the Org
> integration.  I thought it would be a good idea to get some feedback
> from the community on how their experience using emacs-jupyter has been.
> I'm getting back into active maintenance of the project and am looking
> for feedback to get a better idea of what the future of the project
> could look like.  What features of standard Org source blocks do you
> find Jupyter source blocks are lacking?  What potential features do you
> think would be useful for Jupyter source blocks to support, given the
> capabilities of Jupyter?  What would it mean to see Emacs-Jupyter and
> Org more integrated?  Of course, any other thoughts are welcome.
> 
> -- 
> Nathaniel
> 



  parent reply	other threads:[~2022-01-05 17:40 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-01-04 23:24 Feedback on Emacs-Jupyter Nathaniel Nicandro
2022-01-05  4:37 ` Ken Mankoff
2022-01-06 18:44   ` Nathaniel Nicandro
2022-01-05  7:26 ` Colin Baxter 😺
2022-01-05 17:39 ` David Dynerman [this message]
2022-01-07 18:57   ` Nathaniel Nicandro
2022-01-08 11:27 ` Daniel Fleischer
2022-01-12 13:24 ` Ihor Radchenko
2022-01-13 20:56   ` Nathaniel Nicandro
2022-01-14 14:25     ` Ihor Radchenko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.orgmode.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=15302A63-FD67-4754-A32E-46EE40AD4076@block-party.net \
    --to=emperordali@block-party.net \
    --cc=emacs-orgmode@gnu.org \
    --cc=nathanielnicandro@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).