Re: [BUG] FAILED test-ob-python/session-multiline

[Ihor Radchenko <yantar92@posteo.net>]

Jack Kamm <jackkamm@gmail.com> writes:

> This one might take some time to fix, since it's hard to reproduce and
> I'm not sure the cause of it. But here are 3 different solutions I am
> considering now:
>
> 1. In addition to printing `org-babel-python-eoe-indicator' after
>    execution, we could also print out a "beginning of execution"
>    indicator before execution, and then capture the output between the
>    beginning and end indicators. This is how the async session
>    execution works, and should avoid any possibility of capturing
>    prompts.

This idea looks interesting. Although I would not be so sure that it
will fix things - I have learned that comint has many edge cases we may
not easily anticipate.

For example, see the discussion in
https://yhetil.org/emacs-devel/87y1tgqhmc.fsf@localhost/

> 2. Instead of relying on our own custom `org-babel-python-send-string',
>    we could try switching to python.el's
>    `python-shell-send-string-no-output', which is probably more
>    robust. This would also allow removing the ugly
>    `org-babel-python-eoe-indicator' we currently print.

Beware non-standard REPL prompts. When we have something other than >>>,
the logic may have hard time figuring out output boundaries without
deterministic separators like eoe-indicator.

>    Downside is that the output would not be echoed into the session
>    anymore. To fix that, we could manually insert the captured output
>    into the comint session buffer after execution.  Alternatively, we
>    could add an argument to `python-shell-send-string-no-output' to
>    avoid suppressing output, submit it upstream to python.el, and then
>    backport to Org to support older emacs versions.

If we can (eventually) remove some custom code from Org and move it to
Emacs, it will be the best for working towards RMS request
https://orgmode.org/list/E1kIPh1-0001Lu-Rg@fencepost.gnu.org

> 3. Revisit a series of commits I made in 2020, which was supposed to
>    make session evaluation more robust, and was inspired by
>    `python-shell-send-string-no-output':
>    https://git.sr.ht/~bzg/org-mode/commit/4df12ea39
>
>    However, I had to partially revert that work, due to compatibility
>    issue with emacs 26.3:
>    https://list.orgmode.org/871rjcan53.fsf@kyleam.com/
>
>    I think we no longer support emacs 26.3, so I could potentially
>    revisit this now -- but it's been a few years and will take some
>    time to refresh my memory about this.

Note that I changed `org-babel-comint-wait-for-output' and some other
ob-comint machinery since that time. In particular, I tried to address
various edge cases we encountered with bash sessions. See the commentary
in `org-babel-comint-with-output'.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>