RE: org-babel-execute-src-block filters characters from :session *shell* output AND Re: [BUG] conda doesn't work in ob-shell sessions

["Cook, Malcolm" <MEC@stowers.org>]

I have a new workaround, which is to configure conda to not change the prompt.

    conda config --set changeps1 false

As discussed elsewhere:

	Do not modify prompt by default #10928
	https://github.com/conda/conda/issues/10928

I generally would not want to do this in normal interactive use, but in this context, am going to proceed.

I think it is preferable to trying to improve on prompt tracking.  That appears just to lead to more convolutions.

Thoughts?

> # -*-  org-confirm-babel-evaluate: nil; -*-
> 
> Excuse the top-posting.  I’m reviving two old-ish threads and felt it best.
> 
> I have been struggling with interrelated issues raised in
> 
>  - https://list.orgmode.org/87jznda90u.fsf@localhost/#t
>  - https://list.orgmode.org/87le1bc8j3.fsf@localhost/
> 
> I expect I am using all the patches offered in addressing these given my recent
> build from main.  However, in my hands, I find they still easily allow for
> mistakes identifying prompts in code block results.
> 
> In this demonstration, I am extending the approach to inquiry begun by Jack
> in https://list.orgmode.org/87ttzn1mai.fsf@gmail.com/
> 
> #+begin_src emacs-lisp
>    (emacs-version)
> #+end_src
> 
> #+RESULTS:
> : GNU Emacs 31.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo version
> 1.15.12, Xaw3d scroll bars)
> :  of 2024-09-04
> 
> #+begin_src emacs-lisp
>    (org-version nil t)
> #+end_src
> 
> #+RESULTS:
> : Org mode version 9.7.13 (9.7.13-8566bc @ /home/mec/.emacs.d/elpa/org-
> 9.7.13/)
> 
> 
> 
> #+begin_src emacs-lisp
>   (org-babel-do-load-languages
>     'org-babel-load-languages
>     '((shell . t)))
> #+end_src
> 
> #+RESULTS:
> 
> Here I define two org code blocks I will use repeatedly below:
> 
> #+name:test_filter
> #+begin_src shell :session *shell* :results output
>   printf "a\nb\nc\n>d\n<e\nf>\nggg ggg>\nhhh hhh+\na\n"
> #+end_src
> 
> #+name:shell_prompt_info
> #+begin_src elisp
>   (with-current-buffer "*shell*"
>     (format "[comint-prompt-regexp]=[%s]\n[org-babel-comint-prompt-regexp-
> old]=[%s]" comint-prompt-regexp org-babel-comint-prompt-regexp-old))
> #+end_src
> 
> #+caption: The results looks good - the output apparently is not confused as
> being prompt.
> #+call: test_filter()
> 
> #+RESULTS:
> : a
> : b
> : c
> : >d
> : <e
> : f>
> : ggg ggg>
> : hhh hhh+
> : a
> 
> #+caption: take a look at the prompt variables.
> #+call:shell_prompt_info()
> 
> #+RESULTS:
> : [comint-prompt-regexp]=[^org_babel_sh_prompt>  *]
> : [org-babel-comint-prompt-regexp-old]=[^[^#$%>
> : ]*[#$%>] *]
> 
> #+caption: check on conda's availabiity & version #+begin_src shell :session
> *shell* :results output conda --version #+end_src
> 
> #+RESULTS:
> : conda 24.7.1
> 
> #+begin_src shell :session *shell* :results output conda create --yes --name
> myenv python=3.9 #+end_src
> 
> #+RESULTS:
> #+begin_example
> ... abbreviated...
> 
> To activate this environment, use
> 
> conda activate myenv
> 
> To deactivate an active environment, use
> 
> conda deactivate
> #+end_example
> 
> #+begin_src shell :session *shell* :results output
>   conda activate myenv
> #+end_src
> 
> #+RESULTS:
> 
> #+begin_src shell :session *shell* :results output which python #+end_src
> 
> #+RESULTS:
> : /n/projects/mec/SRSCHPC2/local/inst/Mambaforge/24.3.0-
> 0/envs/myenv/bin/python
> 
> #+caption: alas, the output of test_filter is changed.  Some lines are gone
> missing and some are changed.
> #+call: test_filter()
> 
> #+RESULTS:
> : a
> : b
> : c
> : d
> : <e
> :
> : hhh hhh+
> : a
> 
> #+caption: Observe the prompts have changed.  Perhaps this is related issue?
> #+call:shell_prompt_info()
> 
> #+RESULTS:
> : [comint-prompt-regexp]=[^[^#$%>
> : ]*[#$%>] *]
> : [org-babel-comint-prompt-regexp-old]=[^org_babel_sh_prompt>  *]
> 
> #+caption:  can we restore by deactivating the environment?
> #+begin_src shell :session *shell* :results output
>   conda deactivate
> #+end_src
> 
> #+RESULTS:
> 
> #+caption: alas, no:
> #+call: test_filter()
> 
> #+RESULTS:
> : a
> : b
> : c
> : d
> : <e
> :
> : hhh hhh+
> : a
> 
> #+caption: how about by resetting the prompt #+begin_src shell :session
> *shell* :results output
>   PROMPT_COMMAND=;PS1="org_babel_sh_prompt> ";PS2= #+end_src
> 
> #+RESULTS:
> 
> #+caption: alas, again, no
> #+call: test_filter()
> #+RESULTS:
> : a
> : b
> : c
> : d
> : <e
> :
> : hhh hhh+
> : a
> 
> #+caption: perhaps restoring the prompt variables will recover?
> #+begin_src elisp
>     (with-current-buffer "*shell*"
>       (setq-local comint-prompt-regexp "^org_babel_sh_prompt>  *"
>       	      org-babel-comint-prompt-regexp-old "[^[^#$%> ]*[#$%>] *"))
> #+end_src
> 
> #+RESULTS:
> : [^[^#$%>
> : ]*[#$%>] *
> 
> #+caption: YES!
> #+call: test_filter()
> 
> #+RESULTS:
> : a
> : b
> : c
> : >d
> : <e
> : f>
> : ggg ggg>
> : hhh hhh+
> : a
> 
> I'm unsure what change this argues for, but I think it pretty clearly
> demonstrates the ongoing issue.
> 
> In the above, I am exclusively allowing org/ob/comint to "own" the shell
> buffer, and not interact with it, as recommended earlier by Ivor.
> 
> I have tried the above after first calling `(shell)` and find variations on the
> above occur.  I would like to be able to 'share' the
> *shell* buffer with org/ob/comint but expect resolving the non-interactive
> case should possibly lay foundation.
> 
> I would additional like to layer in working with remote shells (e.g. `:dir
> "/ssh:me@host:~/`) and have tried but this is just layering in complexity on
> the localhost case so I'm backing off for now.
> 
> What else can I report or test?