[BUG] org-babel-comint-with-output: incorrect prompt detection [9.6-pre]

[BUG] org-babel-comint-with-output: incorrect prompt detection [9.6-pre]

[Bruno Barbier]

Hi,

The function `org-babel-comint-with-output' (o-b-c-w-o) may not detect
prompts correctly.

The function o-b-c-w-o checks if the incoming arbitrary chunk of text
contains whole prompts, and replace them.

But the incoming text may contain an incomplete prompt at the
end. When this happens, it will miss that real prompt.

For example, using a haskell prompt, let say the prompt is "Prelude> ",
the first incoming text is "Pre", the second incoming text is
"lude> ", o-b-c-w-o will wrongly infer that "lude> " is the prompt,
and that "Pre" was part of the previous value/output.

This is with current main (after commit 1ef420b19d2e310d9ef67e090ef84cdb6da4744d).

I'm randomly hit by this bug, when working on tests for ob-haskell.

Let me know if I need to provide more information,

Thanks,

Bruno

Re: [BUG] org-babel-comint-with-output: incorrect prompt detection [9.6-pre]

[Ihor Radchenko]

Bruno Barbier <perso.bruno.barbier@free.fr> writes:

> For example, using a haskell prompt, let say the prompt is "Prelude> ",
> the first incoming text is "Pre", the second incoming text is
> "lude> ", o-b-c-w-o will wrongly infer that "lude> " is the prompt,
> and that "Pre" was part of the previous value/output.

The problem is that Bash can send incoming text like

"prompt> " "prompt> " "prompt> output\n".

"prompt> prompt> prompt> output\n".

So, we cannot reliably distinguish your case from other possibly comint
quirks based on newline.

One possible fix could be replacing `comint-prompt-regexp' one more time
at the very end in addition to `org-babel-comint-prompt-separator'. We
must also make `comint-prompt-regexp' more precise to avoid "lude> "
being detected as a prompt.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] org-babel-comint-with-output: incorrect prompt detection [9.6-pre]

[Bruno Barbier]

Ihor Radchenko <yantar92@posteo.net> writes:

> The problem is that Bash can send incoming text like
>
> "prompt> " "prompt> " "prompt> output\n".
>
> "prompt> prompt> prompt> output\n".

Yes. And I've seen these "outputs" with ob-haskell too ...

>
> So, we cannot reliably distinguish your case from other possibly comint
> quirks based on newline.
>
> One possible fix could be replacing `comint-prompt-regexp' one more time
> at the very end in addition to `org-babel-comint-prompt-separator'.

If I understand correctly, you suggest to do one last replace on the whole
text (buffer-string). But, for ob-haskell, the current prompt regex is:
     "^[[:alnum:].*_() |λ]*> "
     
(it starts with "^")

> We must also make `comint-prompt-regexp' more precise to avoid "lude> "
> being detected as a prompt.
If we need to do that for ob-haskell (when relying on the interpreter),
the cleanest way might be to redefine the set of possible prompts. ghci
is quite flexible:
      GHCi, version 8.10.7

   :set prompt <prompt>        set the prompt used in GHCi
   :set prompt-cont <prompt>   set the continuation prompt used in GHCi
   :set prompt-function <expr> set the function to handle the prompt
   :set prompt-cont-function <expr>
                               set the function to handle the continuation prompt

I'll keep thinking about how to solve this.

Do we have common design principles and/or generic tests that all
languages based on org-babel/ob-comint must pass ?


Thanks Ihor!

Bruno




>
> -- 
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>

Re: [BUG] org-babel-comint-with-output: incorrect prompt detection [9.6-pre]

[Ihor Radchenko]

Bruno Barbier <perso.bruno.barbier@free.fr> writes:

>> We must also make `comint-prompt-regexp' more precise to avoid "lude> "
>> being detected as a prompt.
> If we need to do that for ob-haskell (when relying on the interpreter),
> the cleanest way might be to redefine the set of possible prompts. ghci
> is quite flexible:
>       GHCi, version 8.10.7
>
>    :set prompt <prompt>        set the prompt used in GHCi
>    :set prompt-cont <prompt>   set the continuation prompt used in GHCi
>    :set prompt-function <expr> set the function to handle the prompt
>    :set prompt-cont-function <expr>
>                                set the function to handle the continuation prompt

This is the approach I used in ob-shell. See
`org-babel-shell-set-prompt-commands' and its usage in the code.

> Do we have common design principles and/or generic tests that all
> languages based on org-babel/ob-comint must pass ?

No tests. (I am not sure what can be tested generically).

The common design is ob-comint.el API and ob-core.el API.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] org-babel-comint-with-output: incorrect prompt detection [9.6-pre]

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

> One possible fix could be replacing `comint-prompt-regexp' one more time
> at the very end in addition to `org-babel-comint-prompt-separator'. We
> must also make `comint-prompt-regexp' more precise to avoid "lude> "
> being detected as a prompt.

This is what I ended up doing because of another similar bug report.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=456462741

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>