emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)]
@ 2022-11-30 13:33 Frédéric Santos
  2022-11-30 22:47 ` Cook, Malcolm
  2022-12-01  5:48 ` Ihor Radchenko
  0 siblings, 2 replies; 15+ messages in thread
From: Frédéric Santos @ 2022-11-30 13:33 UTC (permalink / raw)
  To: emacs-orgmode

Hi everyone,

With Org 9.6 released yesterday, I noticed a bug with ob-R: all R code
blocks with ":results output :session *R*" in the header will return no
output at all (although the code is correctly processed in the inferior
ESS buffer).

Steps to follow to reproduce the issue:

1. Create a minimal init.el file with this content (adapting paths if needed):

(add-to-list 'load-path "~/.emacs.d/elpa/org-9.6/")
(require 'org)
(add-to-list 'load-path "~/.emacs.d/elpa/ess-20221124.756/")
(require 'ess-site)
(org-babel-do-load-languages
 'org-babel-load-languages
 '((org . t)
   (R . t)))

2. Open emacs -Q -l init.el

3. Create an Org document with this content:

#+begin_src R :results output :session *R*
x <- 2:4
print(x)
#+end_src

4. Try to evaluate the R code block: you'll only get an empty output.

Note that external eval (i.e., without the :session arg) still works as
expected, and that other results types (e.g., :results value) still work
as well.

Bug confirmed by other ESS users on the mailing list:
https://stat.ethz.ch/pipermail/ess-help/2022-November/013162.html

Version info:
============
Emacs  : GNU Emacs 28.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.34, cairo version 1.17.6) of 2022-09-12
Org    : Org mode version 9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)
ESS    : ess-version: 18.10.3snapshot [elpa: 20221124.756]


^ permalink raw reply	[flat|nested] 15+ messages in thread

* RE: [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)]
  2022-11-30 13:33 [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)] Frédéric Santos
@ 2022-11-30 22:47 ` Cook, Malcolm
  2022-12-01  5:48 ` Ihor Radchenko
  1 sibling, 0 replies; 15+ messages in thread
From: Cook, Malcolm @ 2022-11-30 22:47 UTC (permalink / raw)
  To: Frédéric Santos, emacs-orgmode@gnu.org


>Bug confirmed by other ESS users on the mailing list:
>https://stat.ethz.ch/pipermail/ess-help/2022-November/013162.html

FWIW: confirmed here too with

GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo version 1.15.12, Xaw3d scroll bars) of 2022-01-19
Org mode version 9.6-pre (release_9.5.5-1136-gf78dfe @ /home/mec/.emacs.d/org-mode/lisp/)
ess-version: 18.10.3snapshot [elpa: 20221124.756] (loaded from /home/mec/.emacs.d/elpa/ess-20221124.756/)

~ malcolm_cook@stowers.org



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)]
  2022-12-01  6:26   ` William Denton
@ 2022-12-01  1:48     ` Ihor Radchenko
  2022-12-01  7:26       ` Frederic Santos
                         ` (2 more replies)
  0 siblings, 3 replies; 15+ messages in thread
From: Ihor Radchenko @ 2022-12-01  1:48 UTC (permalink / raw)
  To: William Denton; +Cc: Frédéric Santos, emacs-orgmode

William Denton <wtd@pobox.com> writes:

> I tried that, but it didn't fix it. :(

Ok. I was able to reproduce on my side.

> With a minimal setup, running the R block with a :session makes it ask where the 
> working directory should be (the default in the prompt is the directory the test 
> Org file is in).  Setting ess-startup-directory to 'default-directory doesn't 
> change that, or the output problem.

It looks like there is something wrong with comint-prompt-regexp in R
buffers. It somehow does not match the prompt.

I will need to look further.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)]
  2022-11-30 13:33 [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)] Frédéric Santos
  2022-11-30 22:47 ` Cook, Malcolm
@ 2022-12-01  5:48 ` Ihor Radchenko
  2022-12-01  6:26   ` William Denton
  1 sibling, 1 reply; 15+ messages in thread
From: Ihor Radchenko @ 2022-12-01  5:48 UTC (permalink / raw)
  To: Frédéric Santos; +Cc: emacs-orgmode

Frédéric Santos <frederic.santos@u-bordeaux.fr> writes:

> Hi everyone,
>
> With Org 9.6 released yesterday, I noticed a bug with ob-R: all R code
> blocks with ":results output :session *R*" in the header will return no
> output at all (although the code is correctly processed in the inferior
> ESS buffer).

May it be a duplicate of
https://orgmode.org/list/87bkoxj50h.fsf@localhost?

I did not test this report itself, but if my guess is correct, setting
`ess-startup-directory' to 'default-directory might be a temporary
workaround until we find the fix to this breaking change in ESS.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)]
  2022-12-01  5:48 ` Ihor Radchenko
@ 2022-12-01  6:26   ` William Denton
  2022-12-01  1:48     ` Ihor Radchenko
  0 siblings, 1 reply; 15+ messages in thread
From: William Denton @ 2022-12-01  6:26 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: Frédéric Santos, emacs-orgmode

On 1 December 2022, Ihor Radchenko wrote:

>> With Org 9.6 released yesterday, I noticed a bug with ob-R: all R code
>> blocks with ":results output :session *R*" in the header will return no
>> output at all (although the code is correctly processed in the inferior
>> ESS buffer).
>
> May it be a duplicate of
> https://orgmode.org/list/87bkoxj50h.fsf@localhost?
>
> I did not test this report itself, but if my guess is correct, setting
> `ess-startup-directory' to 'default-directory might be a temporary
> workaround until we find the fix to this breaking change in ESS.

I tried that, but it didn't fix it. :(

With a minimal setup, running the R block with a :session makes it ask where the 
working directory should be (the default in the prompt is the directory the test 
Org file is in).  Setting ess-startup-directory to 'default-directory doesn't 
change that, or the output problem.

Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.
Toronto, Canada


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)]
  2022-12-01  1:48     ` Ihor Radchenko
@ 2022-12-01  7:26       ` Frederic Santos
  2022-12-01 16:34         ` Greg Minshall
  2022-12-01 21:18       ` Berry, Charles
  2022-12-02  6:02       ` Ihor Radchenko
  2 siblings, 1 reply; 15+ messages in thread
From: Frederic Santos @ 2022-12-01  7:26 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: William Denton, emacs-orgmode

----- Mail original -----
> De: "Ihor Radchenko" <yantar92@posteo.net>
> À: "William Denton" <wtd@pobox.com>
> Cc: "Frederic Santos" <frederic.santos@u-bordeaux.fr>, "emacs-orgmode" <emacs-orgmode@gnu.org>
> Envoyé: Jeudi 1 Décembre 2022 02:48:22
> Objet: Re: [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @
> /home/fsantos/.emacs.d/elpa/org-9.6/)]

> It looks like there is something wrong with comint-prompt-regexp in R
> buffers. It somehow does not match the prompt.

I'm not sure it's the explanation either, since external (i.e., non-session) evaluation works as expected. (Just remove the ":session *R*" part of the header, and you'll see everything is okay.) Thus, I guess this implies that R prompts are correctly recognized (?).
Also, an important difference between external and session eval is that session eval uses ESS, and external eval does not. So it *might* be an ESS-related issue, but since everything works fine with Org 9.5.5 and the latest ESS version, I wouldn't bet on that.


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)]
  2022-12-01  7:26       ` Frederic Santos
@ 2022-12-01 16:34         ` Greg Minshall
  2022-12-02  5:08           ` Ihor Radchenko
  0 siblings, 1 reply; 15+ messages in thread
From: Greg Minshall @ 2022-12-01 16:34 UTC (permalink / raw)
  To: Frederic Santos; +Cc: Ihor Radchenko, William Denton, emacs-orgmode

Frederic,

> > It looks like there is something wrong with comint-prompt-regexp in
> > R buffers. It somehow does not match the prompt.
> 
> I'm not sure it's the explanation either, since external (i.e.,
> non-session) evaluation works as expected. (Just remove the ":session
> *R*" part of the header, and you'll see everything is okay.)

Ihor will know, but i *think* that without :session, there is no R
"shell" running, so the ...-prompt-regexp isn't used.  rather (do i know
this?  assume this?), an R script is prepared and executed.

cheers, Greg


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)]
  2022-12-01  1:48     ` Ihor Radchenko
  2022-12-01  7:26       ` Frederic Santos
@ 2022-12-01 21:18       ` Berry, Charles
  2022-12-02  6:02       ` Ihor Radchenko
  2 siblings, 0 replies; 15+ messages in thread
From: Berry, Charles @ 2022-12-01 21:18 UTC (permalink / raw)
  To: Ihor Radchenko
  Cc: William Denton, Frédéric Santos, emacs-orgmode@gnu.org



> On Nov 30, 2022, at 5:48 PM, Ihor Radchenko <yantar92@posteo.net> wrote:
> 
> William Denton <wtd@pobox.com> writes:
> 
>> I tried that, but it didn't fix it. :(
> 
> Ok. I was able to reproduce on my side.
> 
>> With a minimal setup, running the R block with a :session makes it ask where the 
>> working directory should be (the default in the prompt is the directory the test 
>> Org file is in).  Setting ess-startup-directory to 'default-directory doesn't 
>> change that, or the output problem.
> 
> It looks like there is something wrong with comint-prompt-regexp in R
> buffers. It somehow does not match the prompt.
> 
> I will need to look further.
> 

The older (commit bed47b437d8cde7a98bafdb07996e248b40f70e6e) version of org-babel-comint-with-output used comint-prompt-regexp and works fine.

I don't really follow how `org-babel-comint-prompt-separator` gets set to something useful in the current version. 

Reverting org-babel-comint-with-output to that older commit (and eval-ing the defun for org-babel-R-evaluate-session) seems to fix this case for those who need `:results output` in R src blocks till this gets sorted out.

HTH,

Chuck


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)]
  2022-12-01 16:34         ` Greg Minshall
@ 2022-12-02  5:08           ` Ihor Radchenko
  0 siblings, 0 replies; 15+ messages in thread
From: Ihor Radchenko @ 2022-12-02  5:08 UTC (permalink / raw)
  To: Greg Minshall; +Cc: Frederic Santos, William Denton, emacs-orgmode

Greg Minshall <minshall@umich.edu> writes:

> Ihor will know, but i *think* that without :session, there is no R
> "shell" running, so the ...-prompt-regexp isn't used.  rather (do i know
> this?  assume this?), an R script is prepared and executed.

You are right.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)]
  2022-12-01  1:48     ` Ihor Radchenko
  2022-12-01  7:26       ` Frederic Santos
  2022-12-01 21:18       ` Berry, Charles
@ 2022-12-02  6:02       ` Ihor Radchenko
  2022-12-03 22:06         ` Jeremie Juste
  2022-12-06 23:40         ` Cook, Malcolm
  2 siblings, 2 replies; 15+ messages in thread
From: Ihor Radchenko @ 2022-12-02  6:02 UTC (permalink / raw)
  To: William Denton; +Cc: Frédéric Santos, emacs-orgmode

Ihor Radchenko <yantar92@posteo.net> writes:

> It looks like there is something wrong with comint-prompt-regexp in R
> buffers. It somehow does not match the prompt.
>
> I will need to look further.

Fixed on bugfix.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=456462741

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)]
  2022-12-02  6:02       ` Ihor Radchenko
@ 2022-12-03 22:06         ` Jeremie Juste
  2022-12-06 23:40         ` Cook, Malcolm
  1 sibling, 0 replies; 15+ messages in thread
From: Jeremie Juste @ 2022-12-03 22:06 UTC (permalink / raw)
  To: Ihor Radchenko
  Cc: William Denton, Frédéric Santos, emacs-orgmode,
	Berry, Charles


Hello Ihor,

On Friday,  2 Dec 2022 at 06:02, Ihor Radchenko wrote:>
> Fixed on bugfix.
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=456462741


Many thanks to you all for helping solve this issue.
I apologise for the delayed response. From ffb30cd7373, I added two
tests to make sure this issue does not happen again in ob-R.


Best regards,
Jeremie



^ permalink raw reply	[flat|nested] 15+ messages in thread

* RE: [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)]
  2022-12-02  6:02       ` Ihor Radchenko
  2022-12-03 22:06         ` Jeremie Juste
@ 2022-12-06 23:40         ` Cook, Malcolm
  2022-12-07 12:25           ` Ihor Radchenko
  1 sibling, 1 reply; 15+ messages in thread
From: Cook, Malcolm @ 2022-12-06 23:40 UTC (permalink / raw)
  To: Ihor Radchenko, William Denton
  Cc: Frédéric Santos, emacs-orgmode@gnu.org

[-- Attachment #1: Type: text/plain, Size: 1513 bytes --]

Hi,

I tried applying just the change in https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=456462741 to released Org 9.6 and find that when using `:session *R*` the #+RESULTS are still empty, but when running without :session the results are provided.

Is there any diagnostic or reprex I can provide to demonstrate that it persists?

~ malcolm_cook@stowers.org
From: emacs-orgmode-bounces+mec=stowers.org@gnu.org <emacs-orgmode-bounces+mec=stowers.org@gnu.org> On Behalf Of Ihor Radchenko
Sent: Friday, December 2, 2022 12:03 AM
To: William Denton <wtd@pobox.com>
Cc: Frédéric Santos <frederic.santos@u-bordeaux.fr>; emacs-orgmode@gnu.org
Subject: Re: [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)]

Ihor Radchenko <yantar92@posteo.net<mailto:yantar92@posteo.net>> writes:

> It looks like there is something wrong with comint-prompt-regexp in R
> buffers. It somehow does not match the prompt.
>
> I will need to look further.

Fixed on bugfix.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=456462741<https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=456462741>

--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/<https://orgmode.org>>.
Support Org development at <https://liberapay.com/org-mode<https://liberapay.com/org-mode>>,
or support my work at <https://liberapay.com/yantar92<https://liberapay.com/yantar92>>

[-- Attachment #2: Type: text/html, Size: 40924 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* RE: [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)]
  2022-12-06 23:40         ` Cook, Malcolm
@ 2022-12-07 12:25           ` Ihor Radchenko
  2022-12-07 17:50             ` Cook, Malcolm
  0 siblings, 1 reply; 15+ messages in thread
From: Ihor Radchenko @ 2022-12-07 12:25 UTC (permalink / raw)
  To: Cook, Malcolm
  Cc: William Denton, Frédéric Santos, emacs-orgmode@gnu.org

"Cook, Malcolm" <MEC@stowers.org> writes:

> I tried applying just the change in https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=456462741 to released Org 9.6 and find that when using `:session *R*` the #+RESULTS are still empty, but when running without :session the results are provided.

Could you please elaborate what you mean by "just the change"? This
change is a fix on top of another change we included in Org 9.6.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>


^ permalink raw reply	[flat|nested] 15+ messages in thread

* RE: [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)]
  2022-12-07 12:25           ` Ihor Radchenko
@ 2022-12-07 17:50             ` Cook, Malcolm
  2022-12-07 19:04               ` Jeremie Juste
  0 siblings, 1 reply; 15+ messages in thread
From: Cook, Malcolm @ 2022-12-07 17:50 UTC (permalink / raw)
  To: Ihor Radchenko
  Cc: William Denton, Frédéric Santos, emacs-orgmode@gnu.org

>"Cook, Malcolm" <mailto:MEC@stowers.org> writes:
>
>> I tried applying just the change in https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=456462741 to released Org 9.6 and find that when using `:session *R*` the #+RESULTS are still empty, but when running without :session the results are provided.
>
>Could you please elaborate what you mean by "just the change"? This
>change is a fix on top of another change we included in Org 9.6.

Of course....

I manually edited ob-comint.el and inserted the lines marked with a '+' from https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=456462741 (removing the '+') in the right place, and re-evaluated the now modified function definition (went to end of defun and hit C-x C-e)

Doh!  

It is a macro not a defun, so simply redefining it will not effect where it was used.  

My bad.

Byte-compiling and reloading ob-R.el worked.

So, answering your question revealed my error.

Thanks.

I look forward to the fix finding elpa.

Cheers,

Malcolm



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)]
  2022-12-07 17:50             ` Cook, Malcolm
@ 2022-12-07 19:04               ` Jeremie Juste
  0 siblings, 0 replies; 15+ messages in thread
From: Jeremie Juste @ 2022-12-07 19:04 UTC (permalink / raw)
  To: Cook, Malcolm
  Cc: Ihor Radchenko, William Denton, Frédéric Santos,
	emacs-orgmode@gnu.org


Hello Malcolm,

On Wednesday,  7 Dec 2022 at 17:50, Cook, Malcolm wrote:
>
> Byte-compiling and reloading ob-R.el worked.

Glad that you could solve it. From 08433d9b0, I added a test to make sure we are not
surprised again. 

Best regards,
Jeremie




^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2022-12-07 19:05 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-30 13:33 [BUG] ob-R: session evaluation returns empty ouputs with Org 9.6 [9.6 ( @ /home/fsantos/.emacs.d/elpa/org-9.6/)] Frédéric Santos
2022-11-30 22:47 ` Cook, Malcolm
2022-12-01  5:48 ` Ihor Radchenko
2022-12-01  6:26   ` William Denton
2022-12-01  1:48     ` Ihor Radchenko
2022-12-01  7:26       ` Frederic Santos
2022-12-01 16:34         ` Greg Minshall
2022-12-02  5:08           ` Ihor Radchenko
2022-12-01 21:18       ` Berry, Charles
2022-12-02  6:02       ` Ihor Radchenko
2022-12-03 22:06         ` Jeremie Juste
2022-12-06 23:40         ` Cook, Malcolm
2022-12-07 12:25           ` Ihor Radchenko
2022-12-07 17:50             ` Cook, Malcolm
2022-12-07 19:04               ` Jeremie Juste

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).