* R output with session polluted with ascii color codes
@ 2018-03-16 13:57 Tyler Smith
2018-03-16 15:25 ` William Denton
0 siblings, 1 reply; 5+ messages in thread
From: Tyler Smith @ 2018-03-16 13:57 UTC (permalink / raw)
To: Emacs Org-Mode Help
Hi,
The printed output of some R functions includes ascii color codes (i.e., ^[[3m^[[90m). This causes problems with R code blocks evaluated with the :session option. I've pasted a file below that demonstrates the problem. (should I attach it as a file? not sure what's preferable).
I can't find a way to disable this feature from R. However, I did find a way to filter out these codes from the babel side. The function of interest is ~org-babel-R-evaluate-session` in ob-R.el. Adding a ~(replace-regexp-in-string ansi-color-control-seq-regexp "" ...)~ form around the output strips away the color codes so they don't mess up the output. I've included that code in the example. If that makes sense I can put that together as a patch. If that doesn't make sense, please let me know how to fix this!
Best,
Tyler
* Reproducible Example
Start with ~emacs -Q~, then evaluate each of the following code blocks
in turn.
#+BEGIN_SRC elisp setup
(require 'package)
(setq package-load-list
'((org-plus-contrib t)
(ess t)
(julia-mode t)))
(package-initialize)
(require 'org)
(require 'ess)
(org-babel-do-load-languages
'org-babel-load-languages
'((emacs-lisp . t)
(R . t)
(shell . t)))
#+END_SRC
#+RESULTS:
If you don't already have ~tidyr~ and ~dplyr~ installed, you need to
do that before running the following:
#+BEGIN_SRC R :results output
library(tidyr)
library(dplyr)
as_tibble(iris)
#+END_SRC
#+RESULTS:
#+begin_example
# A tibble: 150 x 5
Sepal.Length Sepal.Width Petal.Length Petal.Width Species
<dbl> <dbl> <dbl> <dbl> <fct>
1 5.10 3.50 1.40 0.200 setosa
2 4.90 3.00 1.40 0.200 setosa
3 4.70 3.20 1.30 0.200 setosa
4 4.60 3.10 1.50 0.200 setosa
5 5.00 3.60 1.40 0.200 setosa
6 5.40 3.90 1.70 0.400 setosa
7 4.60 3.40 1.40 0.300 setosa
8 5.00 3.40 1.50 0.200 setosa
9 4.40 2.90 1.40 0.200 setosa
10 4.90 3.10 1.50 0.100 setosa
# ... with 140 more rows
#+end_example
#+BEGIN_SRC R :results output :session RSESSION
library(tidyr)
library(dplyr)
as_tibble(iris)
#+END_SRC
#+RESULTS:
#+begin_example
Attaching package: ‘dplyr’
The following objects are masked from ‘package:stats’:
filter, lag
The following objects are masked from ‘package:base’:
intersect, setdiff, setequal, union
^[[90m# A tibble: 150 x 5^[[39m
Sepal.Length Sepal.Width Petal.Length Petal.Width Species
^[[3m^[[90m<dbl>^[[39m^[[23m ^[[3m^[[90m<dbl>^[[39m^[[23m ^[[3m^[[90m<dbl>^[[39m^[[23m ^[[3m^[[90m<dbl>^[[39m^[[23m ^[[3m^[[90m<fct>^[[39m^[[23m
^[[90m 1^[[39m 5.10 3.50 1.40 ^[[90m0^[[39m^[[90m.^[[39m200 setosa
^[[90m 2^[[39m 4.90 3.00 1.40 ^[[90m0^[[39m^[[90m.^[[39m200 setosa
^[[90m 3^[[39m 4.70 3.20 1.30 ^[[90m0^[[39m^[[90m.^[[39m200 setosa
^[[90m 4^[[39m 4.60 3.10 1.50 ^[[90m0^[[39m^[[90m.^[[39m200 setosa
^[[90m 5^[[39m 5.00 3.60 1.40 ^[[90m0^[[39m^[[90m.^[[39m200 setosa
^[[90m 6^[[39m 5.40 3.90 1.70 ^[[90m0^[[39m^[[90m.^[[39m400 setosa
^[[90m 7^[[39m 4.60 3.40 1.40 ^[[90m0^[[39m^[[90m.^[[39m300 setosa
^[[90m 8^[[39m 5.00 3.40 1.50 ^[[90m0^[[39m^[[90m.^[[39m200 setosa
^[[90m 9^[[39m 4.40 2.90 1.40 ^[[90m0^[[39m^[[90m.^[[39m200 setosa
^[[90m10^[[39m 4.90 3.10 1.50 ^[[90m0^[[39m^[[90m.^[[39m100 setosa
^[[90m# ... with 140 more rows^[[39m
#+end_example
* Proposed Fix
Evaluating the following replaces the built-in function with my fix:
#+BEGIN_SRC elisp fix
(defun org-babel-R-evaluate-session
(session body result-type result-params column-names-p row-names-p)
"Evaluate BODY in SESSION.
If RESULT-TYPE equals `output' then return standard output as a
string. If RESULT-TYPE equals `value' then return the value of the
last statement in BODY, as elisp."
(cl-case result-type
(value
(with-temp-buffer
(insert (org-babel-chomp body))
(let ((ess-local-process-name
(process-name (get-buffer-process session)))
(ess-eval-visibly-p nil))
(ess-eval-buffer nil)))
(let ((tmp-file (org-babel-temp-file "R-")))
(org-babel-comint-eval-invisibly-and-wait-for-file
session tmp-file
(format org-babel-R-write-object-command
(if row-names-p "TRUE" "FALSE")
(if column-names-p
(if row-names-p "NA" "TRUE")
"FALSE")
".Last.value" (org-babel-process-file-name tmp-file 'noquote)))
(org-babel-R-process-value-result
(org-babel-result-cond result-params
(with-temp-buffer
(insert-file-contents tmp-file)
(org-babel-chomp (buffer-string) "\n"))
(org-babel-import-elisp-from-file tmp-file '(16)))
column-names-p)))
(output
;; strip ansi-color-control-seq-regexp from output!!
(replace-regexp-in-string
ansi-color-control-seq-regexp ""
(mapconcat
'org-babel-chomp
(butlast
(delq nil
(mapcar
(lambda (line) (when (> (length line) 0) line))
(mapcar
(lambda (line) ;; cleanup extra prompts left in output
(if (string-match
"^\\([>+.]\\([ ][>.+]\\)*[ ]\\)"
(car (split-string line "\n")))
(substring line (match-end 1))
line))
(org-babel-comint-with-output (session org-babel-R-eoe-output)
(insert (mapconcat 'org-babel-chomp
(list body org-babel-R-eoe-indicator)
"\n"))
(inferior-ess-send-input)))))) "\n")))))
#+END_SRC
#+RESULTS:
: org-babel-R-evaluate-session
#+BEGIN_SRC R :results output :session RSESSION
as_tibble(iris)
#+END_SRC
#+RESULTS:
#+begin_example
# A tibble: 150 x 5
Sepal.Length Sepal.Width Petal.Length Petal.Width Species
<dbl> <dbl> <dbl> <dbl> <fct>
1 5.10 3.50 1.40 0.200 setosa
2 4.90 3.00 1.40 0.200 setosa
3 4.70 3.20 1.30 0.200 setosa
4 4.60 3.10 1.50 0.200 setosa
5 5.00 3.60 1.40 0.200 setosa
6 5.40 3.90 1.70 0.400 setosa
7 4.60 3.40 1.40 0.300 setosa
8 5.00 3.40 1.50 0.200 setosa
9 4.40 2.90 1.40 0.200 setosa
10 4.90 3.10 1.50 0.100 setosa
# ... with 140 more rows
#+end_example
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: R output with session polluted with ascii color codes
2018-03-16 13:57 R output with session polluted with ascii color codes Tyler Smith
@ 2018-03-16 15:25 ` William Denton
2018-03-16 17:23 ` Tyler Smith
0 siblings, 1 reply; 5+ messages in thread
From: William Denton @ 2018-03-16 15:25 UTC (permalink / raw)
To: Tyler Smith; +Cc: Emacs Org-Mode Help
On 16 March 2018, Tyler Smith wrote:
> ^[[90m# A tibble: 150 x 5^[[39m
> Sepal.Length Sepal.Width Petal.Length Petal.Width Species
> ^[[3m^[[90m<dbl>^[[39m^[[23m ^[[3m^[[90m<dbl>^[[39m^[[23m ^[[3m^[[90m<dbl>^[[39m^[[23m ^[[3m^[[90m<dbl>^[[39m^[[23m ^[[3m^[[90m<fct>^[[39m^[[23m
> ^[[90m 1^[[39m 5.10 3.50 1.40 ^[[90m0^[[39m^[[90m.^[[39m200 setosa
> ^[[90m 2^[[39m 4.90 3.00 1.40 ^[[90m0^[[39m^[[90m.^[[39m200 setosa
I tried the example in my configured setup and didn't get the ASCII codes. I
don't know why and didn't dig into it, but I could; for now I figured I'd just
mention this.
Bill
--
William Denton :: Toronto, Canada --- Listening to Art: https://listeningtoart.org/
https://www.miskatonic.org/ --- GHG.EARTH: http://ghg.earth/
Caveat lector. --- STAPLR: http://staplr.org/
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: R output with session polluted with ascii color codes
2018-03-16 15:25 ` William Denton
@ 2018-03-16 17:23 ` Tyler Smith
2018-03-16 18:55 ` William Denton
0 siblings, 1 reply; 5+ messages in thread
From: Tyler Smith @ 2018-03-16 17:23 UTC (permalink / raw)
To: William Denton; +Cc: Emacs Org-Mode Help
On Fri, Mar 16, 2018, at 11:25 AM, William Denton wrote:
>
> I tried the example in my configured setup and didn't get the ASCII codes. I
> don't know why and didn't dig into it, but I could; for now I figured I'd just
> mention this.
>
Thanks for checking. I should have mentioned that I'm running 27.0.50 with org 9.1.7 and ess 17.11. I see this problem with emacs -Q and regular emacs. If you're using the same versions and don't see this, then there should be something I can do in my config to fix it.
Best,
Tyler
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: R output with session polluted with ascii color codes
2018-03-16 17:23 ` Tyler Smith
@ 2018-03-16 18:55 ` William Denton
2018-03-16 19:22 ` Tyler Smith
0 siblings, 1 reply; 5+ messages in thread
From: William Denton @ 2018-03-16 18:55 UTC (permalink / raw)
To: Tyler Smith; +Cc: Emacs Org-Mode Help
On 16 March 2018, Tyler Smith wrote:
> Thanks for checking. I should have mentioned that I'm running 27.0.50 with org
> 9.1.7 and ess 17.11. I see this problem with emacs -Q and regular emacs. If
> you're using the same versions and don't see this, then there should be
> something I can do in my config to fix it.
I think we're the same---Org says it's 9.1.6, but I just compiled it fresh from
source five minutes ago, so it should be up to date. Not sure what's going on
there. I recompiled Emacs from fresh source, too, and updated the ESS package
(17.11, ELPA 20180314.612).
Anyway, I tried again, and there was still no problem in Emacs. That made me
wonder if the difference was in R. When I run "as_tibble(iris)" in the R
console in a terminal I don't get the ASCII colour codes, so maybe that's where
the difference lies? I'm running R 3.4.2, compiled from source, on Ubuntu
17.10.
Is there an R user on the list who sees the coloured text in the R console and
can try the example in Org?
#+BEGIN_SRC R :results output :session RSESSION
library(tidyr)
library(dplyr)
as_tibble(iris)
#+END_SRC
Bill
--
William Denton :: Toronto, Canada --- Listening to Art: https://listeningtoart.org/
https://www.miskatonic.org/ --- GHG.EARTH: http://ghg.earth/
Caveat lector. --- STAPLR: http://staplr.org/
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: R output with session polluted with ascii color codes
2018-03-16 18:55 ` William Denton
@ 2018-03-16 19:22 ` Tyler Smith
0 siblings, 0 replies; 5+ messages in thread
From: Tyler Smith @ 2018-03-16 19:22 UTC (permalink / raw)
To: William Denton; +Cc: Emacs Org-Mode Help
On Fri, Mar 16, 2018, at 2:55 PM, William Denton wrote:
> On 16 March 2018, Tyler Smith wrote:
>
> Anyway, I tried again, and there was still no problem in Emacs. That made me
> wonder if the difference was in R. When I run "as_tibble(iris)" in the R
> console in a terminal I don't get the ASCII colour codes, so maybe that's where
> the difference lies? I'm running R 3.4.2, compiled from source, on Ubuntu
> 17.10.
I've tried this as well, outside of Emacs. For me, it depends on which terminal emulator I use. xterm displays the colours, while gnome terminal does not. Which is odd, because I use ansi colour codes for my prompt and ls output (etc), and those do display as expected in both xterm and gnome terminal.
Weird!
Thanks,
Tyler
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2018-03-16 19:22 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-03-16 13:57 R output with session polluted with ascii color codes Tyler Smith
2018-03-16 15:25 ` William Denton
2018-03-16 17:23 ` Tyler Smith
2018-03-16 18:55 ` William Denton
2018-03-16 19:22 ` Tyler Smith
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).