* org tables into R?
@ 2015-01-02 19:57 Michael Gauland
2015-01-02 20:13 ` Andreas Leha
2015-01-04 3:19 ` Michael Gauland
0 siblings, 2 replies; 17+ messages in thread
From: Michael Gauland @ 2015-01-02 19:57 UTC (permalink / raw)
To: emacs-orgmode
I want to use an R block to manipulate data from an org table, but I'm not
getting what I expect.
If I define a table like this:
#+NAME: data
| A | B | C |
|-----+-----+----|
| 115 | 76 | 60 |
| 124 | 78 | 55 |
| 118 | 73 | 65 |
| 114 | 75 | 61 |
| 108 | 74 | 82 |
and pass it into R like this:
#+BEGIN_SRC R :results output :exports both :session :var data.table=data
names(data.table)
head(data.table)
#+END_SRC
I expect to get a table with three columns ("A", "B", and C"), but instead I
seem to get a single column named "A.B.C":
#+RESULTS:
: [1] "A.B.C"
: A.B.C
: 1 115 76 60
: 2 124 78 55
: 3 118 73 65
: 4 114 75 61
: 5 108 74 82
What am I doing wrong?
Thanks,
Michael Gauland
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: org tables into R? 2015-01-02 19:57 org tables into R? Michael Gauland @ 2015-01-02 20:13 ` Andreas Leha 2015-01-02 22:45 ` Vikas Rawal 2015-01-04 3:19 ` Michael Gauland 1 sibling, 1 reply; 17+ messages in thread From: Andreas Leha @ 2015-01-02 20:13 UTC (permalink / raw) To: emacs-orgmode Hi Michael, Michael Gauland <mikelygee@gmail.com> writes: > I want to use an R block to manipulate data from an org table, but I'm not > getting what I expect. > > If I define a table like this: > > #+NAME: data > | A | B | C | > > |-----+-----+----| > | 115 | 76 | 60 | > | 124 | 78 | 55 | > | 118 | 73 | 65 | > | 114 | 75 | 61 | > | 108 | 74 | 82 | > > and pass it into R like this: > > #+BEGIN_SRC R :results output :exports both :session :var data.table=data > names(data.table) > head(data.table) > #+END_SRC > > I expect to get a table with three columns ("A", "B", and C"), but instead I > seem to get a single column named "A.B.C": > > #+RESULTS: > : [1] "A.B.C" > : A.B.C > : 1 115 76 60 > : 2 124 78 55 > : 3 118 73 65 > : 4 114 75 61 > : 5 108 74 82 > > > What am I doing wrong? I believe, that this is a bug that has been reported already [1] and is still open. Regards, Andreas [1] http://news.gmane.org/find-root.php?message_id=olulhmmtei5.fsf%40med.uni%2dgoettingen.de ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: org tables into R? 2015-01-02 20:13 ` Andreas Leha @ 2015-01-02 22:45 ` Vikas Rawal 2015-01-04 10:01 ` Andreas Leha 0 siblings, 1 reply; 17+ messages in thread From: Vikas Rawal @ 2015-01-02 22:45 UTC (permalink / raw) To: Andreas Leha; +Cc: org-mode mailing list [-- Attachment #1: Type: text/plain, Size: 504 bytes --] > Michael Gauland <mikelygee@gmail.com <mailto:mikelygee@gmail.com>> writes: >> I want to use an R block to manipulate data from an org table, but I'm not >> getting what I expect. >> >> >> >> What am I doing wrong? > > I believe, that this is a bug that has been reported already [1] and is > still open. > The code sent by Michael works fine for me (that it, the org table is read as a data.frame). It may help to upgrade your org-mode. What version are you on? Vikas [-- Attachment #2: Type: text/html, Size: 1737 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: org tables into R? 2015-01-02 22:45 ` Vikas Rawal @ 2015-01-04 10:01 ` Andreas Leha 2015-01-05 0:10 ` Aaron Ecay 0 siblings, 1 reply; 17+ messages in thread From: Andreas Leha @ 2015-01-04 10:01 UTC (permalink / raw) To: emacs-orgmode Hi Vikas, Thanks for following up on this issue! Vikas Rawal <vikaslists@agrarianresearch.org> writes: > Michael Gauland <mikelygee@gmail.com> writes: > > I want to use an R block to manipulate data from an org table, but > I'm not > getting what I expect. > > > > > > What am I doing wrong? > > > I believe, that this is a bug that has been reported already [1] > and is > still open. > > > > The code sent by Michael works fine for me (that it, the org table is > read as a data.frame). > > It may help to upgrade your org-mode. What version are you on? > > Vikas It turns out I cannot reproduce it either with emacs -Q. I investigated and found that the culprit is an export filter I have defined that replaces tabs with spaces: --8<---------------cut here---------------start------------->8--- (defun my-e-beamer-final-filter (contents backend info) (replace-regexp-in-string "\t" " " contents)) (add-to-list 'org-export-filter-final-output-functions 'my-e-beamer-final-filter) --8<---------------cut here---------------end--------------->8--- Now, I consider that a bug as well. I am not exporting anything, here. But I am merely evaluating a code block. Export filters should not interfere. Plus, it used to work (unfortunately, I do not have the time to bisect right now). So, why are export filters involved? And can that be disabled? Regards, Andreas PS: Here is my environment: #+begin_src emacs-lisp (concat (replace-regexp-in-string " on .*" "" (emacs-version)) "\n" (replace-regexp-in-string " @.*" "" (org-version nil t))) #+end_src #+results: : GNU Emacs 24.4.50.1 (x86_64-apple-darwin13.3.0, NS appkit-1265.21 Version 10.9.4 (Build 13E28)) : of 2014-09-02 : Org-mode version 8.3beta (release_8.3beta-709-g3e64f6 ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: org tables into R? 2015-01-04 10:01 ` Andreas Leha @ 2015-01-05 0:10 ` Aaron Ecay 2015-01-05 11:48 ` Nicolas Goaziou 0 siblings, 1 reply; 17+ messages in thread From: Aaron Ecay @ 2015-01-05 0:10 UTC (permalink / raw) To: Andreas Leha, emacs-orgmode; +Cc: Nicolas Goaziou [-- Attachment #1: Type: text/plain, Size: 1303 bytes --] Hi Andreas, Thanks for following up on this problem. 2015ko urtarrilak 4an, Andreas Leha-ek idatzi zuen: > I investigated and found that the culprit is an export filter I have > defined that replaces tabs with spaces: > > --8<---------------cut here---------------start------------->8--- > (defun my-e-beamer-final-filter (contents backend info) > (replace-regexp-in-string "\t" " " contents)) > (add-to-list 'org-export-filter-final-output-functions 'my-e-beamer-final-filter) > --8<---------------cut here---------------end--------------->8--- > > Now, I consider that a bug as well. I am not exporting anything, here. > But I am merely evaluating a code block. Export filters should not > interfere. Plus, it used to work (unfortunately, I do not have the time > to bisect right now). > > So, why are export filters involved? And can that be disabled? Recently(-ish), Nicolas updated the org-table functions to use the export framework. One of the drawbacks of this approach (as you have brought to our attention) is that export filters are invoked. This should probably be disabled. Another drawback is the detection of unexpanded macros (which will be triggered e.g. if a tabular babel result has macro-like {{{text}}} in it). A proof-of-concept patch for this is attached. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-export-allow-sloppy-usage-of-macros-for-internal-API.patch --] [-- Type: text/x-diff, Size: 1576 bytes --] From 50ce44be96f446272d1e465c01265632f0ed488f Mon Sep 17 00:00:00 2001 From: Aaron Ecay <aaronecay@gmail.com> Date: Sun, 4 Jan 2015 18:14:26 -0500 Subject: [PATCH] [export] allow sloppy usage of macros for internal APIs * lisp/ox.el (org-export-as): Support :sloppy-macros plist entry. * lisp/org-table.el (orgtbl-to-generic): Use it. --- lisp/org-table.el | 2 +- lisp/ox.el | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/lisp/org-table.el b/lisp/org-table.el index 6b33eda..7a53d7a 100644 --- a/lisp/org-table.el +++ b/lisp/org-table.el @@ -4834,7 +4834,7 @@ This may be either a string or a function of two arguments: (table-cell . ,(org-table--to-generic-cell params)) ;; Section. Return contents to avoid garbage around table. (section . (lambda (s c i) c)))) - 'body-only (org-combine-plists params '(:with-tables t)))))) + 'body-only (org-combine-plists params '(:with-tables t :sloppy-macros t)))))) (defun org-table--generic-apply (value name &optional with-cons &rest args) (cond ((null value) nil) diff --git a/lisp/ox.el b/lisp/ox.el index f47baef..0fcfc04 100644 --- a/lisp/ox.el +++ b/lisp/ox.el @@ -2885,7 +2885,8 @@ Return code as a string." (cons "email" (or (plist-get info :email) "")) (cons "title" (org-element-interpret-data (plist-get info :title)))) - 'finalize) + (unless (plist-get info :sloppy-macros) + 'finalize)) ;; Parse buffer. (setq tree (org-element-parse-buffer nil visible-only)) ;; Handle left-over uninterpreted elements or objects in -- 2.2.1 [-- Attachment #3: Type: text/plain, Size: 681 bytes --] I wonder, though, whether the various exceptions like this that will stack up will tip the balance towards its being simpler for org-table to have its own set of functions that operate on the org-element parse tree, but don’t invoke org-export code. For csv this would be something like (pseudocode): (org-element-mapconcat (lambda (table-row) (org-element-mapconcat (lambda (table-cell) (quote-for-csv (org-element-interpret table-cell)) table-row ",")) table "\n") Nicolas, WDYT? If you prefer to continue using the export framework, I can add some commentary to the attached patch and install it. Thanks, -- Aaron Ecay ^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: org tables into R? 2015-01-05 0:10 ` Aaron Ecay @ 2015-01-05 11:48 ` Nicolas Goaziou 2015-01-06 4:27 ` Aaron Ecay 2015-01-06 10:14 ` Andreas Leha 0 siblings, 2 replies; 17+ messages in thread From: Nicolas Goaziou @ 2015-01-05 11:48 UTC (permalink / raw) To: Andreas Leha; +Cc: emacs-orgmode Hello, Aaron Ecay <aaronecay@gmail.com> writes: > Recently(-ish), Nicolas updated the org-table functions to use the > export framework. One of the drawbacks of this approach (as you have > brought to our attention) is that export filters are invoked. This > should probably be disabled. Indeed. I removed user-defined filters and hooks. Thank you. > Another drawback is the detection of unexpanded macros (which will be > triggered e.g. if a tabular babel result has macro-like {{{text}}} in > it). A proof-of-concept patch for this is attached. I don't think it needs to be fixed: macro would then be silently dropped, which is a step backwards. OTOH "ob-R.el" should consider using ":raw t" parameter for its table conversion function. Regards, -- Nicolas Goaziou ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: org tables into R? 2015-01-05 11:48 ` Nicolas Goaziou @ 2015-01-06 4:27 ` Aaron Ecay 2015-01-06 23:08 ` Nicolas Goaziou 2015-01-06 10:14 ` Andreas Leha 1 sibling, 1 reply; 17+ messages in thread From: Aaron Ecay @ 2015-01-06 4:27 UTC (permalink / raw) To: Nicolas Goaziou; +Cc: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 1285 bytes --] Hi Nicolas, 2015ko urtarrilak 5an, Nicolas Goaziou-ek idatzi zuen: > > Hello, > > Aaron Ecay <aaronecay@gmail.com> writes: > >> Recently(-ish), Nicolas updated the org-table functions to use the >> export framework. One of the drawbacks of this approach (as you have >> brought to our attention) is that export filters are invoked. This >> should probably be disabled. > > Indeed. I removed user-defined filters and hooks. Thank you. > >> Another drawback is the detection of unexpanded macros (which will be >> triggered e.g. if a tabular babel result has macro-like {{{text}}} in >> it). A proof-of-concept patch for this is attached. > > I don't think it needs to be fixed: macro would then be silently > dropped, which is a step backwards. You are correct about the silent dropping of macro-like text. However, with current master that case gives an undefined macro error, which is even worse. Try this (in emacs -Q with org and ESS in the load-path) to see it: #+name: foo #+begin_src R c("foo","{{{bar}}}") #+end_src > OTOH "ob-R.el" should consider using ":raw t" parameter for its table > conversion function. I think :raw is needed in ‘org-babel-insert-result’ in addition to my previous patch. New patch attached. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-babel-fix-macros-in-tabular-output.patch --] [-- Type: text/x-diff, Size: 2996 bytes --] From 4d2985ba88d2ba0c35ff715a7285469e8040d4b0 Mon Sep 17 00:00:00 2001 From: Aaron Ecay <aaronecay@gmail.com> Date: Sun, 4 Jan 2015 18:14:26 -0500 Subject: [PATCH] [babel] fix macros in tabular output * lisp/ox.el (org-export-as): Support :sloppy-macros plist entry. * lisp/org-table.el (orgtbl-to-generic): Use it. * lisp/ob-R.el (org-babel-R-assign-elisp): * lisp/ob-core.el (org-babel-insert-result): Use :raw argument to org-table conversion. --- lisp/ob-R.el | 2 +- lisp/ob-core.el | 2 +- lisp/org-table.el | 2 +- lisp/ox.el | 3 ++- 4 files changed, 5 insertions(+), 4 deletions(-) diff --git a/lisp/ob-R.el b/lisp/ob-R.el index 2470b4f..6f76aa5 100644 --- a/lisp/ob-R.el +++ b/lisp/ob-R.el @@ -239,7 +239,7 @@ This function is called by `org-babel-execute-src-block'." (min (if lengths (apply 'min lengths) 0))) ;; Ensure VALUE has an orgtbl structure (depth of at least 2). (unless (listp (car value)) (setq value (list value))) - (let ((file (orgtbl-to-tsv value '(:fmt org-babel-R-quote-tsv-field))) + (let ((file (orgtbl-to-tsv value '(:raw t :fmt org-babel-R-quote-tsv-field))) (header (if (or (eq (nth 1 value) 'hline) colnames-p) "TRUE" "FALSE")) (row-names (if rownames-p "1" "NULL"))) diff --git a/lisp/ob-core.el b/lisp/ob-core.el index 93fcb2a..9ff83f2 100644 --- a/lisp/ob-core.el +++ b/lisp/ob-core.el @@ -2213,7 +2213,7 @@ code ---- the results are extracted in the syntax of the source (lambda (el) (or (listp el) (eq el 'hline))) result) result (list result)) - '(:fmt (lambda (cell) (format "%s" cell)))) "\n")) + '(:raw t :fmt (lambda (cell) (format "%s" cell)))) "\n")) (goto-char beg) (when (org-at-table-p) (org-table-align))) ((and (listp result) (not (funcall proper-list-p result))) (insert (format "%s\n" result))) diff --git a/lisp/org-table.el b/lisp/org-table.el index 6b33eda..7a53d7a 100644 --- a/lisp/org-table.el +++ b/lisp/org-table.el @@ -4834,7 +4834,7 @@ This may be either a string or a function of two arguments: (table-cell . ,(org-table--to-generic-cell params)) ;; Section. Return contents to avoid garbage around table. (section . (lambda (s c i) c)))) - 'body-only (org-combine-plists params '(:with-tables t)))))) + 'body-only (org-combine-plists params '(:with-tables t :sloppy-macros t)))))) (defun org-table--generic-apply (value name &optional with-cons &rest args) (cond ((null value) nil) diff --git a/lisp/ox.el b/lisp/ox.el index f47baef..0fcfc04 100644 --- a/lisp/ox.el +++ b/lisp/ox.el @@ -2885,7 +2885,8 @@ Return code as a string." (cons "email" (or (plist-get info :email) "")) (cons "title" (org-element-interpret-data (plist-get info :title)))) - 'finalize) + (unless (plist-get info :sloppy-macros) + 'finalize)) ;; Parse buffer. (setq tree (org-element-parse-buffer nil visible-only)) ;; Handle left-over uninterpreted elements or objects in -- 2.2.1 [-- Attachment #3: Type: text/plain, Size: 282 bytes --] There are several usages of orgtbl-to-* functions in babel (R, awk, gnuplot, shell, sqlite, sql). Org-plot also uses one. The attached patch adds :raw to the ob-R call, but if I understand correctly probably all of them should add it. Should I do that? Thanks, -- Aaron Ecay ^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: org tables into R? 2015-01-06 4:27 ` Aaron Ecay @ 2015-01-06 23:08 ` Nicolas Goaziou 2015-01-06 23:17 ` Aaron Ecay 0 siblings, 1 reply; 17+ messages in thread From: Nicolas Goaziou @ 2015-01-06 23:08 UTC (permalink / raw) To: emacs-orgmode Aaron Ecay <aaronecay@gmail.com> writes: > You are correct about the silent dropping of macro-like text. However, > with current master that case gives an undefined macro error, which is > even worse. I disagree. If we allow to insert macro-like text in the table, it will become active in the generated table and will, sooner or later, generate an undefined macro error anyway. > Try this (in emacs -Q with org and ESS in the load-path) to > see it: > > #+name: foo > #+begin_src R > c("foo","{{{bar}}}") > #+end_src Yes, an error is thrown, but #+RESULTS: foo | foo | | {{{bar}}} | is as cheesy. See above. Macros are a very special kind of datum in Org. Luckily, their syntax is awkward so you're unlikely to create one unwillingly. How common is your example? >> OTOH "ob-R.el" should consider using ":raw t" parameter for its table >> conversion function. > > I think :raw is needed in ‘org-babel-insert-result’ in addition to my > previous patch. New patch attached. Actually, my advice doesn't stand. ":raw t" is not sufficient to prevent macro expansion, which happens at the beginning of the export process. Regards, ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: org tables into R? 2015-01-06 23:08 ` Nicolas Goaziou @ 2015-01-06 23:17 ` Aaron Ecay 2015-01-06 23:38 ` Nicolas Goaziou 0 siblings, 1 reply; 17+ messages in thread From: Aaron Ecay @ 2015-01-06 23:17 UTC (permalink / raw) To: Nicolas Goaziou, emacs-orgmode Hi Nicolas, 2015ko urtarrilak 6an, Nicolas Goaziou-ek idatzi zuen: > > Aaron Ecay <aaronecay@gmail.com> writes: > >> You are correct about the silent dropping of macro-like text. However, >> with current master that case gives an undefined macro error, which is >> even worse. > > I disagree. If we allow to insert macro-like text in the table, it will > become active in the generated table and will, sooner or later, generate > an undefined macro error anyway. > >> Try this (in emacs -Q with org and ESS in the load-path) to >> see it: >> >> #+name: foo >> #+begin_src R >> c("foo","{{{bar}}}") >> #+end_src > > Yes, an error is thrown, but > > #+RESULTS: foo > | foo | > | {{{bar}}} | > > is as cheesy. See above. > > Macros are a very special kind of datum in Org. Luckily, their syntax is > awkward so you're unlikely to create one unwillingly. How common is your > example? Nothing unwilling about it – I had been deliberately trying to generate a table with macros in it as the result of a babel block. These macros are defined in the document, in order to encapsulate some fiddly typesetting which recurs very commonly. Shouldn’t this workflow be supported? In any case, the point is broader: the orgtbl-* functions cannot cope with macros in their input at all (not just in the context of babel). I think the programmatic interface to org tables ought to be composable with macros. Thanks, -- Aaron Ecay ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: org tables into R? 2015-01-06 23:17 ` Aaron Ecay @ 2015-01-06 23:38 ` Nicolas Goaziou 0 siblings, 0 replies; 17+ messages in thread From: Nicolas Goaziou @ 2015-01-06 23:38 UTC (permalink / raw) To: emacs-orgmode Aaron Ecay <aaronecay@gmail.com> writes: > Nothing unwilling about it – I had been deliberately trying to generate a > table with macros in it as the result of a babel block. These macros are > defined in the document, in order to encapsulate some fiddly typesetting > which recurs very commonly. > > Shouldn’t this workflow be supported? Note that Babel blocks are evaluated after macro expansion. As a consequence, the workflow above is hardly useful, as it requires its user to manually evaluate all blocks generating macros /before/ exporting the document. If the workflow is not used and doesn't have intrinsic value, I don't think we need to invent kludges to support it. > In any case, the point is broader: the orgtbl-* functions cannot cope > with macros in their input at all (not just in the context of babel). I > think the programmatic interface to org tables ought to be composable > with macros. orgtbl-* are not an API for tables, but converters. My point is still valid: you don't have a real use-case, so just don't bother. Regards, ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: org tables into R? 2015-01-05 11:48 ` Nicolas Goaziou 2015-01-06 4:27 ` Aaron Ecay @ 2015-01-06 10:14 ` Andreas Leha 2015-01-06 11:02 ` Rainer M Krug 1 sibling, 1 reply; 17+ messages in thread From: Andreas Leha @ 2015-01-06 10:14 UTC (permalink / raw) To: emacs-orgmode Hi Aaron and Nicolas, Thanks for partly fixing this issue. Unfortunately, when working with source block interactively the issue persists for me. If in the same sample file I view the source block via 'C-c C-v v' and step through the generated code line-by-line, the table is again not split into columns. This time I immediately tried with 'emacs -Q' and I can reproduce the issue. Here is the ecm again: --8<---------------cut here---------------start------------->8--- * test #+NAME: data | A | B | C | |-----+----+----| | 115 | 76 | 60 | | 124 | 78 | 55 | | 118 | 73 | 65 | | 114 | 75 | 61 | | 108 | 74 | 82 | and pass it into R like this, then evaluation (C-c C-c) works, but stepping through the code generated for viewing (C-c C-v v) does not: #+BEGIN_SRC R :results output :exports both :session :var data.table=data names(data.table) head(data.table) #+END_SRC #+results: : [1] "A" "B" "C" : A B C : 1 115 76 60 : 2 124 78 55 : 3 118 73 65 : 4 114 75 61 : 5 108 74 82 #+begin_src emacs-lisp (concat (replace-regexp-in-string " on .*" "" (emacs-version)) "\n" (replace-regexp-in-string " @.*" ")" (org-version nil t)) "\n" (replace-regexp-in-string "].*" "]" (ess-version))) #+end_src #+results: : GNU Emacs 24.4.50.1 (x86_64-apple-darwin13.3.0, NS appkit-1265.21 Version 10.9.4 (Build 13E28)) : of 2014-09-02 : Org-mode version 8.3beta (release_8.3beta-717-ga8096c) : ess-version: 14.05 [git: 4283f1304a54502c42707b6a4ba347703f0992dd] --8<---------------cut here---------------end--------------->8--- Here is what I see in the org babel preview: --8<---------------cut here---------------start------------->8--- data.table <- local({ con <- textConnection( "\"A\" \"B\" \"C\" \"115\" \"76\" \"60\" \"124\" \"78\" \"55\" \"118\" \"73\" \"65\" \"114\" \"75\" \"61\" \"108\" \"74\" \"82\"" ) res <- utils::read.table( con, header = TRUE, row.names = NULL, sep = "\t", as.is = TRUE ) close(con) res }) names(data.table) head(data.table) --8<---------------cut here---------------end--------------->8--- And finally, here is my R session, when I step through the code: --8<---------------cut here---------------start------------->8--- R version 3.1.2 (2014-10-31) -- "Pumpkin Helmet" Copyright (C) 2014 The R Foundation for Statistical Computing Platform: x86_64-apple-darwin10.8.0 (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. Natural language support but running in an English locale R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > > options(STERM='iESS', str.dendrogram.last="'", editor='emacsclient', show.error.locations=TRUE) > data.table <- local({ + con <- textConnection( + "\"A\" \"B\" \"C\" + \"115\" \"76\" \"60\" + \"124\" \"78\" \"55\" + \"118\" \"73\" \"65\" + \"114\" \"75\" \"61\" + \"108\" \"74\" \"82\"" + ) + res <- utils::read.table( + con, + header = TRUE, + row.names = NULL, + sep = "\t", + as.is = TRUE + ) + close(con) + res + }) > names(data.table) [1] "A...B...C" > head(data.table) A...B...C 1 115 76 60 2 124 78 55 3 118 73 65 4 114 75 61 5 108 74 82 --8<---------------cut here---------------end--------------->8--- Regards, Andreas ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: org tables into R? 2015-01-06 10:14 ` Andreas Leha @ 2015-01-06 11:02 ` Rainer M Krug 2015-01-06 12:07 ` Andreas Leha 0 siblings, 1 reply; 17+ messages in thread From: Rainer M Krug @ 2015-01-06 11:02 UTC (permalink / raw) To: Andreas Leha; +Cc: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 6045 bytes --] Andreas Leha <andreas.leha@med.uni-goettingen.de> writes: > Hi Aaron and Nicolas, Sorry for coming so late to this topic - I should try to figure out what is causing the problem as I am the one responsible for this code... > > Thanks for partly fixing this issue. Unfortunately, when working with > source block interactively the issue persists for me. If in the same > sample file I view the source block via 'C-c C-v v' and step through the > generated code line-by-line, the table is again not split into columns. > > This time I immediately tried with 'emacs -Q' and I can reproduce the > issue. > > Here is the ecm again: > > * test > #+NAME: data > | A | B | C | > |-----+----+----| > | 115 | 76 | 60 | > | 124 | 78 | 55 | > | 118 | 73 | 65 | > | 114 | 75 | 61 | > | 108 | 74 | 82 | > > and pass it into R like this, then evaluation (C-c C-c) works, but > stepping through the code generated for viewing (C-c C-v v) does not: > > #+BEGIN_SRC R :results output :exports both :session :var data.table=data > names(data.table) > head(data.table) > #+END_SRC > > #+results: > : [1] "A" "B" "C" > : A B C > : 1 115 76 60 > : 2 124 78 55 > : 3 118 73 65 > : 4 114 75 61 > : 5 108 74 82 > > #+begin_src emacs-lisp > (concat > (replace-regexp-in-string " on .*" "" (emacs-version)) > "\n" > (replace-regexp-in-string " @.*" ")" (org-version nil t)) > "\n" > (replace-regexp-in-string "].*" "]" (ess-version))) > #+end_src > > #+results: > : GNU Emacs 24.4.50.1 (x86_64-apple-darwin13.3.0, NS appkit-1265.21 Version 10.9.4 (Build 13E28)) > : of 2014-09-02 > : Org-mode version 8.3beta (release_8.3beta-717-ga8096c) > : ess-version: 14.05 [git: 4283f1304a54502c42707b6a4ba347703f0992dd] > ,---- | #+RESULTS: | : GNU Emacs 24.4.1 (x86_64-apple-darwin14.0.0, Carbon Version 157 AppKit 1343.16) | : of 2014-11-24 | : Org-mode version 8.3beta (release_8.3beta-695-g390015) | : ess-version: 14.09 [svn: 6041 (2014-09-13)] `---- I get the same in the preview, but it works for me. I do not use -q. ,---- | > data.table <- local({ | + con <- textConnection( | + "\"A\" \"B\" \"C\" | + \"115\" \"76\" \"60\" | + \"124\" \"78\" \"55\" | + \"118\" \"73\" \"65\" | + \"114\" \"75\" \"61\" | + \"108\" \"74\" \"82\"" | + ) | + res <- utils::read.table( | + con, | + header = TRUE, | + row.names = NULL, | + sep = "\t", | + as.is = TRUE | + ) | + close(con) | + res | + }) | > names(data.table) | [1] "A" "B" "C" | > head(data.table) | A B C | 1 115 76 60 | 2 124 78 55 | 3 118 73 65 | 4 114 75 61 | 5 108 74 82 | > `---- > > Here is what I see in the org babel preview: > This might be due to temail, but you don't have tabs (\t) between your entries below - could you verify if this is the case? If yes, this might be a locale issue (would be strange)? > data.table <- local({ > con <- textConnection( > "\"A\" \"B\" \"C\" > \"115\" \"76\" \"60\" > \"124\" \"78\" \"55\" > \"118\" \"73\" \"65\" > \"114\" \"75\" \"61\" > \"108\" \"74\" \"82\"" > ) > res <- utils::read.table( > con, > header = TRUE, > row.names = NULL, > sep = "\t", > as.is = TRUE > ) > close(con) > res > }) > names(data.table) > head(data.table) One solution would be to insert (instead of the tab character) "\t" to separate the fields? You could try and modify the ob-R-transfer-variable-table-with-header - From the help: ,---- | ob-R-transfer-variable-table-with-header is a variable defined in `ob-R.el'. | Its value is | "%s <- local({\n con <- textConnection(\n %S\n )\n res <- utils::read.table(\n con,\n header = %s,\n row.names = %s,\n sep = \"\\t\",\n as.is = TRUE\n )\n close(con)\n res\n })" | | This variable may be risky if used as a file-local variable. | | Documentation: | R code used to transfer a table defined as a variable from org to R. | | This function is used when the table contains a header. `---- This is passed to the (format) function and one might be able to change something there. Cheers, Rainer > > > And finally, here is my R session, when I step through the code: > > > R version 3.1.2 (2014-10-31) -- "Pumpkin Helmet" > Copyright (C) 2014 The R Foundation for Statistical Computing > Platform: x86_64-apple-darwin10.8.0 (64-bit) > > R is free software and comes with ABSOLUTELY NO WARRANTY. > You are welcome to redistribute it under certain conditions. > Type 'license()' or 'licence()' for distribution details. > > Natural language support but running in an English locale > > R is a collaborative project with many contributors. > Type 'contributors()' for more information and > 'citation()' on how to cite R or R packages in publications. > > Type 'demo()' for some demos, 'help()' for on-line help, or > 'help.start()' for an HTML browser interface to help. > Type 'q()' to quit R. > >> > options(STERM='iESS', str.dendrogram.last="'", editor='emacsclient', show.error.locations=TRUE) >> data.table <- local({ > + con <- textConnection( > + "\"A\" \"B\" \"C\" > + \"115\" \"76\" \"60\" > + \"124\" \"78\" \"55\" > + \"118\" \"73\" \"65\" > + \"114\" \"75\" \"61\" > + \"108\" \"74\" \"82\"" > + ) > + res <- utils::read.table( > + con, > + header = TRUE, > + row.names = NULL, > + sep = "\t", > + as.is = TRUE > + ) > + close(con) > + res > + }) >> names(data.table) > [1] "A...B...C" >> head(data.table) > A...B...C > 1 115 76 60 > 2 124 78 55 > 3 118 73 65 > 4 114 75 61 > 5 108 74 82 > > > Regards, > Andreas > > > -- Rainer M. Krug email: Rainer<at>krugs<dot>de PGP: 0x0F52F982 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 494 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: org tables into R? 2015-01-06 11:02 ` Rainer M Krug @ 2015-01-06 12:07 ` Andreas Leha 2015-01-06 13:37 ` Rainer M Krug 0 siblings, 1 reply; 17+ messages in thread From: Andreas Leha @ 2015-01-06 12:07 UTC (permalink / raw) To: emacs-orgmode Hi Rainer, Thanks for chiming in. Rainer M Krug <Rainer@krugs.de> writes: > Andreas Leha <andreas.leha@med.uni-goettingen.de> writes: > >> Hi Aaron and Nicolas, > > Sorry for coming so late to this topic - I should try to figure out what > is causing the problem as I am the one responsible for this code... > >> >> Thanks for partly fixing this issue. Unfortunately, when working with >> source block interactively the issue persists for me. If in the same >> sample file I view the source block via 'C-c C-v v' and step through the >> generated code line-by-line, the table is again not split into columns. >> >> This time I immediately tried with 'emacs -Q' and I can reproduce the >> issue. >> >> Here is the ecm again: >> >> * test >> #+NAME: data >> | A | B | C | >> |-----+----+----| >> | 115 | 76 | 60 | >> | 124 | 78 | 55 | >> | 118 | 73 | 65 | >> | 114 | 75 | 61 | >> | 108 | 74 | 82 | >> >> and pass it into R like this, then evaluation (C-c C-c) works, but >> stepping through the code generated for viewing (C-c C-v v) does not: >> >> #+BEGIN_SRC R :results output :exports both :session :var data.table=data >> names(data.table) >> head(data.table) >> #+END_SRC >> >> #+results: >> : [1] "A" "B" "C" >> : A B C >> : 1 115 76 60 >> : 2 124 78 55 >> : 3 118 73 65 >> : 4 114 75 61 >> : 5 108 74 82 >> >> #+begin_src emacs-lisp >> (concat >> (replace-regexp-in-string " on .*" "" (emacs-version)) >> "\n" >> (replace-regexp-in-string " @.*" ")" (org-version nil t)) >> "\n" >> (replace-regexp-in-string "].*" "]" (ess-version))) >> #+end_src >> >> #+results: >> : GNU Emacs 24.4.50.1 (x86_64-apple-darwin13.3.0, NS appkit-1265.21 Version 10.9.4 (Build 13E28)) >> : of 2014-09-02 >> : Org-mode version 8.3beta (release_8.3beta-717-ga8096c) >> : ess-version: 14.05 [git: 4283f1304a54502c42707b6a4ba347703f0992dd] >> > > ,---- > | #+RESULTS: > | : GNU Emacs 24.4.1 (x86_64-apple-darwin14.0.0, Carbon Version 157 AppKit 1343.16) > | : of 2014-11-24 > | : Org-mode version 8.3beta (release_8.3beta-695-g390015) > | : ess-version: 14.09 [svn: 6041 (2014-09-13)] > `---- > > > I get the same in the preview, but it works for me. I do not use -q. > > ,---- > | > data.table <- local({ > | + con <- textConnection( > | + "\"A\" \"B\" \"C\" > | + \"115\" \"76\" \"60\" > | + \"124\" \"78\" \"55\" > | + \"118\" \"73\" \"65\" > | + \"114\" \"75\" \"61\" > | + \"108\" \"74\" \"82\"" > | + ) > | + res <- utils::read.table( > | + con, > | + header = TRUE, > | + row.names = NULL, > | + sep = "\t", > | + as.is = TRUE > | + ) > | + close(con) > | + res > | + }) > | > names(data.table) > | [1] "A" "B" "C" > | > head(data.table) > | A B C > | 1 115 76 60 > | 2 124 78 55 > | 3 118 73 65 > | 4 114 75 61 > | 5 108 74 82 > | > > `---- > > >> >> Here is what I see in the org babel preview: >> > > This might be due to temail, but you don't have tabs (\t) between your > entries below - could you verify if this is the case? If yes, this might > be a locale issue (would be strange)? Yes, there seem to be no tabs. If there were tabs, R should parse the table correctly, I guess. (BTW: My custom filter replacing tabs with spaces is still disabled, even though I pulled from master, which includes the recent commit 70f965535b07c03a40792237de3a970d1cb57551 Author: Nicolas Goaziou <mail@nicolasgoaziou.fr> Date: Mon Jan 5 12:39:26 2015 +0100 org-table: Ignore filters and hooks in radio tables ) > >> data.table <- local({ >> con <- textConnection( >> "\"A\" \"B\" \"C\" >> \"115\" \"76\" \"60\" >> \"124\" \"78\" \"55\" >> \"118\" \"73\" \"65\" >> \"114\" \"75\" \"61\" >> \"108\" \"74\" \"82\"" >> ) >> res <- utils::read.table( >> con, >> header = TRUE, >> row.names = NULL, >> sep = "\t", >> as.is = TRUE >> ) >> close(con) >> res >> }) >> names(data.table) >> head(data.table) > > One solution would be to insert (instead of the tab character) "\t" to > separate the fields? > > You could try and modify the ob-R-transfer-variable-table-with-header - > From the help: > > ,---- > | ob-R-transfer-variable-table-with-header is a variable defined in `ob-R.el'. > | Its value is > | "%s <- local({\n con <- textConnection(\n %S\n )\n res <- > | utils::read.table(\n con,\n header = %s,\n row.names = %s,\n sep = > | \"\\t\",\n as.is = TRUE\n )\n close(con)\n res\n })" > | > | This variable may be risky if used as a file-local variable. > | > | Documentation: > | R code used to transfer a table defined as a variable from org to R. > | > | This function is used when the table contains a header. > `---- > > This is passed to the (format) function and one might be able to change > something there. Thanks for the hint. Do I assume correctly, that this variable is used when the code block is evaluated as well? In that case, I am reluctant to change it, as the evaluation now works. The surprising thing is the different behaviour for preview and evaluation. I am extremely tight with my time right now, so that I won't be able to do much experimentation/investigation right now, I am afraid. But I will try to still be helpful... Thanks, Andreas > > Cheers, > > Rainer > >> >> >> And finally, here is my R session, when I step through the code: >> >> >> R version 3.1.2 (2014-10-31) -- "Pumpkin Helmet" >> Copyright (C) 2014 The R Foundation for Statistical Computing >> Platform: x86_64-apple-darwin10.8.0 (64-bit) >> >> R is free software and comes with ABSOLUTELY NO WARRANTY. >> You are welcome to redistribute it under certain conditions. >> Type 'license()' or 'licence()' for distribution details. >> >> Natural language support but running in an English locale >> >> R is a collaborative project with many contributors. >> Type 'contributors()' for more information and >> 'citation()' on how to cite R or R packages in publications. >> >> Type 'demo()' for some demos, 'help()' for on-line help, or >> 'help.start()' for an HTML browser interface to help. >> Type 'q()' to quit R. >> >>> > options(STERM='iESS', str.dendrogram.last="'", editor='emacsclient', show.error.locations=TRUE) >>> data.table <- local({ >> + con <- textConnection( >> + "\"A\" \"B\" \"C\" >> + \"115\" \"76\" \"60\" >> + \"124\" \"78\" \"55\" >> + \"118\" \"73\" \"65\" >> + \"114\" \"75\" \"61\" >> + \"108\" \"74\" \"82\"" >> + ) >> + res <- utils::read.table( >> + con, >> + header = TRUE, >> + row.names = NULL, >> + sep = "\t", >> + as.is = TRUE >> + ) >> + close(con) >> + res >> + }) >>> names(data.table) >> [1] "A...B...C" >>> head(data.table) >> A...B...C >> 1 115 76 60 >> 2 124 78 55 >> 3 118 73 65 >> 4 114 75 61 >> 5 108 74 82 >> >> >> Regards, >> Andreas >> >> >> ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: org tables into R? 2015-01-06 12:07 ` Andreas Leha @ 2015-01-06 13:37 ` Rainer M Krug 2015-01-06 17:49 ` Aaron Ecay 0 siblings, 1 reply; 17+ messages in thread From: Rainer M Krug @ 2015-01-06 13:37 UTC (permalink / raw) To: Andreas Leha; +Cc: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 9521 bytes --] Andreas Leha <andreas.leha@med.uni-goettingen.de> writes: > Hi Rainer, > > Thanks for chiming in. Hope I can help... > > Rainer M Krug <Rainer@krugs.de> writes: >> Andreas Leha <andreas.leha@med.uni-goettingen.de> writes: >> >>> Hi Aaron and Nicolas, >> >> Sorry for coming so late to this topic - I should try to figure out what >> is causing the problem as I am the one responsible for this code... >> >>> >>> Thanks for partly fixing this issue. Unfortunately, when working with >>> source block interactively the issue persists for me. If in the same >>> sample file I view the source block via 'C-c C-v v' and step through the >>> generated code line-by-line, the table is again not split into columns. >>> >>> This time I immediately tried with 'emacs -Q' and I can reproduce the >>> issue. >>> >>> Here is the ecm again: >>> >>> * test >>> #+NAME: data >>> | A | B | C | >>> |-----+----+----| >>> | 115 | 76 | 60 | >>> | 124 | 78 | 55 | >>> | 118 | 73 | 65 | >>> | 114 | 75 | 61 | >>> | 108 | 74 | 82 | >>> >>> and pass it into R like this, then evaluation (C-c C-c) works, but >>> stepping through the code generated for viewing (C-c C-v v) does not: >>> >>> #+BEGIN_SRC R :results output :exports both :session :var data.table=data >>> names(data.table) >>> head(data.table) >>> #+END_SRC >>> >>> #+results: >>> : [1] "A" "B" "C" >>> : A B C >>> : 1 115 76 60 >>> : 2 124 78 55 >>> : 3 118 73 65 >>> : 4 114 75 61 >>> : 5 108 74 82 >>> >>> #+begin_src emacs-lisp >>> (concat >>> (replace-regexp-in-string " on .*" "" (emacs-version)) >>> "\n" >>> (replace-regexp-in-string " @.*" ")" (org-version nil t)) >>> "\n" >>> (replace-regexp-in-string "].*" "]" (ess-version))) >>> #+end_src >>> >>> #+results: >>> : GNU Emacs 24.4.50.1 (x86_64-apple-darwin13.3.0, NS appkit-1265.21 Version 10.9.4 (Build 13E28)) >>> : of 2014-09-02 >>> : Org-mode version 8.3beta (release_8.3beta-717-ga8096c) >>> : ess-version: 14.05 [git: 4283f1304a54502c42707b6a4ba347703f0992dd] >>> >> >> ,---- >> | #+RESULTS: >> | : GNU Emacs 24.4.1 (x86_64-apple-darwin14.0.0, Carbon Version 157 AppKit 1343.16) >> | : of 2014-11-24 >> | : Org-mode version 8.3beta (release_8.3beta-695-g390015) >> | : ess-version: 14.09 [svn: 6041 (2014-09-13)] >> `---- >> >> >> I get the same in the preview, but it works for me. I do not use -q. >> >> ,---- >> | > data.table <- local({ >> | + con <- textConnection( >> | + "\"A\" \"B\" \"C\" >> | + \"115\" \"76\" \"60\" >> | + \"124\" \"78\" \"55\" >> | + \"118\" \"73\" \"65\" >> | + \"114\" \"75\" \"61\" >> | + \"108\" \"74\" \"82\"" >> | + ) >> | + res <- utils::read.table( >> | + con, >> | + header = TRUE, >> | + row.names = NULL, >> | + sep = "\t", >> | + as.is = TRUE >> | + ) >> | + close(con) >> | + res >> | + }) >> | > names(data.table) >> | [1] "A" "B" "C" >> | > head(data.table) >> | A B C >> | 1 115 76 60 >> | 2 124 78 55 >> | 3 118 73 65 >> | 4 114 75 61 >> | 5 108 74 82 >> | > >> `---- >> >> >>> >>> Here is what I see in the org babel preview: >>> >> >> This might be due to temail, but you don't have tabs (\t) between your >> entries below - could you verify if this is the case? If yes, this might >> be a locale issue (would be strange)? > > Yes, there seem to be no tabs. If there were tabs, R should parse the > table correctly, I guess. Exactly. So somehow the tabs get replaced in your system by spaces. > > (BTW: My custom filter replacing tabs with spaces is still disabled, > even though I pulled from master, which includes the recent > > commit 70f965535b07c03a40792237de3a970d1cb57551 > Author: Nicolas Goaziou <mail@nicolasgoaziou.fr> > Date: Mon Jan 5 12:39:26 2015 +0100 > > org-table: Ignore filters and hooks in radio tables > ) > > >> >>> data.table <- local({ >>> con <- textConnection( >>> "\"A\" \"B\" \"C\" >>> \"115\" \"76\" \"60\" >>> \"124\" \"78\" \"55\" >>> \"118\" \"73\" \"65\" >>> \"114\" \"75\" \"61\" >>> \"108\" \"74\" \"82\"" >>> ) >>> res <- utils::read.table( >>> con, >>> header = TRUE, >>> row.names = NULL, >>> sep = "\t", >>> as.is = TRUE >>> ) >>> close(con) >>> res >>> }) >>> names(data.table) >>> head(data.table) >> >> One solution would be to insert (instead of the tab character) "\t" to >> separate the fields? >> >> You could try and modify the ob-R-transfer-variable-table-with-header - >> From the help: >> >> ,---- >> | ob-R-transfer-variable-table-with-header is a variable defined in `ob-R.el'. >> | Its value is >> | "%s <- local({\n con <- textConnection(\n %S\n )\n res <- >> | utils::read.table(\n con,\n header = %s,\n row.names = %s,\n sep = >> | \"\\t\",\n as.is = TRUE\n )\n close(con)\n res\n })" >> | >> | This variable may be risky if used as a file-local variable. >> | >> | Documentation: >> | R code used to transfer a table defined as a variable from org to R. >> | >> | This function is used when the table contains a header. >> `---- >> >> This is passed to the (format) function and one might be able to change >> something there. > > Thanks for the hint. Do I assume correctly, that this variable is used > when the code block is evaluated as well? In that case, I am reluctant > to change it, as the evaluation now works. The surprising thing is the > different behaviour for preview and evaluation. Yup - you are right. These variables (ob-R-transfer-variable-table-with-header and ob-R-transfer-variable-table-without-header) are only used in the function org-babel-R-assign-elisp which is only called from org-babel-variable-assignments:R which is doing the variable assignments whenever they are needed. What would definitely solve the issue is if the string containing the tabs would use \t instead - but I have no idea how this could be achieved easily. > > I am extremely tight with my time right now, so that I won't be able to > do much experimentation/investigation right now, I am afraid. But I > will try to still be helpful... The question is really "where does the tab become a space". As you see the spaces after C-c C-v v it seems that the problem also occurs when the function org-babel-expand-src-block is called. So I think to narrow down what is happening, you have to step through this function and see if the assignments contain spaces instead of tabs from the beginning or if they are changed whithin the function. As only you see these errors, I would still guess that it is something with your system (locale?, utf et al issues?). By the way - are you starting R as --vanilla? If not, it might be worth trying it out? ,---- | (setq inferior-R-args "--vanilla") `---- Actually, if you don't see tabs after calling org-babel-expand-src-block on the example block, this is a pure org issue - no ESS or R involved here. One thing you could try is to change the sep value in the variables ob-R-transfer-variable-table-with-header and ob-R-transfer-variable-table-without-header as follow: ,---- | ... | sep = \"\", ;;;;;<<<<<<<<< was before sep = \"\\t\" | ... `---- This will set the separator to "whitespace", i.e. "that is one or more spaces, tabs, newlines or carriage returns" (from R help). This should solve your problem. Let us know if this solves your problem. Cheers, Rianer > > Thanks, > Andreas > > >> >> Cheers, >> >> Rainer >> >>> >>> >>> And finally, here is my R session, when I step through the code: >>> >>> >>> R version 3.1.2 (2014-10-31) -- "Pumpkin Helmet" >>> Copyright (C) 2014 The R Foundation for Statistical Computing >>> Platform: x86_64-apple-darwin10.8.0 (64-bit) >>> >>> R is free software and comes with ABSOLUTELY NO WARRANTY. >>> You are welcome to redistribute it under certain conditions. >>> Type 'license()' or 'licence()' for distribution details. >>> >>> Natural language support but running in an English locale >>> >>> R is a collaborative project with many contributors. >>> Type 'contributors()' for more information and >>> 'citation()' on how to cite R or R packages in publications. >>> >>> Type 'demo()' for some demos, 'help()' for on-line help, or >>> 'help.start()' for an HTML browser interface to help. >>> Type 'q()' to quit R. >>> >>>> > options(STERM='iESS', str.dendrogram.last="'", editor='emacsclient', show.error.locations=TRUE) >>>> data.table <- local({ >>> + con <- textConnection( >>> + "\"A\" \"B\" \"C\" >>> + \"115\" \"76\" \"60\" >>> + \"124\" \"78\" \"55\" >>> + \"118\" \"73\" \"65\" >>> + \"114\" \"75\" \"61\" >>> + \"108\" \"74\" \"82\"" >>> + ) >>> + res <- utils::read.table( >>> + con, >>> + header = TRUE, >>> + row.names = NULL, >>> + sep = "\t", >>> + as.is = TRUE >>> + ) >>> + close(con) >>> + res >>> + }) >>>> names(data.table) >>> [1] "A...B...C" >>>> head(data.table) >>> A...B...C >>> 1 115 76 60 >>> 2 124 78 55 >>> 3 118 73 65 >>> 4 114 75 61 >>> 5 108 74 82 >>> >>> >>> Regards, >>> Andreas >>> >>> >>> > > > -- Rainer M. Krug email: Rainer<at>krugs<dot>de PGP: 0x0F52F982 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 494 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: org tables into R? 2015-01-06 13:37 ` Rainer M Krug @ 2015-01-06 17:49 ` Aaron Ecay 2015-01-07 9:01 ` Rainer M Krug 0 siblings, 1 reply; 17+ messages in thread From: Aaron Ecay @ 2015-01-06 17:49 UTC (permalink / raw) To: Rainer M Krug, Andreas Leha; +Cc: emacs-orgmode Hi Rainer and Andreas, 2015ko urtarrilak 6an, Rainer M Krug-ek idatzi zuen: > What would definitely solve the issue is if the string containing the > tabs would use \t instead - but I have no idea how this could be > achieved easily. This could be achieved by modifying the call to orgtbl-to-csv in org-babel-R-assign-elisp: diff --git i/lisp/ob-R.el w/lisp/ob-R.el index 6f76aa5..ec3f110 100644 --- i/lisp/ob-R.el +++ w/lisp/ob-R.el @@ -239,7 +239,10 @@ This function is called by `org-babel-execute-src-block'." (min (if lengths (apply 'min lengths) 0))) ;; Ensure VALUE has an orgtbl structure (depth of at least 2). (unless (listp (car value)) (setq value (list value))) - (let ((file (orgtbl-to-tsv value '(:fmt org-babel-R-quote-tsv-field))) + (let ((file (replace-regexp-in-string + "\t" "\\t" + (orgtbl-to-tsv value '(:raw t :fmt org-babel-R-quote-tsv-field)) + nil t)) (header (if (or (eq (nth 1 value) 'hline) colnames-p) "TRUE" "FALSE")) (row-names (if rownames-p "1" "NULL"))) (This is just a quick hack; if it works then the code should actually use orgtbl-to-generic to do the conversion in a single step.) > ,---- > | ... > | sep = \"\", ;;;;;<<<<<<<<< was before sep = \"\\t\" > | ... > `---- > > This will set the separator to "whitespace", i.e. "that is one or more > spaces, tabs, newlines or carriage returns" (from R help). This was the approach that Michael took in his original patch. It introduces new bugs related to the handling of quotes, though – see the “Details” section of ?scan in R for the gory details (beginning with “If ‘sep’ is the default”...). Andreas raised a bug a while ago that indicates to me that this code should switch to csv rather than a whitespace-delimited format. But there are lots of corner cases, and I have not had time to work on a patch that I am confident could cover them all. Andreas’s original report (also referenced earlier in this thread): http://news.gmane.org/find-root.php?message_id=olulhmmtei5.fsf%40med.uni%2dgoettingen.de Thanks, -- Aaron Ecay ^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: org tables into R? 2015-01-06 17:49 ` Aaron Ecay @ 2015-01-07 9:01 ` Rainer M Krug 0 siblings, 0 replies; 17+ messages in thread From: Rainer M Krug @ 2015-01-07 9:01 UTC (permalink / raw) To: Andreas Leha; +Cc: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 3258 bytes --] Aaron Ecay <aaronecay@gmail.com> writes: > Hi Rainer and Andreas, > > 2015ko urtarrilak 6an, Rainer M Krug-ek idatzi zuen: >> What would definitely solve the issue is if the string containing the >> tabs would use \t instead - but I have no idea how this could be >> achieved easily. > > This could be achieved by modifying the call to orgtbl-to-csv in > org-babel-R-assign-elisp: > > diff --git i/lisp/ob-R.el w/lisp/ob-R.el > index 6f76aa5..ec3f110 100644 > --- i/lisp/ob-R.el > +++ w/lisp/ob-R.el > @@ -239,7 +239,10 @@ This function is called by `org-babel-execute-src-block'." > (min (if lengths (apply 'min lengths) 0))) > ;; Ensure VALUE has an orgtbl structure (depth of at least 2). > (unless (listp (car value)) (setq value (list value))) > - (let ((file (orgtbl-to-tsv value '(:fmt org-babel-R-quote-tsv-field))) > + (let ((file (replace-regexp-in-string > + "\t" "\\t" > + (orgtbl-to-tsv value '(:raw t :fmt org-babel-R-quote-tsv-field)) > + nil t)) > (header (if (or (eq (nth 1 value) 'hline) colnames-p) > "TRUE" "FALSE")) > (row-names (if rownames-p "1" "NULL"))) > > (This is just a quick hack; if it works then the code should actually > use orgtbl-to-generic to do the conversion in a single step.) That looks like it would work. > >> ,---- >> | ... >> | sep = \"\", ;;;;;<<<<<<<<< was before sep = \"\\t\" >> | ... >> `---- >> >> This will set the separator to "whitespace", i.e. "that is one or more >> spaces, tabs, newlines or carriage returns" (from R help). > > This was the approach that Michael took in his original patch. It > introduces new bugs related to the handling of quotes, though – see the > “Details” section of ?scan in R for the gory details (beginning with “If > ‘sep’ is the default”...). I agree completely - it would not be a final solution but possibly solve the problem temporarily in an emacs.org file. But the real question is: why are on some setups the tabs converted to spaces? > > Andreas raised a bug a while ago that indicates to me that this code > should switch to csv rather than a whitespace-delimited format. But > there are lots of corner cases, and I have not had time to work on a > patch that I am confident could cover them all. Might be a solution - but I do not know how commas are working under different locales. In MS Office, a csv file is a semicolon (;) separated file when in the chosen language settings a comma is used as decimal separator - and and I have no idea if language settings affect emacs in the same way (I actually don't hope so). This makes me think - why not use "|" as a separator (as it is the case in org)? In this way, all tables which are in org can be directly given to R as strings without the hassle of to much of conversion? Rainer > > Andreas’s original report (also referenced earlier in this thread): > http://news.gmane.org/find-root.php?message_id=olulhmmtei5.fsf%40med.uni%2dgoettingen.de > > Thanks, -- Rainer M. Krug email: Rainer<at>krugs<dot>de PGP: 0x0F52F982 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 494 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: org tables into R? 2015-01-02 19:57 org tables into R? Michael Gauland 2015-01-02 20:13 ` Andreas Leha @ 2015-01-04 3:19 ` Michael Gauland 1 sibling, 0 replies; 17+ messages in thread From: Michael Gauland @ 2015-01-04 3:19 UTC (permalink / raw) To: emacs-orgmode I modified the definition of ob-R-transfer-variable-table-with-header in ob-R.el by changing 'sep' from \"\\t\" to \"\". Now #+BEGIN_SRC R :results output :exports both :session :var data.table=data :colnames yes names(data.table) head(data.table) #+END_SRC (note the addition of :colnames=yes) give me: #+RESULTS: : [1] "A" "B" "C" : A B C : 1 115 76 60 : 2 124 78 55 : 3 118 73 65 : 4 114 75 61 : 5 108 74 82 as I expect. What is the current procedure to submit a patch? Thanks, Michael ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2015-01-07 14:31 UTC | newest] Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-01-02 19:57 org tables into R? Michael Gauland 2015-01-02 20:13 ` Andreas Leha 2015-01-02 22:45 ` Vikas Rawal 2015-01-04 10:01 ` Andreas Leha 2015-01-05 0:10 ` Aaron Ecay 2015-01-05 11:48 ` Nicolas Goaziou 2015-01-06 4:27 ` Aaron Ecay 2015-01-06 23:08 ` Nicolas Goaziou 2015-01-06 23:17 ` Aaron Ecay 2015-01-06 23:38 ` Nicolas Goaziou 2015-01-06 10:14 ` Andreas Leha 2015-01-06 11:02 ` Rainer M Krug 2015-01-06 12:07 ` Andreas Leha 2015-01-06 13:37 ` Rainer M Krug 2015-01-06 17:49 ` Aaron Ecay 2015-01-07 9:01 ` Rainer M Krug 2015-01-04 3:19 ` Michael Gauland
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).