[ob-R] table variable passing broken

[ob-R] table variable passing broken

[Andreas Leha]

Hi all,

There seems to be a bug in table passing as variables now using the
tangle-friendly version of passing variables.


Here is an example (I get an error also with emacs -Q):

--8<---------------cut here---------------start------------->8---
* test
#+name: testtab
| variable              | display                       | unit  |
|-----------------------+-------------------------------+-------|
| num_cells             | Number of Cells in Well       |       |
| cell_area             | Cell Area                     | μm²   |
| nucleus_area          | Nucleus Area                  | μm²   |
| roundness             | Cell Roundness                |       |
| ratio_w2l             | Cell Width to Length Ratio    |       |
| inten_nuc_dapi_median | Intensity Nucleus DAPI Median |       |
| dapi_median           | Intensity Nucleus DAPI Median |       |
| edu_median            | Intensity edu Median          |       |
| oct4_median           | Intensity oct4 Median         |       |
| clump_size            | Clump Size                    | cells |
| short_name            | Cell Line                     |       |
| p_col                 | Column                        |       |
| batch                 | Batch                         |       |
| concentration         | Fibronectin Concentration     | ugml  |
| Residual              | Residual                      |       |
| evaluation_guid       | Plate                         |       |
| donor                 | Genotype                      |       |

#+BEGIN_SRC R :session *test* :var test=testtab
  test
#+END_SRC

#+RESULTS:
--8<---------------cut here---------------end--------------->8---



I see this in my R session:

--8<---------------cut here---------------start------------->8---
Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings,  (from testorg.org!917613Wp#22) : 
  line 17 did not have 3 elements
--8<---------------cut here---------------end--------------->8---

This is on
- GNU Emacs 24.4.50.1 (x86_64-apple-darwin13.3.0, NS appkit-1265.21 Version 10.9.4 (Build 13E28)) of 2014-09-02 on mib106584i.local
- Org-mode version 8.3beta (release_8.3beta-470-g087f8e)
- ess-version: 14.05 [git: 4283f1304a54502c42707b6a4ba347703f0992dd]

Best,
Andreas

Re: [ob-R] table variable passing broken

[Charles Berry]

Andreas Leha <andreas.leha <at> med.uni-goettingen.de> writes:

> 
> Hi all,
> 
> There seems to be a bug in table passing as variables now using the
> tangle-friendly version of passing variables.
> 
> Here is an example (I get an error also with emacs -Q):
> 
> --8<---------------cut here---------------start------------->8---
> * test
> #+name: testtab
> | variable              | display                       | unit  |
> |-----------------------+-------------------------------+-------|
> | num_cells             | Number of Cells in Well       |       |
> | cell_area             | Cell Area                     | μm²   |
> | nucleus_area          | Nucleus Area                  | μm²   |
> | roundness             | Cell Roundness                |       |
> | ratio_w2l             | Cell Width to Length Ratio    |       |
> | inten_nuc_dapi_median | Intensity Nucleus DAPI Median |       |
> | dapi_median           | Intensity Nucleus DAPI Median |       |
> | edu_median            | Intensity edu Median          |       |
> | oct4_median           | Intensity oct4 Median         |       |
> | clump_size            | Clump Size                    | cells |
> | short_name            | Cell Line                     |       |
> | p_col                 | Column                        |       |
> | batch                 | Batch                         |       |
> | concentration         | Fibronectin Concentration     | ugml  |
> | Residual              | Residual                      |       |
> | evaluation_guid       | Plate                         |       |
> | donor                 | Genotype                      |       |
> 
> #+BEGIN_SRC R :session *test* :var test=testtab
>   test
> #+END_SRC
> 
> #+RESULTS:
> --8<---------------cut here---------------end--------------->8---
> 
> I see this in my R session:
> 
> --8<---------------cut here---------------start------------->8---
> Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, 
> na.strings,  (from
> testorg.org!917613Wp#22) : 
>   line 17 did not have 3 elements
> --8<---------------cut here---------------end--------------->8---
> 

I think this is the wrong diagnosis.

Did you actually revert to the earlier version of ob-R.el to confirm that 
this would have run correctly? 

The reason I ask is that I just tried this with org-babel-R-assign-elisp
from 

  org-mode-a5686d87786b1d6514ec85959a2188f703346a06/lisp/ob-R.el

and got the same error. Note this:

--8<---------------cut here---------------start------------->8---

#+name: testtab2
| variable | display  | unit |
|----------+----------+------|
| donor    | Genotype |      |

  
#+BEGIN_SRC emacs-lisp :var test=testtab2
(orgtbl-to-tsv test '(:fmt org-babel-R-quote-tsv-field))
#+END_SRC

#+RESULTS:
: "donor"	"Genotype"


#+BEGIN_SRC emacs-lisp :var value=testtab2
;; from org-babel-R-assign-elisp
(mapcar 'length (org-remove-if-not 'sequencep value))
#+END_SRC

#+RESULTS:
| 3 |

--8<---------------cut here---------------end--------------->8---

In particular, the empty table cells are omitted even though 

`value' or `test' has all lengths as 3. This results in 
calling read.table ( ..., fill=FALSE) implicitly.

Not sure if the fix is to retool org-babel-R-assign-elisp or something
in org-table.el.

HTH,

Chuck

Re: [ob-R] table variable passing broken

[Andreas Leha]

Charles Berry <ccberry@ucsd.edu> writes:
> Andreas Leha <andreas.leha <at> med.uni-goettingen.de> writes:
>
>> 
>> Hi all,
>> 
>> There seems to be a bug in table passing as variables now using the
>> tangle-friendly version of passing variables.
>> 
>> Here is an example (I get an error also with emacs -Q):
>> 
>> --8<---------------cut here---------------start------------->8---
>> * test
>> #+name: testtab
>> | variable              | display                       | unit  |
>> |-----------------------+-------------------------------+-------|
>> | num_cells             | Number of Cells in Well       |       |
>> | cell_area             | Cell Area                     | μm²   |
>> | nucleus_area          | Nucleus Area                  | μm²   |
>> | roundness             | Cell Roundness                |       |
>> | ratio_w2l             | Cell Width to Length Ratio    |       |
>> | inten_nuc_dapi_median | Intensity Nucleus DAPI Median |       |
>> | dapi_median           | Intensity Nucleus DAPI Median |       |
>> | edu_median            | Intensity edu Median          |       |
>> | oct4_median           | Intensity oct4 Median         |       |
>> | clump_size            | Clump Size                    | cells |
>> | short_name            | Cell Line                     |       |
>> | p_col                 | Column                        |       |
>> | batch                 | Batch                         |       |
>> | concentration         | Fibronectin Concentration     | ugml  |
>> | Residual              | Residual                      |       |
>> | evaluation_guid       | Plate                         |       |
>> | donor                 | Genotype                      |       |
>> 
>> #+BEGIN_SRC R :session *test* :var test=testtab
>>   test
>> #+END_SRC
>> 
>> #+RESULTS:
>> --8<---------------cut here---------------end--------------->8---
>> 
>> I see this in my R session:
>> 
>> --8<---------------cut here---------------start------------->8---
>> Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, 
>> na.strings,  (from
>> testorg.org!917613Wp#22) : 
>>   line 17 did not have 3 elements
>> --8<---------------cut here---------------end--------------->8---
>> 
>
> I think this is the wrong diagnosis.

I agree.  Saving the table as tsv (via org-table-export) results
in a file that cannot be read from R either.

>
> Did you actually revert to the earlier version of ob-R.el to confirm that 
> this would have run correctly? 

I did not revert.  But that org file used to work.  I won't be able
to bisect any time soon.

>
> The reason I ask is that I just tried this with org-babel-R-assign-elisp
> from 
>
>   org-mode-a5686d87786b1d6514ec85959a2188f703346a06/lisp/ob-R.el
>
> and got the same error. Note this:
>
>
> #+name: testtab2
> | variable | display  | unit |
> |----------+----------+------|
> | donor    | Genotype |      |
>
>   
> #+BEGIN_SRC emacs-lisp :var test=testtab2
> (orgtbl-to-tsv test '(:fmt org-babel-R-quote-tsv-field))
> #+END_SRC
>
> #+RESULTS:
> : "donor"	"Genotype"
>

exactly.  That also causes the org-table-export to fail.

>
> #+BEGIN_SRC emacs-lisp :var value=testtab2
> ;; from org-babel-R-assign-elisp
> (mapcar 'length (org-remove-if-not 'sequencep value))
> #+END_SRC
>
> #+RESULTS:
> | 3 |
>
> In particular, the empty table cells are omitted even though 
>
> `value' or `test' has all lengths as 3. This results in 
> calling read.table ( ..., fill=FALSE) implicitly.
>
> Not sure if the fix is to retool org-babel-R-assign-elisp or something
> in org-table.el.
>

I am the wrong person to answer that.  But it looks to me to be an
issue for org-table.el.

Thanks for your better analysis.

Regards,
Andreas

Re: [ob-R] table variable passing broken

[Andreas Leha]

Andreas Leha <andreas.leha@med.uni-goettingen.de> writes:
> Charles Berry <ccberry@ucsd.edu> writes:
>> Andreas Leha <andreas.leha <at> med.uni-goettingen.de> writes:
>>
>>> 
>>> Hi all,
>>> 
>>> There seems to be a bug in table passing as variables now using the
>>> tangle-friendly version of passing variables.
>>> 
>>> Here is an example (I get an error also with emacs -Q):
>>> 
>>> --8<---------------cut here---------------start------------->8---
>>> * test
>>> #+name: testtab
>>> | variable              | display                       | unit  |
>>> |-----------------------+-------------------------------+-------|
>>> | num_cells             | Number of Cells in Well       |       |
>>> | cell_area             | Cell Area                     | μm²   |
>>> | nucleus_area          | Nucleus Area                  | μm²   |
>>> | roundness             | Cell Roundness                |       |
>>> | ratio_w2l             | Cell Width to Length Ratio    |       |
>>> | inten_nuc_dapi_median | Intensity Nucleus DAPI Median |       |
>>> | dapi_median           | Intensity Nucleus DAPI Median |       |
>>> | edu_median            | Intensity edu Median          |       |
>>> | oct4_median           | Intensity oct4 Median         |       |
>>> | clump_size            | Clump Size                    | cells |
>>> | short_name            | Cell Line                     |       |
>>> | p_col                 | Column                        |       |
>>> | batch                 | Batch                         |       |
>>> | concentration         | Fibronectin Concentration     | ugml  |
>>> | Residual              | Residual                      |       |
>>> | evaluation_guid       | Plate                         |       |
>>> | donor                 | Genotype                      |       |
>>> 
>>> #+BEGIN_SRC R :session *test* :var test=testtab
>>>   test
>>> #+END_SRC
>>> 
>>> #+RESULTS:
>>> --8<---------------cut here---------------end--------------->8---
>>> 
>>> I see this in my R session:
>>> 
>>> --8<---------------cut here---------------start------------->8---
>>> Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, 
>>> na.strings,  (from
>>> testorg.org!917613Wp#22) : 
>>>   line 17 did not have 3 elements
>>> --8<---------------cut here---------------end--------------->8---
>>> 
>>
>> I think this is the wrong diagnosis.
>
> I agree.  Saving the table as tsv (via org-table-export) results
> in a file that cannot be read from R either.
>
>>
>> Did you actually revert to the earlier version of ob-R.el to confirm that 
>> this would have run correctly? 
>
> I did not revert.  But that org file used to work.  I won't be able
> to bisect any time soon.
>
>>
>> The reason I ask is that I just tried this with org-babel-R-assign-elisp
>> from 
>>
>>   org-mode-a5686d87786b1d6514ec85959a2188f703346a06/lisp/ob-R.el
>>
>> and got the same error. Note this:
>>
>>
>> #+name: testtab2
>> | variable | display  | unit |
>> |----------+----------+------|
>> | donor    | Genotype |      |
>>
>>   
>> #+BEGIN_SRC emacs-lisp :var test=testtab2
>> (orgtbl-to-tsv test '(:fmt org-babel-R-quote-tsv-field))
>> #+END_SRC
>>
>> #+RESULTS:
>> : "donor"	"Genotype"
>>
>
> exactly.  That also causes the org-table-export to fail.
>
>>
>> #+BEGIN_SRC emacs-lisp :var value=testtab2
>> ;; from org-babel-R-assign-elisp
>> (mapcar 'length (org-remove-if-not 'sequencep value))
>> #+END_SRC
>>
>> #+RESULTS:
>> | 3 |
>>
>> In particular, the empty table cells are omitted even though 
>>
>> `value' or `test' has all lengths as 3. This results in 
>> calling read.table ( ..., fill=FALSE) implicitly.
>>
>> Not sure if the fix is to retool org-babel-R-assign-elisp or something
>> in org-table.el.
>>
>
> I am the wrong person to answer that.  But it looks to me to be an
> issue for org-table.el.
>
> Thanks for your better analysis.
>
> Regards,
> Andreas

To keep this issue going, here a quick thought:

Since orgtbl-to-csv seems to work, a temporary fix on ob-R's side would
be to use that for passing of tables.

Andreas