Tangling with variables in R

Tangling with variables in R

[Rainer M Krug]

[-- Attachment #1: Type: text/plain, Size: 4776 bytes --]

Hi

I just realized (again) that tangling with variables in R contains many
particularities.

1) it only works with non-tables, i.e. single values.

When defining the following variables:

--8<---------------cut here---------------start------------->8---
#+NAME: YEARS
|   | year          |
|---+---------------|
| 1 | 1990          |
| 2 | 2000          |
#+PROPERTY: var+ YEARS=YEARS
#+PROPERTY: var+ PRESENT=2008
--8<---------------cut here---------------end--------------->8---

I get the following code in the tangled .R file:

,----
| YEARS <- read.table("/var/folders/50/wcr5bjwn75q595n6x82gxj280000gn/T/babel-97151aBD/R-import-97151vpn",
|                       header=TRUE,
|                       row.names=1,
|                       sep="\t",
|                       as.is=TRUE)
| PRESENT <- 2008
`----

Variable transfer from tables does not work, as it is based on a
temporary file, which is not easy to distribute together with the
tangled.R file and requires manual work as the path will be different.

I consider this as a bug which should be fixed.

2) With regards to variables which are defined non-file wide, (i.e. in
properties in a subtree and variables defined per code block and
function call), these are set when they occur in the tangled code, but
they are not unset *for the next code block* or *for R code in the next
subtree* (I hope you know what I mean). They are therefore similar to
the use of a session header argument instead of non-session evaluation
of the code.

This is a (conceptually) a more complex issue, and requires some initial
thought on how this should be dealt with and how the tangled code should
relate to the executed code.

- Should the tangled .R file result in the same output as the execution in
the org file, i.e. an accompanying .R file to a exported pdf, so that
the .R file can be used to reproduce the graphs and analysis in the .pdf
exported from the .org? or

- Is tangling a completely thing to execution, and the resulting R code
in the .R file is not expected to reproduce the results in the org file?

- Finally, does tangling with variables makes sense?

My opinions are 

a) *All* variables should be transferred to the .R file. This can be
already disabled via the :no-expand header argument. Unfortunately, this
is combined with noweb expansion, and it might be useful to split these
two, i.e. to be able to only disable variable expansion but to keep
noweb (I don't use noweb so far, so it is not a problem to me as it is
now).

b) The variable assignments should be per code block / function call. So
a tangled block should look as follow:

--8<---------------cut here---------------start------------->8---
#+NAME: YEARS
|   | year          |
|---+---------------|
| 1 | 1990          |
| 2 | 2000          |
#+PROPERTY: var+ YEARS=YEARS
#+PROPERTY: var+ PRESENT=2008

#+begin_src R
x <- 4
cat(x^2)
#+end_src
--8<---------------cut here---------------end--------------->8---

should result in something like the following:

,----
| ## # Set the variables
| YEARS <- TRANSFER_TABLE()
| PRESENT <- TRANSFER_VALUE()
| ## Here begins the real code
| x <- 4
| cat(x^2)
| ## # Unset all variables previously set
`----

but I prefer the following approach, as the result would be very
similar, only that the variables are still present afterwards which
would make debugging easier:

,----
| ## # Unset all variables previously set
| ## # Set the variables
| YEARS <- TRANSFER_TABLE()
| PRESENT <- TRANSFER_VALUE()
| ## Here begins the real code
| x <- 4
| cat(x^2)
`----

This is effectively already implemented by using R environments. See [1]
and particularly [2] and [3] for how it is implemented. This does not
yet address the concern about the transfer of tables, but I will look at
this.

Apologies for a long post, but I would like to know which direction of
the tangling / variable transfer from org to R should take - I don't
want to spend a lot of time solving a problem which does not really
exist.

So - any comments? Suggestions?

Thanks,

Rainer

Footnotes: 
[1]  https://github.com/rkrug/orgmode-dev

[2]  https://github.com/rkrug/orgmode-dev/blob/R-env/lisp/ob-R.el

[3]  https://github.com/rkrug/orgmode-dev/blob/R-env/etc/R/org_functions.R

-- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Stellenbosch University
South Africa

Tel :       +33 - (0)9 53 10 27 44
Cell:       +33 - (0)6 85 62 59 98
Fax :       +33 - (0)9 58 10 27 44

Fax (D):    +49 - (0)3 21 21 25 22 44

email:      Rainer@krugs.de

Skype:      RMkrug

PGP: 0x0F52F982

[-- Attachment #2: Type: application/pgp-signature, Size: 494 bytes --]

Re: Tangling with variables in R

[Andreas Leha]

Hi Rainer,

Rainer M Krug <Rainer@krugs.de> writes:

> Hi
>
> I just realized (again) that tangling with variables in R contains many
> particularities.
>
> 1) it only works with non-tables, i.e. single values.
>
> When defining the following variables:
>
> #+NAME: YEARS
> |   | year          |
> |---+---------------|
> | 1 | 1990          |
> | 2 | 2000          |
> #+PROPERTY: var+ YEARS=YEARS
> #+PROPERTY: var+ PRESENT=2008
>
>
> I get the following code in the tangled .R file:
>
> ,----
> | YEARS <- read.table("/var/folders/50/wcr5bjwn75q595n6x82gxj280000gn/T/babel-97151aBD/R-import-97151vpn",
> |                       header=TRUE,
> |                       row.names=1,
> |                       sep="\t",
> |                       as.is=TRUE)
> | PRESENT <- 2008
> `----
>
> Variable transfer from tables does not work, as it is based on a
> temporary file, which is not easy to distribute together with the
> tangled.R file and requires manual work as the path will be different.
>
> I consider this as a bug which should be fixed.

Me too, see http://thread.gmane.org/gmane.emacs.orgmode/51067


>
> 2) With regards to variables which are defined non-file wide, (i.e. in
> properties in a subtree and variables defined per code block and
> function call), these are set when they occur in the tangled code, but
> they are not unset *for the next code block* or *for R code in the next
> subtree* (I hope you know what I mean). They are therefore similar to
> the use of a session header argument instead of non-session evaluation
> of the code.
>
> This is a (conceptually) a more complex issue, and requires some initial
> thought on how this should be dealt with and how the tangled code should
> relate to the executed code.
>
> - Should the tangled .R file result in the same output as the execution in
> the org file, i.e. an accompanying .R file to a exported pdf, so that
> the .R file can be used to reproduce the graphs and analysis in the .pdf
> exported from the .org? or
>
> - Is tangling a completely thing to execution, and the resulting R code
> in the .R file is not expected to reproduce the results in the org file?
>
> - Finally, does tangling with variables makes sense?
>
> My opinions are 
>
> a) *All* variables should be transferred to the .R file. This can be
> already disabled via the :no-expand header argument. Unfortunately, this
> is combined with noweb expansion, and it might be useful to split these
> two, i.e. to be able to only disable variable expansion but to keep
> noweb (I don't use noweb so far, so it is not a problem to me as it is
> now).
>
> b) The variable assignments should be per code block / function call. So
> a tangled block should look as follow:
>
> #+NAME: YEARS
> |   | year          |
> |---+---------------|
> | 1 | 1990          |
> | 2 | 2000          |
> #+PROPERTY: var+ YEARS=YEARS
> #+PROPERTY: var+ PRESENT=2008
>
> #+begin_src R
> x <- 4
> cat(x^2)
> #+end_src
>
> should result in something like the following:
>
> ,----
> | ## # Set the variables
> | YEARS <- TRANSFER_TABLE()
> | PRESENT <- TRANSFER_VALUE()
> | ## Here begins the real code
> | x <- 4
> | cat(x^2)
> | ## # Unset all variables previously set
> `----
>
> but I prefer the following approach, as the result would be very
> similar, only that the variables are still present afterwards which
> would make debugging easier:
>
> ,----
> | ## # Unset all variables previously set
> | ## # Set the variables
> | YEARS <- TRANSFER_TABLE()
> | PRESENT <- TRANSFER_VALUE()
> | ## Here begins the real code
> | x <- 4
> | cat(x^2)
> `----
>
> This is effectively already implemented by using R environments. See [1]
> and particularly [2] and [3] for how it is implemented. This does not
> yet address the concern about the transfer of tables, but I will look at
> this.

These are good thoughts!

For the general question on whether tangling should directly reflect
weaving, there was a long (and fruitless) discussion or R-devel lately.
See http://thread.gmane.org/gmane.comp.lang.r.devel/36031 and maybe
http://thread.gmane.org/gmane.comp.lang.r.devel/36031/focus=36075 where
Yihui states that it is very hard to tangle code that produces the
same result as weaving.  This might actually be easier in org than in
Sweave/knitr, but still the org setup will have a big impact, that would
be hard to replicate in the tangled R code.

Setting aside the general question:
In my opinion, it should definitely be possible to tangle with
variables.  I think the approach of the environments to address (2) is a
good one.



>
> Apologies for a long post, but I would like to know which direction of
> the tangling / variable transfer from org to R should take - I don't
> want to spend a lot of time solving a problem which does not really
> exist.
>
> So - any comments? Suggestions?
>

See above.  And thanks for your work on ob-R!  Keep it up!

> Thanks,
>
> Rainer
>
> Footnotes: 
> [1]  https://github.com/rkrug/orgmode-dev
>
> [2]  https://github.com/rkrug/orgmode-dev/blob/R-env/lisp/ob-R.el
>
> [3]  https://github.com/rkrug/orgmode-dev/blob/R-env/etc/R/org_functions.R


Best,
Andreas

Re: Tangling with variables in R

[Rainer M Krug]

[-- Attachment #1: Type: text/plain, Size: 6716 bytes --]

Andreas Leha <andreas.leha@med.uni-goettingen.de> writes:

> Hi Rainer,

Hi Andreas,

>
> Rainer M Krug <Rainer@krugs.de> writes:
>
>> Hi
>>
>> I just realized (again) that tangling with variables in R contains many
>> particularities.
>>
>> 1) it only works with non-tables, i.e. single values.
>>
>> When defining the following variables:
>>
>> #+NAME: YEARS
>> |   | year          |
>> |---+---------------|
>> | 1 | 1990          |
>> | 2 | 2000          |
>> #+PROPERTY: var+ YEARS=YEARS
>> #+PROPERTY: var+ PRESENT=2008
>>
>>
>> I get the following code in the tangled .R file:
>>
>> ,----
>> | YEARS <- read.table("/var/folders/50/wcr5bjwn75q595n6x82gxj280000gn/T/babel-97151aBD/R-import-97151vpn",
>> |                       header=TRUE,
>> |                       row.names=1,
>> |                       sep="\t",
>> |                       as.is=TRUE)
>> | PRESENT <- 2008
>> `----
>>
>> Variable transfer from tables does not work, as it is based on a
>> temporary file, which is not easy to distribute together with the
>> tangled.R file and requires manual work as the path will be different.
>>
>> I consider this as a bug which should be fixed.
>
> Me too, see http://thread.gmane.org/gmane.emacs.orgmode/51067
>

OK - then I will include a fix to the tangling of tables issue in one of
my first patches.

>
>>
>> 2) With regards to variables which are defined non-file wide, (i.e. in
>> properties in a subtree and variables defined per code block and
>> function call), these are set when they occur in the tangled code, but
>> they are not unset *for the next code block* or *for R code in the next
>> subtree* (I hope you know what I mean). They are therefore similar to
>> the use of a session header argument instead of non-session evaluation
>> of the code.
>>
>> This is a (conceptually) a more complex issue, and requires some initial
>> thought on how this should be dealt with and how the tangled code should
>> relate to the executed code.
>>
>> - Should the tangled .R file result in the same output as the execution in
>> the org file, i.e. an accompanying .R file to a exported pdf, so that
>> the .R file can be used to reproduce the graphs and analysis in the .pdf
>> exported from the .org? or
>>
>> - Is tangling a completely thing to execution, and the resulting R code
>> in the .R file is not expected to reproduce the results in the org file?
>>
>> - Finally, does tangling with variables makes sense?
>>
>> My opinions are 
>>
>> a) *All* variables should be transferred to the .R file. This can be
>> already disabled via the :no-expand header argument. Unfortunately, this
>> is combined with noweb expansion, and it might be useful to split these
>> two, i.e. to be able to only disable variable expansion but to keep
>> noweb (I don't use noweb so far, so it is not a problem to me as it is
>> now).
>>
>> b) The variable assignments should be per code block / function call. So
>> a tangled block should look as follow:
>>
>> #+NAME: YEARS
>> |   | year          |
>> |---+---------------|
>> | 1 | 1990          |
>> | 2 | 2000          |
>> #+PROPERTY: var+ YEARS=YEARS
>> #+PROPERTY: var+ PRESENT=2008
>>
>> #+begin_src R
>> x <- 4
>> cat(x^2)
>> #+end_src
>>
>> should result in something like the following:
>>
>> ,----
>> | ## # Set the variables
>> | YEARS <- TRANSFER_TABLE()
>> | PRESENT <- TRANSFER_VALUE()
>> | ## Here begins the real code
>> | x <- 4
>> | cat(x^2)
>> | ## # Unset all variables previously set
>> `----
>>
>> but I prefer the following approach, as the result would be very
>> similar, only that the variables are still present afterwards which
>> would make debugging easier:
>>
>> ,----
>> | ## # Unset all variables previously set
>> | ## # Set the variables
>> | YEARS <- TRANSFER_TABLE()
>> | PRESENT <- TRANSFER_VALUE()
>> | ## Here begins the real code
>> | x <- 4
>> | cat(x^2)
>> `----
>>
>> This is effectively already implemented by using R environments. See [1]
>> and particularly [2] and [3] for how it is implemented. This does not
>> yet address the concern about the transfer of tables, but I will look at
>> this.
>
> These are good thoughts!

Thanks.

>
> For the general question on whether tangling should directly reflect
> weaving, there was a long (and fruitless) discussion or R-devel lately.
> See http://thread.gmane.org/gmane.comp.lang.r.devel/36031 and maybe
> http://thread.gmane.org/gmane.comp.lang.r.devel/36031/focus=36075 where
> Yihui states that it is very hard to tangle code that produces the
> same result as weaving.  This might actually be easier in org than in
> Sweave/knitr, but still the org setup will have a big impact, that would
> be hard to replicate in the tangled R code.

I agree completely - but it might be worth to try to go into this
direction rather then blocking it with choosing the wrong approach
initially.

>
> Setting aside the general question:
> In my opinion, it should definitely be possible to tangle with
> variables.  I think the approach of the environments to address (2) is a
> good one.

OK - I will put some thoughts into this and see how it could be done
easily and with limited rewuirements to supplementary files.

>
>
>
>>
>> Apologies for a long post, but I would like to know which direction of
>> the tangling / variable transfer from org to R should take - I don't
>> want to spend a lot of time solving a problem which does not really
>> exist.
>>
>> So - any comments? Suggestions?
>>
>
> See above.  And thanks for your work on ob-R!  Keep it up!

Thanks - I will do. Your email comes at the right moment, as I was
wondering if I am the only one using R and ESS and org. But I guess it
is rather the inner workings do not impact on most users.

I will see that I get the first set of patches together before the end
of June (after this it is holidays).

Cheers,

Rainer

>
>> Thanks,
>>
>> Rainer
>>
>> Footnotes: 
>> [1]  https://github.com/rkrug/orgmode-dev
>>
>> [2]  https://github.com/rkrug/orgmode-dev/blob/R-env/lisp/ob-R.el
>>
>> [3]  https://github.com/rkrug/orgmode-dev/blob/R-env/etc/R/org_functions.R
>
>
> Best,
> Andreas
>
>

-- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Stellenbosch University
South Africa

Tel :       +33 - (0)9 53 10 27 44
Cell:       +33 - (0)6 85 62 59 98
Fax :       +33 - (0)9 58 10 27 44

Fax (D):    +49 - (0)3 21 21 25 22 44

email:      Rainer@krugs.de

Skype:      RMkrug

PGP: 0x0F52F982

[-- Attachment #2: Type: application/pgp-signature, Size: 494 bytes --]

Re: Tangling with variables in R

[Sebastien Vauban]

Rainer M Krug wrote:
>> See above.  And thanks for your work on ob-R!  Keep it up!
>
> Thanks - I will do. Your email comes at the right moment, as I was
> wondering if I am the only one using R and ESS and org. But I guess it
> is rather the inner workings do not impact on most users.

No, you're not. But, yes, I'm only using that combination once every so
many months.

Still, when I do use it, I'm glad that it works and produces wonderful
results!

Best regards,
  Seb

-- 
Sebastien Vauban