org-babel-R export parameters

org-babel-R export parameters

[Russell Adams]

I needed the ability to view what parameters were sent to plot in
R. Turns out that you can't query some things like the active filename
in R, so I went back to org.

This patch takes each parameter and converts it to a variable in R,
including the filename (org_babel_filename).

My intent is to parse the filename in R to create a dev.copy() to a
pdf while creating a PNG. Ideally then I can see the png inline, and
export to latex with the PDF.

Thanks.

------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com

PGP Key ID:     0x1160DCB3           http://www.adamsinfoserv.com/

Fingerprint:    1723 D8CA 4280 1EC9 557F  66E8 1154 E018 1160 DCB3




$ diff -Narub org-babel-R.el.orig org-babel-R.el
--- org-babel-R.el.orig           2010-06-06 17:42:46.000000000 -0500
+++ org-babel-R.el                2010-06-06 17:53:59.000000000 -0500
@@ -161,7 +161,12 @@
                      (if (member (car pair) allowed-args)
                                       (format ",%s=%s" (substring (symbol-name (car pair)) 1) (cdr pair)) ""))
                                                          params ""))
-    (format "%s(%s=\"%s\"%s%s%s)\n" device filearg out-file args (if extra-args "," "") (or extra-args ""))))
+    (setq babel-vars (mapconcat (lambda (pair)
+                                  (if (member (car pair) allowed-args)
+                                      (format "org_babel_%s=\"%s\"\n" (substring (symbol-name (car pair)) 1) (cdr pair)) ""))
+                                params ""))
+    (setq babel-vars (concat babel-vars (format "org_babel_filename=\"%s\"\n" out-file)))
+    (format "%s(%s=\"%s\"%s%s%s)\n%s\n" device filearg out-file args (if extra-args "," "") (or extra-args "") babel-vars)))
 
 (defvar org-babel-R-eoe-indicator "'org_babel_R_eoe'")
 (defvar org-babel-R-eoe-output "[1] \"org_babel_R_eoe\"")

Re: org-babel-R export parameters

[Dan Davison]

Russell Adams <RLAdams@AdamsInfoServ.Com> writes:

> I needed the ability to view what parameters were sent to plot in
> R. Turns out that you can't query some things like the active filename
> in R, so I went back to org.
>
> This patch takes each parameter and converts it to a variable in R,
> including the filename (org_babel_filename).
>
> My intent is to parse the filename in R to create a dev.copy() to a
> pdf while creating a PNG. Ideally then I can see the png inline, and
> export to latex with the PDF.

Hi Russell,

I haven't completely understood paras 1 and 3 above. Would you mind
posting an example that illustrates the problem that this patch solves?

Dan


>
> Thanks.
>
> ------------------------------------------------------------------
> Russell Adams                            RLAdams@AdamsInfoServ.com
>
> PGP Key ID:     0x1160DCB3           http://www.adamsinfoserv.com/
>
> Fingerprint:    1723 D8CA 4280 1EC9 557F  66E8 1154 E018 1160 DCB3
>
>
>
>
> $ diff -Narub org-babel-R.el.orig org-babel-R.el
> --- org-babel-R.el.orig           2010-06-06 17:42:46.000000000 -0500
> +++ org-babel-R.el                2010-06-06 17:53:59.000000000 -0500
> @@ -161,7 +161,12 @@
>                       (if (member (car pair) allowed-args)
>                                        (format ",%s=%s" (substring (symbol-name (car pair)) 1) (cdr pair)) ""))
>                                                           params ""))
> -    (format "%s(%s=\"%s\"%s%s%s)\n" device filearg out-file args (if extra-args "," "") (or extra-args ""))))
> +    (setq babel-vars (mapconcat (lambda (pair)
> +                                  (if (member (car pair) allowed-args)
> +                                      (format "org_babel_%s=\"%s\"\n" (substring (symbol-name (car pair)) 1) (cdr pair)) ""))
> +                                params ""))
> +    (setq babel-vars (concat babel-vars (format "org_babel_filename=\"%s\"\n" out-file)))
> +    (format "%s(%s=\"%s\"%s%s%s)\n%s\n" device filearg out-file args (if extra-args "," "") (or extra-args "") babel-vars)))
>  
>  (defvar org-babel-R-eoe-indicator "'org_babel_R_eoe'")
>  (defvar org-babel-R-eoe-output "[1] \"org_babel_R_eoe\"")
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: org-babel-R export parameters

[Russell Adams]

On Mon, Jun 07, 2010 at 12:27:46AM +0100, Dan Davison wrote:
> Russell Adams <RLAdams@AdamsInfoServ.Com> writes:
> 
> > I needed the ability to view what parameters were sent to plot in
> > R. Turns out that you can't query some things like the active filename
> > in R, so I went back to org.

Really, you can't query the filename of the active device in R. I
exported all the parameters to the R code block to R variables so I
can use them in R.

> > This patch takes each parameter and converts it to a variable in R,
> > including the filename (org_babel_filename).
> >
> > My intent is to parse the filename in R to create a dev.copy() to a
> > pdf while creating a PNG. Ideally then I can see the png inline, and
> > export to latex with the PDF.

See example.

> 
> Hi Russell,
> 
> I haven't completely understood paras 1 and 3 above. Would you mind
> posting an example that illustrates the problem that this patch solves?
> 
> Dan
> 

Here's a non-functional example that I hope can illustrate the point:

----------------------------------------------------------------------
* HEading

  blah blah blah blah blah blah blah blah blah blah blah blah blah


#+BEGIN_SRC R :file whatever.png :width 300 :height 200 :exports none

a = c(1,2,3,4)
plot(a)

dev.copy(pdf,gsub(".png",".pdf",org_babel_filename))
#+END_SRC

#+results:
[[file:whatever.png]]

#+LATEX \\include[whatever.pdf]

----------------------------------------------------------------------

The idea is I want to be able to see the inline PNG image of my graph
while writing, and when I export I'll point the latex exporter to the
PDF. This provides a vector format for Latex, instead of a low
resolution bitmap. Yes you can use PDF's as includes, and they look
great because they are a vector format.

Does that help?

I'm having issues with the R dev.copy, but that's a separate issue.

Thanks.

------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com

PGP Key ID:     0x1160DCB3           http://www.adamsinfoserv.com/

Fingerprint:    1723 D8CA 4280 1EC9 557F  66E8 1154 E018 1160 DCB3

Re: org-babel-R export parameters

[Dan Davison]

Russell Adams <RLAdams@AdamsInfoServ.Com> writes:

> On Mon, Jun 07, 2010 at 12:27:46AM +0100, Dan Davison wrote:
>> Russell Adams <RLAdams@AdamsInfoServ.Com> writes:
>> 
>> > I needed the ability to view what parameters were sent to plot in
>> > R. Turns out that you can't query some things like the active filename
>> > in R, so I went back to org.
>
> Really, you can't query the filename of the active device in R. I
> exported all the parameters to the R code block to R variables so I
> can use them in R.
>
>> > This patch takes each parameter and converts it to a variable in R,
>> > including the filename (org_babel_filename).
>> >
>> > My intent is to parse the filename in R to create a dev.copy() to a
>> > pdf while creating a PNG. Ideally then I can see the png inline, and
>> > export to latex with the PDF.
>
> See example.
>
>> 
>> Hi Russell,
>> 
>> I haven't completely understood paras 1 and 3 above. Would you mind
>> posting an example that illustrates the problem that this patch solves?
>> 
>> Dan
>> 
>
> Here's a non-functional example that I hope can illustrate the point:
>
> ----------------------------------------------------------------------
> * HEading
>
>   blah blah blah blah blah blah blah blah blah blah blah blah blah
>
>
> #+BEGIN_SRC R :file whatever.png :width 300 :height 200 :exports none
>
> a = c(1,2,3,4)
> plot(a)
>
> dev.copy(pdf,gsub(".png",".pdf",org_babel_filename))
> #+END_SRC
>
> #+results:
> [[file:whatever.png]]
>
> #+LATEX \\include[whatever.pdf]
>
> ----------------------------------------------------------------------
>
> The idea is I want to be able to see the inline PNG image of my graph
> while writing, and when I export I'll point the latex exporter to the
> PDF. 

Hi Russell,

Thanks, that's clear. You may well be right that it would be appropriate
to expose further information about the babel source block
(e.g. the :file argument) to the external language. However, one general
design consideration is that where possible we do want to avoid
implementing ad-hoc language-specific behaviour. If we do go your route
I would suggest that everything should be wrapped up in a single list
object (hash in perl, dict in python, etc etc), and that that list/hash
object should have a reasonably consistent name across languages
(__org_babel_header_args__ or something).

For now however, I would suggest taking the view that what you are doing
is slightly non-standard org-babel usage, and therefore that it should
be achieved more explicitly. E.g. how about the following approach?

#+begin_src R :results file :var basename="myplot"
  a <- 1:4
  pngfile <- sprintf("%s.png", basename)
  pdffile <- sprintf("%s.pdf", basename)
  
  png(pngfile)
  plot(a)
  dev.off()
  
  pdf(pdffile)
  plot(a)
  dev.off()
  
  pngfile
#+end_src

It should be possible to avoid repetitive code by abstracting that
functionality. One possibility would be an R function
plot.png.and.pdf.returning.png.filename(); alternatively, I suspect
analogous abstraction could be achieved in org-babel using noweb block
references or a library of babel function.

Dan

> This provides a vector format for Latex, instead of a low
> resolution bitmap. Yes you can use PDF's as includes, and they look
> great because they are a vector format.
>
> Does that help?
>
> I'm having issues with the R dev.copy, but that's a separate issue.
>
> Thanks.
>
> ------------------------------------------------------------------
> Russell Adams                            RLAdams@AdamsInfoServ.com
>
> PGP Key ID:     0x1160DCB3           http://www.adamsinfoserv.com/
>
> Fingerprint:    1723 D8CA 4280 1EC9 557F  66E8 1154 E018 1160 DCB3
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: org-babel-R export parameters

[Russell Adams]

On Mon, Jun 07, 2010 at 10:03:46AM +0100, Dan Davison wrote:
> Hi Russell,
> 
> Thanks, that's clear. You may well be right that it would be appropriate
> to expose further information about the babel source block
> (e.g. the :file argument) to the external language. However, one general
> design consideration is that where possible we do want to avoid
> implementing ad-hoc language-specific behaviour. If we do go your route
> I would suggest that everything should be wrapped up in a single list
> object (hash in perl, dict in python, etc etc), and that that list/hash
> object should have a reasonably consistent name across languages
> (__org_babel_header_args__ or something).

I already thought of that, but R was my target here. I didn't want to
jump into a major design decision. ;]


> For now however, I would suggest taking the view that what you are doing
> is slightly non-standard org-babel usage, and therefore that it should
> be achieved more explicitly. E.g. how about the following approach?
> 
> #+begin_src R :results file :var basename="myplot"
>   a <- 1:4
>   pngfile <- sprintf("%s.png", basename)
>   pdffile <- sprintf("%s.pdf", basename)
>   
>   png(pngfile)
>   plot(a)
>   dev.off()
>   
>   pdf(pdffile)
>   plot(a)
>   dev.off()
>   
>   pngfile
> #+end_src

I like this better, and I can use dev.copy here too. The issue is a
plot is rarely one line. Most of my plots are a half page of code
between the data set, legend, etc.

------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com

PGP Key ID:     0x1160DCB3           http://www.adamsinfoserv.com/

Fingerprint:    1723 D8CA 4280 1EC9 557F  66E8 1154 E018 1160 DCB3

Re: Re: org-babel-R export parameters

[Erik Iverson]

<snip>
> The idea is I want to be able to see the inline PNG image of my graph
> while writing, and when I export I'll point the latex exporter to the
> PDF. This provides a vector format for Latex, instead of a low
> resolution bitmap. Yes you can use PDF's as includes, and they look
> great because they are a vector format.

I have this exact same issue.  One idea I haven't explored, but that might be 
possible, is to generate PDFs in org-babel, and then display the pdf using emacs 
doc-view, which converts to png for display in an emacs buffer, if I understand 
correctly.

Re: Re: org-babel-R export parameters

[Erik Iverson]

<snip>
>>
>> #+begin_src R :results file :var basename="myplot"
>>   a <- 1:4
>>   pngfile <- sprintf("%s.png", basename)
>>   pdffile <- sprintf("%s.pdf", basename)
>>   
>>   png(pngfile)
>>   plot(a)
>>   dev.off()
>>   
>>   pdf(pdffile)
>>   plot(a)
>>   dev.off()
>>   
>>   pngfile
>> #+end_src
> 
> I like this better, and I can use dev.copy here too. The issue is a
> plot is rarely one line. Most of my plots are a half page of code
> between the data set, legend, etc.
> 

You could just make your complex plots functions, and then call your 
function in in one line. Or if you're using lattice or ggplot2, create 
objects representing the plot, and then plot them in one line.

Re: Re: org-babel-R export parameters

[Russell Adams]

On Mon, Jun 07, 2010 at 01:40:32PM -0500, Erik Iverson wrote:
> You could just make your complex plots functions, and then call your  
> function in in one line. Or if you're using lattice or ggplot2, create  
> objects representing the plot, and then plot them in one line.

Funny you should mention that, I already did. ;]

So my current issue is that it appears that even though I set :export
none, the png is still included in my Latex output.

Suggestions?

======================================================================
* Heading

  blah blah blah blah blah blah blah blah blah blah blah blah blah

#+BEGIN_SRC R :results file :exports none :var basename="MyFile"

doublePlot = function (base, plotFunc) {

  # basename from org-babel
  myPng = paste(base,"png",sep=".")
  myPdf = paste(base,"pdf",sep=".")

  png(myPng,
      width  = 1024,
      height = 500)

  plotFunc()

  dev.off()

  pdf(myPdf,
      paper = 'usr')

  plotFunc()

  dev.off()

  print(myPng)

}

a = c(1,2,3,4)

doublePlot(base=basename, plotFunc=function(){
     plot(a)
     })

#+END_SRC

#+results:
[[file:MyFile.png]]

#+LATEX: \includegraphics[width=\textwidth]{MyFile.pdf}
======================================================================




------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com

PGP Key ID:     0x1160DCB3           http://www.adamsinfoserv.com/

Fingerprint:    1723 D8CA 4280 1EC9 557F  66E8 1154 E018 1160 DCB3

Re: Re: org-babel-R export parameters

[Eric Schulte]

Hi Russel,

Russell Adams <RLAdams@AdamsInfoServ.Com> writes:

[...]
>
> So my current issue is that it appears that even though I set :export
> none, the png is still included in my Latex output.
>
> Suggestions?
>

The png is still included your output because the link to the png is
part of your org-mode buffer.

>
> #+results:
> [[file:MyFile.png]]
>

Currently org-babel isn't bold enough to remove existing results during
export.  The :exports none argument simply means that babel will not run
the source-code block on export.

I personally find this useful because it allows me to use code blocks to
generate results, and then when I'm content with the file I can set
":exports none" to avoid re-generating the file on every export -- while
retaining the existing link keeps the file included in my export.

I do see how this could be confusing, and maybe it would be appropriate
to begin stripping out the results of code blocks on export.

Best -- Eric

Re: Re: org-babel-R export parameters

[Erik Iverson]

> I personally find this useful because it allows me to use code blocks to
> generate results, and then when I'm content with the file I can set
> ":exports none" to avoid re-generating the file on every export -- while
> retaining the existing link keeps the file included in my export.
> 
> I do see how this could be confusing, and maybe it would be appropriate
> to begin stripping out the results of code blocks on export.

1 vote for that from me, for the exact same situation as this thread 
describes, i.e., I want to preview graphics in emacs, but have PDFs 
included in LaTeX upon export.

I have cobbled together something this afternoon that uses doc-view to 
overlay a png over a PDF results file.  There are a couple things to 
sort out, but I'll post that when I get home, it could possibly help 
here, at least with not having to generate both types of files from 
R/org-mode.

Re: Re: org-babel-R export parameters

[Russell Adams]

On Mon, Jun 07, 2010 at 12:44:43PM -0700, Eric Schulte wrote:
> The png is still included your output because the link to the png is
> part of your org-mode buffer.


That explains it. So here's the fix, add # at the beginning of the
line with the inline image.

======================================================================


* HEading

  blah blah blah blah blah blah blah blah blah blah blah blah blah


#+BEGIN_SRC R :results raw :exports none :var basename="MyFile"
# Data from org
doublePlot = function (base, plotFunc) {

  # basename from org-babel
  myPng = paste(base,"png",sep=".")
  myPdf = paste(base,"pdf",sep=".")

  png(myPng,
      width  = 1024,
      height = 500)

  plotFunc()

  dev.off()

  pdf(myPdf,
      paper = 'usr')

  plotFunc()

  dev.off()

  print(paste("# [[file:",myPng,"]]",sep=""))

}

a = c(1,2,3,4)

doublePlot(base=basename, plotFunc=function(){
     plot(a)
     })

#+END_SRC

#+results:
# [[file:MyFile.png]]


#+LATEX: \includegraphics[width=\textwidth]{MyFile.pdf}
======================================================================


------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com

PGP Key ID:     0x1160DCB3           http://www.adamsinfoserv.com/

Fingerprint:    1723 D8CA 4280 1EC9 557F  66E8 1154 E018 1160 DCB3

Re: Re: org-babel-R export parameters

[Eric Schulte]

Erik Iverson <eriki@ccbr.umn.edu> writes:

>> I personally find this useful because it allows me to use code blocks to
>> generate results, and then when I'm content with the file I can set
>> ":exports none" to avoid re-generating the file on every export -- while
>> retaining the existing link keeps the file included in my export.
>>
>> I do see how this could be confusing, and maybe it would be appropriate
>> to begin stripping out the results of code blocks on export.
>
> 1 vote for that from me, for the exact same situation as this thread
> describes, i.e., I want to preview graphics in emacs, but have PDFs
> included in LaTeX upon export.
>

Upon reflection, I agree with you (and Russel) that just because results
were lying around in the buffer before export doesn't mean that the
':exports none' and ':exports code' header arguments should have their
meaning subverted.

I've made this change to the exporter (in the latest git HEAD) and
thanks to the way Org-mode handles export, the removal of results during
export does *not* mean that the results will be permanently removed
after export -- said differently, exporting will have no lasting effect
on the contents of the org-mode buffer.

Thanks to both of you for bringing this up.

>
> I have cobbled together something this afternoon that uses doc-view to
> overlay a png over a PDF results file.  There are a couple things to
> sort out, but I'll post that when I get home, it could possibly help
> here, at least with not having to generate both types of files from
> R/org-mode.

sounds great, I look forward to seeing it.  It seems to be that such a
pdf inline preview would be useful across general org-mode usage, not
just in the context of babel.

Best -- Eric