Re: Org-mode and ESS

From: Austin Frank <austin.frank@gmail.com>
To: Eric Schulte <schulte.eric@gmail.com>
Cc: emacs-orgmode@gnu.org
Subject: Re: Org-mode and ESS
Date: Thu, 13 Nov 2008 00:40:24 -0500	[thread overview]
Message-ID: <m0prl06v5j.fsf@gmail.com> (raw)
In-Reply-To: <87myg4z4wv.fsf@gmail.com> (Eric Schulte's message of "Wed, 12 Nov 2008 19:22:08 -0800")

[-- Attachment #1.1.1: Type: text/plain, Size: 3355 bytes --]

On Wed, Nov 12 2008, Eric Schulte wrote:

> In the above example how would you/Sweave handle the case where there
> are multiple R blocks in the same file?  For example

Sweave puts all variables defined in a document into the same
environment.  So assigning to a twice overwrites the first assignment.

,----[ tmp2.Rorg ]
| #+BEGIN_R
| a <- 3
| a
| #+END_R
| 
| - the value of a is \R{a}
| - the value of a + 3 is \R{a + 3}  
| 
| #+BEGIN_R
| a <- 8
| a
| #+END_R
| 
| - the value of a is \R{a}
| - the value of a + 3 is \R{a + 3}  
`----

,----[ tmp2.org ]
| #+BEGIN_LaTeX
| \begin{Schunk}
| \begin{Sinput}
| > a <- 3
| > a
| \end{Sinput}
| \begin{Soutput}
| [1] 3
| \end{Soutput}
| \end{Schunk}
| #+END_LaTeX
| 
| - the value of a is 3
| - the value of a + 3 is 6  
| 
| #+BEGIN_LaTeX
| \begin{Schunk}
| \begin{Sinput}
| > a <- 8
| > a
| \end{Sinput}
| \begin{Soutput}
| [1] 8
| \end{Soutput}
| \end{Schunk}
| #+END_LaTeX
| 
| - the value of a is 8
| - the value of a + 3 is 11  
`----

> With that questions answered it should be relatively straightforward
> to implement exporting as you described using only org-mode's block
> processing, and letting ESS do the actual calculations all without any
> dependence on Sweave.

There is a Sweave.sty that handles formatting of the Schunk, Sinput, and
Soutput environments.  While you may not need to depend on Sweave for
the preprocessing step, it would be nice if the formatted document at
least had the option of having the same appearance as Sweave-processed
files.  Still, I suppose I could just add a #+LATEX_HEADER line for
these files.

> if you have examples of graphic creation, I'd be interested to see how
> they work, and relatedly how difficult it would be to move that
> functionality into org-mode.

OK, here comes the first attempt.  The additional feature demonstrated
in this example is the inclusion of options to the Sweave preprocessor
at the beginning of the block.  In this case I've specified fig=TRUE to
tell the preprocessor to generate image files for the plot commands
contained in the block.  By default, Sweave creates a .eps and a .pdf
file for each block that contains a plotting or printing command and has
the fig=TRUE argument, and then inserts an \includegraphics{} command in
the output file.

,----[ tmp3.Rorg ]
| * first figure
| #+BEGIN_R fig=TRUE
| x <- rnorm(100)
| y <- rnorm(100)
| 
| plot(x,y)
| #+END_R
`----

,----[ tmp3.org ]
| * first figure
| #+BEGIN_LaTeX
| \begin{Schunk}
| \begin{Sinput}
| > x <- rnorm(100)
| > y <- rnorm(100)
| > plot(x, y)
| \end{Sinput}
| \end{Schunk}
| #+END_LaTeX
| #+LaTeX:  \includegraphics{tmp3-001}
`----

,----[ tmp3.tex ]
| % Created 2008-11-12 Wed 23:46
| \documentclass[11pt,a4paper]{article}
| \usepackage[utf8]{inputenc}
| \usepackage[T1]{fontenc}
| \usepackage{graphicx}
| \usepackage{hyperref}
| \usepackage{Sweave}
| 
| \title{tmp3}
| \author{Austin Frank}
| \date{12 November 2008}
| 
| \begin{document}
| 
| \maketitle
| 
| 
| \section*{first figure}
| \label{sec-1}
| 
| \begin{Schunk}
| \begin{Sinput}
| > x <- rnorm(100)
| > y <- rnorm(100)
| > plot(x, y)
| \end{Sinput}
| \end{Schunk}
| \includegraphics{tmp3-001}
| 
| \end{document}
`----

And the pdf is attached.

[-- Attachment #1.1.2: tmp3.pdf --]
[-- Type: application/pdf, Size: 30353 bytes --]

[-- Attachment #1.1.3: Type: text/plain, Size: 3021 bytes --]

I haven't looked into it too closely, but I would guess that when
fig=TRUE is specified, first the R block is run, and then the additional
R commands

  dev.copy2eps()
  dev.copy2pdf()

are run.  This results in the creation of eps and pdf files based on the
last thing plotted.  The functions could each take a file="foo" argument
(the filename defaults to Rplots.[eps|pdf]).  Sweave uses automatically
incremented names for its generated graphics files.  If your block
system could have an after-block hook, you could evaluate the original
block, then the appropriate dev.copy function, and then add an
additional line to the exported file for inclusion of the graphics file.

> However I think the rest of the point you make immediately below would
> be easily addressed through using an org-mode only approach.

Exciting!  If you're interested in what other options the Sweave
preprocessor contains, the manual can be found at
http://www.stat.auckland.ac.nz/~dscott/782/Sweave-manual-20060104.pdf.

Sweave also has a processing option called "tangle".  This extracts all
of the R code from a document and puts into its own file.  This would be
a useful feature in a generic block exporter as well, I would think.
Take all of the blocks of a certain type and dump the content into a
file with the appropriate extension for whatever the major mode was for
that block.

Finally , a couple of systems for caching Sweave output have sprung up.
I think these use the strategy of hashing the content of each block,
saving the output of each block, and only re-running the block if the
content has changed.  A general-purpose option like this for the block
exporting system in org could be very nice to have.  It's possible that
the attachment system with git integration already gets us very close to
a working version of this.

The more I think about this last possibility, the more potential I think
it has.  With the caching turned on, at export time you create files
containing the contents of each block to be cached, one file per block.
You create a second set of files that has the results of block export,
again one file per block.  You add the directory of block files and
results using org-attach.  On the next export, block files are generated
again.  You use git to ask which ones have changed.  For those that have
changed, you go on to do the normal block export, saving the results
again.  Finally, you pull the results files back into the original
document in the right places.  It would be neat if something like this
could work.

> If you happen to use yasnippets, I find it makes the creation of
> blocks in org files a very quick/easy experience.  See
> http://legito.net/worg/org-configs/index.php#sec-2.1 for details.

I'll check it out, thanks.  Sorry to ramble, hope this was useful!

Looking forward to seeing what you come up with!
/au

-- 
Austin Frank
http://aufrank.net
GPG Public Key (D7398C2F): http://aufrank.net/personal.asc

[-- Attachment #1.2: Type: application/pgp-signature, Size: 193 bytes --]

[-- Attachment #2: Type: text/plain, Size: 204 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode