* Re: Babel - simple getting started problem
2009-12-07 17:36 ` Graham Smith
@ 2009-12-07 20:13 ` Dan Davison
2009-12-07 20:19 ` Graham Smith
0 siblings, 1 reply; 10+ messages in thread
From: Dan Davison @ 2009-12-07 20:13 UTC (permalink / raw)
To: Graham Smith; +Cc: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 1596 bytes --]
Graham Smith <myotisone@gmail.com> writes:
> Dan
>
>> Here's a version of your example that works.
>
> Thanks, this all works for me now.
>
> In addition to the tutorial on the worg, is there a manual somewhere,
> because the var option isn't mentioned in the tutorial.
>
> And judging by the trouble I am having, I'm going to need a lot more
> help to get this working for me.
Hi Graham,
The Worg pages is the current documentation, but please just ask when
you have any problems -- include the string [babel] in the subject line
so people can filter it out.
One suggestion: are you looking at the HTML version of the tutorial? If
so, how about getting hold of the source org file, and working through
that? There you can see exactly ehat the syntax is and you can play
around with changing things and re-evaluating, etc. It should be a lot
clearer that way. If you use git, then just clone Worg; if not, I'm
attaching the current version of org-babel.org.
To get the latest version of the org document in the future, I can't
remember off-hand what the recommended way is to access the plain-text
versions of Worg files via a web browser (anyone?), but one reliable way
is to browse the webpages of the git repository at
http://repo.or.cz/w/Worg.git
Try clicking on the "tree" link near the top, then a few links later you
get to this URL which seems to produce the file in question
http://repo.or.cz/w/Worg.git/blob_plain/HEAD:/org-contrib/babel/org-babel.org
Dan
p.s. Please bear in mind that the name of that file is likely to change
to "org-babel-doc.org" in the nearish future.
[-- Attachment #2: org-babel-doc.org --]
[-- Type: text/plain, Size: 54343 bytes --]
#+OPTIONS: H:3 num:nil toc:2 \n:nil @:t ::t |:t ^:{} -:t f:t *:t TeX:t LaTeX:t skip:nil d:(HIDE) tags:not-in-toc
#+STARTUP: align fold nodlcheck hidestars oddeven lognotestate hideblocks
#+SEQ_TODO: TODO(t) INPROGRESS(i) WAITING(w@) | DONE(d) CANCELED(c@)
#+TAGS: Write(w) Update(u) Fix(f) Check(c) noexport(n)
#+TITLE: Org-babel
#+AUTHOR: Eric Schulte, Dan Davison
#+EMAIL: schulte.eric at gmail dot com, davison at stats dot ox dot ac dot uk
#+LANGUAGE: en
#+STYLE: <style type="text/css">#outline-container-introduction{ clear:both; }</style>
#+begin_html
<div id="subtitle" style="float: center; text-align: center;">
<p>executable source code blocks in org-mode</p>
</div>
<div id="logo" style="float: left; text-align: center; max-width: 340px; font-size: 8pt; margin-left: 1em;">
<p>
<img src="../../images/babel/tower-of-babel.png" alt="Tower of Babel"/>
<div id="attr" style="margin: -0.5em;">
The Tower of Babel by
<a href="http://commons.wikimedia.org/wiki/Pieter_Brueghel_the_Elder" title="">
<b>Pieter Brueghel the Elder</b>
</a>
</div>
<p>
And the Lord said, Behold, the people is one, and they have all
one language; and this they begin to do; and now nothing will be
restrained from them, which they have imagined to do. Genesis
11:1-9
</p>
</p>
</div>
#+end_html
* Improving this document :noexport:
** TODO Document slice indexing of tables
** TODO Document synonymous alternatives
{call,lob}, {source, function, srcname}, {results, resname}
** TODO Describe useful functions
- `org-babel-execute-buffer'
- `org-babel-execute-subtree'
** TODO Language support
Hopefully we will be breaking out a separate section for
each language, and expanding the portion which documents the actual
usage of header-arguments and noweb references as those sections are
woefully out of date.
** TODO Developments
- org-babel can now cache the results of source block execution to avoid
rerunning the same calculation. The cache uses a sha1 hash key of the
source code body and the header arguments to determine if
recalculation is required. These hash keys are kept mostly hidden in
the #+resname line of the results of the block. This behavior is
turned off by default. It is controlled through the :cache
and :nocache header arguments. To enable caching on a single block
add the :cache header argument, to enable global caching change the
value of your `org-babel-default-header-args' variable as follows
(setq org-babel-default-header-args
(cons '(:cache)
(assq-delete-all :nocache org-babel-default-header-args)))
- It is now possible to fold results by tabbing on the beginning of the
#+resname line. This can be done automatically to all results on
opening of a file by adding the following to your org-mode hook
(add-hook 'org-mode-hook 'org-babel-result-hide-all)
- allow header argument values to be lisp forms, for example the
following is now valid
:file (format "%s/images/pca-scatter.png" dir)
- aliases
- 'call' can now be used as an alias for 'lob'
- 'results' can now be used as an alias for 'resname'
- 'source' or 'function' can now be used as aliases for 'srcname'
** TODO Useful variables
- # -*- org-src-preserve-indentation: t -*-
** TODO Language specific header arguments
- org-babel: capture graphical output from R
If a [:file filename.ext] header arg is provided, then all graphical
output from the source block is captured on disk, and output of the
source block is a link to the resulting file, as with the
graphics-only languages such as gnuplot, ditaa, dot, asymptote. An
attempt is made to find a graphics device corresponding to the file
extension (currently .png, .jpg, .jpeg, .tiff, .bmp, .pdf, .ps,
.postscript are recognised); if that fails, png format output is
created.
Additionally, values for several arguments to the R graphics
device can be passed using header args:
:width :height :bg :units :pointsize
:antialias :quality :compression :res :type
:family :title :fonts :version :paper :encoding
:pagecentre :colormodel :useDingbats :horizontal
Arguments to the R graphics device that are not supported as header
args can be passed as a string in R argument syntax, using the header
arg :R-dev-args
An example block is (although both bg and fg can be passed directly as
header args)
\#+begin_src R :file z.pdf :width 8 :height 8 :R-dev-args bg="olivedrab", fg="hotpink"
plot(matrix(rnorm(100), ncol=2), type="l")
\#+end_src
- Yes, I think we do want a version of this for python and ruby et al. In
your example, the filename is created in python. I suggest doing it
slightly differently, something like this.
#+srcname: fileoutput
#+begin_src python :file outfile.txt
def savetofile(result, filename):
with open(filename, 'w') as f:
f.write(str(result))
savetofile(78, 'outfile.txt')
55
#+end_src
#+resname: fileoutput
[[file:outfile.txt]]
This functionality is now available for ruby & python in branch
ded-babel of git://repo.or.cz/org-mode/babel.git.
So here, if you specify :file <filepath> ruby/python blindly outputs a
link to <filepath>, regardless of the contents of the
code. Responsibility for creating useful contents of <filepath> lies
with the code. Notice that with this you have to specify the output file
twice: once as an org-babel directive, and once in the python code. This
is in contrast to the graphics languages (dot, ditaa, asymptote), where
the results *automatically* get sent to the file specified by :file. The
same is also true now for graphical output from R.
The difference with python, ruby et al is that they might create file
output in a variety of ways which we can't anticipate, so we can't
automatically send output to the file. In contrast, the graphics
language *always* create file output and always do it in the same
way. [And in R it is possible to divert all graphical output to file] A
possible extension of the above might be to use a "magic variable" so
that a python variable is created e.g. __org_babel_output_file__ that
always holds a string corresponding to the file specified by :file. Eric
may have further ideas / views here.
* Introduction
:PROPERTIES:
:CUSTOM_ID: introduction
:END:
Org-babel extends the very excellent [[http://orgmode.org/][Org-mode]] with the ability to
execute source code within Org-mode documents. Org-mode is an [[http://www.gnu.org/software/emacs/][Emacs]]
major mode for doing almost anything with plain text. If you are
not familiar with Org-mode please take a moment to read [[http://orgmode.org/][the Org-mode
homepage]] before continuing.
As its name implies, Org-babel will execute code in many different
languages. The results of code execution --- text, tables and
graphics --- can be used as input to other source code blocks or
integrated into the powerful publishing facilities of Org-mode.
Org-babel augments [[http://orgmode.org/manual/Literal-examples.html][Org-mode support for source code blocks]]
by providing:
1) interactive and on-export execution of source code;
2) code blocks as functions that can be parameterised, refer to
other code blocks, and be called remotely; and
3) export to files for literate programming.
* Getting Started
:PROPERTIES:
:CUSTOM_ID: getting-started
:results: silent
:END:
If you have a working Emacs installation, then getting started with
Org-babel is a simple five-step process.
1) It is strongly recommended that you update to the latest version
of Org-mode by [[file:../../org-faq.org::keeping-current-with-Org-mode-development][keeping current with Org-mode development]].
Org-babel is included in the contrib directory of Org-mode.
2) Make sure that the path to Org-mode's contrib/lisp directory is in
your load-path and add the following Emacs Lisp expression to your .emacs.
#+begin_src emacs-lisp
(require 'org-babel-init)
#+end_src
3) Activate the subset of [[#reference-and-documentation][supported Org-babel languages]] that you
will be using by adding Emacs Lisp expressions to your
.emacs. As an example, the following three Emacs Lisp
expressions activate R, Ruby and Python. Shell script and
Emacs-lisp are enabled by default.
#+begin_src emacs-lisp
(require 'org-babel-R) ;; requires R and ess-mode
(require 'org-babel-ruby) ;; requires ruby, irb, ruby-mode, and inf-ruby
(require 'org-babel-python) ;; requires python, and python-mode
#+end_src
4) Load the Library of Babel by adding the following Emacs Lisp
expression to your .emacs. The Library of Babel will make
pre-built helper functions available in the languages you will be
using.
#+begin_src emacs-lisp
(org-babel-load-library-of-babel)
#+end_src
5) Evaluate your modified .emacs.
* Basic Org-babel Functionality
:PROPERTIES:
:CUSTOM_ID: basic-functionality
:END:
*** Source Code Blocks
:PROPERTIES:
:CUSTOM_ID: source-code-blocks
:END:
Org-babel is all about [[http://orgmode.org/manual/Literal-examples.html][source code blocks]] in Org-mode. If you are
unfamiliar with the notion of a source code block in Org-mode,
please have a look at the [[http://orgmode.org/manual/Literal-examples.html][Org-mode manual]] before proceeding.
Source code blocks in [[#reference-and-documentation][supported languages]] can occur anywhere in an
Org-mode file. Source code blocks can be entered directly into
the Org-mode file, but it is often easier to enter code with the
function =org-edit-src-code=, which is called with the keyboard
shortcut, C-c '. This places the source code block in a new
buffer with the appropriate mode activated.
For example, a source code block of [[http://www.ruby-lang.org/][ruby]] code looks like this in
an Org-mode file:
: #+begin_src ruby
: require 'date'
: "This file was last evaluated on #{Date.today}"
: #+end_src
We had to take [[http://orgmode.org/manual/Literal-examples.html#Literal-examples][special steps]] to make it look that way in the HTML
output. Normally, when exported to HTML, source blocks are
fontified according to their language, and the =begin_src=
... =end_src= mark-up is omitted, like this:
#+begin_src ruby
require 'date'
"This file was last evaluated on #{Date.today}"
#+end_src
#+resname:
: This file was last evaluated on 2009-10-03
In general, if you are viewing the HTML version, you will see the
HTML output only. However, much of this document consists of
interactive examples, and therefore in order to get a feeling for the
mechanics of Org-babel it might make most sense to grab the Org-mode
version of this file
#+html: <a href="http://eschulte.github.com/org-babel/org-babel.org">org-babel.org</a>
and work through it in Emacs. Alternatively the HTMLized version of
the plain text of this file at
#+html: <a href="http://eschulte.github.com/org-babel/org-babel.org.html">org-babel.org.html</a>
allows the plain text version to be viewed (non-interactively) in a
web browser.
If you don't have ruby installed on your system, then you can run this
shell script alternative from the Org-mode version of the file:
#+begin_src sh
echo "This file was last evaluated on `date`"
#+end_src
*** Source Code Execution
:PROPERTIES:
:CUSTOM_ID: source-code-execution
:END:
Org-babel executes source code blocks for *interpreted* languages such
as shell, python, R, etc. by passing code to the interpreter, which
must be installed on your system. You control what is done with the
results of execution.
Here are examples of code blocks in three different languages,
followed by their output. If you are viewing the Org-mode version of
this document in Emacs, place point anywhere inside a block and press
C-c C-c to run the code[fn:1] (and feel free to alter it!).
**** Ruby
In the Org-mode file:
: #+begin_src ruby
: "This file was last evaluated on #{Date.today}"
: #+end_src
HTML export of code:
#+begin_src ruby
"This file was last evaluated on #{Date.today}"
#+end_src
HTML export of the resulting string:
#+resname:
: This file was last evaluated on 2009-08-09
**** Shell
In the Org-mode file:
: #+begin_src sh
: echo "This file takes up `du -h org-babel.org |sed 's/\([0-9k]*\)[ ]*org-babel.org/\1/'`"
: #+end_src
HTML export of code:
#+begin_src sh
echo "This file takes up `du -h org-babel.org |sed 's/\([0-9k]*\)[ ]*org-babel.org/\1/'`"
#+end_src
HTML export of the resulting string:
#+resname:
: This file takes up 36K
**** [[http://www.r-project.org/][R]]
In the Org-mode file:
: #+begin_src R :colnames t
: t(sort(table(tolower(scan("org-babel.org", what="", na.strings="|"))), decreasing=TRUE)[1:10])
: #+end_src
HTML export of code:
#+begin_src R :colnames t
t(sort(table(tolower(scan("org-babel.org", what="", na.strings="|"))), decreasing=TRUE)[1:10])
#+end_src
HTML export of the resulting table:
#+resname:
| "the" | "of" | "to" | "in" | "a" | "is" | "and" | "code" | ":" | "be" |
|-------+------+------+------+-----+------+-------+--------+-----+------|
| 303 | 130 | 100 | 94 | 90 | 84 | 75 | 69 | 60 | 51 |
**** [[http://ditaa.sourceforge.net/][ditaa]]
In the Org-mode file:
: #+begin_src ditaa :file blue.png :cmdline -r
: +---------+
: | cBLU |
: | |
: | +----+
: | |cPNK|
: | | |
: +----+----+
: #+end_src
HTML export of code:
#+begin_src ditaa :file blue.png :cmdline -r
+---------+
| cBLU |
| |
| +----+
| |cPNK|
| | |
+----+----+
#+end_src
HTML export of the resulting image:
#+resname:
[[file:../../images/babel/blue.png]]
*** Source Code Block Syntax
The basic syntax of source code blocks in Org-babel is as follows:
: #+srcname: name(arguments)
: #+begin_src language header-arguments
: body
: #+end_src
- name :: This name is associated with the source code block. This is
similar to the =#+tblname= lines which can be used to name tables
in Org-mode files. By referencing the srcname of a source-code
block it is possible to evaluate the block from other places in
the file, from other files, or from inside Org-mode tables.
- arguments :: Source code blocks can have [[#arguments-to-source-code-blocks][arguments]] using a familiar
function-call syntax similar to languages such as python or R.
- language :: The language of the code in the source-code block. Valid
values must be members of =org-babel-interpreters=.
- header-arguments :: Header arguments control many facets of the
evaluation and output of source-code blocks. See the [[* Header Arguments][Header
Arguments]] section for a complete review of available header
arguments.
- body :: The source code to be evaluated. An important key-binding
is C-c '. This calls =org-edit-special=, a function that
brings up an edit buffer containing the code using the Emacs
major mode appropriate to the language. You can edit your
source code block as you regularly would in Emacs.
*** Capturing the Results of Code Evaluation
:PROPERTIES:
:CUSTOM_ID: results
:END:
Org-babel provides two fundamentally different modes for capturing
the results of code evaluation: functional mode and scripting
mode. The choice of mode is specified by the =:results= header
argument.
**** =:results value= (functional mode)
The 'result' of code evaluation is the *value* of the last
statement in the source code block. In functional mode, the
source code block is a function with a return value. The return
value of one source code block can be used as input for another
source code block, even one in a different language. In this
way, Org-babel becomes a [[meta-programming-language]]. This setting
is the default.
For example, consider the following block of python code and its
output.
#+begin_src python :results value
import time
print("Hello, today's date is %s" % time.ctime())
print('Two plus two is')
return 2 + 2
#+end_src
#+resname:
: 4
Notice that, in functional mode, the output consists of the value of
the last statement and nothing else.
**** =:results output= (scripting mode)
In scripting mode, Org-babel captures the text output of the
source code block and places it in the Org-mode buffer. It is
called scripting mode because the code block contains a series of
commands, and the output of each command is returned. Unlike
functional mode, the source code block itself has no return value
apart from the output of the commands it contains. (This mode
will be familiar to Sweave users).
Consider the result of evaluating this source code block with scripting mode.
#+srcname: name
#+begin_src python :results output
import time
print("Hello, today's date is %s" % time.ctime())
print('Two plus two is')
2 + 2
#+end_src
#+resname: name
: Hello, today's date is Wed Nov 11 18:50:36 2009
: Two plus two is
Here, scripting mode returned the text that python sent to stdout. Because
the source code block doesn't include a =print()= statement for the last
value (2 + 2), 4 does not appear in the results.
*** Session-based Evaluation
For some languages, such as python, R, ruby and shell, it is
possible to run an interactive session as an "inferior process"
within Emacs. This means that an environment is created containing
data objects that persist between different source code
blocks. Org-babel supports evaluation of code within such sessions
with the =:session= header argument. If the header argument is
given a value then that will be used as the name of the session.
Thus, it is possible to run separate simultaneous sessions in the
same language.
With R, the session will be under the control of [[http://ess.r-project.org/][Emacs Speaks
Statistics]] as usual, and the full power of ESS is thus still
available, both in the R session, and when switching to the R code
edit buffer with C-c '.
*** Arguments to Source Code Blocks
:PROPERTIES:
:CUSTOM_ID: arguments-to-source-code-blocks
:END:
Org-babel supports parameterisation of source code blocks, i.e.,
arguments can be passed to source code blocks, which gives them
the status of *functions*. Arguments can be passed to source code blocks in
both functional and scripting modes.
**** Simple example of using a source block as a function
First let's look at a very simple example. The following source
code block defines a function, using python, that squares its argument.
#+srcname: square(x)
#+begin_src python
x*x
#+end_src
In the Org-mode file, the function looks like this:
: #+srcname: square(x)
: #+begin_src python
: x*x
: #+end_src
Now we use the source block:
: #+lob: square(x=6)
(/for information on the/ =lob= /syntax see/ [[library-of-babel]])
#+lob: square(x=6)
#+resname: square(x=6)
: 36
**** A more complex example using an Org-mode table as input
In this example we define a function called =fibonacci-seq=, using
Emacs Lisp. The function =fibonacci-seq= computes a Fibonacci
sequence. The function takes a single argument, in this case, a
reference to an Org-mode table.
Here is the Org-mode table that is passed to =fibonacci-seq=:
#+tblname: fibonacci-inputs
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
| 2 | 4 | 6 | 8 | 10 | 12 | 14 | 16 | 18 | 20 |
The table looks like this in the Org-mode buffer:
: #+tblname: fibonacci-inputs
: | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
: | 2 | 4 | 6 | 8 | 10 | 12 | 14 | 16 | 18 | 20 |
The [[http://www.gnu.org/software/emacs/manual/elisp.html][Emacs Lisp]] source code:
#+srcname: fibonacci-seq(fib-inputs=fibonacci-inputs)
#+begin_src emacs-lisp
(defun fibonacci (n)
(if (or (= n 0) (= n 1))
n
(+ (fibonacci (- n 1)) (fibonacci (- n 2)))))
(mapcar (lambda (row)
(mapcar #'fibonacci row)) fib-inputs)
#+end_src
In the Org-mode buffer the function looks like this:
: #+srcname: fibonacci-seq(fib-inputs=fibonacci-inputs)
: #+begin_src emacs-lisp
: (defun fibonacci (n)
: (if (or (= n 0) (= n 1))
: n
: (+ (fibonacci (- n 1)) (fibonacci (- n 2)))))
:
: (mapcar (lambda (row)
: (mapcar #'fibonacci row)) fib-inputs)
: #+end_src
The return value of =fibonacci-seq= is a table:
#+resname:
| 1 | 1 | 2 | 3 | 5 | 8 | 13 | 21 | 34 | 55 |
| 1 | 3 | 8 | 21 | 55 | 144 | 377 | 987 | 2584 | 6765 |
*** In-line Source Blocks
Code can be evaluated in-line using the following syntax:
: Without header args: src_lang{code} or with header args: src_lang[args]{code},
: for example src_python[:session]{10*x}, where x is a variable existing in the
: python session.
* A Meta-programming Language for Org-mode
:PROPERTIES:
:CUSTOM_ID: meta-programming-language
:END:
Because the return value of a function written in one language can be
passed to a function written in another language, or to an Org-mode
table, which is itself programmable, Org-babel can be used as a
meta-functional programming language. With Org-babel, functions from
many languages can work together. You can mix and match languages,
using each language for the tasks to which it is best suited.
For example, let's take some system diagnostics in the shell and graph them with R.
1. Create a source code block, using shell code, to list
directories in our home directory, together with their
sizes. Org-babel automatically converts the output into an Org-mode
table.
#+srcname: directories
#+begin_src sh :results replace
cd ~ && du -sc * |grep -v total
#+end_src
#+resname: directories
| 72 | "Desktop" |
| 12156104 | "Documents" |
| 3482440 | "Downloads" |
| 2901720 | "Library" |
| 57344 | "Movies" |
| 16548024 | "Music" |
| 120 | "News" |
| 7649472 | "Pictures" |
| 0 | "Public" |
| 152224 | "Sites" |
| 8 | "System" |
| 56 | "bin" |
| 3821872 | "mail" |
| 10605392 | "src" |
| 1264 | "tools" |
2. A function, written with a single line of R code, plots the data
in the Org-mode table as a
pie-chart. Note how this source code block uses the =srcname=
of the previous source code block to obtain the data.
#+srcname: directory-pie-chart(dirs = directories)
#+begin_src R :session R-pie-example :file ../../images/babel/dirs.png
pie(dirs[,1], labels = dirs[,2])
#+end_src
[[file:../../images/babel/dirs.png]]
* Multilingual Spreadsheet Plugins for Org-mode
:PROPERTIES:
:CUSTOM_ID: spreadsheet
:END:
Not only can Org-babel pass entire tables of data as [[arguments-to-source-code-blocks][arguments to
source code blocks]], Org-babel can also be used to call source code
blocks from *within* Org-mode tables using Org-mode's [[http://orgmode.org/manual/The-spreadsheet.html#The-spreadsheet][existing spreadsheet
functionality]].
*** Example 1: Data Summaries Using R
As a simple example, we'll fill in a cell in an Org-mode table with the
average value of a few numbers. First, let's make some data. The
following source block creates an Org-mode table filled with five random
numbers between 0 and 1.
#+srcname: tbl-example-data()
#+begin_src R
runif(n=5, min=0, max=1)
#+end_src
#+resname: tbl-example-data
| 0.836685163900256 |
| 0.696652316721156 |
| 0.382423302158713 |
| 0.987541858805344 |
| 0.994794291909784 |
Now we define a source block to calculate the mean.
#+srcname: R-mean(x)
#+begin_src R
mean(x)
#+end_src
Finally, we create the table which is going to make use of the R
code. This is done using the =sbe= ('source block evaluate') macro in
the table formula line.
#+tblname: summaries
| mean |
|-------------------|
| 0.779619386699051 |
#+TBLFM: @2$1='(sbe "R-mean" (x "tbl-example-data()"))
To recalculate the table formula, use C-u C-c C-c in the
table. Notice that as things stand the calculated value doesn't
change, because the data (held in the table above named
=tbl-example-data=) are static. However, if you delete that data table,
then the reference will be interpreted as a reference to the source
block responsible for generating the data; each time the table formula
is recalculated the source block will be evaluated again, and
therefore the calculated average value will change.
*** Example 2: Org-babel Test Suite
While developing Org-babel, we used a suite of tests implemented
as a large Org-mode table. To run the entire test suite we simply
evaluate the table with C-u C-c C-c: all of the tests are run,
the results are compared with expectations, and the table is updated
with results and pass/fail statistics.
Here's a sample of our test suite.
#+TBLNAME: org-babel-tests
| functionality | block | arg | expected | results | pass |
|------------------+--------------+-----+-------------+-------------+------|
| basic evaluation | | | | | pass |
|------------------+--------------+-----+-------------+-------------+------|
| emacs lisp | basic-elisp | 2 | 4 | 4 | pass |
| shell | basic-shell | | 6 | 6 | pass |
| ruby | basic-ruby | | org-babel | org-babel | pass |
| python | basic-python | | hello world | hello world | pass |
| R | basic-R | | 13 | 13 | pass |
#+TBLFM: $5='(if (= (length $3) 1) (sbe $2 (n $3)) (sbe $2)) :: $6='(if (string= $4 $5) "pass" (format "expected %S but was %S" $4 $5))
**** code blocks for tests
#+srcname: basic-elisp(n)
#+begin_src emacs-lisp
(* 2 n)
#+end_src
#+srcname: basic-shell
#+begin_src sh :results silent
expr 1 + 5
#+end_src
#+srcname: date-simple
#+begin_src sh :results silent
date
#+end_src
#+srcname: basic-ruby
#+begin_src ruby :results silent
"org-babel"
#+end_src
#+srcname: basic-python
#+begin_src python :results silent
'hello world'
#+end_src
#+srcname: basic-R
#+begin_src R :results silent
b <- 9
b + 4
#+end_src
* The Library of Babel
:PROPERTIES:
:CUSTOM_ID: library-of-babel
:END:
As we saw above with the [[*Simple%20example%20of%20using%20a%20source%20block%20as%20a%20function][=square=]] example, once a source block
function has been defined it can be called using the =lob= notation:
: #+lob: square(x=6)
But what about source code blocks that you want to make available to
every Org-mode buffer?
In addition to the current buffer, Org-babel searches for
pre-defined source code block functions in the Library of
Babel. This is a user-extensible collection of ready-made source
code blocks for handling common tasks. One use for the Library of
Babel (not yet done!) will be to provide a choice of data graphing
procedures for data held in Org-mode tables, using languages such as
R, gnuplot, asymptote, etc. If you implement something that might be
of use to other Org-mode users, please consider adding it to the
Library of Babel; similarly, feel free to request help solving a
problem using external code via Org-babel -- there's always a chance
that other Org-bable users will be able to contribute some helpful
code.
Org-babel comes pre-populated with the source code blocks located in
the [[file:library-of-babel.org][Library of Babel]] file -- raw file at
#+html: <a href="http://eschulte.github.com/org-babel/library-of-babel.org">library-of-babel.org</a>
--. It is possible to add a source code block to the library
from any Org-mode file by naming it =add-file-to-lob=:
#+srcname: add-file-to-lob
#+begin_src emacs-lisp
(org-babel-lob-ingest "path/to/file.org")
#+end_src
Note that it is possible to pass table values or the output of
a source-code block to Library of Babel functions. It is also possible to
reference Library of Babel functions in arguments to source code blocks.
* Reproducible Research
:PROPERTIES:
:CUSTOM_ID: reproducable-research
:END:
#+begin_quote
An article about computational science in a scientific publication is
not the scholarship itself, it is merely advertising of the
scholarship. The actual scholarship is the complete software
development environment and the complete set of instructions which
generated the figures.
-- D. Donoho
#+end_quote
[[http://reproducibleresearch.net/index.php/Main_Page][Reproducible Research]] (RR) is the practice of distributing, along with
a research publication, all data, software source code, and tools
required to reproduce the results discussed in the publication. As
such the RR package not only describes the research and its results,
but becomes a complete laboratory in which the research can be
reproduced and extended.
Org-mode already has exceptional support for [[http://orgmode.org/manual/Exporting.html#Exporting][exporting to HTML and
LaTeX]]. Org-babel makes Org-mode a tool for RR by *activating* the
data and source code embedded in Org-mode documents; the
entire document becomes executable. This makes it possible, and natural, to
distribute research in a format that encourages readers to recreate
results and perform their own analyses.
One notable existing RR tool is [[http://en.wikipedia.org/wiki/Sweave][Sweave]], which provides a mechanism for
embedding [[http://www.r-project.org/][R]] code into LaTeX documents. Sweave is a mature
and very useful tool, but we believe that Org-babel has several
advantages:
- it supports multiple languages (we're not aware of other RR tools that do this);
- the [[http://orgmode.org/manual/Exporting.html#Exporting][export process]] is flexible and powerful, including HTML as a
target in addition to LaTeX; and
- the document can make use of Org-mode features that support [[http://orgmode.org/manual/Agenda-Views.html#Agenda-Views][project
planning]] and [[http://orgmode.org/manual/TODO-Items.html#TODO-Items][task management]].
* Literate Programming
:PROPERTIES:
:CUSTOM_ID: literate-programming
:END:
#+begin_quote
Let us change our traditional attitude to the construction of
programs: Instead of imagining that our main task is to instruct a
/computer/ what to do, let us concentrate rather on explaining to
/human beings/ what we want a computer to do.
The practitioner of literate programming can be regarded as an
essayist, whose main concern is with exposition and excellence of
style. Such an author, with thesaurus in hand, chooses the names of
variables carefully and explains what each variable means. He or she
strives for a program that is comprehensible because its concepts have
been introduced in an order that is best for human understanding,
using a mixture of formal and informal methods that reinforce each
other.
-- Donald Knuth
#+end_quote
Org-babel supports [[http://en.wikipedia.org/wiki/Literate_programming][Literate Programming]] (LP) by allowing the act of
programming to take place inside of Org-mode documents. The Org-mode
file can then be exported (*woven* in LP speak) to HTML or LaTeX for
consumption by a human, and the embedded source code can be extracted
(*tangled* in LP speak) into structured source code files for
consumption by a computer.
To support these operations Org-babel relies on Org-mode's [[http://orgmode.org/manual/Exporting.html#Exporting][existing
exporting functionality]] for *weaving* of documentation, and on the
=org-babel-tangle= function which makes use of [[http://www.cs.tufts.edu/~nr/noweb/][Noweb]] [[noweb-reference-syntax][reference syntax]]
for *tangling* of code files.
The [[literate-programming-example][following example]] demonstrates the process of *tangling* in
Org-babel.
*** Simple Literate Programming Example (Noweb syntax)
:PROPERTIES:
:CUSTOM_ID: literate-programming-example
:END:
Tangling functionality is controlled by the =tangle= family of
[[header-arguments]]. These arguments can be used to turn tangling on or
off (the default), either for the source code block or the Org-mode
heading level.
The following source code blocks demonstrate how to tangle them into a
single source code file using =org-babel-tangle=.
The following two source code blocks have no =tangle= header arguments
and so will not, by themselves, create source code files. They are
included in the source code file by the third source code block, which
does have a =tangle= header argument.
#+srcname: hello-world-prefix
#+begin_src sh :exports none
echo "/-----------------------------------------------------------\\"
#+end_src
: #+srcname: hello-world-prefix
: #+begin_src sh :exports none
: echo "/-----------------------------------------------------------\\"
: #+end_src
#+srcname: hello-world-postfix
#+begin_src sh :exports none
echo "\-----------------------------------------------------------/"
#+end_src
: #+srcname: hello-world-postfix
: #+begin_src sh :exports none
: echo "\-----------------------------------------------------------/"
: #+end_src
The third source code block does have a =tangle= header argument
indicating the name of the file to which the tangled source code will
be written. It also has [[http://www.cs.tufts.edu/~nr/noweb/][Noweb]] style references to the two previous
source code blocks. These references will be expanded during tangling
to include them in the output file as well.
#+srcname: hello-world
#+begin_src sh :tangle hello :exports none
# <<hello-world-prefix>>
echo "| hello world |"
# <<hello-world-postfix>>
#+end_src
: #+srcname: hello-world
: #+begin_src sh :tangle hello :exports none
: # <<hello-world-prefix>>
: echo "| hello world |"
: # <<hello-world-postfix>>
: #+end_src
Calling =org-babel-tangle= will result in the following shell source
code being written to the hello.sh file:
#+srcname: hello-world-output
#+begin_src sh
#!/usr/bin/env sh
# generated by org-babel-tangle
# [[file:~/src/org-babel/org-babel-worg.org::#literate-programming-example][block-16]]
# <<hello-world-prefix>>
echo "/-----------------------------------------------------------\\"
echo "| hello world |"
# <<hello-world-postfix>>
echo "\-----------------------------------------------------------/"
# block-16 ends here
#+end_src
In addition, the following syntax can be used to insert the *results*
of evaluating a source code block, in this case one named =example-block=.
: # <<example-block()>>
Any optional arguments can be passed to =example-block()= by placing the
arguments inside the parentheses following the convention defined when
calling source block functions (see the [[library-of-babel][Library of babel]]). For example,
: # <<example-block(a=9)>>
sets the value of argument \"a\" equal to \"9\". Note that
these arguments are not evaluated in the current source-code
block but are passed literally to =example-block()=.
#+end_src
*** Emacs Initialization with Org-babel
:PROPERTIES:
:CUSTOM_ID: literate-emacs-initialization
:END:
#+attr_html: style="float:left;"
[[file:../../images/babel/dot-emacs.png]]
Org-babel has special support for embedding your Emacs initialization
into Org-mode files. The =org-babel-load-file= function can be used
to load the Emacs Lisp source code blocks embedded in a literate
Org-mode file in the same way that you might load a regular Emacs Lisp
file, such as .emacs.
This allows you to make use of the nice features of Org-mode, such as folding, tags,
notes, HTML export, etc., to organize and maintain your Emacs initialization.
To try this out, either see the simple [[literate-emacs-init][Literate Emacs Initialization]]
example, or check out the Org-babel Literate
Programming version of Phil Hagelberg's excellent [[http://github.com/technomancy/emacs-starter-kit/tree/master][emacs-starter-kit]]
available at [[http://github.com/eschulte/emacs-starter-kit/tree/master][Org-babel-emacs-starter-kit]].
***** Literate Emacs Initialization
:PROPERTIES:
:CUSTOM_ID: literate-emacs-init
:END:
For a simple example of usage, follow these 5 steps:
1) create a directory named =.emacs.d= in the base of your home
directory;
#+begin_src sh
mkdir ~/.emacs.d
#+end_src
2) checkout the latest version of Org-mode into the src subdirectory
of this new directory;
#+begin_src sh
cd ~/.emacs.d
mkdir src
cd src
git clone git://repo.or.cz/org-mode.git
#+end_src
3) place the following source code block in a file called =init.el= in your Emacs
initialization directory (=~/.emacs.d=).
#+srcname: emacs-init
#+begin_src emacs-lisp
;;; init.el --- Where all the magic begins
;;
;; This file loads both
;; - Org-mode : http://orgmode.org/ and
;; - Org-babel: http://orgmode.org/worg/org-contrib/babel/org-babel.php#library-of-babel
;;
;; It then loads the rest of our Emacs initialization from Emacs lisp
;; embedded in literate Org-mode files.
;; Load up Org Mode and Org Babel for elisp embedded in Org Mode files
(setq dotfiles-dir (file-name-directory (or (buffer-file-name) load-file-name)))
(let* ((org-dir (expand-file-name
"lisp" (expand-file-name
"org" (expand-file-name
"src" dotfiles-dir))))
(org-contrib-dir (expand-file-name
"lisp" (expand-file-name
"contrib" (expand-file-name
".." org-dir))))
(load-path (append (list org-dir org-contrib-dir)
(or load-path nil))))
;; load up Org-mode and Org-babel
(require 'org-install)
(require 'org-babel-init))
;; load up all literate org-mode files in this directory
(mapc #'org-babel-load-file (directory-files dotfiles-dir t "\\.org$"))
;;; init.el ends here
#+end_src
4) implement all of your Emacs customizations inside of Emacs Lisp
source code blocks embedded in Org-mode files in this directory;
and
5) re-start Emacs to load the customizations.
* Reference / Documentation
:PROPERTIES:
:CUSTOM_ID: reference-and-documentation
:END:
*** Languages
:PROPERTIES:
:CUSTOM_ID: languages
:END:
The following table lists the languages currently supported by
Org-babel. There is a [[file:org-babel-template.org][template]] to create Emacs Lisp support files for other languages.
| Language | Support file | Org-babel identifier | Requirements |
|----------------+-------------------------+----------------------+---------------------------------------------|
| R | org-babel-R.el | R | [[http://www.r-project.org/][R]], [[http://ess.r-project.org/][ess-mode]] |
| Asymptote | org-babel-asymptote.el | asymptote | [[http://asymptote.sourceforge.net/][asymptote]], [[http://asymptote.sourceforge.net/doc/Editing-modes.html][asy-mode]] |
| Clojure | org-babel-clojure.el | clojure | [[http://clojure.org/][clojure]], [[http://www.emacswiki.org/emacs/clojure-mode.el][clojure-mode]], [[http://common-lisp.net/project/slime/][slime]], [[http://clojure.codestuffs.com/][swank-clojure]] |
| css | org-babel-css.el | css | none |
| ditaa | org-babel-ditaa.el | ditaa | [[http://ditaa.org/ditaa/][ditaa]] (bundled with Org-mode) |
| Graphviz | org-babel-dot.el | dot | [[http://www.graphviz.org/][dot]] |
| Emacs Lisp | org-babel-emacs-lisp.el | emacs-lisp | none |
| gnuplot | org-babel-gnuplot.el | gnuplot | [[http://www.gnuplot.info/][gnuplot]], [[http://cars9.uchicago.edu/~ravel/software/gnuplot-mode.html][gnuplot-mode]] |
| Haskell | org-babel-haskell.el | haskell | [[http://www.haskell.org/][haskell]], [[http://projects.haskell.org/haskellmode-emacs/][haskell-mode]], [[http://www.haskell.org/haskellwiki/Haskell_mode_for_Emacs#inf-haskell.el:_the_best_thing_since_the_breadknife][inf-haskell]], [[http://people.cs.uu.nl/andres/lhs2tex/][lhs2tex]] |
| LaTeX | org-babel-latex.el | latex | [[http://www.latex-project.org/][latex]], [[http://www.gnu.org/software/auctex/][auctex]], [[http://www.gnu.org/software/auctex/reftex.html][reftex]] |
| Objective Caml | org-babel-ocaml.el | ocaml | [[http://caml.inria.fr/][ocaml]], [[http://www-rocq.inria.fr/~acohen/tuareg/][tuareg-mode]] |
| Perl | org-babel-perl.el | perl | [[http://www.perl.org/][perl]], [[http://www.emacswiki.org/emacs/CPerlMode][cperl-mode]] (optional) |
| Python | org-babel-python.el | python | [[http://www.python.org/][python]], [[https://launchpad.net/python-mode][python-mode]] (optional) |
| Ruby | org-babel-ruby.el | ruby | [[http://www.ruby-lang.org/][ruby]], [[http://www.ruby-lang.org/][irb]], [[http://github.com/eschulte/rinari/raw/master/util/ruby-mode.el][ruby-mode]], [[http://github.com/eschulte/rinari/raw/master/util/inf-ruby.el][inf-ruby mode]] |
| Sass | org-babel-sass.el | sass | [[http://sass-lang.com/][sass]], [[http://github.com/nex3/haml/blob/master/extra/sass-mode.el][sass-mode]] |
| GNU Screen | org-babel-screen.el | screen | [[http://www.gnu.org/software/screen/][screen]], a terminal |
| shell | org-babel-sh.el | sh | a shell |
| SQL | org-babel-sql.el | sql | none |
The following Emacs Lisp code block can be added to your .emacs
and used to activate languages by uncommenting the appropriate
lines.
#+begin_src emacs-lisp
;; Uncomment each of the following require lines if you want Org-babel
;; to support that language. Each language has a comment explaining
;; its dependencies. See the related files in lisp/langs for more
;; detailed explanations of requirements.
;; (require 'org-babel-R) ;; R and ess-mode
;; (require 'org-babel-asymptote) ;; asymptote
;; (require 'org-babel-css) ;; none
;; (require 'org-babel-ditaa) ;; ditaa
;; (require 'org-babel-dot) ;; dot
;; (require 'org-babel-gnuplot) ;; gnuplot, and gnuplot-mode
;; (require 'org-babel-haskell) ;; haskell, haskell-mode, inf-haskell
;; (require 'org-babel-ocaml) ;; ocaml, and tuareg-mode
;; (require 'org-babel-clojure) ;; clojure, clojure-mode, slime, swank-clojure
;; (require 'org-babel-python) ;; python, and python-mode
;; (require 'org-babel-perl) ;; perl
;; (require 'org-babel-ruby) ;; ruby, irb, ruby-mode, and inf-ruby
;; (require 'org-babel-sass) ;; sass, sass-mode
;; (require 'org-babel-screen) ;; screen and a terminal
;; (require 'org-babel-sql) ;; none
#+end_src
**** Language specific documentation
Examples, special header arguments, and guides on how to use them
are available for the following languages:
- [[file:org-babel-doc-R.org][R]]
- [[file:org-babel-doc-screen.org][screen]]
- [[file:org-babel-doc-C.org][C and Make]]
*** Header Arguments
:PROPERTIES:
:CUSTOM_ID: header-arguments
:END:
**** =:results=
There are three types of results header argument:
1) *collection* header arguments specify how the results should be collected from
the source code block;
2) *type* header arguments specify what type of result the source code block
will return -- which has implications for how they will be
inserted into the Org-mode buffer; and
3) *handling* header arguments specify how the results of
evaluating the source code block should be handled.
Only one option from each type may be supplied per source code
block.
***** collection
The following options are mutually exclusive, and specify how the
results should be collected from the source-code block.
- value :: The result is the value of the last statement in the
source code block. This header argument places Org-babel in
functional mode. Note that in some languages, e.g., python, use of this
result type requires that a =return= statement be
included in the body of the source code block. E.g.,
=:results value=.
- output :: The result is the collection of everything printed
to stdout during the execution of the source code
block. This header argument places Org-babel in scripting
mode. E.g., =:results output=.
***** type
The following options are mutually exclusive and specify what
type of results the code block will return.
- table, vector :: The results should be interpreted as an Org-mode table.
If a single value is returned, Org-babel will convert it
into a table with one row and one column. E.g., =:results
value table=.
- scalar, verbatim :: The results should be interpreted
literally -- meaning they will not be converted into a table.
The results will be inserted into the Org-mode buffer as
quoted text. E.g., =:results value verbatim=.
- file :: The results will be interpreted as the path to a file,
and will be inserted into the Org-mode buffer as a file
link. E.g., =:results value file=.
- raw, org :: The results are interpreted as raw Org-mode code and
are inserted directly into the buffer. If the results look
like a table they will be aligned as such by Org-mode.
E.g., =:results value raw=.
- html :: Results are assumed to be HTML and will be enclosed in
a =begin_html= block. E.g., =:results value html=.
- latex :: Results assumed to be LaTeX and are enclosed in a
=begin_latex= block. E.g., =:results value latex=.
- code :: Result are assumed to be parseable code and are
enclosed in a code block. E.g., =:results value code=.
- pp :: The result is converted to pretty-printed code and is
enclosed in a code block. This option currently supports
Emacs Lisp, python, and ruby. E.g., =:results value pp=.
***** handling
The following results options indicate what Org-babel should do
with the results once they are collected.
- silent :: The results will be echoed in the minibuffer but
will not be inserted into the Org-mode buffer. E.g.,
=:results output silent=.
- replace :: The results will be inserted into the Org-mode
buffer. E.g., =:results output replace=.
**** =:exports=
Specify what should be included in HTML or LaTeX exports of the
Org-mode file.
- code :: The body of code is included into the exported file.
E.g., =:exports code=.
- results :: The result of evaluating the code is included in the
exported file. E.g., =:exports results=.
- both :: Both the code and results are included in the exported
file. E.g., =:exports both=.
- none :: Nothing is included in the exported file. E.g.,
=:exports none=.
**** =:tangle=
Specify whether or not the source-code block should be included
in tangled extraction of source-code files.
- yes :: The source-code block is exported to a source-code file
named after the basename (name w/o extension) of the
Org-mode file. E.g., =:tangle yes=.
- no (default) :: The source-code block is not exported to a
source-code file. E.g., =:tangle no=.
- other :: Any other string passed to the =:tangle= header argument
is interpreted as a file basename to which the block will
be exported. E.g., =:tangle basename.ext=.
**** =:session=
Start a session for an interpreted language where state is
preserved. This applies particularly to the supported languages
perl, python, R and ruby.
- other :: A string passed to the =:session= header argument will
give the session a name. This makes it possible to
have multiple sessions for each interpreted language.
Results are handled somewhat differently if a session is invoked.
| | non-session (default) | =:session= |
|----------+--------------------------+-------------------------------------|
| =value= | value of last expression | value of last expression |
| =output= | contents of stdout | concatenation of interpreter output |
Note that in =:results value= the result in both sessions and
non-sessions is imported into Org-mode as a table (a one- or
two-dimensional vector of strings or numbers) when appropriate.
***** Non-session
****** =:results value=
This is the default. Internally, the value is obtained by
wrapping the code in a function definition in the external
language, and evaluating that function. Therefore, code should be
written as if it were the body of such a function. In particular,
note that python does not automatically return a value from a
function unless a =return= statement is present, and so a
'return' statement will usually be required in python :results
value (non-session).
This is the only one of the four evaluation contexts in which the
code is automatically wrapped in a function definition.
****** =:results output=
The code is passed to the interpreter as an external process, and
the contents of the standard output stream is returned as
text. (In certain languages this also contains the error output
stream; this is an area for future work.)
***** =:session=
****** =:results value=
The code is passed to the interpreter running as an interactive
Emacs inferior process. The result returned is the result of the
last evaluation performed by the interpreter. (This is obtained in
a language-specific manner: the value of the variable =_= in
python and ruby, and the value of =.Last.value= in R).
****** =:results output=
The code is passed to the interpreter running as an interactive
Emacs inferior process. The result returned is the concatenation
of the sequence of (text) output from the interactive
interpreter. Notice that this is not necessarily the same as what
would be sent to stdout if the same code were passed to a
non-interactive interpreter running as an external process. For
example, compare the following two blocks:
#+begin_src python :results output
print "hello"
2
print "bye"
#+end_src
#+resname:
: hello
: bye
In non-session mode, the '2' is not printed and does not appear.
#+begin_src python :results output :session
print "hello"
2
print "bye"
#+end_src
#+resname:
: hello
: 2
: bye
But in =:session= mode, the interactive interpreter receives input '2'
and prints out its value, '2'. (Indeed, the other print statements are
unnecessary here).
*** Noweb reference syntax
:PROPERTIES:
:CUSTOM_ID: noweb-reference-syntax
:END:
The [[http://www.cs.tufts.edu/~nr/noweb/][Noweb]] Literate Programming system allows named blocks of code to
be referenced by using the
: <<code-block-name>>
syntax. When a document is tangled, these references are replaced with
the named code. An example is provided in the
[[literate programming example]].
*** Requirements Documentation
The original requirements documentation is available at
[[file:requirements.org][requirements]]. Because the project has evolved since this
documentation was written, it is here mainly for historical reasons.
* Development
** Contributing
Org-babel contributions are welcome! As for Org-mode, authors will
need to sign copyright papers with the FSF; please see the
[[http://orgmode.org/worg/org-contribute.php][instructions]] for contributing to Org-mode.
** Development documents
The development of Org-babel is tracked in [[file:development.org][development]], a large
Org-mode file. This file contains:
- lists of pending development tasks;
- lists of known and resolved bugs; and
- the Org-babel functional test suite, which is implemented as a large
table and is run using Org-babel's spreadsheet functionality.
** In the pipeline
Support for Clojure has recently been contributed, and support for
[[http://www.mozart-oz.org/][Oz]] is being planned.
* Footnotes
[fn:1] Calling C-c C-o on a source code block will open the
block's results in a separate buffer.
[-- Attachment #3: Type: text/plain, Size: 263 bytes --]
>
> But it looks very promising.
>
> Graham
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
[-- Attachment #4: Type: text/plain, Size: 201 bytes --]
_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode
^ permalink raw reply [flat|nested] 10+ messages in thread