Re: [babel] Babel as a test harness?

From: Nathan Neff <nathan.neff@gmail.com>
To: Dan Davison <davison@stats.ox.ac.uk>
Cc: emacs-orgmode@gnu.org
Subject: Re: [babel] Babel as a test harness?
Date: Sat, 20 Mar 2010 14:38:05 -0500	[thread overview]
Message-ID: <211769421003201238t4ddd21b0mc2b6af9610f78d45@mail.gmail.com> (raw)
In-Reply-To: <874okblt2p.fsf@stats.ox.ac.uk>

[-- Attachment #1.1: Type: text/plain, Size: 3545 bytes --]

On Fri, Mar 19, 2010 at 7:45 PM, Dan Davison <davison@stats.ox.ac.uk> wrote:

> Nathan Neff <nathan.neff@gmail.com> writes:
>
> > I think it would be cool to use Org-babel as a simple test harness.
> >
> > I'd like to have an org file, with various sections that demonstrate how
> to do
> > something in Groovy.  I'd like to be able to run all the code in the org
> file
> > and make sure
> > they all run successfully (return code 0).
> >
> > In Groovy, the "assert" function will exit with a non-zero code if it
> fails.
> > How
> > would I use org-babel to generate a "summary" table with the name of
> > each patch of code and whether or not it succeeded?
>
> Like below? This is all Eric's doing. For quite a while now org-babel
> has used a table like this to validate itself. It took me a while to
> understand it, but basically Eric designed a special function (actually,
> a macro) called sbe (source block evaluate) to be used in table formulas
> to call org-babel source blocks. Use C-u C-c C-c to update the table.
>
> * Tests
> | functionality  | block        | arg | expected | results              |
> pass                                       |
>
> |----------------+--------------+-----+----------+----------------------+--------------------------------------------|
> | simple regexp  | simple_regex |     |          |                      |
> pass                                       |
> | regexp with or | regex_or     |     |          | A pretend problem... |
> expected "" but was "A pretend problem..." |
> #+TBLFM: $5='(if (= (length $3) 1) (sbe $2 (n $3)) (sbe $2)) :: $6='(if
> (string= $4 $5) "pass" (format "expected %S but was %S" $4 $5))
>
> * Code blocks for tests
> ** Here's how to match "foo"
> #+source: simple_regex
> #+begin_src groovy
> assert "foo" =~ /foo/
> #+end_src
>
> ** Here's how to match "bar" or "baz"
> #+source: regex_or
> #+begin_src groovy
> assert "bar" =~ "ba(z|r)"
> println "A pretend problem"
> #+end_src
>
> You can put any arguments to your source blocks in the arg column. See
>
> http://orgmode.org/worg/org-contrib/babel/intro.php#spreadsheet
>
> (It often makes most sense to clone Worg and view the org files
> themseleves)
>
> And for a more complex example, including how to pass arguments to
> source blocks to sbe, see our full test suite in the file
> development.org in the devel repo
>
> http://github.com/eschulte/babel-dev/
>
> Dan
>
>

This is really cool Dan!  I simplified the test table -- it runs the test in
column 1, then
prints the output to the second column, and puts "pass" or "fail" in the 3rd
column
if the 2nd column is blank.

* Tests, Simpler

Output from each source block should be nothing.
| block        |
output                                                        | result |
|--------------+---------------------------------------------------------------+--------|
| simple_regex
|                                                               | pass   |
| regex_or
|                                                               | pass   |
| regex_fail   | Caught: java.lang.AssertionError: Expression: (bar =~
foo)... | fail   |
#+TBLFM: $2='(sbe $1) :: $3='(if (string= $2 "") "pass" "fail")

* Code blocks for tests
** Here's how to match "foo"
#+source: simple_regex
#+begin_src groovy results: output
assert "foo" =~ /foo/
#+end_src

** Here's how to match "bar" or "baz"
#+source: regex_or
#+begin_src groovy
assert "bar" =~ "ba(z|r)"
#+end_src

** A failing test
#+source: regex_fail
#+begin_src groovy
assert "bar" =~ "foo"
#+end_src

[-- Attachment #1.2: Type: text/html, Size: 4775 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode