Re: [babel] Babel as a test harness?

From: Dan Davison <davison@stats.ox.ac.uk>
To: Nathan Neff <nathan.neff@gmail.com>
Cc: emacs-orgmode@gnu.org
Subject: Re: [babel] Babel as a test harness?
Date: Fri, 19 Mar 2010 20:45:02 -0400	[thread overview]
Message-ID: <874okblt2p.fsf@stats.ox.ac.uk> (raw)
In-Reply-To: <211769421003191643l2d00b8cai63028b369408552f@mail.gmail.com> (Nathan Neff's message of "Fri, 19 Mar 2010 18:43:35 -0500")

Nathan Neff <nathan.neff@gmail.com> writes:

> I think it would be cool to use Org-babel as a simple test harness.
>
> I'd like to have an org file, with various sections that demonstrate how to do
> something in Groovy.  I'd like to be able to run all the code in the org file
> and make sure
> they all run successfully (return code 0).
>
> In Groovy, the "assert" function will exit with a non-zero code if it fails. 
> How
> would I use org-babel to generate a "summary" table with the name of
> each patch of code and whether or not it succeeded?

Like below? This is all Eric's doing. For quite a while now org-babel
has used a table like this to validate itself. It took me a while to
understand it, but basically Eric designed a special function (actually,
a macro) called sbe (source block evaluate) to be used in table formulas
to call org-babel source blocks. Use C-u C-c C-c to update the table.

* Tests
| functionality  | block        | arg | expected | results              | pass                                       |
|----------------+--------------+-----+----------+----------------------+--------------------------------------------|
| simple regexp  | simple_regex |     |          |                      | pass                                       |
| regexp with or | regex_or     |     |          | A pretend problem... | expected "" but was "A pretend problem..." |
#+TBLFM: $5='(if (= (length $3) 1) (sbe $2 (n $3)) (sbe $2)) :: $6='(if (string= $4 $5) "pass" (format "expected %S but was %S" $4 $5))

* Code blocks for tests
** Here's how to match "foo"
#+source: simple_regex
#+begin_src groovy
assert "foo" =~ /foo/
#+end_src

** Here's how to match "bar" or "baz"
#+source: regex_or
#+begin_src groovy
assert "bar" =~ "ba(z|r)"
println "A pretend problem"
#+end_src

You can put any arguments to your source blocks in the arg column. See

http://orgmode.org/worg/org-contrib/babel/intro.php#spreadsheet

(It often makes most sense to clone Worg and view the org files
themseleves)

And for a more complex example, including how to pass arguments to
source blocks to sbe, see our full test suite in the file
development.org in the devel repo

http://github.com/eschulte/babel-dev/

Dan

p.s. groovy seems to be funny about hyphens in the input file name. I
think you want to apply this:

diff --git a/contrib/babel/lisp/langs/org-babel-groovy.el b/contrib/babel/lisp/langs/org-babel-groovy.el
index 02b3272..d6a28a7 100644
--- a/contrib/babel/lisp/langs/org-babel-groovy.el
+++ b/contrib/babel/lisp/langs/org-babel-groovy.el
@@ -88,7 +88,7 @@ print o join(\"\\n\", @r), \"\\n\"")
 (defun org-babel-groovy-evaluate (session body &optional result-type)
   "Evaluate Groovy code in BODY"
  ;; external process evaluation
- (let ((infile (make-temp-file "org_babel_groovy_input_")))
+  (let ((infile (replace-regexp-in-string "-" "_" (make-temp-file "org_babel_groovy_input_"))))
    (save-excursion
      (with-temp-buffer
        (with-temp-file infile (insert body))



>
> Something like this would be cool:
>
> * Regular Expressions
> ** Here's how to match "foo"
>
> #+source simple_regex
> #+begin_src groovy
> assert "foo" =~ /foo/
> #+end_src
>
> ** Here's how to match "bar" or "baz"
> #+source regex_or
> #+begin_src groovy
> assert "bar" =~ "ba(z|r)"
> #+end_src
>
> * Results:
> | simple_regex | success |
> | regex_or        | success |
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
