On Mon, Sep 5, 2011 at 6:54 PM, Eric Schulte
<schulte.eric@gmail.com> wrote:
Hi Rainer,
> Just as a sideline: would it be possible, to use this file to test babel (on
> a high level)?, i.e. collect all results and compare them with previous sets
> of results?
>
Two problems I see with running these examples in a single large block
1. I'm not sure how to collect all of the results, both of the two
options that occur to me
- mapping over each example and evaluating the first code block or
Probably would be the better solution, and also just skip tests for which the languages are not installed.
- exporting to some backend and checking the resulting export file
would either leave out some results or possibly be very brittle to
small changes (e.g., changes in the exporter backend)
This might be actually be an advantage - if changes are detected, the exported file can be evaluated visually and added to the "aceptable results", either generally or for a user.
2. executing the whole set would require a wide range of languages be
installed on the users system many of which most people would never
use
Point taken - I haven't considered that.
>
> Especially, as this file is high-level and contains a whole range of
> "examples for the real world", I think that might be useful. It would
> definitely not be usable t pinpoint errors, but at identify unintended side
> effects.
I do agree that this collection holds great promise as fodder for Babel
unit tests (unit tests which are sorely needed). Each example could be
wrapped in an ert test method and then one-by-one copied over into the
Org-mode test suite. This should be a fairly easy way to greatly
increase Babel's test suite coverage -- and most importantly it would be
explicitly testing topics that have arisen previously on the mailing
list so are of importance to users.
Exactly.
> And if this could be done by the user to test .emacs configuration
> settings, that would be really great.
>
That's a good idea. Simply running the Org-mode test suite is a good
way for users to test the sanity of their local config.
Yes - I have a few times run into problems due to my configuration file and only realized it later - if there would be ab org file which I could use to test easily my configuration directly after making the changes, life would have been much easier for me. This would also be useful for questions on the mailing list and for a kind of "quality control" for published configs on the web.
Best -- Eric
Cheers,
Rainer