Re: Re: Test framework needed

[Aankhen <aankhen@gmail.com>]

On Wed, Mar 30, 2011 at 20:43, Manuel Giraud
<manuel.giraud@univ-nantes.fr> wrote:
> Rainer M Krug <r.m.krug@gmail.com> writes:
> [snip]
>
>> Please correct me if I am missing something.
>>
>> This suite should actually be updated with effectively each patch which
>> introduces new features and run after each patch.
>
> Which renders this framework far less automatic. I think that having a
> set of org files against which one could try any export and *see* that
> the results are almost correct would be enough.

I think the “automated” part refers to running the tests.  What you
suggest—having a set of files that you could manually export to verify
the results—wouldn’t be of much help, IMHO.  First, it’d require a lot
more time than executing a single command and checking the summary at
the end.  Second, it’d be very error-prone.

A comprehensive automated test suite gives people writing patches an
easier way to perform regression testing and catch any unintended
consequences.  On the other hand, it /does/ take a lot of effort to
keep it in sync with the codebase… maybe we need a Test Fairy. ;-)

Aankhen