Re: Test framework needed

[Eric Abrahamsen <eric@ericabrahamsen.net>]

On Wed, Mar 30 2011, Rainer M Krug wrote:

> On 30/03/11 15:46, Eric Abrahamsen wrote:
>> On Wed, Mar 30 2011, Rainer M Krug wrote:
>> 
>>> Hi
>>>
>>> I was bitten again from an unintended regression in org-mode, and that
>>> the second time in two weeks.
>>>
>>> I am probably not the right person to suggest this, but I think it is
>>> time to introduce a test framework for org-mode, to ensure that the
>>> (without doubt useful) approach to develop org-mode does not lead to
>>> regressions.
>> 
>> This would be the page to start with, though the most likely candidate
>> (Elisp Regression Testing) is only available in Emacs trunk at the
>> moment…
>> 
>> http://www.emacswiki.org/emacs/UnitTesting
>
> Am I right in assuming, that all of the possible test frameworks would
> require org files and the expected output (tengle, export to ...,
> agenda, ...)? In this case, would it make sense to start collecting
> those, as they can easily be user contributed, consequently representing
> a cross section of the use cases (even not intended use cases)?

Yup, what you would need is some org source files that exercise all of
the possible export options (for testing export, for example), including
weird edge cases, and then ERT (if that's what we ended up using) would
provide handy functions for making sure the export output matches
expectations. The excellent gentleman who created the ODT exporter,
whose name currently escapes me, has already created test files for his
exporter—that would be a perfect place to start.

Covering all of org's various functions would end up being a bit of a
PITA, though you're quite right that it's an excellent idea, and will
become more and more necessary.

E