Re: Re: How you can help

[Sebastian Rose <sebastian_rose@gmx.de>]

Hi Richard,


Richard Riley <rileyrgdev@googlemail.com> writes:
>> Added this one to the Clippboard section on new org-tests/index.org in
>> Worg.git. (this section will be temporary...)
>
> Something like the above should only be a link (at most) to the emacs
> manual. Reproducing standard info is bad in the long run in case things
> in the base product (emacs) change for example.


On the long run, yes. I just added this section, to start the page. And
since it's just STARTing... this was in the first or second reaction
that came in. Feel free to edit ('go wild' ;-))!

I wouldn't bother to have several pages like this one will be (it's
supposed to be an index on the long run), each covering another way of
testing.

I just meant to take _ACT_. Scepticism is a good thing, as long as it
doesn't stop you from doing.


>>
>>> Some kind of regression testing framework would be awesome.  Org-mode is
>>> large enough that this is almost a necessity to keep things stable and
>>> bug-free.
>>
>> It's big and feature-RICH.
>
> The nature of OSS means that the community using the product keep it
> stable and bug free. I dont think the efforts to produce meaningful
> regression tests would be beneficial in an ever morphing product like
> org-mode. Clearly my humble opinion on that one :-;


We do test our software by using it. But the bug in the HTML-exporter
that Carsten has fixed two days ago, was introduced in early September
and would be in 6.10, which is supposed to be in the emacs 23 release.

A very simple test plan would have revealed it.



>>> Maybe something can be put together from the git testing framework and
>>> use of emacs -batch to process test org files and verify the output is
>>> as expected (with diff or some other tool).
>>
>>
>> Hey, diff is a good idea!!
>>
>> I didn't take the verification of the output into account yet :-)
>>
>> I just pushed a change of Worgs start page, and added a directory
>> 'org-tests'. I've placed an index.org there, which now is just a
>> collection of ideas (I'm on my day job, so I can't really work on it
>> now).
>>
>> Don't know how often the git repo is published.
>> Bernt and Ben, are you 'worgers' allready?
>>
>> Do you think it makes sense to add snippets and ideas to the new page in
>> Worg? I think while the list great to exchange ideas, it's good to have
>> a place, where all those ideas are destilled to one-liners.
>
> I must say I am dubious about this. It means, for the tests to be
> meaningful, that the output must be  a fixed format in base org.


Why? If the test bails out 'ERROR', I will have to look for the
reason. If the format changed in a legal way => adjust the test.


> I doubt
> this will ever be the case. The presentation will fluctuate while the
> core information (dates, schedules periods etc) will remain pretty much
> constant.
>
> The majority of bugs that I see are often down to people misusing or
> using things in the base which are not fully explored. No amount of
> regression testing can cover things like that unless the regression
> tests include everyones customisations.

Yes, because Carsten add features en masse :-)
I see the testing differently. In the first place we need THINK of
testing.

New Org-revision out?
Ahh, OK, I have to the HTML-exports, I want to be working.

To do this, I need several different setups, several different data
directories (Org-files) and an easy way to test, that doesn't eat my
time and gives a result. The quality may vary, but ERRORs will be
detected for shure.

Not one or mounths later - immediately.

If someone installs emacs 23, tries to export to HTML and gets an
error.... 


> Do I think regression testing is important? Yes - in certain
> environments. But every time Carsten, you, myself or anyone else fires
> up org-mode we are already doing just that.

Yes, but we can do better, easier and more complete.



-- 
Sebastian Rose, EMMA STIL - mediendesign, Niemeyerstr.6, 30449 Hannover

Tel.:  +49 (0)511 - 36 58 472
Fax:   +49 (0)1805 - 233633 - 11044
mobil: +49 (0)173 - 83 93 417
Email: s.rose@emma-stil.de, sebastian_rose@gmx.de
Http:  www.emma-stil.de