Re: How you can help

[Sebastian Rose <sebastian_rose@gmx.de>]

Ben,

I added this to the new page on worg. Please tell me: is it OK for you,
if I simply add your ideas there??

What do the others think?


Ben Alexander <bva@alexanderonline.org> writes:
> The tutorial I'd need to write a test is one which lays out code I could copy
> and paste to do the following
>
> * setup the test environment
>   - create a test directory
>   - create a sample test.org file
>   - put the cursor in a particular place
>
> * run the command we need to test
>   - hit the 'TAB' key, or C-c C-c (some folks might need to be reminded how to
> find out exactly what command is actually being run  when you hit a keystroke.
> And some of me might need to be told what  lisp-code to use when the keystroke
> runs different commands at  different places in a file)
>   - reformat a table
>   - clock in/out
>   - create the agenda
>   - export .html .ics .dvi file
> * How do we specify the correct result???
>   - check that the headline folded properly.  What's the lisp code for getting
> the folded string as displayed?
>   - check that the cursor is where it should be? especially when the cursor is
> near elipses...
>   - check that the agenda is built properly.  What's the lisp code for getting
> the agenda as a string?
>   - check that the exported files are correct.  Maybe the right suggestion is to
> run the export on two different files, so the test  can focus on the 'diff'
> between them.  That way different people who  run the same test on different
> hosts can get the same result.


Please, if you find time, add the rest somewhere there?
If not, I'll take a closer look tonight and check in what's interesting
(which seems to be all of it, actually).


http://www.emacswiki.org/emacs-fr/UnitTesting:

Does this mean to add code to Org's *.el files? This would mean a lot of
work for Carsten :-( OK, once done... 

But:
sh> grep -r defun lisp/ | wc -l
1290

Checking in/out of every function makes 2580 places to check. I can't
estimate the effort, actually, because I never did Unit testing. I guess
this number much higher than needed?

Could anyone with testing expierience estimate?



While I think automated tests are a great thing to do and I'm happy to
see this reaction on the list :-), I still think we should also provide
the 'simple' tests without additional dependencies (when will package
management make it into emacs...). And I'd like to keep the effort and
impact for Carsten as small as possible. He will have to check each and
every patch because of feeling, and actually being, responsible for the
code.

Another question: will the code compile in emacs without the testing
framework installed? Probably yes.

Should we try to think of those 'simple test packages' as the more
_DATA_ part of other tests? This would mean one could code an automated
test without the need to create data too, since we have the data
seperated. Again, I'd like to keep the hurdles low for everyone
involved. When automated testing turns out to be a foolish easy and rock
solid, this could change.


* Three basic approaches:

  1 Simple data and setup packages.
  2 Code, that executes org commands and test for correct output,
    no changes in Org's code needed.
  3 Test framework with changes to Org's code.

I'd promote a combination from 1 and 2, and see, how far it goes.




DATA
  +--- d1
  +--- d2
  +--- d3

TESTS
  +---t-a
  +---t-b
  +---t-c

Now one would run test t-a over d1, d2, d3.
Still a programmer could use d1 to fix a bug in d1 and finally push the
fix for automated testing against d2 and d3.


bug X
tester verifies (is ist bug? how to reproduce...)
tester adds X to dX (message -> list or programmer)
programmer installs dX
programmer fixes bugX
programmer push   (message -> list or tester)
tester tests against d1, d2, d3

where tester = [human|machine] preferable human (simplicity)


Should we track the dependencies between code and tests?

Suppose one has worked on a.el.
Shure more than one test has to run.
Example: org-publish.el contains code common to LaTeX-export,
HTML-export... 



Regards,

-- 
Sebastian Rose, EMMA STIL - mediendesign, Niemeyerstr.6, 30449 Hannover

Tel.:  +49 (0)511 - 36 58 472
Fax:   +49 (0)1805 - 233633 - 11044
mobil: +49 (0)173 - 83 93 417
Email: s.rose@emma-stil.de, sebastian_rose@gmx.de
Http:  www.emma-stil.de