From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sebastian Rose Subject: Re: How you can help Date: Thu, 23 Oct 2008 18:26:22 +0200 Message-ID: <87r6679t1t.fsf@kassiopeya.MSHEIMNETZ> References: <967CE7ED-05E9-4031-9F3B-CFB826511554@alexanderonline.org> <87ljwfbdga.fsf@kassiopeya.MSHEIMNETZ> <752DA813-1B66-4FD1-B28E-3C23176BA13D@alexanderonline.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Kt2yh-0004WP-4k for emacs-orgmode@gnu.org; Thu, 23 Oct 2008 12:24:03 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Kt2yf-0004VP-91 for emacs-orgmode@gnu.org; Thu, 23 Oct 2008 12:24:02 -0400 Received: from [199.232.76.173] (port=53558 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Kt2ye-0004VI-Tv for emacs-orgmode@gnu.org; Thu, 23 Oct 2008 12:24:00 -0400 Received: from mail.gmx.net ([213.165.64.20]:54261) by monty-python.gnu.org with smtp (Exim 4.60) (envelope-from ) id 1Kt2ye-0001JP-Ri for emacs-orgmode@gnu.org; Thu, 23 Oct 2008 12:24:01 -0400 In-Reply-To: <752DA813-1B66-4FD1-B28E-3C23176BA13D@alexanderonline.org> (Ben Alexander's message of "Thu, 23 Oct 2008 15:55:41 +0100") List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Ben Alexander Cc: emacs-orgmode Org-Mode Ben, I added this to the new page on worg. Please tell me: is it OK for you, if I simply add your ideas there?? What do the others think? Ben Alexander writes: > The tutorial I'd need to write a test is one which lays out code I could copy > and paste to do the following > > * setup the test environment > - create a test directory > - create a sample test.org file > - put the cursor in a particular place > > * run the command we need to test > - hit the 'TAB' key, or C-c C-c (some folks might need to be reminded how to > find out exactly what command is actually being run when you hit a keystroke. > And some of me might need to be told what lisp-code to use when the keystroke > runs different commands at different places in a file) > - reformat a table > - clock in/out > - create the agenda > - export .html .ics .dvi file > * How do we specify the correct result??? > - check that the headline folded properly. What's the lisp code for getting > the folded string as displayed? > - check that the cursor is where it should be? especially when the cursor is > near elipses... > - check that the agenda is built properly. What's the lisp code for getting > the agenda as a string? > - check that the exported files are correct. Maybe the right suggestion is to > run the export on two different files, so the test can focus on the 'diff' > between them. That way different people who run the same test on different > hosts can get the same result. Please, if you find time, add the rest somewhere there? If not, I'll take a closer look tonight and check in what's interesting (which seems to be all of it, actually). http://www.emacswiki.org/emacs-fr/UnitTesting: Does this mean to add code to Org's *.el files? This would mean a lot of work for Carsten :-( OK, once done... But: sh> grep -r defun lisp/ | wc -l 1290 Checking in/out of every function makes 2580 places to check. I can't estimate the effort, actually, because I never did Unit testing. I guess this number much higher than needed? Could anyone with testing expierience estimate? While I think automated tests are a great thing to do and I'm happy to see this reaction on the list :-), I still think we should also provide the 'simple' tests without additional dependencies (when will package management make it into emacs...). And I'd like to keep the effort and impact for Carsten as small as possible. He will have to check each and every patch because of feeling, and actually being, responsible for the code. Another question: will the code compile in emacs without the testing framework installed? Probably yes. Should we try to think of those 'simple test packages' as the more _DATA_ part of other tests? This would mean one could code an automated test without the need to create data too, since we have the data seperated. Again, I'd like to keep the hurdles low for everyone involved. When automated testing turns out to be a foolish easy and rock solid, this could change. * Three basic approaches: 1 Simple data and setup packages. 2 Code, that executes org commands and test for correct output, no changes in Org's code needed. 3 Test framework with changes to Org's code. I'd promote a combination from 1 and 2, and see, how far it goes. DATA +--- d1 +--- d2 +--- d3 TESTS +---t-a +---t-b +---t-c Now one would run test t-a over d1, d2, d3. Still a programmer could use d1 to fix a bug in d1 and finally push the fix for automated testing against d2 and d3. bug X tester verifies (is ist bug? how to reproduce...) tester adds X to dX (message -> list or programmer) programmer installs dX programmer fixes bugX programmer push (message -> list or tester) tester tests against d1, d2, d3 where tester = [human|machine] preferable human (simplicity) Should we track the dependencies between code and tests? Suppose one has worked on a.el. Shure more than one test has to run. Example: org-publish.el contains code common to LaTeX-export, HTML-export... Regards, -- Sebastian Rose, EMMA STIL - mediendesign, Niemeyerstr.6, 30449 Hannover Tel.: +49 (0)511 - 36 58 472 Fax: +49 (0)1805 - 233633 - 11044 mobil: +49 (0)173 - 83 93 417 Email: s.rose@emma-stil.de, sebastian_rose@gmx.de Http: www.emma-stil.de