From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sebastian Rose Subject: Re: Re: How you can help Date: Thu, 23 Oct 2008 19:02:30 +0200 Message-ID: <87iqrj9rdl.fsf@kassiopeya.MSHEIMNETZ> References: <967CE7ED-05E9-4031-9F3B-CFB826511554@alexanderonline.org> <878wsfpgtp.fsf@gollum.intra.norang.ca> <877i7zbbe4.fsf@kassiopeya.MSHEIMNETZ> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Kt3Xe-0008Km-WC for emacs-orgmode@gnu.org; Thu, 23 Oct 2008 13:00:11 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Kt3Xd-0008KU-DE for emacs-orgmode@gnu.org; Thu, 23 Oct 2008 13:00:10 -0400 Received: from [199.232.76.173] (port=49976 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Kt3Xd-0008KP-6a for emacs-orgmode@gnu.org; Thu, 23 Oct 2008 13:00:09 -0400 Received: from mail.gmx.net ([213.165.64.20]:39398) by monty-python.gnu.org with smtp (Exim 4.60) (envelope-from ) id 1Kt3Xc-0005gc-Fq for emacs-orgmode@gnu.org; Thu, 23 Oct 2008 13:00:08 -0400 In-Reply-To: (Richard Riley's message of "Thu, 23 Oct 2008 17:49:51 +0200") List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Richard Riley Cc: Bernt Hansen , emacs-orgmode@gnu.org, Ben Alexander Hi Richard, Richard Riley writes: >> Added this one to the Clippboard section on new org-tests/index.org in >> Worg.git. (this section will be temporary...) > > Something like the above should only be a link (at most) to the emacs > manual. Reproducing standard info is bad in the long run in case things > in the base product (emacs) change for example. On the long run, yes. I just added this section, to start the page. And since it's just STARTing... this was in the first or second reaction that came in. Feel free to edit ('go wild' ;-))! I wouldn't bother to have several pages like this one will be (it's supposed to be an index on the long run), each covering another way of testing. I just meant to take _ACT_. Scepticism is a good thing, as long as it doesn't stop you from doing. >> >>> Some kind of regression testing framework would be awesome. Org-mode is >>> large enough that this is almost a necessity to keep things stable and >>> bug-free. >> >> It's big and feature-RICH. > > The nature of OSS means that the community using the product keep it > stable and bug free. I dont think the efforts to produce meaningful > regression tests would be beneficial in an ever morphing product like > org-mode. Clearly my humble opinion on that one :-; We do test our software by using it. But the bug in the HTML-exporter that Carsten has fixed two days ago, was introduced in early September and would be in 6.10, which is supposed to be in the emacs 23 release. A very simple test plan would have revealed it. >>> Maybe something can be put together from the git testing framework and >>> use of emacs -batch to process test org files and verify the output is >>> as expected (with diff or some other tool). >> >> >> Hey, diff is a good idea!! >> >> I didn't take the verification of the output into account yet :-) >> >> I just pushed a change of Worgs start page, and added a directory >> 'org-tests'. I've placed an index.org there, which now is just a >> collection of ideas (I'm on my day job, so I can't really work on it >> now). >> >> Don't know how often the git repo is published. >> Bernt and Ben, are you 'worgers' allready? >> >> Do you think it makes sense to add snippets and ideas to the new page in >> Worg? I think while the list great to exchange ideas, it's good to have >> a place, where all those ideas are destilled to one-liners. > > I must say I am dubious about this. It means, for the tests to be > meaningful, that the output must be a fixed format in base org. Why? If the test bails out 'ERROR', I will have to look for the reason. If the format changed in a legal way => adjust the test. > I doubt > this will ever be the case. The presentation will fluctuate while the > core information (dates, schedules periods etc) will remain pretty much > constant. > > The majority of bugs that I see are often down to people misusing or > using things in the base which are not fully explored. No amount of > regression testing can cover things like that unless the regression > tests include everyones customisations. Yes, because Carsten add features en masse :-) I see the testing differently. In the first place we need THINK of testing. New Org-revision out? Ahh, OK, I have to the HTML-exports, I want to be working. To do this, I need several different setups, several different data directories (Org-files) and an easy way to test, that doesn't eat my time and gives a result. The quality may vary, but ERRORs will be detected for shure. Not one or mounths later - immediately. If someone installs emacs 23, tries to export to HTML and gets an error.... > Do I think regression testing is important? Yes - in certain > environments. But every time Carsten, you, myself or anyone else fires > up org-mode we are already doing just that. Yes, but we can do better, easier and more complete. -- Sebastian Rose, EMMA STIL - mediendesign, Niemeyerstr.6, 30449 Hannover Tel.: +49 (0)511 - 36 58 472 Fax: +49 (0)1805 - 233633 - 11044 mobil: +49 (0)173 - 83 93 417 Email: s.rose@emma-stil.de, sebastian_rose@gmx.de Http: www.emma-stil.de