From mboxrd@z Thu Jan 1 00:00:00 1970 From: Eric Abrahamsen Subject: Re: Test framework needed Date: Wed, 30 Mar 2011 22:11:23 +0800 Message-ID: <87hbakg0l0.fsf@ericabrahamsen.net> References: <4D9329BC.6000106@gmail.com> <87r59og1pt.fsf@ericabrahamsen.net> <4D93369C.7010608@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Return-path: Received: from [140.186.70.92] (port=42798 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Q4w7k-00062e-Ja for emacs-orgmode@gnu.org; Wed, 30 Mar 2011 10:11:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Q4w7a-0001qz-F0 for emacs-orgmode@gnu.org; Wed, 30 Mar 2011 10:11:52 -0400 Received: from lo.gmane.org ([80.91.229.12]:56085) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Q4w7a-0001op-0x for emacs-orgmode@gnu.org; Wed, 30 Mar 2011 10:11:42 -0400 Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Q4w7W-0002Ic-M8 for emacs-orgmode@gnu.org; Wed, 30 Mar 2011 16:11:38 +0200 Received: from 123.121.225.181 ([123.121.225.181]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 30 Mar 2011 16:11:38 +0200 Received: from eric by 123.121.225.181 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 30 Mar 2011 16:11:38 +0200 List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: emacs-orgmode@gnu.org On Wed, Mar 30 2011, Rainer M Krug wrote: > On 30/03/11 15:46, Eric Abrahamsen wrote: >> On Wed, Mar 30 2011, Rainer M Krug wrote: >> >>> Hi >>> >>> I was bitten again from an unintended regression in org-mode, and that >>> the second time in two weeks. >>> >>> I am probably not the right person to suggest this, but I think it is >>> time to introduce a test framework for org-mode, to ensure that the >>> (without doubt useful) approach to develop org-mode does not lead to >>> regressions. >> >> This would be the page to start with, though the most likely candidate >> (Elisp Regression Testing) is only available in Emacs trunk at the >> moment… >> >> http://www.emacswiki.org/emacs/UnitTesting > > Am I right in assuming, that all of the possible test frameworks would > require org files and the expected output (tengle, export to ..., > agenda, ...)? In this case, would it make sense to start collecting > those, as they can easily be user contributed, consequently representing > a cross section of the use cases (even not intended use cases)? Yup, what you would need is some org source files that exercise all of the possible export options (for testing export, for example), including weird edge cases, and then ERT (if that's what we ended up using) would provide handy functions for making sure the export output matches expectations. The excellent gentleman who created the ODT exporter, whose name currently escapes me, has already created test files for his exporter—that would be a perfect place to start. Covering all of org's various functions would end up being a bit of a PITA, though you're quite right that it's an excellent idea, and will become more and more necessary. E