From mboxrd@z Thu Jan 1 00:00:00 1970 From: Carsten Dominik Subject: Re: Testing --- again... Date: Sat, 2 Oct 2010 10:09:03 +0200 Message-ID: <0CDCC1C3-3D93-465E-827C-A722978F4D13@gmail.com> References: <87vd5li75s.fsf@gmx.de> Mime-Version: 1.0 (Apple Message framework v936) Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Return-path: Received: from [140.186.70.92] (port=58164 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1P1x9a-0003cW-IK for emacs-orgmode@gnu.org; Sat, 02 Oct 2010 04:09:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1P1x9Y-0006XP-Io for emacs-orgmode@gnu.org; Sat, 02 Oct 2010 04:09:10 -0400 Received: from mail-ew0-f41.google.com ([209.85.215.41]:33681) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1P1x9Y-0006XL-AQ for emacs-orgmode@gnu.org; Sat, 02 Oct 2010 04:09:08 -0400 Received: by ewy19 with SMTP id 19so1841918ewy.0 for ; Sat, 02 Oct 2010 01:09:06 -0700 (PDT) In-Reply-To: <87vd5li75s.fsf@gmx.de> List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Sebastian Rose Cc: Emacs-orgmode mailing list Hi Sebastian, the lack of a testing suite for Org-mode is really frustrating, and even more frustrating is that we have had like seven attempts to start one, and each of these lead to nothing. So I would be perfectly happy to give a free hand, write access to the repo and a full directory in the distribution to implement one. Once there is a framework, I am sure many people would be willing to contribute tests. More comments below. On Oct 2, 2010, at 5:51 AM, Sebastian Rose wrote: > Hi, > > > I thought about testing again recently. This is something, that never > really got started. For a reason: there's no framework for testing. > > I therefore wrote a very rough proposal, found on > http://github.com/SebastianRose/org-test > > The idea is, to provide two simple commands: > > > * org-test-test-current-defun > will search for tests for the defun point is in or behind > (`beginning-of-defun') and execute them surrounded by > > (let ((select (or selector "^org")) > (deactivate-mark nil)) > (save-excursion > (save-match-data > > > * org-test-test-buffer-file > will search for tests for the entire file and execute them the > same > way. FIrst: I have *no* clue about testing. Second, I am surprised that you want to structure it by function. I would have thought that it could be structure by file at the most. And then there will be tests that involve code from many files. But I guess > > If you use one of these commands, all currently registered ERT tests > are > deleted, and files are reloaded (since you're likely to work on the > tests, too). To repeat the tests without reloading, you will use the > ERT commands like `ert-results-rerun-all-tests', bound to `r' in the > ERT > results buffer. > > > > I choose ERT (git clone http://github.com/ohler/ert.git) because > that's > likely to go into Emacs core (or elpa.gnu.org). > > > > > The idea is to search the directory structure from the current source > file upwards for a directory named "tests/" if it exists. Else ask > the > user. Similar to what `add-change-log-entry' does. > > Below that directory, a tree like the source tree exists: > > project > +-- lisp/ > | +-- a.el > | `-- b/ > | +-- b.el > | > `-- tests/ > +-- a.el/ > | +-- tests.el > | `-- a-defun.el > `-- b/ > +-- b.el/ > +-- tests.el > `-- b-defun.el > > If this setup exists, when editing defun-x in lisp/a.el, > `M-x org-test-test-current-defun' will load tests/a.el/defun-x.el > (fallback: tests.el there) and execute all tests with selector > "^a-defun". Well, OK, this is fine. But under a.el and b.el there should also be general tests that are not function dependent, and there should be a place to put tests that you do not want to assign to a specific file. We do have a "testing" directory already, you can use that. I would prefer the tests to be in testing, not in lisp/testing if possible. I would like to have the lisp directory contain only code. If possible. It would be OK to have a lisp subdirectory in testing, just as it would be OK to have contrib/lisp in testing for the contributed packages. PLEASE, go ahead. I do not think you have write access yet on repo - give me your user name and I'll activate you. - Carsten > `M-x org-test-test-buffer-file' in that same source file will load all > *.el files in tests/a.el/ and execute all ERT tests for selector "^a". > > > Thus tests for > org-mode/lisp/org-protocol.el > will be searched in the directory > org-mode/tests/lisp/org-protocol.el/*.el > > > Once the basic route of testing is clear, I'd like to "translate" the > existing tests for org-html.el to work with ERT, which will involve > writing more tools (create output buffers, compare output with control > files using ediff etc.). I know Lennart Borgman has wrote that stuff > for nXhtml already. I hope we can use his stuff and help here. > > The directory org-mode/lisp/tests/ would not need to be part of the > "official" Org mode package. It could as well be checked out > separately, if "tests" is part of org-mode/lisp/.gitignore (e.g.). > > > Any thoughts? > > > > > Sebastian