From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Eric Schulte" Subject: Re: Re: Testing --- again... Date: Sun, 03 Oct 2010 19:26:39 -0600 Message-ID: <87vd5ibve8.fsf@gmail.com> References: <87vd5li75s.fsf@gmx.de> <0CDCC1C3-3D93-465E-827C-A722978F4D13@gmail.com> <87fwwoihuy.fsf@gmail.com> <87fwwobd9k.fsf@gmx.de> <871v88ibhh.fsf@gmail.com> <87sk0ofef3.fsf@gmail.com> <87ocbbvuig.fsf@gmx.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Received: from [140.186.70.92] (port=48315 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1P2ZpG-0004BR-Tq for emacs-orgmode@gnu.org; Sun, 03 Oct 2010 21:26:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1P2ZpF-0006xi-F4 for emacs-orgmode@gnu.org; Sun, 03 Oct 2010 21:26:46 -0400 Received: from mail-iw0-f169.google.com ([209.85.214.169]:47903) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1P2ZpF-0006xe-9D for emacs-orgmode@gnu.org; Sun, 03 Oct 2010 21:26:45 -0400 Received: by iwn33 with SMTP id 33so9660944iwn.0 for ; Sun, 03 Oct 2010 18:26:43 -0700 (PDT) In-Reply-To: <87ocbbvuig.fsf@gmx.de> (Sebastian Rose's message of "Sun, 03 Oct 2010 02:46:43 +0200") List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Sebastian Rose Cc: Emacs-orgmode mailing list , Carsten Dominik Hi, I've taken the liberty of merging our two testing branches into the new "combined-testing" branch. This now includes both ert and jump.el as git submodules, which can be installed with git submodule init git submodule update after checking out the branch. In merging the two org-test.el files, I was able to remove much of the existing code through using which-func, jump.el. The new navigation functionality will work regardless of the directory structure, so tests for file lisp/org-foo.el can be located in either of the following testing/lisp/test-org-foo.el testing/lisp/org-foo.el/test.el Additionally the navigation functions defined in jump.el should be easily extensible to accommodate new naming schemas, so ideally every test author for a particular file can use whatever naming system they prefer. Note that the navigation function `org-test-jump' when called with a prefix argument uses your code from `org-test-edit-buffer-file-tests' to create the test file if it is not already present. I think I retained most of the functionality from your version of org-test.el in the merge but please let me know if I broke something. In my mind this merged version should be small clean and maintainable and hopefully flexible enough to handle whatever needs we identify during the course of writing the tests. How would you feel about using this as the new base of test development? Best -- Eric also I have some inline comments below Sebastian Rose writes: > "Eric Schulte" writes: >> "Eric Schulte" writes: >> To illustrate my suggestions, I've thrown together a couple simple Babel >> tests roughly following this outline, currently up in the >> `schulte-testing' branch of the Org-mode repo. >> >> To try it out >> 1) load the testing/org-test.el file >> 2) run `org-load-tests' to load up the entire org-mode test suite >> 3) run `ert' to run the test suite. >> 4) or jump to the definition of `org-babel-get-src-block-info' and run >> `org-test-current-function' to just run the tests for that function >> >> Best -- Eric > > Yes. I know how running tests looks like: It looks good :) > > Hahaa, I like that code. It looks so simple :) > > > To load all files below testing/ is a good suggestion probably. The > unloading and loading as I implemented it is simply superfluous, if we > suggest that testers no how to eval a buffer or defun (and if they can > change it, they can eval it). > > > > * Question: > > (defun org-test-current-function () > "Test the current function." > (interactive) > (ert (car (which-function)))) > > `org-test-test-current-defun' in the ert-testing branch does that, too. > (But with that superfluous loading/unloading stuff though :-/) But it > encloses each test inside save-excursion and Co. > I believe ert takes care of the save-excursion stuff > > What does which-func.el that this function does not: > > <#part type="application/emacs-lisp" disposition=inline> > (defun org-test-which-func () > "Return the name of the current defun." > (save-excursion > (save-match-data > (end-of-line) > (beginning-of-defun) > (if (looking-at "(defun[[:space:]]+\\([^([:space:]]*\\)[[:space:]]*(") > (match-string-no-properties 1) > (error "No defun found around point."))))) > <#/part> > > ?? > I'm not sure that it does include anything new, but it's nice to re-use existing packages. > > > > * Keymap > > We should add keys to the org-mode-map. > > C-c t > > is still free here. Or is it in you use it for babel somehow? > > C-c t f org-test-test-current-function > C-c t b org-test-test-current-buffer-file > > Wouldn't we want the keys in the elisp-mode key map, since we'll be doing the testing work from elisp? Although I guess we may want to run tests from within Org-mode files, but then we could do that with a Babel block and dump the results to a table. :) > > > > * ERT Selectors > > I see a little namespace problem coming up. Imagine testing org.el. > Which ert selector would we use? > > "^org" ??? :-/ > > Should we create hashes of filenames as selectors (just kidding)? > Or use the entire filename "^org.el"? The relative path > "^lisp/org.el"? > oh, good point, maybe we'd need to use the "eql" or "tag" ert selectors in this case. > > > > * About the directory structure: > > It does not burden the user, as tests are loaded and executed > automatically (per function, per file or all, depending on the command > used). But imagine the entire thing grows and someone would add tests > for all the stuff in emacs/lisp/. Or add tests files for single > functions simply because there is so much to test that one file would > be hard to handle. > This could cause a lot of clutter. > > Is this too hypothetical? Hmmm - it might be... > > If you checkout ert-testing, eval testing/org-test.el and do > > M-x org-test-edit-buffer-file-tests > In the combined-testing branch you'd do this by calling org-test-jump with a prefix argument. > > you're in your test file in the correct directory (which is created > if it doesn't exist). > > M-x org-test-edit-current-defuns-tests > > creates a file named after the defun you're in. You'll need to be in > an elisp file of course. The testing/ directory will resemble the > directory structure of the project. > > Every one who saw that directory structure simply asked "don't you > think it's overkill?" :D I'll probably drop that. > heh, yea I'd lean towards getting into the writing of tests and then leaving our implementation flexible enough so that we can adopt this more fine-grained directory structure when/if it becomes necessary. > > > > We could go on on two rails for a while. I'll need this week to dig > deeper into ERT and come to conclusions. > > It would be a good thing if we could agree about the keys and the > selectors (again: "^org"), so that it's painless to switch branches ;) > > > > > > > Sebastian