From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Maus Subject: Re: [test] Mark tests with missing dependencies as "expected to fail" Date: Sun, 13 Nov 2011 17:18:48 +0100 Message-ID: <87zkg0ov13.wl%dmaus@ictsoc.de> References: <87fwiqnclu.wl%dmaus@ictsoc.de> <87fwiq2rsl.fsf@gmail.com> Mime-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: multipart/signed; boundary="pgp-sign-Multipart_Sun_Nov_13_17:18:48_2011-1"; micalg=pgp-sha256; protocol="application/pgp-signature" Content-Transfer-Encoding: 7bit Return-path: Received: from eggs.gnu.org ([140.186.70.92]:41869) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RPcli-0004Q4-5j for emacs-orgmode@gnu.org; Sun, 13 Nov 2011 11:18:55 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RPclg-00058t-Gb for emacs-orgmode@gnu.org; Sun, 13 Nov 2011 11:18:53 -0500 Received: from app1a.xlhost.de ([213.202.242.161]:54411) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RPclg-00058d-4n for emacs-orgmode@gnu.org; Sun, 13 Nov 2011 11:18:52 -0500 In-Reply-To: <87fwiq2rsl.fsf@gmail.com> List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Eric Schulte Cc: David Maus , emacs-orgmode@gnu.org --pgp-sign-Multipart_Sun_Nov_13_17:18:48_2011-1 Content-Type: text/plain; charset=US-ASCII Hi Eric, At Tue, 18 Oct 2011 10:22:34 -0600, Eric Schulte wrote: > Hi David, > > I agree it would be preferable to note that not all tests are run when > dependencies are missing, although I don't think it is extremely > important. I think some version of the above would be worthwhile if it > could be done in a file-wide manner (as are the current dependency > checks) and wouldn't require duplicating the dependency check or > changing every test form individually. Perhaps a file-local-variable > could be used to expect failures for every form defined in the file? I tried the approach with a file-local variable but it didn't work out: A macro can be expanded at any time, i.e. looks like there is no way to obtain a reference to the buffer where the macro is defined at expansion time. But finally came up with this one: #+begin_src emacs-lisp (defmacro org-test-with-dependencies (dependencies &rest body) "Mark `ert-deftest' forms in BODY with a expected result depending on DEPENDENCIES. DEPENDENCIES is an alist with a human-readable name of the dependency as key. The second element of each member should be a form that evaluates to a non-nil value if the dependency is met. All `ert-deftest' forms in BODY are marked as expected to pass if all dependencies are met. Otherwise the expected result is set to `:failed' and the test's bodies modified to signal an error with an error message indicating the first failing dependency." (macrolet ((define-dependencies () `(cond ,@(mapcar (lambda (dependency) `((not ,(second dependency)) ,(first dependency))) dependencies)))) (let ((missing-dependency (define-dependencies))) `(progn ,@(mapcar (lambda (sexp) (if (and (consp sexp) (eq (first sexp) 'ert-deftest)) (let* ((docstring (if (stringp (fourth sexp)) (list (fourth sexp)))) (deftest-body (nthcdr (if docstring 4 3) sexp))) `(,@(append (list (first sexp) (second sexp) (third sexp)) docstring) :expected-result ,@(if missing-dependency `(:failed (error "Missing dependency: %s" ,missing-dependency)) '(:passed)) ,@deftest-body)) sexp)) body))))) #+end_src You wrap it around ert-deftest forms, e.g. #+begin_src emacs-lisp (org-test-with-dependencies (("ESS" (featurep 'ess))) (ert-deftest test-ob-R/simple-session () (org-test-with-temp-text "#+begin_src R :session R\n paste(\"Yep!\")\n#+end_src\n" (should (string= "Yep!" (org-babel-execute-src-block)))))) #+end_src #+begin_quote ELISP> (macroexpand '(org-test-with-dependencies (("ESS" (featurep 'ess))) (ert-deftest foo () t))) (progn (ert-deftest foo nil :expected-result :failed (error "Missing dependency: %s" "ESS") t)) ELISP> #end_quote If this is acceptable I'd push it and start to adjust the test definitions. Best, -- David -- OpenPGP... 0x99ADB83B5A4478E6 Jabber.... dmjena@jabber.org Email..... dmaus@ictsoc.de --pgp-sign-Multipart_Sun_Nov_13_17:18:48_2011-1 Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iF4EABEIAAYFAk6/7egACgkQma24O1pEeOYO6QD/bmkc+KCzOJo5Dt0bxgIM+Olo Ow9cgyaDLmBzdsuarbgBANb0Fxq6u8HB6CSDFcu3ET3LOt/86tKhhbw+XeXix/7Q =RmfM -----END PGP SIGNATURE----- --pgp-sign-Multipart_Sun_Nov_13_17:18:48_2011-1--