emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* [test] Mark tests with missing dependencies as "expected to fail"
@ 2011-10-18  4:32 David Maus
  2011-10-18 16:22 ` Eric Schulte
  0 siblings, 1 reply; 6+ messages in thread
From: David Maus @ 2011-10-18  4:32 UTC (permalink / raw)
  To: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 1832 bytes --]

Hi all,

Currently tests with missing dependency are silently skipped -- it
might be worth changing this behavior to not skip them, but mark them
as expected to fail. You can do this in ERT by placing the
keyword :expected-result followed by either :passed or :failed before
the test's body.

Benefit of this is that the tests w/ missing dependencies will show up
in the ERT result page (with a small letter f) but (obviously) don't
count as failures.

The following macro is a first shot at a convenient way to define
tests with dependencies:

#+begin_src emacs-lisp
  (defmacro org-test-deftest-with-dependency (dependency &rest body)
    (let* ((docstring (if (stringp (third body)) (list (third body))))
           (deftest (nthcdr (if docstring 3 2) body))
           (dependency-p (eval dependency)))
      `(ert-deftest ,@(append (list (first body) (second body)) docstring)
           ,@(if dependency-p
                 '(:expected-result :passed)
               '(:expected-result :failed (error "Missing dependency")))

Here DEPENDENCY is a simple form that evaluates to non-nil if the
dependency is met. If marking the tests this way is the agreed way to
go we can extend the syntax of a dependency to an alist with a
human-readable description of the dependency as key and a list of
forms that all must eval to non-nil as value. E.g.

#+begin_src emacs-lisp
  (defvar test-ob-R-dependencies
    '(("R executable" (org-test-for-executable "R"))
      ("Package ESS" (featurep 'ess))))

And change the expander code to map through this alist and as soon one
dependency evals to nil sets a variable `dependency-missing' to the
respective dependency.

Any comments on this?

  -- David
OpenPGP... 0x99ADB83B5A4478E6
Jabber.... dmjena@jabber.org
Email..... dmaus@ictsoc.de

[-- Attachment #2: Type: application/pgp-signature, Size: 230 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2011-11-22  6:10 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-10-18  4:32 [test] Mark tests with missing dependencies as "expected to fail" David Maus
2011-10-18 16:22 ` Eric Schulte
     [not found]   ` <CALbR62330+VdrP5sRHLa3Mre5HAi8+LkbkvkuxDGbyEjAApcJQ@mail.gmail.com>
2011-10-18 17:03     ` Fwd: " Brian Wightman
2011-11-13 16:18   ` David Maus
2011-11-13 19:31     ` Eric Schulte
2011-11-22  6:10       ` David Maus

Code repositories for project(s) associated with this inbox:


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).