emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed

From: Eric Schulte <schulte.eric@gmail.com>
To: David Maus <dmaus@ictsoc.de>
Cc: emacs-orgmode@gnu.org
Subject: Re: [test] Mark tests with missing dependencies as "expected to fail"
Date: Tue, 18 Oct 2011 10:22:34 -0600	[thread overview]
Message-ID: <87fwiq2rsl.fsf@gmail.com> (raw)
In-Reply-To: <87fwiqnclu.wl%dmaus@ictsoc.de> (David Maus's message of "Tue, 18 Oct 2011 06:32:45 +0200")

David Maus <dmaus@ictsoc.de> writes:

> Hi all,
>
> Currently tests with missing dependency are silently skipped -- it
> might be worth changing this behavior to not skip them, but mark them
> as expected to fail. You can do this in ERT by placing the
> keyword :expected-result followed by either :passed or :failed before
> the test's body.
>
> Benefit of this is that the tests w/ missing dependencies will show up
> in the ERT result page (with a small letter f) but (obviously) don't
> count as failures.
>
> The following macro is a first shot at a convenient way to define
> tests with dependencies:
>
> #+begin_src emacs-lisp
>   (defmacro org-test-deftest-with-dependency (dependency &rest body)
>     (let* ((docstring (if (stringp (third body)) (list (third body))))
>            (deftest (nthcdr (if docstring 3 2) body))
>            (dependency-p (eval dependency)))
>       `(ert-deftest ,@(append (list (first body) (second body)) docstring)
>            ,@(if dependency-p
>                  '(:expected-result :passed)
>                '(:expected-result :failed (error "Missing dependency")))
>          ,@deftest)))
> #+end_src
>
> Here DEPENDENCY is a simple form that evaluates to non-nil if the
> dependency is met. If marking the tests this way is the agreed way to
> go we can extend the syntax of a dependency to an alist with a
> human-readable description of the dependency as key and a list of
> forms that all must eval to non-nil as value. E.g.
>
> #+begin_src emacs-lisp
>   (defvar test-ob-R-dependencies
>     '(("R executable" (org-test-for-executable "R"))
>       ("Package ESS" (featurep 'ess))))
> #+end_src
>
> And change the expander code to map through this alist and as soon one
> dependency evals to nil sets a variable `dependency-missing' to the
> respective dependency.
>
> Any comments on this?
>

Hi David,

I agree it would be preferable to note that not all tests are run when
dependencies are missing, although I don't think it is extremely
important.  I think some version of the above would be worthwhile if it
could be done in a file-wide manner (as are the current dependency
checks) and wouldn't require duplicating the dependency check or
changing every test form individually.  Perhaps a file-local-variable
could be used to expect failures for every form defined in the file?

Cheers -- Eric

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

next prev parent reply	other threads:[~2011-10-18 16:22 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-10-18  4:32 [test] Mark tests with missing dependencies as "expected to fail" David Maus
2011-10-18 16:22 ` Eric Schulte [this message]
     [not found]   ` <CALbR62330+VdrP5sRHLa3Mre5HAi8+LkbkvkuxDGbyEjAApcJQ@mail.gmail.com>
2011-10-18 17:03     ` Fwd: " Brian Wightman
2011-11-13 16:18   ` David Maus
2011-11-13 19:31     ` Eric Schulte
2011-11-22  6:10       ` David Maus

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.orgmode.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87fwiq2rsl.fsf@gmail.com \
    --to=schulte.eric@gmail.com \
    --cc=dmaus@ictsoc.de \
    --cc=emacs-orgmode@gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).