Re: [O] [test] Mark tests with missing dependencies as "expected to fail"

[David Maus]

Hi Eric,

At Tue, 18 Oct 2011 10:22:34 -0600,
Eric Schulte wrote:
> Hi David,
>
> I agree it would be preferable to note that not all tests are run when
> dependencies are missing, although I don't think it is extremely
> important.  I think some version of the above would be worthwhile if it
> could be done in a file-wide manner (as are the current dependency
> checks) and wouldn't require duplicating the dependency check or
> changing every test form individually.  Perhaps a file-local-variable
> could be used to expect failures for every form defined in the file?

I tried the approach with a file-local variable but it didn't work
out: A macro can be expanded at any time, i.e. looks like there is no
way to obtain a reference to the buffer where the macro is defined at
expansion time.

But finally came up with this one:

#+begin_src emacs-lisp
    (defmacro org-test-with-dependencies (dependencies &rest body)
      "Mark `ert-deftest' forms in BODY with a expected result
    depending on DEPENDENCIES.  DEPENDENCIES is an alist with a
    human-readable name of the dependency as key. The second element
    of each member should be a form that evaluates to a non-nil value
    if the dependency is met.

    All `ert-deftest' forms in BODY are marked as expected to pass if
    all dependencies are met. Otherwise the expected result is set to
    `:failed' and the test's bodies modified to signal an error with
    an error message indicating the first failing dependency."
      (macrolet ((define-dependencies ()
                   `(cond
                     ,@(mapcar (lambda (dependency)
                                 `((not ,(second dependency)) ,(first 
dependency))) dependencies))))
        (let ((missing-dependency (define-dependencies)))
          `(progn
             ,@(mapcar (lambda (sexp)
                         (if (and (consp sexp)
                                  (eq (first sexp) 'ert-deftest))
                             (let* ((docstring (if (stringp (fourth sexp)) 
(list (fourth sexp))))
                                    (deftest-body (nthcdr (if docstring 4 3) 
sexp)))
                               `(,@(append (list (first sexp) (second sexp) 
(third sexp)) docstring)
                                 :expected-result ,@(if missing-dependency
                                                        `(:failed (error 
"Missing dependency: %s" ,missing-dependency))
                                                      '(:passed))
                                 ,@deftest-body))
                           sexp)) body)))))
#+end_src

You wrap it around ert-deftest forms, e.g.

#+begin_src emacs-lisp
(org-test-with-dependencies (("ESS" (featurep 'ess)))
  (ert-deftest test-ob-R/simple-session ()
   (org-test-with-temp-text
       "#+begin_src R :session R\n  paste(\"Yep!\")\n#+end_src\n"
     (should (string= "Yep!" (org-babel-execute-src-block))))))
#+end_src

#+begin_quote
ELISP> (macroexpand '(org-test-with-dependencies (("ESS" (featurep 'ess))) 
(ert-deftest foo () t)))
(progn
  (ert-deftest foo nil :expected-result :failed
               (error "Missing dependency: %s" "ESS")
               t))

ELISP>
#end_quote

If this is acceptable I'd push it and start to adjust the test
definitions.

Best,
  -- David
--
OpenPGP... 0x99ADB83B5A4478E6
Jabber.... address@hidden
Email..... address@hidden

pgp49n_cbVr5M.pgp
Description: PGP signature