Re: [PATCH] make test: Make failure results more verbose

[Ihor Radchenko <yantar92@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 368 bytes --]

Max Nikulin <manikulin@gmail.com> writes:

> I am attaching a tentative patch for Org that should make 
> EMACS_TEST_VERBOSE setting more transparent.

Thanks! Your patch looks cleaner.
In addition, I am attaching a tentative hack to make ERT pretty-print
the failure reason as you wished. WDYT?

Though it may also be something to consider in Emacs core.

Best,
Ihor

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-make-test-Pretty-print-failure-reason-in-test-summar.patch --]
[-- Type: text/x-patch, Size: 1745 bytes --]

From 1a8520b9cb2883672b1a6d2385d539a158658890 Mon Sep 17 00:00:00 2001
Message-Id: <1a8520b9cb2883672b1a6d2385d539a158658890.1642771846.git.yantar92@gmail.com>
From: Ihor Radchenko <yantar92@gmail.com>
Date: Fri, 21 Jan 2022 21:30:21 +0800
Subject: [PATCH] make test: Pretty print failure reason in test summary

---
 testing/org-test.el | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/testing/org-test.el b/testing/org-test.el
index 0f1e254aa..a00007b6b 100644
--- a/testing/org-test.el
+++ b/testing/org-test.el
@@ -344,6 +344,13 @@ (defun org-test-strip-text-props (s)
   (let ((noprop (copy-sequence s)))
     (set-text-properties 0 (length noprop) nil noprop)
     noprop))
+
+(require 'cl-macs)
+(defun org-test-ert-reason-for-test-result:pretty-func (func result)
+  "Call `ert-reason-for-test-result' (as FUNC(RESULT)).
+Pretty-print the failure reason."
+  (cl-letf (((symbol-function 'format) (lambda (_ reason) (concat "\n" (pp-to-string reason)))))
+    (funcall func result)))
 \f
 
 (defun org-test-string-exact-match (regex string &optional start)
@@ -435,7 +442,11 @@ (defun org-test-run-batch-tests (&optional org-test-selector)
     (org-test-update-id-locations)
     (org-test-load)
     (message "selected tests: %s" org-test-selector)
-    (ert-run-tests-batch-and-exit org-test-selector)))
+    (advice-add 'ert-reason-for-test-result :around
+                #'org-test-ert-reason-for-test-result:pretty-func)
+    (ert-run-tests-batch-and-exit org-test-selector)
+    (advice-remove 'ert-reason-for-test-result
+                   #'org-test-ert-reason-for-test-result:pretty-func)))
 
 (defun org-test-run-all-tests ()
   "Run all defined tests matching \"\\(org\\|ob\\)\".
-- 
2.34.1