From: Max Nikulin <manikulin@gmail.com>
To: emacs-orgmode@gnu.org
Subject: Re: [PATCH] make test: Make failure results more verbose
Date: Fri, 21 Jan 2022 22:01:05 +0700 [thread overview]
Message-ID: <ssehrk$14pl$1@ciao.gmane.io> (raw)
In-Reply-To: <87h79xz015.fsf@localhost>
[-- Attachment #1: Type: text/plain, Size: 2078 bytes --]
On 21/01/2022 20:33, Ihor Radchenko wrote:
> Max Nikulin <manikulin@gmail.com> writes:
>
>> I am attaching a tentative patch for Org that should make
>> EMACS_TEST_VERBOSE setting more transparent.
>
> Thanks! Your patch looks cleaner.
In the meanwhile Lars fixed ERT in Emacs-29, so empty string is
considered as false now, see https://debbugs.gnu.org/53313
Values like "no" should not be considered as false by Org makefiles, so
I am attaching an updated version.
> In addition, I am attaching a tentative hack to make ERT pretty-print
> the failure reason as you wished. WDYT?
Sorry, I was not clear enough. Summary should remain single-line. There
are pretty-printed failure reason in the main part of the log. I was
complained concerning the following brick
> signal(ert-test-failed (((should (equal "aB " (org-test-with-parsed-
> ert-fail(((should (equal "aB " (org-test-with-parsed-data "* Headlin
> (if (unwind-protect (setq value-5061 (apply fn-5059 args-5060)) (set
> (let (form-description-5063) (if (unwind-protect (setq value-5061 (a
> (let ((value-5061 'ert-form-evaluation-aborted-5062)) (let (form-des
> (let* ((fn-5059 #'equal) (args-5060 (condition-case err (let ((signa
> (closure (t) nil (let* ((fn-5059 #'equal) (args-5060 (condition-case
> ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
> ert-run-test(#s(ert-test :name test-org-export/get-previous-element
> ert-run-or-rerun-test(#s(ert--stats :selector "test-org-export/get-p
> ert-run-tests("test-org-export/get-previous-element" #f(compiled-fun
> ert-run-tests-batch("test-org-export/get-previous-element")
> ert-run-tests-batch-and-exit("test-org-export/get-previous-element")
> (let ((org-id-track-globally t) (org-test-selector (if org-test-sele
> org-test-run-batch-tests("test-org-export/get-previous-element")
> eval((org-test-run-batch-tests org-test-select-re) t)
> command-line-1(("-L" "/home/ubuntu/ert" "--eval" "(setq vc-handled-b
> command-line()
> normal-top-level()
However it will be too long when pretty printed.
[-- Attachment #2: 0001-make-test-Make-failure-summary-more-verbose.patch --]
[-- Type: text/x-patch, Size: 1365 bytes --]
From a53095fa1c6679376708dcc1a05a446c3967d914 Mon Sep 17 00:00:00 2001
From: Max Nikulin <manikulin@gmail.com>
Date: Sat, 15 Jan 2022 22:54:30 +0700
Subject: [PATCH] make test: Make failure summary more verbose
* mk/default.mk: By default enable verbose failure summary for Emacs-28
or newer.
Set or unset EMACS_TEST_VERBOSE environment to control reporting of failure
reasons in summary since in Emacs-28 ERT switches to verbose mode even by
an empty string (fixed in Emacs-29).
---
mk/default.mk | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/mk/default.mk b/mk/default.mk
index c8a15bdd2..804089280 100644
--- a/mk/default.mk
+++ b/mk/default.mk
@@ -32,6 +32,20 @@ TMPDIR ?= /tmp
testdir = $(TMPDIR)/tmp-orgtest
# Configuration for testing
+# Verbose ERT summary by default for Emacs-28 and above.
+# To override:
+# - Add to local.mk
+# EMACS_TEST_VERBOSE =
+# - Export EMACS_TEST_VERBOSE environment variable with empty value
+# - Run tests as
+# EMACS_TEST_VERBOSE= make test [OTHER_ARGUMENTS...]
+# or as
+# make test EMACS_TEST_VERBOSE= [OTHER_ARGUMENTS...]
+EMACS_TEST_VERBOSE ?= yes
+ifeq (,$(EMACS_TEST_VERBOSE))
+# Emacs-28 considers empty value as true, fixed in Emacs-29
+unexport EMACS_TEST_VERBOSE
+endif
# add options before standard load-path
BTEST_PRE =
# add options after standard load path
--
2.25.1
next prev parent reply other threads:[~2022-01-21 15:02 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-02 13:12 [PATCH] make test: Make failure results more verbose Ihor Radchenko
2022-01-03 5:35 ` Max Nikulin
2022-01-07 15:04 ` Ihor Radchenko
2022-01-11 16:46 ` Max Nikulin
2022-01-15 12:52 ` Max Nikulin
2022-01-15 13:06 ` Max Nikulin
2022-01-15 15:58 ` Max Nikulin
2022-01-21 13:33 ` Ihor Radchenko
2022-01-21 15:01 ` Max Nikulin [this message]
2022-01-23 13:31 ` Ihor Radchenko
2022-01-21 13:31 ` Ihor Radchenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='ssehrk$14pl$1@ciao.gmane.io' \
--to=manikulin@gmail.com \
--cc=emacs-orgmode@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).