emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
From: Max Nikulin <manikulin@gmail.com>
To: emacs-orgmode@gnu.org
Subject: Re: [PATCH] make test: Make failure results more verbose
Date: Fri, 21 Jan 2022 22:01:05 +0700	[thread overview]
Message-ID: <ssehrk$14pl$1@ciao.gmane.io> (raw)
In-Reply-To: <87h79xz015.fsf@localhost>

[-- Attachment #1: Type: text/plain, Size: 2078 bytes --]

On 21/01/2022 20:33, Ihor Radchenko wrote:
> Max Nikulin <manikulin@gmail.com> writes:
> 
>> I am attaching a tentative patch for Org that should make
>> EMACS_TEST_VERBOSE setting more transparent.
> 
> Thanks! Your patch looks cleaner.

In the meanwhile Lars fixed ERT in Emacs-29, so empty string is 
considered as false now, see https://debbugs.gnu.org/53313
Values like "no" should not be considered as false by Org makefiles, so 
I am attaching an updated version.

> In addition, I am attaching a tentative hack to make ERT pretty-print
> the failure reason as you wished. WDYT?

Sorry, I was not clear enough. Summary should remain single-line. There 
are pretty-printed failure reason in the main part of the log. I was 
complained concerning the following brick

>   signal(ert-test-failed (((should (equal "aB " (org-test-with-parsed-
>   ert-fail(((should (equal "aB " (org-test-with-parsed-data "* Headlin
>   (if (unwind-protect (setq value-5061 (apply fn-5059 args-5060)) (set
>   (let (form-description-5063) (if (unwind-protect (setq value-5061 (a
>   (let ((value-5061 'ert-form-evaluation-aborted-5062)) (let (form-des
>   (let* ((fn-5059 #'equal) (args-5060 (condition-case err (let ((signa
>   (closure (t) nil (let* ((fn-5059 #'equal) (args-5060 (condition-case
>   ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
>   ert-run-test(#s(ert-test :name test-org-export/get-previous-element 
>   ert-run-or-rerun-test(#s(ert--stats :selector "test-org-export/get-p
>   ert-run-tests("test-org-export/get-previous-element" #f(compiled-fun
>   ert-run-tests-batch("test-org-export/get-previous-element")
>   ert-run-tests-batch-and-exit("test-org-export/get-previous-element")
>   (let ((org-id-track-globally t) (org-test-selector (if org-test-sele
>   org-test-run-batch-tests("test-org-export/get-previous-element")
>   eval((org-test-run-batch-tests org-test-select-re) t)
>   command-line-1(("-L" "/home/ubuntu/ert" "--eval" "(setq vc-handled-b
>   command-line()
>   normal-top-level()

However it will be too long when pretty printed.

[-- Attachment #2: 0001-make-test-Make-failure-summary-more-verbose.patch --]
[-- Type: text/x-patch, Size: 1365 bytes --]

From a53095fa1c6679376708dcc1a05a446c3967d914 Mon Sep 17 00:00:00 2001
From: Max Nikulin <manikulin@gmail.com>
Date: Sat, 15 Jan 2022 22:54:30 +0700
Subject: [PATCH] make test: Make failure summary more verbose

* mk/default.mk: By default enable verbose failure summary for Emacs-28
or newer.

Set or unset EMACS_TEST_VERBOSE environment to control reporting of failure
reasons in summary since in Emacs-28 ERT switches to verbose mode even by
an empty string (fixed in Emacs-29).
---
 mk/default.mk | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/mk/default.mk b/mk/default.mk
index c8a15bdd2..804089280 100644
--- a/mk/default.mk
+++ b/mk/default.mk
@@ -32,6 +32,20 @@ TMPDIR ?= /tmp
 testdir = $(TMPDIR)/tmp-orgtest
 
 # Configuration for testing
+# Verbose ERT summary by default for Emacs-28 and above.
+# To override:
+# - Add to local.mk
+#   EMACS_TEST_VERBOSE =
+# - Export EMACS_TEST_VERBOSE environment variable with empty value
+# - Run tests as
+#   EMACS_TEST_VERBOSE= make test [OTHER_ARGUMENTS...]
+#   or as
+#   make test EMACS_TEST_VERBOSE= [OTHER_ARGUMENTS...]
+EMACS_TEST_VERBOSE ?= yes
+ifeq (,$(EMACS_TEST_VERBOSE))
+# Emacs-28 considers empty value as true, fixed in Emacs-29
+unexport EMACS_TEST_VERBOSE
+endif
 # add options before standard load-path
 BTEST_PRE   =
 # add options after standard load path
-- 
2.25.1


  reply	other threads:[~2022-01-21 15:02 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-01-02 13:12 Ihor Radchenko
2022-01-03  5:35 ` Max Nikulin
2022-01-07 15:04   ` Ihor Radchenko
2022-01-11 16:46     ` Max Nikulin
2022-01-15 12:52       ` Max Nikulin
2022-01-15 13:06         ` Max Nikulin
2022-01-15 15:58           ` Max Nikulin
2022-01-21 13:33             ` Ihor Radchenko
2022-01-21 15:01               ` Max Nikulin [this message]
2022-01-23 13:31                 ` Ihor Radchenko
2022-01-21 13:31         ` Ihor Radchenko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.orgmode.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='ssehrk$14pl$1@ciao.gmane.io' \
    --to=manikulin@gmail.com \
    --cc=emacs-orgmode@gnu.org \
    --subject='Re: [PATCH] make test: Make failure results more verbose' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Code repositories for project(s) associated with this inbox:

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).