From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 2Jb+D6bC6mGT9QAAgWs5BA (envelope-from ) for ; Fri, 21 Jan 2022 15:26:46 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id kE0zDabC6mFmRAAA9RJhRA (envelope-from ) for ; Fri, 21 Jan 2022 15:26:46 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id DA9032E1DF for ; Fri, 21 Jan 2022 15:26:45 +0100 (CET) Received: from localhost ([::1]:35784 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nAusG-0000FU-FP for larch@yhetil.org; Fri, 21 Jan 2022 09:26:44 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37740) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nAtyK-0001T5-G9 for emacs-orgmode@gnu.org; Fri, 21 Jan 2022 08:28:57 -0500 Received: from [2a00:1450:4864:20::12b] (port=46732 helo=mail-lf1-x12b.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nAtyH-000401-DX for emacs-orgmode@gnu.org; Fri, 21 Jan 2022 08:28:56 -0500 Received: by mail-lf1-x12b.google.com with SMTP id d3so33810383lfv.13 for ; Fri, 21 Jan 2022 05:28:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=35eehEhlBes51TJuAz3sr7OkNOXexAAX0/WI1e1snYk=; b=RgWBhWrb54YizohTBDM8gU1RH7jsZlQfkDT4MW4NcqWrl9msVakjHAMYvAE86b0P/c kbxVyqxrQLU2VCfjAvI9QfZN05f7tQ4Grwa3Fkiaexki8c0q8YDjucTTydMkedXSx78d 3+9GkaYvoCiczFlwVTO5WFvjk2+YCjfBd9pX2t34O5V2GNGpec9RioH+S/O77oQIx7Qg Ar7DZl6njIsCbPyhQMtGpAtji+5xUq4jfVGF720N4f0cyJYdRUugN9PF15qxpQwD1C9i cIbc4SvsmSBpAwSxFprEMwgxNDvy2oeGGWJGhpKZFKt8TBGBA+bt4Hy34+e2tk49nRQI 4arQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=35eehEhlBes51TJuAz3sr7OkNOXexAAX0/WI1e1snYk=; b=BkTjGDg0YF2ibIV6ahWDoVUUqqcJALNUslv5Rgpfh1dzrJQWRoU4c4zYcxDts0rF+Z ZBdYLQ6sNABNmudSGu2Rn0y3QnUzxgTx/mT/qijCIULDoBnw3188EBheMOmcpvS9c6gB YttegpMr3rRed3gAVBXz9P/RNUjjKiHqUnbgl6W9vwLLHSgWjxsy3wladsTqBLLy3XVu DS+WfhgaJhMg2W5WOLf8exc7ozy/pqHDy0+M9HMhp50wPSK5zZsbdmA8FeTDcmDEMUUt e0m096hlRTx7wy+DL7l5tEJ53fkoYcKhthKQdBuDnldvv9iGlSg5mXh4BKcDSamstZAI 9kHA== X-Gm-Message-State: AOAM533EQmn2+GLwGM6jZbWKgtX0MsBNfrM8yVyp6jIxlRijqEEGa8Oj J+MsYdQ19aepCsg3oZxZAiA= X-Google-Smtp-Source: ABdhPJwn1hvRlGb1DbsKVzHXcTsTN20c5QQeVnFVJ5dC0MAgJTyG5EaQmdw73fOZ4LHyQ60VAuODOA== X-Received: by 2002:a05:6512:348f:: with SMTP id v15mr3672227lfr.498.1642771728312; Fri, 21 Jan 2022 05:28:48 -0800 (PST) Received: from localhost ([141.105.67.194]) by smtp.gmail.com with ESMTPSA id m9sm252985lfk.233.2022.01.21.05.28.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jan 2022 05:28:47 -0800 (PST) From: Ihor Radchenko To: Max Nikulin Subject: Re: [PATCH] make test: Make failure results more verbose In-Reply-To: References: <87ee5q5lic.fsf@localhost> <87lezrr3i7.fsf@localhost> Date: Fri, 21 Jan 2022 21:33:10 +0800 Message-ID: <87h79xz015.fsf@localhost> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::12b (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::12b; envelope-from=yantar92@gmail.com; helo=mail-lf1-x12b.google.com X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: emacs-orgmode@gnu.org Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1642775205; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=727WRQkexBad7f5YU0tPJMimbiAJrMFlyqsiIa9o0xI=; b=SRHaCRCOjDVLYrF3OM7odNhi9gibjyMC26zwdZVN6St9ydE1G0zlc4TDlxGwchGnrqIEnL MMy8x4OHUZwpLScx26jee/s6kWmzBKYaPO1ubm2ctJVUQ3+CiYf9wtDI/0Z++0mqUwhdMY rndCXcPoyzRRyuyjJjys3ioiCstwhJOLMoBu0OzsH9CZfpSR2II4DbVHZAP/AB3OfwF/JW oYQ6MF1c+AurB319n6ybIEJJivQW48LEK6ACvxT2MB2GESy9d5qmiKYDix8Hy9+Dkxwu0x XTw8+y9mUItGtKw8GLbGzvH4rNkaS4tXBTp1U4vqsi7ik50s961n+79ODfKyIQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1642775205; a=rsa-sha256; cv=none; b=NuguGmHH7U4Iocz578pZ7tTPVLOD0RGjZAZlH+H9tSowEY0iax+K9iWM07q+HPYrlmTyBg KBZNWAARMBDNGYr/RxPf94Z58C6/eSaRBeFdgV1UXD5IHFrAjVThsVPA0PXiVvuWMgSxRZ +Kaj8LxAOu5JQINK+LHMmfpkXKSknuIAkfL+WdWNDi5kldD6PKcCojIdgAdec7VQ+2TUX4 La+r9BpEusV78Z3yEX7Qd4mMavPQ03uEb/v03sd/LMAP4HqcRXUJdngy7OcfG2Oq3E71m9 ymQfm5WyMrWtkbnj/HkSRmlsaFB0Uk84Ty1Xfb6MLvrXwpg0tIbmkG7qFuJ8kA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=gmail.com header.s=20210112 header.b=RgWBhWrb; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -1.52 Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=gmail.com header.s=20210112 header.b=RgWBhWrb; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: DA9032E1DF X-Spam-Score: -1.52 X-Migadu-Scanner: scn1.migadu.com X-TUID: 85iPAQAJQ0l5 --=-=-= Content-Type: text/plain Max Nikulin writes: > I am attaching a tentative patch for Org that should make > EMACS_TEST_VERBOSE setting more transparent. Thanks! Your patch looks cleaner. In addition, I am attaching a tentative hack to make ERT pretty-print the failure reason as you wished. WDYT? Though it may also be something to consider in Emacs core. Best, Ihor --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-make-test-Pretty-print-failure-reason-in-test-summar.patch >From 1a8520b9cb2883672b1a6d2385d539a158658890 Mon Sep 17 00:00:00 2001 Message-Id: <1a8520b9cb2883672b1a6d2385d539a158658890.1642771846.git.yantar92@gmail.com> From: Ihor Radchenko Date: Fri, 21 Jan 2022 21:30:21 +0800 Subject: [PATCH] make test: Pretty print failure reason in test summary --- testing/org-test.el | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/testing/org-test.el b/testing/org-test.el index 0f1e254aa..a00007b6b 100644 --- a/testing/org-test.el +++ b/testing/org-test.el @@ -344,6 +344,13 @@ (defun org-test-strip-text-props (s) (let ((noprop (copy-sequence s))) (set-text-properties 0 (length noprop) nil noprop) noprop)) + +(require 'cl-macs) +(defun org-test-ert-reason-for-test-result:pretty-func (func result) + "Call `ert-reason-for-test-result' (as FUNC(RESULT)). +Pretty-print the failure reason." + (cl-letf (((symbol-function 'format) (lambda (_ reason) (concat "\n" (pp-to-string reason))))) + (funcall func result))) (defun org-test-string-exact-match (regex string &optional start) @@ -435,7 +442,11 @@ (defun org-test-run-batch-tests (&optional org-test-selector) (org-test-update-id-locations) (org-test-load) (message "selected tests: %s" org-test-selector) - (ert-run-tests-batch-and-exit org-test-selector))) + (advice-add 'ert-reason-for-test-result :around + #'org-test-ert-reason-for-test-result:pretty-func) + (ert-run-tests-batch-and-exit org-test-selector) + (advice-remove 'ert-reason-for-test-result + #'org-test-ert-reason-for-test-result:pretty-func))) (defun org-test-run-all-tests () "Run all defined tests matching \"\\(org\\|ob\\)\". -- 2.34.1 --=-=-=--