From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id yIwmDGaLqmWePgEA62LTzQ:P1 (envelope-from ) for ; Fri, 19 Jan 2024 15:47:02 +0100 Received: from aspmx1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id yIwmDGaLqmWePgEA62LTzQ (envelope-from ) for ; Fri, 19 Jan 2024 15:47:02 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=luLZ1JCC; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1705675622; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=2XGvlD6AYTlRcKKi+C/oEfo6xRg5EHlyX6eUyiQ6tvk=; b=iuYw+ejOm60p3BTICKg1RlYcHjTyo6xUb7XlbX1uhv/DZdqDXyDvQwhIy+Jk4helKlC/x5 NhlF4K40aNLzx82Bgil3zrGojd0WwPZEx/z5fGyuoXbFrcDwT55fPfP1OxyrJ0XlGTxdwA a79e+sFI5nAQBMuqpcmX3JyTxwzxS0Q5qJzHZkfxaA3Bmwv3SzcdWJHZ9P3wHa5Wi0rEVz /62lcjinF94HOAbCOCWMGBvLBmLqXMpPSjmsYNvo6KphqAOQyrlfzMwxJ6/xVhFQ+3D45I Mdp2rRr7MsR64ayzlsEGLf6aA9xF9A5kN0FfOFMo5XY/Ujr7cKvQRhUHnvUFGw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=luLZ1JCC; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1705675622; a=rsa-sha256; cv=none; b=KIa2xeTeWH7uFrQ81uXumraZjFnZm8vtavxR9xPB4INjeVsMBDvE6d0aThRRW+OfjjV+qj 2o1KD6klUE9cXyasdjDSlmTccUywZV/biz4wqoqcdiEmKEYd36XiYjvjpGpqFp+2/dJWDq I91IObffDVPdlY95w9HLrXORDNTHq/kw7Uec/vzgT3TsinjL/hkaKWnxsJwPtDR2yZ+/Qx /ANHqtHynbUR0e+M3zOHSoPScENHbzX5xwp9XjcEOuuPVqgl6L3reJ7+qtF/a6gIJTN02A RqbkGNO/c9e1ZeCg5LAihmyQ21ZzO9fDfAmPiSDhc1/FbPSh1i1TpFFJHMLRKw== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id F245E1B0F5 for ; Fri, 19 Jan 2024 15:47:01 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rQq8k-0005qR-0n; Fri, 19 Jan 2024 09:46:39 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQq82-0005W3-3j for emacs-orgmode@gnu.org; Fri, 19 Jan 2024 09:46:01 -0500 Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQq7z-0006x8-FP for emacs-orgmode@gnu.org; Fri, 19 Jan 2024 09:45:53 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id ED063240101 for ; Fri, 19 Jan 2024 15:45:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1705675549; bh=+UHpOHVVhrNG7Xz8uj+U3k6gf9TBnkfKRxacqElJrEI=; h=MIME-Version:Content-Type:Content-Transfer-Encoding:Date:From:To: Cc:Subject:Message-ID:From; b=luLZ1JCCs2PzCoxXWaGXrQ6VRQGhHd1T6VQd7hbwLgic/cVgpVwJrEvZ3BNorwAFJ a/WBmDRCCddkZ0fcg8gzlJ9aaZjUOuV67Y9U50QsMK2KMh0W08Ddk5hPw7gWSnt31S RIaAcE9DMK69aYvhBzbgTPEopKzI7X6zYswxWpaH6Heg5kzEPVuzBY4R3dVdkr2Lui RLpuasf4Gu14hndxTsrFYWnFy1OQs+c3yf2SS4VyT//czCcJa8YyzBCXTv1V9OG+qT i8C0X4sB/sZFgAU3thsK+c54e9DYpLYYoSURRFZ6qw9Tvk2f0FHBFXo2AT+hQSaSIO l/5oS3o/+MaWQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4TGj8r1sc4z9rxB; Fri, 19 Jan 2024 15:45:48 +0100 (CET) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 19 Jan 2024 14:45:48 +0000 From: gerard.vermeulen@posteo.net To: Ihor Radchenko Cc: Emacs orgmode Subject: Re: [PATCH] Patches for problems discovered with interactive testing In-Reply-To: <874jf9pjw4.fsf@localhost> References: <7815c5360c35a1794db076a646518035@posteo.net> <874jf9pjw4.fsf@localhost> Message-ID: Received-SPF: pass client-ip=185.67.36.66; envelope-from=gerard.vermeulen@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -10.40 X-Spam-Score: -10.40 X-Migadu-Queue-Id: F245E1B0F5 X-Migadu-Scanner: mx12.migadu.com X-TUID: hNrYLcPQ06zc On 19.01.2024 13:50, Ihor Radchenko wrote: > gerard.vermeulen@posteo.net writes: > >> This patch addresses a number of glitches discovered during >> interactive >> testing and contains possible fixes (sub-patch test-ob-maxima fixes >> batch testing on my system, but not interactive testing). > > Thanks for your interest in improving Org mode tests! > >> The test-ob-maxima patch may be due to a different Maxima version, but >> I do not think so. In batch testing test-ob-maxima passes contrary to >> interactive mode. > > On my side, Maxima tests pass both interactively and non-interactively. > I use Maxima 5.47.0 > What is your version? The same: maxima --version Maxima 5.47.0 In batch testing without the patch I get: Test ob-maxima/batch+verbatim condition: (ert-test-failed ((should (equal (org-babel-execute-src-block) "(assume(z > 0),integrate(exp(-t)*t^z,t,0,inf))\n gamma(z + 1)")) :form (equal "(linenum:0,\n(assume(z > 0),integrate(exp(-t)*t^z,t,0,inf))\n gamma(z + 1)" "(assume(z > 0),integrate(exp(-t)*t^z,t,0,inf))\n gamma(z + 1)") :value nil :explanation (arrays-of-different-length 104 92 "(linenum:0,\n(assume(z > 0),integrate(exp(-t)*t^z,t,0,inf))\n gamma(z + 1)" "(assume(z > 0),integrate(exp(-t)*t^z,t,0,inf))\n gamma(z + 1)" first-mismatch-at 1))) FAILED 162/1252 ob-maxima/batch+verbatim (0.694916 sec) at ../lisp/test-ob -maxima.el:71 I do not know where the difference comes from. I am not a Maxima user and not inclined to track this down unless you tell me it is important. > >> The test-ob-python patch unsets org-src-preserve-indentation which I >> have set. >> >> The test-ob patch also unsets org-src-preserve-indentation. > >> The test-org patch also unsets org-src-preserve-indentation. This >> reduces the number of failures to 2 while testing test-org >> interactively. > > It is generally not expected that Org tests will pass when you try to > run them on Emacs instance with your personal config. This is because > you can have arbitrary customizations, hooks, or advices that change > the > Org mode defaults. > > let-binding a couple of variables cannot fix this general problem for > all scenarios. So, I do not see any reason to accept patches that make > the tests run with personal config of a single user. > Before I made the patch, I had reverted changes that worked interactively in my personal config but caused failures with make test. Because of the Maxima test failure (also in batch), I was not sure that all tests pass on a reference system. > What might be useful, however, is making sure that tests do not fail > interactively with a clean Emacs config (make repro). > Check out testing/README file. > I have a super-weird system where make repro opens Emacs with as default-directory my home directory. So I have to make symbolic links from my home to the testing and lisp directories. But make test works. Because of testing my 0001-org-babel-demarcate-block-split-using-element-API.patch interactively in my personal config (before make test which takes quite some time), I figured out that it would guess to suppress false failures. > >> (ert-deftest ob-maxima/string-input () >> "Test of string input" >> (org-test-at-id "b5842ed4-8e8b-4b18-a1c9-cef006b6a6c8" >> (org-babel-next-src-block 2) >> - (should (equal "- sin(x)" (org-babel-execute-src-block))))) >> + (should (equal '(("(linenum:0," "") ("-" "sin(x)")) >> + (org-babel-execute-src-block))))) > > The test in question is testing the output of > > #+begin_src maxima :var fun="sin(x)" :var q=2 :results silent > print(diff(fun, x, q))$ > #+end_src > > It is certainly not expected that such code block returns "linenum:0". I would say that it is an unexplained difference, I still get (with "linenum:0") : #+header: :wrap "src text -n" #+name: lst:maxima-example #+begin_src maxima -n :batch batch :cmdline --quiet :exports both :results raw rat(1/(x+1) + x/(x-1)); #+end_src #+name: lst:maxima-example-results #+results: lst:maxima-example #+begin_src text -n (%i1) (linenum:0, (%i1) rat(1/(x+1)+x/(x-1)) 2 x + 2 x - 1 (%o1)/R/ ------------ 2 x - 1 #+end_src > > Tests in Org mode are verifying that Org mode behaves correctly. It > makes no sense to change the test when Org mode result is not correct - > we should instead fix Org mode. > > In the majority of cases, if the test is failing, it is not something > wrong with the test, but something wrong with Org mode code that test > is > verifying. > >> (ert-deftest test-ob/does-not-replace-a-block-with-the-results () >> (org-test-with-temp-text "#+NAME: foo >> diff --git a/testing/lisp/test-org-element.el >> b/testing/lisp/test-org-element.el >> index ca7d77e28..f0958ff59 100644 >> --- a/testing/lisp/test-org-element.el >> +++ b/testing/lisp/test-org-element.el >> @@ -4457,7 +4457,7 @@ Text >> '(paragraph nil " Two spaces\n\n \n Two spaces")) >> '(paragraph nil "Two spaces\n\n\nTwo spaces"))) >> (should >> - (equal >> + (equal ;; Fails in test. Why? (equal ...) form works in ielm. > > It would help if you provided the error ERT displayed when the test > failed. I cannot reproduce it. Now it works in interactive testing (personal config as when I first saw this). If you ask me, I am still willing to collect the 2 sub-patches with the let bindings which improve testing interactively in personal configs somewhat. If not, it is OK (I understand now that it is not important). Regards -- Gerard