From: Ihor Radchenko <yantar92@posteo.net>
To: gerard.vermeulen@posteo.net
Cc: Emacs orgmode <emacs-orgmode@gnu.org>
Subject: Re: [PATCH] Patches for problems discovered with interactive testing
Date: Fri, 19 Jan 2024 12:50:19 +0000 [thread overview]
Message-ID: <874jf9pjw4.fsf@localhost> (raw)
In-Reply-To: <7815c5360c35a1794db076a646518035@posteo.net>
gerard.vermeulen@posteo.net writes:
> This patch addresses a number of glitches discovered during interactive
> testing and contains possible fixes (sub-patch test-ob-maxima fixes
> batch testing on my system, but not interactive testing).
Thanks for your interest in improving Org mode tests!
> The test-ob-maxima patch may be due to a different Maxima version, but
> I do not think so. In batch testing test-ob-maxima passes contrary to
> interactive mode.
On my side, Maxima tests pass both interactively and non-interactively.
I use Maxima 5.47.0
What is your version?
> The test-ob-python patch unsets org-src-preserve-indentation which I
> have set.
>
> The test-ob patch also unsets org-src-preserve-indentation.
> The test-org patch also unsets org-src-preserve-indentation. This
> reduces the number of failures to 2 while testing test-org
> interactively.
It is generally not expected that Org tests will pass when you try to
run them on Emacs instance with your personal config. This is because
you can have arbitrary customizations, hooks, or advices that change the
Org mode defaults.
let-binding a couple of variables cannot fix this general problem for
all scenarios. So, I do not see any reason to accept patches that make
the tests run with personal config of a single user.
What might be useful, however, is making sure that tests do not fail
interactively with a clean Emacs config (make repro).
Check out testing/README file.
> (ert-deftest ob-maxima/string-input ()
> "Test of string input"
> (org-test-at-id "b5842ed4-8e8b-4b18-a1c9-cef006b6a6c8"
> (org-babel-next-src-block 2)
> - (should (equal "- sin(x)" (org-babel-execute-src-block)))))
> + (should (equal '(("(linenum:0," "") ("-" "sin(x)"))
> + (org-babel-execute-src-block)))))
The test in question is testing the output of
#+begin_src maxima :var fun="sin(x)" :var q=2 :results silent
print(diff(fun, x, q))$
#+end_src
It is certainly not expected that such code block returns "linenum:0".
Tests in Org mode are verifying that Org mode behaves correctly. It
makes no sense to change the test when Org mode result is not correct -
we should instead fix Org mode.
In the majority of cases, if the test is failing, it is not something
wrong with the test, but something wrong with Org mode code that test is
verifying.
> (ert-deftest test-ob/does-not-replace-a-block-with-the-results ()
> (org-test-with-temp-text "#+NAME: foo
> diff --git a/testing/lisp/test-org-element.el b/testing/lisp/test-org-element.el
> index ca7d77e28..f0958ff59 100644
> --- a/testing/lisp/test-org-element.el
> +++ b/testing/lisp/test-org-element.el
> @@ -4457,7 +4457,7 @@ Text
> '(paragraph nil " Two spaces\n\n \n Two spaces"))
> '(paragraph nil "Two spaces\n\n\nTwo spaces")))
> (should
> - (equal
> + (equal ;; Fails in test. Why? (equal ...) form works in ielm.
It would help if you provided the error ERT displayed when the test failed.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
next prev parent reply other threads:[~2024-01-19 12:47 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-19 9:48 [PATCH] Patches for problems discovered with interactive testing gerard.vermeulen
2024-01-19 12:50 ` Ihor Radchenko [this message]
2024-01-19 14:45 ` gerard.vermeulen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=874jf9pjw4.fsf@localhost \
--to=yantar92@posteo.net \
--cc=emacs-orgmode@gnu.org \
--cc=gerard.vermeulen@posteo.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).