From: Martyn Jago <martyn.jago@btinternet.com>
To: emacs-orgmode@gnu.org
Subject: Re: [RFC] testing the org exporters
Date: Mon, 4 Jul 2011 09:39:44 +0000 (UTC) [thread overview]
Message-ID: <loom.20110704T113754-3@post.gmane.org> (raw)
In-Reply-To: 87k4c1lpx9.fsf@gmail.com
Hi Jambunathan
>> I have been using the attached test.org file as part of my org-odt
>> related work.
>>
>> The usual steps I follow for testing are quite simple:
>> 1. Visit the file
>> 2. Export it to xhtml or odt
>> 3. Visually make sure that everything is OK.
>>
>> I would like to add this file to the testing dir of the trunk. This file
>> would be a good starting point for not only validating but also
>> documenting (the quirks?) of various exporters.
>>
>> At the moment, I am not very inclined to have it integrated with
>> ert[1]. Based on my experience, the standalone test.org file has proven
>> it's worth multiple times over during the development phase.
>>
>> What do you think? I am happy to hear feedback on
>>
>> 1. what to put in that test.org
>> 2. where to put the test.org
>> 3. how to have it integrated with test framework.
>>
>> Footnotes:
>> [1] This is mostly because I don't know nothing about ert.
>
>Hi Jambunathan,
>
>I would very much like to see this test file [1] integrated into the testing
>repository of org-mode in (testing/examples) _and_ integrated into
>Org-mode's ert test framework.
>
>ERT is easy to use, and will more than repay any time spent applying it
>to your test file through the ability to quickly programmatically and
>reproducibly evaluate export results. There are many tests [2] which
>already implement a
>1. visit example file
>2. export to backend
>3. confirm properties of backend (e.g., does or does not
>contain some string)
>
>Please try running the Org-mode test suite and look at the examples in
>testing/lisp. I see no reason not to take this final step and begin
>writing ERT tests which can serve as both documentation of issues and
>insurance against regression.
I also think that this is a great idea.
In fact I have recently discovered that the LilyPond
(Music Score Engravement) community (gnu) do something like this to
ensure their high-quality rendering is not regressively broken or degraded.
AFAICR they generate visual snippets by continuous integration, and have
a team of (non-programmer but heavy user) contributers analyse the visual
results in order to report any regressions to the development team. These
tests are performed weekly, and are intended to take no more than 20 minutes
per week of each quality-assurance members time. Sign-offs happen once per
week.
I also agree with you that this could aid in providing highly maintained
visual snippets (with simple source) for the provision of examples for
documentation purposes. And even where ert can be configured to do the
actual testing, it may be nice to generate the visual result for
documentation purposes.
Perhaps this visual output could live in worg?
Being a user of ert I would be happy helping to set this up if the community
thought it would be of use.
( just my 10 pence ;)
Regards
Martyn
prev parent reply other threads:[~2011-07-04 9:40 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-07-01 12:25 [RFC] testing the org exporters Jambunathan K
2011-07-01 20:08 ` Eric Schulte
2011-07-04 9:39 ` Martyn Jago [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=loom.20110704T113754-3@post.gmane.org \
--to=martyn.jago@btinternet.com \
--cc=emacs-orgmode@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).