From: Ben Alexander <bva@alexanderonline.org>
To: Richard Riley <rileyrgdev@googlemail.com>
Cc: emacs-orgmode Org-Mode <emacs-orgmode@gnu.org>
Subject: Re: Re: How you can help
Date: Thu, 23 Oct 2008 17:22:22 +0100 [thread overview]
Message-ID: <024FF120-2C9B-449D-A2FA-279D53A47892@alexanderonline.org> (raw)
In-Reply-To: <um63njqpk0.fsf@development.richardriley.net>
On 2008-Oct-23, at 16:49, Richard Riley wrote:
> Sebastian Rose <sebastian_rose@gmx.de> writes:
>
>> Bernt Hansen <bernt@norang.ca> writes:
>>> Running a minimal emacs should suppress custom config files:
>>> emacs -q -l yourtest.el
>>
>> Added this one to the Clippboard section on new org-tests/index.org
>> in
>> Worg.git. (this section will be temporary...)
>
> Something like the above should only be a link (at most) to the emacs
> manual. Reproducing standard info is bad in the long run in case
> things
> in the base product (emacs) change for example.
>
>>
>>> Some kind of regression testing framework would be awesome. Org-
>>> mode is
>>> large enough that this is almost a necessity to keep things stable
>>> and
>>> bug-free.
>>
>> It's big and feature-RICH.
>
> The nature of OSS means that the community using the product keep it
> stable and bug free. I dont think the efforts to produce meaningful
> regression tests would be beneficial in an ever morphing product like
> org-mode. Clearly my humble opinion on that one :-;
>
I'm a bit clue-free on what the words mean. I think of regression
tests as ways of defining the way "Things Should Work(TM)" so if
someone commits as fix to org-mode that works fine on Debian, but
breaks on Windows, there's a way for someone on Windows to easily
(brainlessly) run the test and report back to the developer.
I don't think of org-mode as 'ever morphing'. There are some features
that are stable, and should remain so. A regression test would be
valuable, presumably, for developers working on a new feature (don't
screw up important existing functionality!)
>
> I must say I am dubious about this. It means, for the tests to be
> meaningful, that the output must be a fixed format in base org. I
> doubt
> this will ever be the case. The presentation will fluctuate while the
> core information (dates, schedules periods etc) will remain pretty
> much
> constant.
>
I agree that the presentation is fluid and it was hard for me to
imagine how to test it in an automated way. But I disagree (I think)
in that having a test on the presentation is extremely powerful.
I'm thinking that one example of how to write such a test could be
copied when someone is trying to explain what they want to happen, and
how it is not happening right now.
Perhaps such a test would have a very limited useful life. I'd like
to think that it could live and be useful for a very long time (that's
why thinking about configuration/user customization issues in a test
is important). But even if the test is only useful until the bug is
fixed (or the feature implemented) , so be it.
> The majority of bugs that I see are often down to people misusing or
> using things in the base which are not fully explored. No amount of
> regression testing can cover things like that unless the regression
> tests include everyones customisations.
>
Not everyone's customizations, just every customization org-mode has.
I mean, if the feature is there, it should work. If the feature is
important, it should work in the same way on every platform.
The trick -- and I hope it's either solved or easy -- is exactly how
to show that "the feature works the same way as it should, right now,
on this platform"
If I understand you properly, you're just saying that the user
customization creates so much variability in the output, there isn't
anyway you could capture all the possible presentations in a single
test. Well, yeah, so user customizations need to be left out of the
test case.
Except that in the scenario I'm thinking of where someone (a mere
mortal using org-mode, with limited lisp knowledge) is trying to
report a bug, user customization will be critical. My first thought
was asking org-mode to dump all the customization variables it has
into a buffer (as buffer-local variables maybe?) would help someone
else recreate the bug. (By starting org-mode with NO customizations,
except what's found in the test case)
Or are you suggesting that the source of presentation variation is due
mostly to OS, Emacs version, or something else beyond the scope of our
tests, so they couldn't be replicated on another machine?
> Do I think regression testing is important? Yes - in certain
> environments. But every time Carsten, you, myself or anyone else fires
> up org-mode we are already doing just that.
>
>
> --
> Inventor: A person who makes an ingenious arrangement of wheels,
> levers and springs, and believes it civilization. ~Ambrose Bierce,
> The Devil's Dictionary
next prev parent reply other threads:[~2008-10-23 16:22 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-10-23 12:04 How you can help Ben Alexander
2008-10-23 13:43 ` Bernt Hansen
2008-10-23 15:04 ` Sebastian Rose
2008-10-23 15:49 ` Richard Riley
2008-10-23 16:22 ` Ben Alexander [this message]
2008-10-23 17:02 ` Sebastian Rose
2008-10-24 12:13 ` Richard Riley
2008-10-24 15:39 ` Sebastian Rose
2008-10-24 16:27 ` Manish
2008-10-24 18:41 ` Avdi Grimm
2008-10-23 19:13 ` Avdi Grimm
2008-10-24 12:19 ` Richard Riley
2008-10-23 16:19 ` Bernt Hansen
2008-10-24 5:05 ` Carsten Dominik
2008-10-23 17:01 ` Jason F. McBrayer
2008-10-23 23:46 ` Eric Schulte
2008-10-23 14:20 ` Sebastian Rose
2008-10-23 14:50 ` Manish
2008-10-23 15:46 ` Eric Schulte
2008-10-23 16:18 ` Avdi Grimm
2008-10-23 14:55 ` Ben Alexander
2008-10-23 16:26 ` Sebastian Rose
2008-10-23 16:42 ` Avdi Grimm
2008-10-23 17:33 ` Sebastian Rose
2008-10-23 19:10 ` Avdi Grimm
2008-10-24 21:09 ` Tom Breton (Tehom)
2008-10-24 18:33 ` Ben Alexander
2008-10-24 18:44 ` Avdi Grimm
2008-10-24 19:02 ` Jeff Mickey
2008-10-26 19:49 ` org-cycle broken when cursor is at ellipses Ben Alexander
2008-10-26 21:31 ` Cameron Horsburgh
2008-10-27 8:47 ` Carsten Dominik
2008-10-27 8:47 ` Carsten Dominik
[not found] ` <D43ED86C-EFD4-4BA8-8528-4F82DB11D625@alexanderonline.org>
2008-10-23 17:12 ` How you can help Sebastian Rose
2008-10-23 15:08 ` Sebastian Rose
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=024FF120-2C9B-449D-A2FA-279D53A47892@alexanderonline.org \
--to=bva@alexanderonline.org \
--cc=emacs-orgmode@gnu.org \
--cc=rileyrgdev@googlemail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).