From: Richard Riley <rileyrgdev@googlemail.com>
To: Sebastian Rose <sebastian_rose@gmx.de>
Cc: Bernt Hansen <bernt@norang.ca>,
emacs-orgmode@gnu.org, Richard Riley <rileyrgdev@googlemail.com>,
Ben Alexander <bva@alexanderonline.org>
Subject: Re: Re: How you can help
Date: Fri, 24 Oct 2008 14:13:09 +0200 [thread overview]
Message-ID: <okej262nu2.fsf@development.richardriley.net> (raw)
In-Reply-To: <87iqrj9rdl.fsf@kassiopeya.MSHEIMNETZ> (Sebastian Rose's message of "Thu, 23 Oct 2008 19:02:30 +0200")
Sebastian Rose <sebastian_rose@gmx.de> writes:
> Hi Richard,
>
>
> Richard Riley <rileyrgdev@googlemail.com> writes:
>>> Added this one to the Clippboard section on new org-tests/index.org in
>>> Worg.git. (this section will be temporary...)
>>
>> Something like the above should only be a link (at most) to the emacs
>> manual. Reproducing standard info is bad in the long run in case things
>> in the base product (emacs) change for example.
>
>
> On the long run, yes. I just added this section, to start the page. And
> since it's just STARTing... this was in the first or second reaction
> that came in. Feel free to edit ('go wild' ;-))!
>
> I wouldn't bother to have several pages like this one will be (it's
> supposed to be an index on the long run), each covering another way of
> testing.
>
> I just meant to take _ACT_. Scepticism is a good thing, as long as it
> doesn't stop you from doing.
Of course not.
>
>
>>>
>>>> Some kind of regression testing framework would be awesome. Org-mode is
>>>> large enough that this is almost a necessity to keep things stable and
>>>> bug-free.
>>>
>>> It's big and feature-RICH.
>>
>> The nature of OSS means that the community using the product keep it
>> stable and bug free. I dont think the efforts to produce meaningful
>> regression tests would be beneficial in an ever morphing product like
>> org-mode. Clearly my humble opinion on that one :-;
>
>
> We do test our software by using it. But the bug in the HTML-exporter
> that Carsten has fixed two days ago, was introduced in early September
> and would be in 6.10, which is supposed to be in the emacs 23 release.
>
> A very simple test plan would have revealed it.
The question is : define simple. I am not against test plans, but the
nature of such means a process in place where such test plans are
carried out. Carsten knows his SW inside out and would find it hard to
apply test cases every time a bug is fixed for time alone. Bugs happen
in SW and the nature (excuse the repetition) of OSS is that we, the
users, are kind of the testers and users.
>>>> Maybe something can be put together from the git testing framework and
>>>> use of emacs -batch to process test org files and verify the output is
>>>> as expected (with diff or some other tool).
>>>
>>>
>>> Hey, diff is a good idea!!
>>>
>>> I didn't take the verification of the output into account yet :-)
>>>
>>> I just pushed a change of Worgs start page, and added a directory
>>> 'org-tests'. I've placed an index.org there, which now is just a
>>> collection of ideas (I'm on my day job, so I can't really work on it
>>> now).
>>>
>>> Don't know how often the git repo is published.
>>> Bernt and Ben, are you 'worgers' allready?
>>>
>>> Do you think it makes sense to add snippets and ideas to the new page in
>>> Worg? I think while the list great to exchange ideas, it's good to have
>>> a place, where all those ideas are destilled to one-liners.
>>
>> I must say I am dubious about this. It means, for the tests to be
>> meaningful, that the output must be a fixed format in base org.
>
>
> Why? If the test bails out 'ERROR', I will have to look for the
> reason. If the format changed in a legal way => adjust the test.
What is an error? And for whom? That the categories are one more tab to
the right? I know I am probably sounding a bit of a killjoy here but
having been involved in closed source SW development with dedicated QA
and testing teams it was hard enough then. Maybe I am just being a bit
of an old curmudgeon here :-(
>
>
>> I doubt
>> this will ever be the case. The presentation will fluctuate while the
>> core information (dates, schedules periods etc) will remain pretty much
>> constant.
>>
>> The majority of bugs that I see are often down to people misusing or
>> using things in the base which are not fully explored. No amount of
>> regression testing can cover things like that unless the regression
>> tests include everyones customisations.
>
> Yes, because Carsten add features en masse :-)
> I see the testing differently. In the first place we need THINK of
> testing.
I think we all do. And the main contributors do too. But it is
unrealistic to expect a full regression test for each and every release
unless there is someone willing to step up to the mark and take the time
to perform them. I personally feel that enough take the "cutting edge"
and report back before the "official" cuts. We are all beat testers. Its
the nature of being an emacs and OSS user in the first place :-;
>
> New Org-revision out?
> Ahh, OK, I have to the HTML-exports, I want to be working.
>
> To do this, I need several different setups, several different data
> directories (Org-files) and an easy way to test, that doesn't eat my
> time and gives a result. The quality may vary, but ERRORs will be
> detected for shure.
>
> Not one or mounths later - immediately.
If you don't find it immediately then how often do you use the feature?
Now I know I probably sound negative. I assure I am not. Maybe the
earlier word "sceptic" is more applicable ..
>
> If someone installs emacs 23, tries to export to HTML and gets an
> error....
I have been there too. In fact you found a bug that I left hanging for
ages because I found a work around ( dont publish from anywhere other
than the root of the project) - so thanks for your perseverance on that.
I think the best thing people can do is keep a stable version and also
test the CVS (oops) GIT version too very now and again.
>
>
>> Do I think regression testing is important? Yes - in certain
>> environments. But every time Carsten, you, myself or anyone else fires
>> up org-mode we are already doing just that.
>
> Yes, but we can do better, easier and more complete.
Possibly. I look forward to seeing proposed solutions and would gladly
lend some time to applying them.
--
God never made his work for man to mend.
~John Dryden
next prev parent reply other threads:[~2008-10-24 12:13 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-10-23 12:04 How you can help Ben Alexander
2008-10-23 13:43 ` Bernt Hansen
2008-10-23 15:04 ` Sebastian Rose
2008-10-23 15:49 ` Richard Riley
2008-10-23 16:22 ` Ben Alexander
2008-10-23 17:02 ` Sebastian Rose
2008-10-24 12:13 ` Richard Riley [this message]
2008-10-24 15:39 ` Sebastian Rose
2008-10-24 16:27 ` Manish
2008-10-24 18:41 ` Avdi Grimm
2008-10-23 19:13 ` Avdi Grimm
2008-10-24 12:19 ` Richard Riley
2008-10-23 16:19 ` Bernt Hansen
2008-10-24 5:05 ` Carsten Dominik
2008-10-23 17:01 ` Jason F. McBrayer
2008-10-23 23:46 ` Eric Schulte
2008-10-23 14:20 ` Sebastian Rose
2008-10-23 14:50 ` Manish
2008-10-23 15:46 ` Eric Schulte
2008-10-23 16:18 ` Avdi Grimm
2008-10-23 14:55 ` Ben Alexander
2008-10-23 16:26 ` Sebastian Rose
2008-10-23 16:42 ` Avdi Grimm
2008-10-23 17:33 ` Sebastian Rose
2008-10-23 19:10 ` Avdi Grimm
2008-10-24 21:09 ` Tom Breton (Tehom)
2008-10-24 18:33 ` Ben Alexander
2008-10-24 18:44 ` Avdi Grimm
2008-10-24 19:02 ` Jeff Mickey
2008-10-26 19:49 ` org-cycle broken when cursor is at ellipses Ben Alexander
2008-10-26 21:31 ` Cameron Horsburgh
2008-10-27 8:47 ` Carsten Dominik
2008-10-27 8:47 ` Carsten Dominik
[not found] ` <D43ED86C-EFD4-4BA8-8528-4F82DB11D625@alexanderonline.org>
2008-10-23 17:12 ` How you can help Sebastian Rose
2008-10-23 15:08 ` Sebastian Rose
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=okej262nu2.fsf@development.richardriley.net \
--to=rileyrgdev@googlemail.com \
--cc=bernt@norang.ca \
--cc=bva@alexanderonline.org \
--cc=emacs-orgmode@gnu.org \
--cc=sebastian_rose@gmx.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).