From: Martin Pohlack <mp26@os.inf.tu-dresden.de>
To: Carsten Dominik <carsten.dominik@gmail.com>
Cc: org-mode <emacs-orgmode@gnu.org>
Subject: Re: A testing framework
Date: Sun, 21 Mar 2010 20:48:19 +0100 [thread overview]
Message-ID: <4BA67803.8040701@os.inf.tu-dresden.de> (raw)
In-Reply-To: <45E96F29-7015-47AD-9271-9B9914C968E8@gmail.com>
Hi Carsten, all,
On 21.03.2010 08:26, Carsten Dominik wrote:
> we had an earlier threads about testing frameworks:
>
> http://orgmode.org/worg/org-tests/index.php
>
> http://thread.gmane.org/gmane.emacs.orgmode/8759/focus=8775
>
> http://thread.gmane.org/gmane.emacs.orgmode/8743/focus=8743
Very interesting, I just went through all this.
From above material, two general approaches have been considered for
testing org-mode:
* Lisp based unit testing.
* Example-file based feature testing.
I am targeting the second approach here for now as I think it may
provide more benefit per time spent and would allow more people to
participate easily.
I hope that every emacs user will be able to write tests with my
framework. Basically only an example test case and the expected
result should be need for tests, together with a minimal description
of the setup.
If only an example file triggering a bug is available, a corresponding
test can be crafted with minimal effort.
For now I have a minimal framework running with two tests
demonstrating expected behavior for folding and a bug in the version
I'm using.
I would like to enquire comments regarding the following open issues,
where I have no satisfying solution yet:
* I would like to start the test in something like a temporally stable
setup, such that generated timestamps would be somewhat predictable
(basically with the system clock set to 1.1.2005 20:00, or something
like that).
There are several levels where this could be achieved, ranging from:
1. Experimental patches to the Linux kernel,
2. Library wrappers using LD_PRELOAD around emacs to virtualize the
time and gettimeofday system calls,
3. Advised lisp functions for obtaining time stamps, and
4. Post processing the test output to generalized time stamps.
Currently, I'm going with 4, 3 may make sense if someone could
identify the hopefully 2-3 functions to be advised.
2 and 1 are probably too Linux specific and I'm not sure if 2 works
with emacs.
Comments?
* Currently I use a list of lisp functions for specifying the behavior
to be executed in a test, for example:
(org-todo)
or
(org-end-of-line)
(org-cycle)
This is not very intuitive for users. I would rather like something
like a recorded macro to execute.
For example:
"end, tab"
Does anyone here has a tip how to specify a list of user-input
actions in a file and play that back?
* For visual comparison I currently simply save the visible part of
buffers.
One may get a step further here and also capture properties, faces and
the cursor position. I remember vaguely that there is such a
facility somewhat related to rich text in emacs, but wasn't able to
find it again. Do you guys know of such a thing?
> I think it would be great to have a testing framework and a
> lot of tests, but I do not have time to make this happen.
> Maybe you can try to get the people who were active in those
> threads and energize them?
I assumed everyone still interested in contributing to org-mode is
still lurking in this list. So speak up if you are interested in
contributing to the tests :-).
Cheers,
Martin
prev parent reply other threads:[~2010-03-21 19:46 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-03-01 8:54 Testing Martin Pohlack
2010-03-20 0:18 ` A testing framework (was: Testing) Martin Pohlack
2010-03-21 7:26 ` Carsten Dominik
2010-03-21 19:48 ` Martin Pohlack [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4BA67803.8040701@os.inf.tu-dresden.de \
--to=mp26@os.inf.tu-dresden.de \
--cc=carsten.dominik@gmail.com \
--cc=emacs-orgmode@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).