From: "Eric Schulte" <schulte.eric@gmail.com>
To: Carsten Dominik <carsten.dominik@gmail.com>
Cc: Emacs-orgmode mailing list <emacs-orgmode@gnu.org>
Subject: Re: Re: Testing --- again...
Date: Sat, 02 Oct 2010 12:12:37 -0600 [thread overview]
Message-ID: <87fwwoihuy.fsf@gmail.com> (raw)
In-Reply-To: <0CDCC1C3-3D93-465E-827C-A722978F4D13@gmail.com> (Carsten Dominik's message of "Sat, 2 Oct 2010 10:09:03 +0200")
Hi,
This is exciting.
Rather than impose a complete directory/layout schema before-hand I'd
lean towards starting with a little more chaos and then letting the
structure of the test directory develop /naturally/. From the
discussion below it sounds like an initial structure of
testing/lisp/
testing/contrib/lisp/
may make sense, reserving the top level for "meta" testing stuff, like
functions for running tests, common fixtures, example files, etc...
I have two questions.
1) while waiting for ert to be included into Emacs, should we include an
ert distribution as part of the Org-mode repository (maybe using git
sub-modules) or should we just agree that users should have a certain
version of ert installed locally? I'm honestly not sure which of
these options sounds preferable.
2) should the initial population of the testing/ directory take place in
a separate branch of the repository or in the master branch? Again I
don't know which I would prefer, branches add complication but could
result in cleaner commit histories.
Best -- Eric
Carsten Dominik <carsten.dominik@gmail.com> writes:
> Hi Sebastian,
>
> the lack of a testing suite for Org-mode is really frustrating,
> and even more frustrating is that we have had like seven attempts
> to start one, and each of these lead to nothing. So I would
> be perfectly happy to give a free hand, write access to the repo
> and a full directory in the distribution to implement one.
> Once there is a framework, I am sure many people would be
> willing to contribute tests.
>
> More comments below.
>
> On Oct 2, 2010, at 5:51 AM, Sebastian Rose wrote:
>
>> Hi,
>>
>>
>> I thought about testing again recently. This is something, that never
>> really got started. For a reason: there's no framework for testing.
>>
>> I therefore wrote a very rough proposal, found on
>> http://github.com/SebastianRose/org-test
>>
>> The idea is, to provide two simple commands:
>>
>>
>> * org-test-test-current-defun
>> will search for tests for the defun point is in or behind
>> (`beginning-of-defun') and execute them surrounded by
>>
>> (let ((select (or selector "^org"))
>> (deactivate-mark nil))
>> (save-excursion
>> (save-match-data
>>
>>
>> * org-test-test-buffer-file
>> will search for tests for the entire file and execute them the
>> same
>> way.
>
> FIrst: I have *no* clue about testing.
>
> Second, I am surprised that you want to structure it by function. I
> would have
> thought that it could be structure by file at the most. And then
> there will
> be tests that involve code from many files.
>
> But I guess
>
>>
>> If you use one of these commands, all currently registered ERT tests
>> are
>> deleted, and files are reloaded (since you're likely to work on the
>> tests, too). To repeat the tests without reloading, you will use the
>> ERT commands like `ert-results-rerun-all-tests', bound to `r' in the
>> ERT
>> results buffer.
>>
>>
>>
>> I choose ERT (git clone http://github.com/ohler/ert.git) because
>> that's
>> likely to go into Emacs core (or elpa.gnu.org).
>>
>>
>>
>>
>> The idea is to search the directory structure from the current source
>> file upwards for a directory named "tests/" if it exists. Else ask
>> the
>> user. Similar to what `add-change-log-entry' does.
>>
>> Below that directory, a tree like the source tree exists:
>>
>> project
>> +-- lisp/
>> | +-- a.el
>> | `-- b/
>> | +-- b.el
>> |
>> `-- tests/
>> +-- a.el/
>> | +-- tests.el
>> | `-- a-defun.el
>> `-- b/
>> +-- b.el/
>> +-- tests.el
>> `-- b-defun.el
>>
>> If this setup exists, when editing defun-x in lisp/a.el,
>> `M-x org-test-test-current-defun' will load tests/a.el/defun-x.el
>> (fallback: tests.el there) and execute all tests with selector
>> "^a-defun".
>
> Well, OK, this is fine. But under a.el and b.el there should also be
> general tests that are not function dependent, and there should be a
> place
> to put tests that you do not want to assign to a specific file.
>
> We do have a "testing" directory already, you can use that.
> I would prefer the tests to be in testing, not in lisp/testing
> if possible. I would like to have the lisp directory contain
> only code. If possible.
>
> It would be OK to have a lisp subdirectory in testing,
> just as it would be OK to have contrib/lisp in testing
> for the contributed packages.
>
> PLEASE, go ahead. I do not think you have write access
> yet on repo - give me your user name and I'll activate you.
>
> - Carsten
>
>> `M-x org-test-test-buffer-file' in that same source file will load all
>> *.el files in tests/a.el/ and execute all ERT tests for selector "^a".
>>
>>
>> Thus tests for
>> org-mode/lisp/org-protocol.el
>> will be searched in the directory
>> org-mode/tests/lisp/org-protocol.el/*.el
>>
>>
>> Once the basic route of testing is clear, I'd like to "translate" the
>> existing tests for org-html.el to work with ERT, which will involve
>> writing more tools (create output buffers, compare output with control
>> files using ediff etc.). I know Lennart Borgman has wrote that stuff
>> for nXhtml already. I hope we can use his stuff and help here.
>>
>> The directory org-mode/lisp/tests/ would not need to be part of the
>> "official" Org mode package. It could as well be checked out
>> separately, if "tests" is part of org-mode/lisp/.gitignore (e.g.).
>>
>>
>> Any thoughts?
>>
>>
>>
>>
>> Sebastian
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
next prev parent reply other threads:[~2010-10-02 18:12 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-10-02 3:51 Testing --- again Sebastian Rose
2010-10-02 8:09 ` Carsten Dominik
2010-10-02 14:53 ` Sebastian Rose
2010-10-02 15:01 ` Sebastian Rose
2010-10-02 17:05 ` Carsten Dominik
2010-10-02 18:12 ` Eric Schulte [this message]
2010-10-02 19:33 ` Sebastian Rose
2010-10-02 20:30 ` Eric Schulte
2010-10-02 21:55 ` Eric Schulte
2010-10-03 0:46 ` Sebastian Rose
2010-10-04 1:26 ` Eric Schulte
2010-10-04 3:59 ` Sebastian Rose
2010-10-04 15:26 ` Eric Schulte
2010-10-05 0:21 ` Sebastian Rose
2010-10-05 18:57 ` Eric Schulte
2010-10-06 0:48 ` Bernt Hansen
2010-10-06 2:57 ` Eric Schulte
2010-10-02 22:14 ` Sebastian Rose
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87fwwoihuy.fsf@gmail.com \
--to=schulte.eric@gmail.com \
--cc=carsten.dominik@gmail.com \
--cc=emacs-orgmode@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).