emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
From: Max Nikulin <manikulin@gmail.com>
To: emacs-orgmode@gnu.org
Subject: Re: [PATCH] org-test: Create a collaborative test set for Org buffer parser
Date: Tue, 14 Dec 2021 23:16:38 +0700	[thread overview]
Message-ID: <spag1d$mb7$1@ciao.gmane.io> (raw)
In-Reply-To: <87fsqzi4tw.fsf@localhost>

On 11/12/2021 21:39, Ihor Radchenko wrote:
> The attached patch is an attempt to create something like shared repo
> for Org element parser tests.

"[PATCH]" prefix in the subject might be a reason why you message 
received less attention than it should.

> The test set is essentially a series of .org files alongside .el files
> containing normalised output of `org-element-parse-buffer'. (see the
> patch)

I think, the set should contain hundreds of tests to be helpful, thus 2 
files per test will likely be inconvenient since most of samples will be 
short. I suggest to group test input and results into large files. Such 
tests should be augmented by some metadata: keywords (labels, tags). 
E.g. besides heading

- sample: "* Simple Heading"
   keywords: heading

it should be possible to filter related cases with similar markup

- sample: "*Bold* emphasis"
   keywords: emphasis, heading
   description: Despite line is started from a star,
     there is no space after it, so it is not a heading.

- sample: " * Unordered list item"
   keywords: list, heading
   description: Due to leading space it is a list item,
     not a heading.

- sample: "*"
   keywords: text, heading
   description: Not a heading since there is no space after the star.

I omitted test IDs above.

Version of Org and test set should be included into metadata for the 
whole suite.

Since partial compliance is assumed, format of test results should be 
declared as well to be able to publish overview or comparison.

Are properties like :begin and :end mandatory for reference results of 
parsing? They make structures more verbose and harder to read. Often it 
is enough to compare content and similar properties.

> Any comments or suggestions?
> I am particularly looking for thoughts about licensing and possible
> distribution of the test set in separate repository.

Since these tests will unlikely become a part of some software, I do not 
think that GPL may be an obstacle for any project. Requirement of signed 
consent will likely prevent contributing of new cases from some developers.

  reply	other threads:[~2021-12-14 16:17 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-12-11 14:39 Ihor Radchenko
2021-12-14 16:16 ` Max Nikulin [this message]
2021-12-14 20:27   ` Tim Cross
2021-12-15 13:44     ` Ihor Radchenko
2021-12-15 13:23   ` Ihor Radchenko
2021-12-19  8:23     ` Max Nikulin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information: https://www.orgmode.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='spag1d$mb7$1@ciao.gmane.io' \
    --to=manikulin@gmail.com \
    --cc=emacs-orgmode@gnu.org \
    --subject='Re: [PATCH] org-test: Create a collaborative test set for Org buffer parser' \


* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Code repositories for project(s) associated with this inbox:


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).