From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id QNGaEgZCnmGi4wAAgWs5BA (envelope-from ) for ; Wed, 24 Nov 2021 14:45:42 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id sKVVDgZCnmE/FwAA1q6Kng (envelope-from ) for ; Wed, 24 Nov 2021 13:45:42 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id DACB47866 for ; Wed, 24 Nov 2021 14:45:41 +0100 (CET) Received: from localhost ([::1]:34690 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mpsai-00038N-UV for larch@yhetil.org; Wed, 24 Nov 2021 08:45:40 -0500 Received: from eggs.gnu.org ([209.51.188.92]:40898) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mps2M-0003an-Fe for emacs-orgmode@gnu.org; Wed, 24 Nov 2021 08:10:11 -0500 Received: from [2607:f8b0:4864:20::62c] (port=34632 helo=mail-pl1-x62c.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mps2K-00081V-6G for emacs-orgmode@gnu.org; Wed, 24 Nov 2021 08:10:09 -0500 Received: by mail-pl1-x62c.google.com with SMTP id y8so1829344plg.1 for ; Wed, 24 Nov 2021 05:10:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:mime-version; bh=wd+jCfiAWkFszbqpNiFiovZ3X+t5cXBgb1IXyJWqVAw=; b=kxRbptBOgG6hK6/96ytBhyB2LP4XcFU9BmoXHF3UehX1synytu5oJ6/Z0qOFVPt9EF WcO5mU+oJmG2RZbcpf0nkVBQgl075CNeET8Wv0iRQDaw3mwOvlLFwQeyU1O/4XRS38gr PmJwQMWuUDRM7vX2q04XD7vbuUwZB3O9b13mQhCep4IqAUkfgoXGeOARjQ+6FAMXexZL 7O7p5pdLAEPRvwhPVBFJJ/Ald3TOSP0kpTWw80aCB8V1Ezmi1nXp1oqV/UKd7F0JUwkt QpxG9H0MnRYQ2d2Z6p989skRnPhXEuFujLnivW65fb4pLLfVWa9kXvRqMpP6AKryYWtP rA5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:mime-version; bh=wd+jCfiAWkFszbqpNiFiovZ3X+t5cXBgb1IXyJWqVAw=; b=VeA7bhHF0fTclFMRtzBjMyG2FYi5HZ3VosL0JJjYmVRat4hNMdk5orAYAjIs6ahWOB P1yBkS3XcScQRgVnDZ+j3Eu2s48zJ0/+IRfZlS7li5yjIDZyHYPTW8ROqJ0uJqsbW9xe RKu+dfmjgUlI/JMH1+FPPASojemoexGYEu+T0A+iD/8Uh84OfuhVelIaRLDDg2Yck1NT uucLD4mcdK96Ql1WsN6st2xxM2nwkGNlvhbT3Poy0D2D6wfN0ETqVCmkGKn9v5VUBK9M GdI+MSbK6F96ilzXTjVHLuY26HuBqWTI+ZY9UKoEH72ENiI6tj+i1OySnHBu1J51Hrjt VZ5w== X-Gm-Message-State: AOAM5331qj530e4RqfzDpI27MDftxlUo+2s4B10bVVEcX2N2LRCcB1/E QMdck3e3Sa/pbBBJtv8an8dutQrjbxbkZw== X-Google-Smtp-Source: ABdhPJx2TKLRFe+WUOjd30rYR8p3w+QLbh+r5bsBKvs3xxUuD0VYVOo1uO/UKMi7dWkpTRjducpCVA== X-Received: by 2002:a17:90a:e389:: with SMTP id b9mr15157347pjz.235.1637759406436; Wed, 24 Nov 2021 05:10:06 -0800 (PST) Received: from localhost ([60.234.42.250]) by smtp.gmail.com with ESMTPSA id g9sm18061567pfj.160.2021.11.24.05.10.05 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Nov 2021 05:10:06 -0800 (PST) From: Ihor Radchenko To: emacs-orgmode@gnu.org Subject: [HELP] Request for patches to improve Org test coverage Date: Wed, 24 Nov 2021 21:11:28 +0800 Message-ID: <874k81bsvz.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Host-Lookup-Failed: Reverse DNS lookup failed for 2607:f8b0:4864:20::62c (failed) Received-SPF: pass client-ip=2607:f8b0:4864:20::62c; envelope-from=yantar92@gmail.com; helo=mail-pl1-x62c.google.com X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1637761541; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=wd+jCfiAWkFszbqpNiFiovZ3X+t5cXBgb1IXyJWqVAw=; b=G28nOQm6VAlh7p3fDGpbP7U3bDigiWwoG6xxydQc/U3+EwEzI4D07PKvXLlaeqfOVthLL4 t9G+mDJt7LTlJI+cNo/J9Oi12eIBppFtxo5VqjDnwMhmDUrs397SXfj5yHu2uYHNXKlYJb zSoGxt4HEVJawqRPuJngEmPzkvu7UIv3eDkH3R+53c5wn671sbe6EwXrKptlBIOrGCmpII OrGhETwCCj2xsoyI+8Mql8JLR7Man9CqvXyfdZlnxexKIs/dECffGdXFtrVfsH8na2+dse C5WgsY7zXkCqeOeFWXNMfHRiLUqjIMMWsZva0tDJ85PqVATuW17kW+DcjYhi0A== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1637761541; a=rsa-sha256; cv=none; b=Mn293syDAfuySfao83AB2Rpx8ICVSEr/jOUc/rxpkBAe3IVAnTIG0pPNfNhZhJ0iLMbEj0 pmGB+x8aM1PXdQuf2kwnZQiDTLpCjBBpXh0Im3cthz7z418hX1HcL1JEt2JLkeUVHq+BtZ FLvjtIj4E6WgBbzwIICr55IGr2lp1Tan/1vVR3pkLzXcD1cK/A3bLIm6oR50ubdBT83Om9 UBhQLR+DxXij6PIZsq0cHBAS9/845FqcaUWLdU6FZWy4u2or57YSlCVv8K63AOPgadOEsM wLztgt1i8Ao9TWzQFdXnueab5PRKYK/3wUTj+zwABkht7oH1sRxrDC/qTwW4gw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=kxRbptBO; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Spam-Score: -3.59 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=kxRbptBO; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Queue-Id: DACB47866 X-Spam-Score: -3.59 X-Migadu-Scanner: scn1.migadu.com X-TUID: LLy0PpSEKTlZ Dear Fellow Orgers, Despite being not directly visible to users, Org mode testing suite is critical to maintain overall codebase stability. It helps to avoid unintentional breakages as we keep adding new features to Org. Yet, writing tests is probably not the most exciting part for volunteer contributors. Not many new patches also include tests and the core Org maintainers do not have a bandwidth to supply every new significant patch with tests. I am writing this help request in a hope that someone interested and/or experienced with writing tests can help improving Org tests without a need to spend a lot of time doing something unfamiliar (e.g. I personally wrote my very first test as a patch for Org, not knowing much about code testing beforehand). Some of the possible improvements to Org test suite are listed below: - Tests on bugfix branch fail to run using Emacs 26. Not because of main Org code, but because test-org re-defines decode-time in a way not supported by older Emacs versions. (This has been fixed on main and simply need to be ported back to bugfix). - Tests always run in a single process and cannot benefit from make -j Supporting parallel execution of multiple test sets would make Org development much faster by reducing make test runtime. - Testing native-compiled Org is missing. Having it would be nice. - Tests covering state logging are missing. I wrote an example patch how to write such tests, but never got around for more https://orgmode.org/list/87tul1v11c.fsf@localhost - Most of the tests are written assuming default values of customised variables. We have very basic test coverage for non-standard settings. A way to run all possible tests under most common user customisations would improve the test coverage a lot. - Generally, more and better tests would be welcome. Best, Ihor