From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id +J9UDstRYmNj2wAAbAwnHQ (envelope-from ) for ; Wed, 02 Nov 2022 12:17:31 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id oENQDctRYmOZHQAAG6o9tA (envelope-from ) for ; Wed, 02 Nov 2022 12:17:31 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id DEEBA2976C for ; Wed, 2 Nov 2022 12:17:30 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqBje-00043E-CD; Wed, 02 Nov 2022 07:16:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqBjV-00041C-RN for emacs-orgmode@gnu.org; Wed, 02 Nov 2022 07:16:33 -0400 Received: from mail-vk1-xa36.google.com ([2607:f8b0:4864:20::a36]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oqBjT-0001jw-Gp for emacs-orgmode@gnu.org; Wed, 02 Nov 2022 07:16:32 -0400 Received: by mail-vk1-xa36.google.com with SMTP id t85so8771197vkb.7 for ; Wed, 02 Nov 2022 04:16:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=gyvCOPZBz+7KGP07oRUNCEkYKHlDeyWyw171wxx/3Cg=; b=OiMOsLJ/W1YHhGnZm9mO8d7TqPZ2fC3AVKcjXFQXRaYKObvA4EIqJX1YhlR0MyjCv1 VAG/9jKZzjWnwUbcytuMEgt5ypUkfZGTq0We8Td2kXiHA+K7h/7rXEsH8Vc0reFH8V6h bqVfsB1XmsEwHF6d2CysvG4nIRbVi4lQCkgEHay9iAgWs0i9/D9GktzW8/UywqFGXrev sp5+bH/b1T6x/TdmCZKi5pCizhpLdWDeDrfpmaXkOzkMURYpknZX7KBp39mnY53t99o/ oOndCB2OQ54T9glyIoUtvw40BqZdxrvBg+PllUMf//9u7d17vvoYR6fa+3lDqTSOJ4Xl J0ZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gyvCOPZBz+7KGP07oRUNCEkYKHlDeyWyw171wxx/3Cg=; b=fQY+mkXASkByIgk3Tl5rbaBJNYOtSjfJ2vE/CQGt4XEkhs+mQFQOoxyhdlyFFTH903 CsrxdSP6x+3as/jeetQhTmsx9csVZlKGz4b/ciZUsxL57oFq8Q8uGH/FeYMlXVzlDn0G pl9fKM6Lci4X2tw3qYe5e3/Spk1x/6LmeE5n+bFuq/9S7nyItPfV1voX0eDbBdRYM1UI 0GEIyzI80bP/GNyXGIKT509HBAKyJ2euGMpi6Vd0oW8iZI4YnaUHJ03oR0+WSI7B8tHA 5szSl22ALR+GHLW94nQHJtqE7MSt7vhi0wRyHjgRgq1F28/A8arbmIkYNQWjhkCz+zQ1 sMgQ== X-Gm-Message-State: ACrzQf3l1DsAtCDFP1Tl/5gFsPtjVNVVZIptFotK3xulEOyxzJe6fsM1 t2MstR6LnDhb5f/f83d3RvzcETglJUknU5VOaOU= X-Google-Smtp-Source: AMsMyM47vdULQm0Rt94M1cE03AYuCYZC9/u/7DYpGwksAmNAl/pfGPc7Kcze1crxFpTB5ynxKNgqJAcTaK31my8JVhc= X-Received: by 2002:a1f:200e:0:b0:3b7:6558:52a2 with SMTP id g14-20020a1f200e000000b003b7655852a2mr11517911vkg.20.1667387790415; Wed, 02 Nov 2022 04:16:30 -0700 (PDT) MIME-Version: 1.0 References: <878rktluej.fsf@localhost> In-Reply-To: <878rktluej.fsf@localhost> From: =?UTF-8?Q?Christian_K=C3=B6stlin?= Date: Wed, 2 Nov 2022 12:15:53 +0100 Message-ID: Subject: Re: [PATCH] Ignore flaky ob-python tests To: Ihor Radchenko Cc: emacs-orgmode@gnu.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2607:f8b0:4864:20::a36; envelope-from=christian.koestlin@gmail.com; helo=mail-vk1-xa36.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Emacs-orgmode" Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1667387850; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=gyvCOPZBz+7KGP07oRUNCEkYKHlDeyWyw171wxx/3Cg=; b=qDYRLnVl1+mZWWOIioPjj1JUIsHCkidPdcupQD/ZvgB/lpJXmu55B33hywv8PYXj/lk9ge lYgkDxqF3rT3n0bR5BRVdT+qvM5mLrcqBe64fYvOcdbGDtl3EXt5yBp1MdwMuvxF6jglz4 iRZ/qQTHk1Do1GdYRdbnJwnZclUNS4JDTCTetXTNrZwysPhm8QRyiKIrL2no+mHaAdm4XR HYaij7eeAg3FgAaHu//OADgZcQVBymSW4XpvyQFOqqwm2S/9SxY6xbwsWw3+r9YUMTrkmL X2/Rof5jvpUDHtDlnAK3IwwegI6EkMl9gM6riwzs7rfSLK+CyiG5Bwh45j3SfA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1667387850; a=rsa-sha256; cv=none; b=R0ZqTbmyoprNiLT7a/PdkG6kkho1lcwvc1HaWeB2GgEXYqRp8T49pqeKuQceBF9W/gAWwv U9loy9UUQcBqRfly2xc+jvZBFm0jSzv75HGGMpX5kLq7hi5Wnn44q4fk6WcF4CiOlAMzis 2sPaR+XAuutsDUC3+uyR/fj7MLP9z5wCTxojJF1GNIS50eznV+tkC5vpVFdQcdaRvGdhwa Q0bZYJJsoc2H46N+zQ4kbG0c3mDBTcyTfvxNmgS0KZitMKDX0ZLrqT77bjW1/I5edTDXhu yMn5DJynrhFNef6cCy4hhP6cLmSMfwSdMA4/oAP2FK/4qVLwJwRAIkL4EenuVA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="OiMOsLJ/"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -3.99 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="OiMOsLJ/"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: DEEBA2976C X-Spam-Score: -3.99 X-Migadu-Scanner: scn1.migadu.com X-TUID: VaSKw8XEFB35 On Wed, Nov 2, 2022 at 7:52 AM Ihor Radchenko wrote: > > Christian K=C3=B6stlin writes: > > > The new async feature of python seems to be implemented in a way > > that can break with high CPU load. This patch still runs the 3 tests in > > question, but does not fail the complete testsuite if any of those fail= . > > Thanks for the patch, but I'm afraid that it will drive the tests useless= . > After the patch, even if the tests fail for a valid reason, it will not > be visible upon running make test, AFAIU. I think we know of the async problem, and you reported that as well, so I would expect if someone is able to fix it, he will also move the tests to "sharp" again. With my patch the tests are still failing but the testsuite continues to run and returns with 0 as status code (one can see the failed test though). > Maybe we can instead provide some variable that can turn-on ignoring > some problematic tests only during CI tests? Yes ... that would also be possible e.g. skip the test or test nothing if it's running on the ci. (I am talking only about the 3 tests!). Kind regards, Christian