From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id cLvQOkCQ8GZvFQAAqHPOHw:P1 (envelope-from ) for ; Sun, 22 Sep 2024 21:46:41 +0000 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id cLvQOkCQ8GZvFQAAqHPOHw (envelope-from ) for ; Sun, 22 Sep 2024 23:46:41 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=gGDv3IDu; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1727041599; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=9Z/IAGT3cBtb0ohOSMNheksT4aKm9uGu9B0m7QjCXAA=; b=FFzBSkB1iSJNwjdzOdaY50eCVWeo2hBHBEEXgIN9b7iGLG77jrWZVgtU8ipMvwvcehJuuK GdbxMLj2h71S/DW3uS3UrXhdupA+YBY5Bpwh3wfgCDxmdKSTGF6yB6K/NOV0ney8dvzQbK UBpIc8esloUXzeN8F85SF7klkcmBs+r5TP6nZ3DE/ehqv6ZH4lARYRv6FcroS7sJRLmjNn 3anXRttX5HOCyu48JGqAUom1ZJJQcJEZCAVSqwtg8BJb2stTum+mlfbnyYITjRwxAyeGOF I4uO8rQks+5rYVTuTw775+hZDWsr6OwKRyoSk+a1pHnhbIydEqg6VQvzP5bAWw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=gGDv3IDu; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=key1; d=yhetil.org; t=1727041599; a=rsa-sha256; cv=none; b=P3l57rlOYGcXbTrF+7M1wNRpr4abHw5TZ6mjLDZGeNtIsl26w4uxo2Zv9sA7yu5+xznwcO kq+QicNXysiM9me1RpgM3EhHSvibmvXpJs52Yp01IxSg/N9TiunYehBBvzCJfdJ2ek5fOI mqMqa/o3/6/lTaSovjIm5YwZowXh4Z0kN9NEFZGUF/AvEepdOreQ0gZD3fmWgosLjpeS1I u2Ln4CdbEXFto4tLpKOFpZSbA7NcRRnsUNOiSI/jSGHKP1Hnm7eTbVcq169OuNjVCO20Uw Ti0iRFwZt2k0qSyGsVqsK7O8Z+yak0YdBgItV9eWqUXHWIahcJrZ+0dgZwS0xg== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 9B94776CE2 for ; Sun, 22 Sep 2024 23:46:39 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ssUP4-0004UO-2S; Sun, 22 Sep 2024 17:46:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ssUP1-0004U6-0o for emacs-orgmode@gnu.org; Sun, 22 Sep 2024 17:45:59 -0400 Received: from mail-pg1-x52a.google.com ([2607:f8b0:4864:20::52a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ssUOw-00077z-6C for emacs-orgmode@gnu.org; Sun, 22 Sep 2024 17:45:58 -0400 Received: by mail-pg1-x52a.google.com with SMTP id 41be03b00d2f7-7db1f13b14aso3001771a12.1 for ; Sun, 22 Sep 2024 14:45:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727041551; x=1727646351; darn=gnu.org; h=mime-version:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=9Z/IAGT3cBtb0ohOSMNheksT4aKm9uGu9B0m7QjCXAA=; b=gGDv3IDuMdyzrA23+tCTsfyGy1RP4ezo1QxLEq5nx0wgqyqc4EEtAIRznXzyrTbDM+ Xod9htNLMPdbGaVeKZTuaUUKbXLrNpgO32CwLO/X/Nim+8JEf3MsxDJeA6UKjNAjtlZf k1LzYn61OU4mt18H3BTHvQsmYh05YjvIGG8MUYa0LNDS1f2NIq2/9fuSkRTkYp0frBk1 rl7OydaYlsmDi/2TRM865NceFGVcZg0o4jX61Hi9rSeUtFm97rfvYTdhhssuiMseOH47 pWNP4arcyN1+rjKC2h9CXuwez7I7Y/SnoHIuHqtXq3LWTPxNu1NPgNnZG42nEBdpPFxs lY0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727041551; x=1727646351; h=mime-version:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=9Z/IAGT3cBtb0ohOSMNheksT4aKm9uGu9B0m7QjCXAA=; b=rB9Nf8sxsbnAd80Y25WRDMQxSvefGq5VtEjb4YADEBm9JQaOx4RHG4XvxVj9f6IoDu QXjevYF1SKYmoGCmT0d5EfIzMpHBNO0r+iQfCKmZdE+WBQMgxuuk2WI9zAldfIEaJEDQ VIpoh7nMy2tPoiQUOWB0vYVoDzEdACIODgaYvdcqykKpD152MB8TfEzG1DPyRwHXfwNM VswlGhkusTlWA3snLRNe9z3qNQFwaLSbvHsyu+03Pj0hbVelt1u0HMDZzcG7Z4TNzxWI W11kUO89pQCH7KWWgtvYeX1nXC4pkRfELdEnyeHlnQtQcZsX14rzF+WDl+QZgj9+xqI2 uINg== X-Gm-Message-State: AOJu0YyYdphBZhBZ4SXGe30kjBHXuMloIbSFB0L3o/PjZzngWv/4urFs OT7Ee6b5MMh43H827OYKJHZR/4Jxgh1MzWkK8jkLLbzyTfG22vI3QyM1WA== X-Google-Smtp-Source: AGHT+IG1YwdiawZPeuXaoBhWubbgDcuf4rPb2LORv9uVjrqlDBHcN973+HkbYxU2F0U3FRvK3RJmpg== X-Received: by 2002:a05:6a20:d498:b0:1d2:d001:5677 with SMTP id adf61e73a8af0-1d30c9fa966mr12842120637.17.1727041551524; Sun, 22 Sep 2024 14:45:51 -0700 (PDT) Received: from localhost ([198.27.183.102]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7db498de2d5sm12016502a12.15.2024.09.22.14.45.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Sep 2024 14:45:49 -0700 (PDT) From: Jack Kamm To: emacs-orgmode@gnu.org Cc: yantar92@posteo.net, matt@excalamus.com, jeremiejuste@gmail.com Subject: [PATCH] Async sessions: Fix prompt removal regression in ob-R Date: Sun, 22 Sep 2024 14:45:48 -0700 Message-ID: <87setrqs4z.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=2607:f8b0:4864:20::52a; envelope-from=jackkamm@gmail.com; helo=mail-pg1-x52a.google.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, TRACKER_ID=0.1 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Queue-Id: 9B94776CE2 X-Migadu-Scanner: mx11.migadu.com X-Spam-Score: -9.83 X-Migadu-Spam-Score: -9.83 X-TUID: FH84nPMmsd2L --=-=-= Content-Type: text/plain Consider the following R block, which prints the occurrences of each element in a list, including NAs: #+begin_src R :session :results output :async table(c("ab","ab","c",NA,NA), useNA='always') #+end_src #+RESULTS: : ab c : 2 1 2 Since Org 9.7, it instead prints: #+RESULTS: : ab c < : 2 1 2 The regression happens in commit: e9c288dfaccc2960e5b6889e6aabea700ad4e05a which made the prompt filtering more consistent between `org-babel-comint-with-output' and `org-babel-comint-async-filter'. However, it causes ob-R async session blocks to be over-aggressive in removing the prompt. Note that non-async ob-R blocks don't suffer from this problem, because ob-R let-binds `comint-prompt-regexp' around `org-babel-comint-with-output' (specifically, it adds a beginning-of-line at the front of the regexp). However, I don't see a good way to let-bind this around `org-babel-comint-async-filter'. The best solution I could think of was to define a new buffer-local variable, `org-babel-comint-prompt-regexp-override', which could be used to override `comint-prompt-regexp' for the purpose of filtering. I attach a patch with this solution. Additionally, the regression causes causes misalignment of the output due to removal of indentation. The fix for this is simpler, and involves replacing a call of `org-trim' with `org-babel-chomp'. I'm not sure if my patch is the best solution. But whatever solution we arrive at, I would like to request that it be applied to bugfix branch. --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-ob-R-Fix-over-aggressive-async-prompt-removal.patch >From 11177e57f8a0c77b6c6541b852c5d105d70afec0 Mon Sep 17 00:00:00 2001 From: Jack Kamm Date: Sun, 22 Sep 2024 13:48:45 -0700 Subject: [PATCH] ob-R: Fix over-aggressive async prompt removal * lisp/ob-comint.el (org-babel-comint-prompt-regexp-override): New variable to override `comint-prompt-regexp' in `org-babel-comint--prompt-filter'. (org-babel-comint-async-filter): Replace `org-trim' with `org-babel-chomp' to avoid removing leading indentation. * lisp/ob-R.el (org-babel-R-evaluate): Set `org-babel-comint-regexp-override' in session evaluation. (org-babel-R-evaluate-session): Remove let binding of `comint-prompt-regexp', since `org-babel-comint-regexp-override' is now set. * testing/lisp/test-ob-R.el (test-ob-R/async-prompt-filter): Test for over-aggressive prompt removal. --- lisp/ob-R.el | 25 ++++++++++++++----------- lisp/ob-comint.el | 18 +++++++++++++++--- testing/lisp/test-ob-R.el | 28 ++++++++++++++++++++++++++++ 3 files changed, 57 insertions(+), 14 deletions(-) diff --git a/lisp/ob-R.el b/lisp/ob-R.el index de2d27a9a..a9a58d0e4 100644 --- a/lisp/ob-R.el +++ b/lisp/ob-R.el @@ -375,11 +375,15 @@ (defun org-babel-R-evaluate (session body result-type result-params column-names-p row-names-p async) "Evaluate R code in BODY." (if session - (if async - (ob-session-async-org-babel-R-evaluate-session - session body result-type column-names-p row-names-p) - (org-babel-R-evaluate-session - session body result-type result-params column-names-p row-names-p)) + (progn + (with-current-buffer session + (setq org-babel-comint-prompt-regexp-override + (concat "^" comint-prompt-regexp))) + (if async + (ob-session-async-org-babel-R-evaluate-session + session body result-type column-names-p row-names-p) + (org-babel-R-evaluate-session + session body result-type result-params column-names-p row-names-p))) (org-babel-R-evaluate-external-process body result-type result-params column-names-p row-names-p))) @@ -456,12 +460,11 @@ (defun org-babel-R-evaluate-session (substring line (match-end 1)) line)) (with-current-buffer session - (let ((comint-prompt-regexp (concat "^" comint-prompt-regexp))) - (org-babel-comint-with-output (session org-babel-R-eoe-output) - (insert (mapconcat 'org-babel-chomp - (list body org-babel-R-eoe-indicator) - "\n")) - (inferior-ess-send-input)))))))) "\n")))) + (org-babel-comint-with-output (session org-babel-R-eoe-output) + (insert (mapconcat 'org-babel-chomp + (list body org-babel-R-eoe-indicator) + "\n")) + (inferior-ess-send-input))))))) "\n")))) (defun org-babel-R-process-value-result (result column-names-p) "R-specific processing of return value. diff --git a/lisp/ob-comint.el b/lisp/ob-comint.el index 764927af7..7f1686035 100644 --- a/lisp/ob-comint.el +++ b/lisp/ob-comint.el @@ -75,11 +75,17 @@ (defun org-babel-comint--set-fallback-prompt () (setq comint-prompt-regexp org-babel-comint-prompt-regexp-old org-babel-comint-prompt-regexp-old tmp)))) +(defvar-local org-babel-comint-prompt-regexp-override nil + "Overrides `comint-prompt-regexp' in `org-babel-comint--prompt-filter.'") + (defun org-babel-comint--prompt-filter (string &optional prompt-regexp) "Remove PROMPT-REGEXP from STRING. -PROMPT-REGEXP defaults to `comint-prompt-regexp'." - (let* ((prompt-regexp (or prompt-regexp comint-prompt-regexp)) +PROMPT-REGEXP defaults to `comint-prompt-regexp', which can be +overridden with `org-babel-comint-prompt-regexp-override'." + (let* ((prompt-regexp (or prompt-regexp + org-babel-comint-prompt-regexp-override + comint-prompt-regexp)) ;; We need newline in case if we do progressive replacement ;; of agglomerated comint prompts with `comint-prompt-regexp' ;; containing ^. @@ -327,7 +333,13 @@ (defun org-babel-comint-async-filter (string) (equal (match-string 2) uuid)) finally return (+ 1 (match-end 0))))) ;; Remove prompt - (res-promptless (org-trim (string-join (mapcar #'org-trim (org-babel-comint--prompt-filter res-str-raw)) "\n") "\n")) + (res-promptless + (org-trim (string-join + (mapcar #'org-babel-chomp + (org-babel-comint--prompt-filter + res-str-raw)) + "\n") + t)) ;; Apply user callback (res-str (funcall org-babel-comint-async-chunk-callback res-promptless))) ;; Search for uuid in associated org-buffers to insert results diff --git a/testing/lisp/test-ob-R.el b/testing/lisp/test-ob-R.el index 9ffbf3afd..05b91afd6 100644 --- a/testing/lisp/test-ob-R.el +++ b/testing/lisp/test-ob-R.el @@ -316,6 +316,34 @@ (org-test-with-temp-text-in-file (string= (concat text result) (buffer-string))))))) +(ert-deftest test-ob-R/async-prompt-filter () + "Test that async evaluation doesn't remove spurious prompts and leading indentation." + (let* (ess-ask-for-ess-directory + ess-history-file + org-confirm-babel-evaluate + (session-name "*R:test-ob-R/session-async-results*") + (kill-buffer-query-functions nil) + (start-time (current-time)) + (wait-time (time-add start-time 3)) + uuid-placeholder) + (org-test-with-temp-text + (concat "#+begin_src R :session " session-name " :async t :results output +table(c('ab','ab','c',NA,NA), useNA='always') +#+end_src") + (setq uuid-placeholder (org-trim (org-babel-execute-src-block))) + (catch 'too-long + (while (string-match uuid-placeholder (buffer-string)) + (progn + (sleep-for 0.01) + (when (time-less-p wait-time (current-time)) + (throw 'too-long (ert-fail "Took too long to get result from callback")))))) + (search-forward "#+results") + (beginning-of-line 2) + (when (should (re-search-forward "\ +:\\([ ]+ab\\)[ ]+c[ ]+[ ]* +:\\([ ]+2\\)[ ]+1[ ]+2")) + (should (equal (length (match-string 1)) (length (match-string 2)))) + (kill-buffer session-name))))) (provide 'test-ob-R) -- 2.46.0 --=-=-=--