From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id ELgfHQ3tVV+PWwAA0tVLHw (envelope-from ) for ; Mon, 07 Sep 2020 08:19:25 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id cJP8GA3tVV8RRQAAbx9fmQ (envelope-from ) for ; Mon, 07 Sep 2020 08:19:25 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id DE5939401AE for ; Mon, 7 Sep 2020 08:19:24 +0000 (UTC) Received: from localhost ([::1]:60124 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kFCN0-00051r-FZ for larch@yhetil.org; Mon, 07 Sep 2020 04:19:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43142) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kFCMg-00051d-1d for emacs-orgmode@gnu.org; Mon, 07 Sep 2020 04:19:02 -0400 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]:33474) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kFCMd-00022j-SL for emacs-orgmode@gnu.org; Mon, 07 Sep 2020 04:19:01 -0400 Received: by mail-pj1-x1034.google.com with SMTP id np15so7168919pjb.0 for ; Mon, 07 Sep 2020 01:18:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=8F80ZoCq/7yFBo8HbFHCVfJZQREvAEkPjuMWOkNKQzU=; b=g08Bqv8s+lrCwOLF6+0zL2ENwrO2b32s2mjIHPuAJ8azDyz2cA688Txd6Qt2EvcqHl FVH1vbheM1HO0Fg55NUIYupitotUYGO5JrAS5WfCCHBf4fcLBsdVpt/qqXEr6HsmKFL/ 7Bcawm+9/WyzwLSsTvRzBQUeBjXCDbo0WJ0jQOaY6LQGz/JvqY9esNpyzJHj3ZtayWHu hbKWUe5cjVlbqCSELpD9lHOG6ZhX9R+dEbxOT0LcbjpJsPJVNV45ssDWnhMKPUv1xO0H 2lORlFwM1dchBKtKcX53pYR6Ef1yD5LgKV708JWeDMLkyJiLVpUTTV2ePhnLTxq7H/fJ LLRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=8F80ZoCq/7yFBo8HbFHCVfJZQREvAEkPjuMWOkNKQzU=; b=Hf1HuoNH8GpWfsVkjTrpgT8G9ZhpZ9uFq32EAKp47eX4Vc8ST3z6/5dEIH2dzHQOR6 XTgYP/bw5CswNaxBEL8XN2rjSWarqteZDdLjmHXyEaLYGwi27NfLFPlHHvib3+QuqaSq 2IgZox1TJMuOHrAmYZ4jlvS2eT3tT/C7jKZyh1A3kFJGVMWZUs4QWYxds2h/7EgX7opG tAVa03kyaK711AzSZMdvupe6O33XEp2n94Ip/tLUGCYZPluQzz5UMCi906FNH+hNxeDN 88FzGaopYtKZcc39sbm7aJbKtwLg0Kh2FEPdU6QkwxWCZ9a/DRP0NafWWbLZnDo0YU6d 8DmA== X-Gm-Message-State: AOAM533oGilNL7hOzPEES4wut1mXej6nsEw5EfLZ2qKVFhtufWMVLOYT gKzvv/vBFzy4ukXaUT/ph8E= X-Google-Smtp-Source: ABdhPJy4Y1iqFrExqF7ux1hRcGfz4RHX/awMS2cwsyvNcucGcujrpSkA0rJbata2VYKWSZ1LxEBIHQ== X-Received: by 2002:a17:902:d354:: with SMTP id l20mr10224618plk.81.1599466737747; Mon, 07 Sep 2020 01:18:57 -0700 (PDT) Received: from localhost (199-83-220-90.PUBLIC.monkeybrains.net. [199.83.220.90]) by smtp.gmail.com with ESMTPSA id c4sm14113424pfo.163.2020.09.07.01.18.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Sep 2020 01:18:56 -0700 (PDT) From: Jack Kamm To: "Berry, Charles" Subject: Re: Help debugging R source code block output problem with :session In-Reply-To: <352C7149-743F-4944-ACA5-7A1242B5A3AA@health.ucsd.edu> References: <87r1rqled0.fsf@havana> <87k0xifa9h.fsf@gmail.com> <87d03art6a.fsf@havana> <87h7slgbi5.fsf@gmail.com> <352C7149-743F-4944-ACA5-7A1242B5A3AA@health.ucsd.edu> Date: Mon, 07 Sep 2020 01:18:55 -0700 Message-ID: <87tuwa9ez4.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=2607:f8b0:4864:20::1034; envelope-from=jackkamm@gmail.com; helo=mail-pj1-x1034.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "emacs-orgmode@gnu.org" , Dylan Schwilk Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=gmail.com header.s=20161025 header.b=g08Bqv8s; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Spam-Score: 0.09 X-TUID: posh+gO1J+32 --=-=-= Content-Type: text/plain Hi Chuck and Dylan, "Berry, Charles" writes: > This problem has been bugging people for years and previous attempts to solve it have always run up against creating more problems in the process of solving this one. > > If you do decide to dig into solving this, please be sure that remote sessions and graphical outputs are not broken. test-ob-R.el does not cover those cases. In fact, it is pretty short, so there are probably other things that could break without `make test' complaining. I think I have found a robust solution to this issue, which I've attached. The solution avoids parsing for prompts at all, thus it should be robust even for the case when the beginning of an output line looks like a prompt. I tested that the unit tests all pass, and that remote sessions and graphical outputs still work. I also added Dylan's examples as unit tests. Cheers, Jack --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-ob-R-Fix-session-output-with-substrings-matching-pro.patch >From 76d0eaa31506ce8a2f81f64eae43161db5721317 Mon Sep 17 00:00:00 2001 From: Jack Kamm Date: Mon, 7 Sep 2020 00:41:52 -0700 Subject: [PATCH] ob-R: Fix session output with substrings matching prompts * lisp/ob-R.el (ess-send-string): Declare external function. (org-babel-R-evaluate-session): New implementation for session output results, that replaces calls to org-babel-comint-with-output with custom code. * testing/lisp/test-ob-R.el (test-ob-R/prompt-output): New test for output results containing angle brackets. (test-ob-R/output-nonprinted): New test for output results that aren't explicitly printed. Fixes issue reported in https://orgmode.org/list/875zgjh8wn.fsf@gmail.com/, https://orgmode.org/list/87r1rqled0.fsf@havana/ --- lisp/ob-R.el | 37 +++++++++++++++++++------------------ testing/lisp/test-ob-R.el | 13 +++++++++++++ 2 files changed, 32 insertions(+), 18 deletions(-) diff --git a/lisp/ob-R.el b/lisp/ob-R.el index 5e9d35f58..b37e3965a 100644 --- a/lisp/ob-R.el +++ b/lisp/ob-R.el @@ -38,6 +38,8 @@ (declare-function ess-make-buffer-current "ext:ess-inf" ()) (declare-function ess-eval-buffer "ext:ess-inf" (vis)) (declare-function ess-wait-for-process "ext:ess-inf" (&optional proc sec-prompt wait force-redisplay)) +(declare-function ess-send-string "ext:ess-inf" + (process string &optional visibly message _type)) (defconst org-babel-header-args:R '((width . :any) @@ -437,24 +439,23 @@ (defun org-babel-R-evaluate-session (org-babel-import-elisp-from-file tmp-file '(16))) column-names-p))) (output - (mapconcat - 'org-babel-chomp - (butlast - (delq nil - (mapcar - (lambda (line) (when (> (length line) 0) line)) - (mapcar - (lambda (line) ;; cleanup extra prompts left in output - (if (string-match - "^\\([>+.]\\([ ][>.+]\\)*[ ]\\)" - (car (split-string line "\n"))) - (substring line (match-end 1)) - line)) - (org-babel-comint-with-output (session org-babel-R-eoe-output) - (insert (mapconcat 'org-babel-chomp - (list body org-babel-R-eoe-indicator) - "\n")) - (inferior-ess-send-input)))))) "\n")))) + (let* ((tmp-file (org-babel-temp-file "R-"))) + (with-temp-file tmp-file + (insert (concat body "\n" org-babel-R-eoe-indicator))) + (with-current-buffer session + (let* ((process (get-buffer-process (current-buffer))) + (string-buffer "") + (comint-output-filter-functions + (cons (lambda (text) (setq string-buffer + (concat string-buffer text))) + comint-output-filter-functions))) + (ess-send-string + process (format "source('%s', print.eval=TRUE)" + (org-babel-process-file-name tmp-file 'noquote))) + (while (not (string-match (regexp-quote org-babel-R-eoe-output) + string-buffer)) + (accept-process-output process)) + (substring string-buffer 0 (match-beginning 0)))))))) (defun org-babel-R-process-value-result (result column-names-p) "R-specific processing of return value. diff --git a/testing/lisp/test-ob-R.el b/testing/lisp/test-ob-R.el index 7ce340ba4..ff7ea19d5 100644 --- a/testing/lisp/test-ob-R.el +++ b/testing/lisp/test-ob-R.el @@ -97,6 +97,19 @@ (ert-deftest test-ob-R/results-file () (org-babel-goto-named-result "TESTSRC") (forward-line 1) (should (string= "[[file:junk/test.org]]" (buffer-substring-no-properties (point-at-bol) (point-at-eol))))))) + +(ert-deftest test-ob-R/prompt-output () + (let (ess-ask-for-ess-directory ess-history-file) + (org-test-with-temp-text + "#+begin_src R :results output :session\nprint(\" \")\nprint(\"one three\")\nprint(\"end\")\n#+end_src\n" + (should (string= "[1] \" \"\n[1] \"one three\"\n[1] \"end\"\n" (org-babel-execute-src-block)))))) + +(ert-deftest test-ob-R/output-nonprinted () + (let (ess-ask-for-ess-directory ess-history-file) + (org-test-with-temp-text + "#+begin_src R :results output :session\n4.0 * 3.5\nlog(10)\nlog10(10)\n(3 + 1) * 5\n3^-1\n1/0\n#+end_src\n" + (should (string= "[1] 14\n[1] 2.302585\n[1] 1\n[1] 20\n[1] 0.3333333\n[1] Inf\n" (org-babel-execute-src-block)))))) + (provide 'test-ob-R) ;;; test-ob-R.el ends here -- 2.28.0 --=-=-=--