From mboxrd@z Thu Jan 1 00:00:00 1970 From: Aaron Ecay Subject: Re: R code block produces only partial output Date: Mon, 01 Sep 2014 01:00:55 -0400 Message-ID: <87k35orl2w.fsf@gmail.com> References: <87iom8zd24.fsf@gmail.com> <877g2oz9gv.fsf@gmail.com> <87lhr27oap.fsf@gmail.com> <87r40uwavs.fsf@gmail.com> <8761i5kg8f.fsf@gmail.com> <87ppgcrg8n.fsf@gmail.com> <87lhr0qimr.fsf@gmail.com> <87wqa9owhv.fsf@gmail.com> <87oavkp2xa.fsf@gmail.com> <87lhq9tcdj.fsf@gmail.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:54173) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XOJjk-0000E7-TE for emacs-orgmode@gnu.org; Mon, 01 Sep 2014 01:01:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XOJjf-0003D6-Cw for emacs-orgmode@gnu.org; Mon, 01 Sep 2014 01:01:04 -0400 Received: from mail-qa0-x235.google.com ([2607:f8b0:400d:c00::235]:51198) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XOJjf-0003Cp-6c for emacs-orgmode@gnu.org; Mon, 01 Sep 2014 01:00:59 -0400 Received: by mail-qa0-f53.google.com with SMTP id w8so4500969qac.26 for ; Sun, 31 Aug 2014 22:00:58 -0700 (PDT) In-Reply-To: <87lhq9tcdj.fsf@gmail.com> List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: "Charles C. Berry" Cc: emacs-orgmode@gnu.org --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Chuck, Attached is a new version of the patch, including a few more tests. I have one question: 2014ko abuztuak 28an, Aaron Ecay-ek idatzi zuen: >> but the patch here uses on.exit(file.create(...)) to ensure that that >> file is created. One hiccup (not sure if it exists in master,too) is >> that starting a remote session and then trying to run src blocks from >> a buffer for a local file will hang (because a local temp file is used >> for sentinel). So there is still stuff to do. >=20 > Hmm, OK. Can you give a recipe to reproduce this? Everything seems to work for me using the following test code, although there is an unexplained ~10s delay while babel sits in the =E2=80=98(while (not (file-exists-p file)) ..= .)=E2=80=99 loop at the end of =E2=80=98org-babel-comint-eval-invisibly-and-wait-for-fi= le=E2=80=99. ,---- | #+name: foo | #+begin_src R :session *foo* :dir /ssh:aecay@foo:/home/aecay :results out= put | 1+1 | #+end_src |=20 | #+RESULTS: foo | : [1] 2 `---- Thanks, --=20 Aaron Ecay --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: inline; filename=0001-ob-R-overhaul-handling-of-output-results-type-in-a-s.patch Content-Transfer-Encoding: quoted-printable >From 65aa82b71e709250717896ab9bc85bf144a8ee30 Mon Sep 17 00:00:00 2001 From: Aaron Ecay Date: Sat, 16 Aug 2014 00:49:05 -0400 Subject: [PATCH] ob-R: overhaul handling of :output results type in a sessi= on MIME-Version: 1.0 Content-Type: text/plain; charset=3DUTF-8 Content-Transfer-Encoding: 8bit * lisp/ob-R.el (org-babel-R-check-evaluate-package): New function. (org-babel-R-initiate-session): Use it. (org-babel-R-evaluate-session): Use the evaluate package to capture session output. This uses the =E2=80=9Cevaluate=E2=80=9D R package[1] to capture the output (incl. warnings, errors, and messages) from a babel R session. This avoids the output showing up in the session buffer, and dodges some previous issues with removing R prompts (>) when scraping the output from the session buffer. Thanks to Charles C. Berry for assistance with this code. [1] --- lisp/ob-R.el | 73 +++++++++++++++++++++++++++++++++++--------= ---- testing/lisp/test-ob-R.el | 61 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 115 insertions(+), 19 deletions(-) diff --git a/lisp/ob-R.el b/lisp/ob-R.el index 41b943c..81b3290 100644 --- a/lisp/ob-R.el +++ b/lisp/ob-R.el @@ -245,6 +245,22 @@ This function is called by `org-babel-execute-src-bloc= k'." ((stringp value) (format "%s <- %S" name (org-no-properties value))) (t (format "%s <- %S" name (prin1-to-string value)))))) =20 +(defvar ess-execute-in-process-buffer) +(defun org-babel-R-check-evaluate-package (&optional recursive) + (save-window-excursion + (let ((ess-execute-in-process-buffer nil) + (r-buff (current-buffer))) + (ess-execute "library(evaluate)" nil "org-babel-R-auto") + (when (with-current-buffer "*org-babel-R-auto*" + (goto-char (point-min)) + (search-forward "Error" nil t)) + (if (and (not recursive) + (y-or-n-p "Cannot load the evaluate package required for babel session = support, would you like to install it from CRAN?")) + (progn + (message "Downloading and installing package (may take some time)") + (ess-execute "install.packages(\"evaluate\")" nil "org-babel-R-auto= ") + (org-babel-R-check-evaluate-package t)) + (user-error "R package evaluate is required, but not available.")))))) =20 (defvar ess-ask-for-ess-directory) ; dynamically scoped (defun org-babel-R-initiate-session (session params) @@ -261,7 +277,9 @@ This function is called by `org-babel-execute-src-block= '." (when (get-buffer session) ;; Session buffer exists, but with dead process (set-buffer session)) - (require 'ess) (R) + (require 'ess) + (R) + (org-babel-R-check-evaluate-package) (rename-buffer (if (bufferp session) (buffer-name session) @@ -365,8 +383,27 @@ last statement in BODY, as elisp." =20 (defvar ess-eval-visibly-p) =20 +(defconst org-babel-R-session-cmd + "local({ + on.exit(file.create(%S)) + sink(%S) + withCallingHandlers( + replay( + Filter(Negate(is.source), + evaluate(%S, envir=3Dparent.frame(2), new_device =3D = FALSE, + stop_on_error =3D 0L))), + message =3D function (x) { + cat(x$message); + invokeRestart(\"muffleMessage\") + }, + error =3D function (e) { + cat(e$message) + }) + sink()})" +"format string for an `R :session :results output' regime.") + (defun org-babel-R-evaluate-session - (session body result-type result-params column-names-p row-names-p) + (session body result-type result-params column-names-p row-names-p) "Evaluate BODY in SESSION. If RESULT-TYPE equals 'output then return standard output as a string. If RESULT-TYPE equals 'value then return the value of the @@ -396,23 +433,21 @@ last statement in BODY, as elisp." (org-babel-import-elisp-from-file tmp-file '(16))) column-names-p))) (output - (mapconcat - 'org-babel-chomp - (butlast - (delq nil - (mapcar - (lambda (line) (when (> (length line) 0) line)) - (mapcar - (lambda (line) ;; cleanup extra prompts left in output - (if (string-match - "^\\([ ]*[>+\\.][ ]?\\)+\\([[0-9]+\\|[ ]\\)" line) - (substring line (match-end 1)) - line)) - (org-babel-comint-with-output (session org-babel-R-eoe-output) - (insert (mapconcat 'org-babel-chomp - (list body org-babel-R-eoe-indicator) - "\n")) - (inferior-ess-send-input)))))) "\n")))) + (let* ((output-file (org-babel-temp-file "R-")) + (sentinel-file (concat output-file "-sentinel"))) + (org-babel-comint-eval-invisibly-and-wait-for-file + session sentinel-file + (format + org-babel-R-session-cmd + (org-babel-local-file-name sentinel-file) + (org-babel-local-file-name output-file) + (org-babel-chomp body))) + (with-temp-buffer + (insert-file-contents output-file) + (goto-char (point-min)) + (flush-lines "^$") + (delete-trailing-whitespace) + (buffer-string)))))) =20 (defun org-babel-R-process-value-result (result column-names-p) "R-specific processing of return value. diff --git a/testing/lisp/test-ob-R.el b/testing/lisp/test-ob-R.el index e3f13f1..3d7ebef 100644 --- a/testing/lisp/test-ob-R.el +++ b/testing/lisp/test-ob-R.el @@ -79,6 +79,67 @@ x (should (equal '(("col") ("a") ("b")) (org-babel-execute-src-block))))) =20 +(ert-deftest test-ob-R/session-output () + (org-test-with-temp-text "#+begin_src R :results output :session *foo* + 1 + message(\"hi\") + 2 + warning(\"hi2\") + 3 + stop(\"hi3\") + 4 +#+end_src +" + (should (string=3D (org-babel-execute-src-block) + "[1] 1 +hi +[1] 2 +Warning message: +hi2 +[1] 3 +Error: hi3 +[1] 4")))) + +(ert-deftest test-ob-R/session-output-stray-open-paren () + (org-test-with-temp-text "#+begin_src R :results output :session *foo* + ( +#+end_src +" + (should (string=3D (org-babel-execute-src-block) + ":2:0: unexpected end of input +1: ( + ^")))) + +(ert-deftest test-ob-R/session-output-stray-close-paren () + (org-test-with-temp-text "#+begin_src R :results output :session *foo* + ) +#+end_src +" + (should (string=3D (org-babel-execute-src-block) + ":1:1: unexpected ')' +1: ) + ^")))) + +(ert-deftest test-ob-R/session-output-stray-single-quote () + (org-test-with-temp-text "#+begin_src R :results output :session *foo* + ' +#+end_src +" + (should (string=3D (org-babel-execute-src-block) + ":1:1: unexpected INCOMPLETE_STRING +1: ' + ^")))) + +(ert-deftest test-ob-R/session-output-stray-dbl-quote () + (org-test-with-temp-text "#+begin_src R :results output :session *foo* + \" +#+end_src +" + (should (string=3D (org-babel-execute-src-block) + ":1:1: unexpected INCOMPLETE_STRING +1: \" + ^")))) + (provide 'test-ob-R) =20 ;;; test-ob-R.el ends here --=20 2.1.0 --=-=-=--