From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id iJrqM3nf214+MQAA0tVLHw (envelope-from ) for ; Sat, 06 Jun 2020 18:24:57 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id UIqlL3nf215jNgAAB5/wlQ (envelope-from ) for ; Sat, 06 Jun 2020 18:24:57 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 5ABB594001C for ; Sat, 6 Jun 2020 18:24:57 +0000 (UTC) Received: from localhost ([::1]:43156 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jhdV2-0005Nt-5h for larch@yhetil.org; Sat, 06 Jun 2020 14:24:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56646) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jhdUc-0005MF-JX for emacs-orgmode@gnu.org; Sat, 06 Jun 2020 14:24:30 -0400 Received: from mail-pj1-x102e.google.com ([2607:f8b0:4864:20::102e]:38658) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jhdUb-00012q-Gj for emacs-orgmode@gnu.org; Sat, 06 Jun 2020 14:24:30 -0400 Received: by mail-pj1-x102e.google.com with SMTP id d6so4225146pjs.3 for ; Sat, 06 Jun 2020 11:24:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:in-reply-to:references:date:message-id:mime-version; bh=ZtY6sibyhdw50e4gYPVeoRKWUAUe7+aD7ED2iVMbeyw=; b=LevA2dnvTr6CyS5EK+mQeNz3GUL5TU3To8o2LSouAiYke0snPWrtbCoN5uwXet5KWl VIs/1O7uKgy9TkLV4bLuc8qKjaIFjnwhwt+NPKVVM8YM93Cod1sSJDPn+ZeM3uCcoweh 6IsjojXaHdLqn7dXWLZsXJTJyRvrYSJ3qcKPQWKRA5WWmir5dW3XXrSzhhFSBqj8gt3u NxJX1EfM13EEx8hwDop9WuB0zMxyGqPLBSEyc9Xi+cKFUUnK0CwEyGnbypVCRWpqJjgT NCmQT3qZ5Iy+dPl/fKpcJeZ/8sqWa8iazxsa7uACyOmFqcCeKbtWiBNx4UCrPIv30s0k PlcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:in-reply-to:references:date :message-id:mime-version; bh=ZtY6sibyhdw50e4gYPVeoRKWUAUe7+aD7ED2iVMbeyw=; b=dNdQdVtmN+ogcGOCbm0Yh2olwCyXKe6YtjgZMSdOR69cQ11+H+mCWBq8PpTyYLaWiV DCdotBww5JeKI773/N/DodELv+xW/yY6yNQiEqnvPulwvmBTWFcSAQiCMBveETt170+m R2nwjeZfAZL9Ihx2DzJEMURGrhWvpISJZnrtBvNBATTMlcRqBqT36PX4WE9YHiyVPpoM lWbOeGuhDck15f9rWIeHwQZAh5v7V9ItFtIzPVMU3z4pqNBRlzYiuiKCVLUlody1Wk45 i4XBUtlpJ267+zcNZqlJdj86S2k212a2FRAioVL4pVyeu5BHvWXSAzM0ePOT0F50GO8g 3kCQ== X-Gm-Message-State: AOAM530BKFtmb7DrPOXv/m7DvHLLsK9OqUcGK8vX9qVnPjjVF8hf68hW hK6yc4w0QT9t2dL9AL//FCJMiQOf X-Google-Smtp-Source: ABdhPJzdCeuxVodYmJzr+444IDtj1Q6cvJPy57pQmkXFMmH6xz1KN7stE+yP0Ot3LlWhpYnuQW/N9A== X-Received: by 2002:a17:90a:6b08:: with SMTP id v8mr9511618pjj.162.1591467867499; Sat, 06 Jun 2020 11:24:27 -0700 (PDT) Received: from localhost (199-83-220-90.PUBLIC.monkeybrains.net. [199.83.220.90]) by smtp.gmail.com with ESMTPSA id 10sm2835789pfn.6.2020.06.06.11.24.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 06 Jun 2020 11:24:26 -0700 (PDT) From: Jack Kamm To: Matthew Lundin , =?utf-8?B?xaB0xJtww6FuIE7Em21lYw==?= , emacs-orgmode@gnu.org Subject: Re: Bug: ob-python mangles multiline :var values [9.3.6 (release_9.3.6-397-ga089600)] In-Reply-To: <87tv009l9a.fsf@gmail.com> References: <87sgico0ne.fsf@gmail.com> <87a71xtmh0.fsf@gmail.com> <87h7w18ca1.fsf@fastmail.fm> <87zh9t9hkw.fsf@gmail.com> <87tv009l9a.fsf@gmail.com> Date: Sat, 06 Jun 2020 11:24:25 -0700 Message-ID: <87r1usxdba.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=2607:f8b0:4864:20::102e; envelope-from=jackkamm@gmail.com; helo=mail-pj1-x102e.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=LevA2dnv; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Spam-Score: -1.71 X-TUID: /PXJJatTFX3x --=-=-= Content-Type: text/plain Hi, Here's my second attempt to fix the issue. I'll wait a few days for comments before merging, to try and avoid causing a bug like last time. It turns out the problem was deeper than just the :var argument, multiline strings in the body were also getting mangled. For example, this block had the same problem as the original report: #+begin_src python text = """a b c """ return text #+end_src #+RESULTS: : a : b : c My fix this time was to use functions from python.el to indent, and to detect whether we are in a string and shouldn't be indented. I also added a couple more unit tests, one for multiline strings, and one for the variable scope/assignment issue that Matt reported. Cheers, Jack --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-ob-python.el-Fix-multiline-strings-in-non-session-re.patch >From 179178d39f6216172e1a070f570cf941f99b1a89 Mon Sep 17 00:00:00 2001 From: Jack Kamm Date: Sat, 6 Jun 2020 10:59:23 -0700 Subject: [PATCH] ob-python.el: Fix multiline strings in non-session :results value * lisp/ob-python.el (org-babel-python-evaluate-external-process): Use functions from python.el to indent lines, avoiding multiline strings. * testing/lisp/test-ob-python.el (test-ob-python/multiline-var): Set test as expected to succeed. (test-ob-python/multiline-str): Add test for multiline string in body. (test-ob-python/header-var-assignment): Test that :var is in correct scope and can be assigned to. cf. https://orgmode.org/list/87tv009l9a.fsf@gmail.com/#t --- lisp/ob-python.el | 14 +++++++++----- testing/lisp/test-ob-python.el | 20 +++++++++++++++++++- 2 files changed, 28 insertions(+), 6 deletions(-) diff --git a/lisp/ob-python.el b/lisp/ob-python.el index dbcfac08d..622f69ce3 100644 --- a/lisp/ob-python.el +++ b/lisp/ob-python.el @@ -296,11 +296,15 @@ (defun org-babel-python-evaluate-external-process (if (member "pp" result-params) org-babel-python-pp-wrapper-method org-babel-python-wrapper-method) - (mapconcat - (lambda (line) (format "\t%s" line)) - (split-string (org-remove-indentation (org-trim body)) - "[\r\n]") - "\n") + (with-temp-buffer + (insert body) + (goto-char (point-min)) + (while (< (point) (point-max)) + (unless (python-syntax-context 'string) + (python-indent-shift-right (line-beginning-position) + (line-end-position))) + (forward-line 1)) + (buffer-string)) (org-babel-process-file-name tmp-file 'noquote)))) (org-babel-eval-read-file tmp-file)))))) (org-babel-result-cond result-params diff --git a/testing/lisp/test-ob-python.el b/testing/lisp/test-ob-python.el index 763942b16..e3f0571a1 100644 --- a/testing/lisp/test-ob-python.el +++ b/testing/lisp/test-ob-python.el @@ -174,7 +174,6 @@ (ert-deftest test-ob-python/assign-underscore () (org-babel-execute-src-block))))) (ert-deftest test-ob-python/multiline-var () - :expected-result :failed (should (equal "a\nb\nc" (org-test-with-temp-text "#+begin_src python :var text=\"a\\nb\\nc\" @@ -182,6 +181,25 @@ (ert-deftest test-ob-python/multiline-var () #+end_src" (org-babel-execute-src-block))))) +(ert-deftest test-ob-python/multiline-str () + (should + (equal "a\nb\nc" + (org-test-with-temp-text "#+begin_src python +text=\"a\\nb\\nc\" +return text +#+end_src" + (org-babel-execute-src-block))))) + +(ert-deftest test-ob-python/header-var-assignment () + (should + (equal "success" + (org-test-with-temp-text "#+begin_src python :var text=\"failure\" +text +text=\"success\" +return text +#+end_src" + (org-babel-execute-src-block))))) + (provide 'test-ob-python) ;;; test-ob-python.el ends here -- 2.27.0 --=-=-=--