From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id sB3EIl4frmJpvwAAbAwnHQ (envelope-from ) for ; Sat, 18 Jun 2022 20:54:22 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id mLicIV4frmIKvQAAG6o9tA (envelope-from ) for ; Sat, 18 Jun 2022 20:54:22 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id EC996352A for ; Sat, 18 Jun 2022 20:54:21 +0200 (CEST) Received: from localhost ([::1]:52552 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o2daO-0006x4-A8 for larch@yhetil.org; Sat, 18 Jun 2022 14:54:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53696) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2dZt-0006wh-Gt for emacs-orgmode@gnu.org; Sat, 18 Jun 2022 14:53:49 -0400 Received: from smtp1-g21.free.fr ([2a01:e0c:1:1599::10]:59094) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2dZp-0000rx-W3 for emacs-orgmode@gnu.org; Sat, 18 Jun 2022 14:53:49 -0400 Received: from tosh-laptop (unknown [IPv6:2a01:e0a:505:3460:1a16:a0c4:3f89:c0d9]) by smtp1-g21.free.fr (Postfix) with ESMTPS id 02E79B00539; Sat, 18 Jun 2022 20:53:40 +0200 (CEST) Received: by tosh-laptop (sSMTP sendmail emulation); Sat, 18 Jun 2022 20:54:57 +0200 From: Bruno Barbier To: Felix Freeman , emacs-orgmode@gnu.org Subject: Re: [BUG] ob-shell: cmdline and stdin broken when used with TRAMP In-Reply-To: References: Date: Sat, 18 Jun 2022 20:54:57 +0200 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: softfail client-ip=2a01:e0c:1:1599::10; envelope-from=brubar.cs@gmail.com; helo=smtp1-g21.free.fr X-Spam_score_int: 12 X-Spam_score: 1.2 X-Spam_bar: + X-Spam_report: (1.2 / 5.0 requ) BAYES_00=-1.9, DKIM_ADSP_CUSTOM_MED=0.001, FORGED_GMAIL_RCVD=1, FREEMAIL_FROM=0.001, MISSING_MID=0.497, NML_ADSP_CUSTOM_MED=0.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action Message-Id: X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1655578462; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=20OLapTHDXVCmz0rTnGaveggL+6e1z5xWLXozW5ttXM=; b=l+pyGV4w3T6voxvqk+2NQfpbzDpFtaXk8q89yOfP81MAW1UtVfMqZV74XmtAbk3+XnKo5b ezTkeTsSqrlHXaxGjHyQYd/GG1GSsmHxIYoSH0cF5oYfEcsjncECe1Q8amC0dDzFSX9bUm pcS3dsIV9ovAXb5ki15pM2EwElDOnH+vEphrpyx+eTmmhrXivjucKVACguDRuZmE459Bm8 BOKAFiaydiUGP7bE0+6mup5M+GIyO1tZdZj7cLC6F9eXWb71X2U/57EwriTzC8eJLQDimo +E8d3Y8yOuaI9rNC5DsaBuAj/7oddzdzKiDztRkJdW9ruwzm3siUQ5Fxj6r5VA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1655578462; a=rsa-sha256; cv=none; b=NqdPxf8e4XgE0B2tE7eHAYhkhBMWrX748slLczYrNv1JShTaSpDneTqNIiV+LdqNV0s6d8 MB4N3yb1kFM74hbC3rnEjARoibBQNCAlU4Srny1VILf2XWVffQlisgd24/uYAmbhlxin2m 5LXSqV9v24+PUo1WKKm+7dXo7WFo2s2VEAIQlS1HOVG0ZKpViou1mjL1DHTyCmX4qFXBrL QBIGX+mWYYCasc77YqhLsPbOJdoXR3uZ+Z0FGvgw53eU6oD72vvsVLe9s2d8Q2QvL1h8We DaTl3onvbNUiX1UhFJaRf7yf2ZmIfEMCHXNoVZaFdpm89tRS45Onh9fh6nrxvw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: 2.21 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: EC996352A X-Spam-Score: 2.21 X-Migadu-Scanner: scn1.migadu.com X-TUID: LrJTVQXN0vvv --=-=-= Content-Type: text/plain "Felix Freeman" via "General discussions about Org-mode." writes: > When using TRAMP, ob-shell's :cmdline and :stdin header options are > broken on org-babel. > I can reproduce the problem (thanks for your nice MCE). I'm using: Org mode version 9.5.4 (release_9.5.4-32-g82036c) GNU Emacs 29.0.50 >From what I understand, the function 'org-babel-sh-evaluate' relies on 'call-process'; and that function ignores file name handlers; as TRAMP relies on those file name handlers, it just cannot do the right thing. Using 'process-file' instead works for me. See the attached patch. I've also included a test, as the problem is reproducible with TRAMP "/mock::" connection. But, that test will only work on GNU/Linux systems. Warning: that's my first attempt to write a patch, and I don't have (yet) signed the copyright papers. Bruno --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-ob-shell-Use-process-file-when-stdin-or-cmdline.patch Content-Description: Patch for ob-shell: cmdline and stdin broken when used with TRAMP >From 9e8c114b738ddc633b50ca48e828676b92be784f Mon Sep 17 00:00:00 2001 From: Bruno BARBIER Date: Sat, 18 Jun 2022 09:48:01 +0200 Subject: [PATCH] ob-shell: Use 'process-file' when stdin or cmdline lib/ob-shell.el (org-babel-sh-evaluate): Use 'process-file' (instead of 'call-process-shell-command') so that 'org-babel-sh-evaluate' will invoke file name handlers based on 'default-directory', if needed, like when using a remote directory. testing/lisp/test-ob-shell.el (ob-shell/remote-with-stdin-or-cmdline): New test. testing/org-test.el (org-test-tramp-remote-dir): New constant. --- lisp/ob-shell.el | 17 ++++++++---- testing/lisp/test-ob-shell.el | 52 +++++++++++++++++++++++++++++++++++ testing/org-test.el | 4 +++ 3 files changed, 67 insertions(+), 6 deletions(-) diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el index 4454e3b5d..515095f9b 100644 --- a/lisp/ob-shell.el +++ b/lisp/ob-shell.el @@ -249,12 +249,17 @@ (defun org-babel-sh-evaluate (session body &optional params stdin cmdline) (set-file-modes script-file #o755) (with-temp-file stdin-file (insert (or stdin ""))) (with-temp-buffer - (call-process-shell-command - (concat (if shebang script-file - (format "%s %s" shell-file-name script-file)) - (and cmdline (concat " " cmdline))) - stdin-file - (current-buffer)) + (with-connection-local-variables + (apply #'process-file + (if shebang (file-local-name script-file) + shell-file-name) + stdin-file + (current-buffer) + nil + (if shebang (when cmdline (list cmdline)) + (list shell-command-switch + (concat (file-local-name script-file) " " cmdline))) + )) (buffer-string)))) (session ; session evaluation (mapconcat diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el index 2f346f699..05ee810a0 100644 --- a/testing/lisp/test-ob-shell.el +++ b/testing/lisp/test-ob-shell.el @@ -106,6 +106,58 @@ (ert-deftest ob-shell/simple-list () "#+BEGIN_SRC sh :results output :var l='(1 2)\necho ${l}\n#+END_SRC" (org-trim (org-babel-execute-src-block)))))) +(ert-deftest ob-shell/remote-with-stdin-or-cmdline () + "Test :stdin and :cmdline with a remote directory." + ;; We assume 'default-directory' is a local directory. + (dolist (spec `( () + (:dir ,org-test-tramp-remote-dir) + (:dir ,org-test-tramp-remote-dir :cmdline t) + (:dir ,org-test-tramp-remote-dir :stdin t) + (:dir ,org-test-tramp-remote-dir :cmdline t :shebang t) + (:dir ,org-test-tramp-remote-dir :stdin t :shebang t) + (:dir ,org-test-tramp-remote-dir :cmdline t :stdin t :shebang t) + (:cmdline t) + (:stdin t) + (:cmdline t :shebang t) + (:stdin t :shebang t) + (:cmdline t :stdin t :shebang t) + )) + (let ((default-directory (or (plist-get :dir spec) default-directory)) + (org-confirm-babel-evaluate nil) + (params-line "") + (who-line " export who=tramp") + (args-line " echo ARGS: --verbose 23 71") + ) + (when-let ((dir (plist-get :dir spec))) + (setq params-line (concat params-line " " ":dir " dir))) + (when (plist-get :stdin spec) + (setq who-line " read -r who") + (setq params-line (concat params-line " :stdin input"))) + (when (plist-get :cmdline spec) + (setq args-line " echo \"ARGS: $*\"") + (setq params-line (concat params-line " :cmdline \"--verbose 23 71\""))) + (when (plist-get :shebang spec) + (setq params-line (concat params-line " :shebang \"#!/bin/sh\""))) + (let* ((result (org-test-with-temp-text + (mapconcat #'identity + (list "#+name: input" + "tramp" + "" + (concat "" + "#+begin_src sh :results output " params-line) + args-line + who-line + " echo \"hello $who from $(pwd)/\"" + "#+end_src") + "\n") + (org-trim (org-babel-execute-src-block)))) + (expected (concat "ARGS: --verbose 23 71" + "\nhello tramp from " (file-local-name default-directory))) + (correct (equal result expected)) + ) + (should (equal result expected)) + )))) + (provide 'test-ob-shell) ;;; test-ob-shell.el ends here diff --git a/testing/org-test.el b/testing/org-test.el index 0f1e254aa..7212544f6 100644 --- a/testing/org-test.el +++ b/testing/org-test.el @@ -96,6 +96,10 @@ (defconst org-test-link-in-heading-file (defconst org-id-locations-file (expand-file-name ".test-org-id-locations" org-test-dir)) +(defconst org-test-tramp-remote-dir "/mock::/tmp/" + "Remote tramp directory. +We really should use 'tramp-test-temporary-file-directory', but that would require TRAMP sources.") + ;;; Functions for writing tests (put 'missing-test-dependency -- 2.35.1 --=-=-=--