From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id 2GTnMPr1xGXTGQEA62LTzQ:P1 (envelope-from ) for ; Thu, 08 Feb 2024 16:40:42 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id 2GTnMPr1xGXTGQEA62LTzQ (envelope-from ) for ; Thu, 08 Feb 2024 16:40:42 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=FwHXM4Hr; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1707406842; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=6wDZ3216agJ+1RUPJ9U87fb5DP2RV3ofa5ZqbBdq2sU=; b=XVKJjSfS3BswVPjQ3n4ueZMsOuV2blVrMDYgJ/Pl3ueQwK5kLzysFq1AfnwGXj9syGGVDu XnJEC2EOubDFY9lRkAcfJxIv6ZXFUoDaoDBe2iLpVkmB2pNWbWKNq4/3tGxpSHkr2YikS4 d68+IyAZWWy9DDIaroP+kye0Rnro8ad9iJWzgklBH0A7uDZo+iWPxg3R4bNoIeM+0wocLI z5D1O57FlPtRxik507Usbgz3GhwA4E0urg53jtmMEBNQ0z/OrgaPofnE2AWNnwG6cJeRSt DUEbtUah8aCrxFz4jQbVBLxaMJlTJjvBiYCe+pgRO15rXDyCYGEV+yrxCdTZaA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=FwHXM4Hr; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Seal: i=1; s=key1; d=yhetil.org; t=1707406842; a=rsa-sha256; cv=none; b=Q3xDJDLOC4rw771Fus6/nvMqqLlN4Nj+t1Kan3A5Li9IUCP3ktUt0mSyNKzWVU3DfZi31D j/qab9QnzFa2uoGV5FHCsa3vDoNYscYQXi/Un8lhx689gg+oRZh9zIimddWvUtIPWe9aM8 6U0uGmnxbR+1GRIXuGK+AAx07XUjJ4oA+BIW1nsa02OVMusBZe7bLnMzndKyUErAxlvAIW TtB5yeAhbRiYHdtpGtjWyZLm3XmgxapKqzusRqFE9LX8IaGq0yU8UCg0XgM2tIcE0J5cJW Wa6Oqryt5FVMjiHGaOTpzrlG2qbyqW6O1HqJufl04L3lPRJGw10/2/6urnSueg== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3D9B06DCF0 for ; Thu, 8 Feb 2024 16:40:42 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rY6VF-0004OR-UN; Thu, 08 Feb 2024 10:39:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rY6VD-0004NL-PQ for emacs-orgmode@gnu.org; Thu, 08 Feb 2024 10:39:51 -0500 Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rY6VA-0004Hm-Ph for emacs-orgmode@gnu.org; Thu, 08 Feb 2024 10:39:51 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 7BF1C240104 for ; Thu, 8 Feb 2024 16:39:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1707406786; bh=I72+ItQQ93lciuwYxeduWMoZj0oRxEsIFw7cHrAemEE=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type: From; b=FwHXM4HrkRoQxwQQXIye/Q8v4uMczUxNVy0oV3zMjrdHXr2/zupaKaPdsZ6Ikd5/F 9hhfHZ1nCf8fA67G/dw7nPQ0cllp3K1OFa5ZMkPXOItFf+81IoMF32w+BPXs11NJlz +ngWjdQ0ioBnUEyxgCz7wFxBd+9KpUQ1rJsNVHDjy1QL4cj5izhm6WKpWh9QjqFdCZ T+pBjZMp/jwoKVt/lSIU9zpO+4/aMbxCHR431xOK8/UZ7A0COW4i99AF0a6/fZpsQB ZBSmi8TMl8nBcbTeFA/k/W3YjtPy1MmxZW9SMeYLNZwYfSwab2RQy1s9bFVXoTH50e WsC+QgOMPu82A== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4TW1Ps58rSz6tm4; Thu, 8 Feb 2024 16:39:45 +0100 (CET) From: Ihor Radchenko To: Jack Kamm Cc: Nasser Alkmim , emacs-orgmode@gnu.org Subject: Re: Async Python src block behavior with :dir header property In-Reply-To: <87v86z7iv0.fsf@gmail.com> References: <87il3a55bg.fsf@gmail.com> <8734udvjai.fsf@localhost> <86plxhely9.fsf@gmail.com> <8734udoejo.fsf@localhost> <86y1c5cztv.fsf@gmail.com> <875xz9o4nj.fsf@localhost> <87plxg95u4.fsf@gmail.com> <87msskmm47.fsf@localhost> <87mssi8ht2.fsf@gmail.com> <87plxdmtq3.fsf@localhost> <87bk8x81g3.fsf@gmail.com> <87y1c019a7.fsf@localhost> <877cjj915y.fsf@gmail.com> <87r0hr9f3b.fsf@localhost> <871q9q84id.fsf@gmail.com> <87mssdg00o.fsf@localhost> <87v86z7iv0.fsf@gmail.com> Date: Thu, 08 Feb 2024 15:43:17 +0000 Message-ID: <87plx7ynai.fsf@localhost> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=185.67.36.66; envelope-from=yantar92@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -9.89 X-Migadu-Queue-Id: 3D9B06DCF0 X-Spam-Score: -9.89 X-Migadu-Scanner: mx11.migadu.com X-TUID: jVR/QKDiR3/M --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Jack Kamm writes: > On executing any python session block I am getting the following error > which I think is caused by the above: > > Debugger entered--Lisp error: (void-variable buffer-name) That's a mystery. > Also, make shows a byte-compiler warning about this: > > Compiling /home/jack/src/org-mode/2024-01-async-file-results-dir/lisp/o= b-core.el... >=20=20=20 > In org-babel-session-buffer: > ob-core.el:785:15: Warning: reference to free variable =E2=80=98buffer-= name=E2=80=99 But this is a real problem. See the attached updated version of the patch set. --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=v2-0001-org-babel-execute-src-block-Force-dir-according-t.patch >From 1185e7a3569048b465b697fe173ce50a3ab7f0cc Mon Sep 17 00:00:00 2001 Message-ID: <1185e7a3569048b465b697fe173ce50a3ab7f0cc.1707406936.git.yantar92@posteo.net> From: Ihor Radchenko Date: Sat, 3 Feb 2024 16:47:57 +0100 Subject: [PATCH v2 1/3] org-babel-execute-src-block: Force :dir according to live session * lisp/ob-core.el (org-babel-execute-src-block): Force eval directory to follow live session buffer, if any. This is consistent with what we promise in the manual section "16.4 Environment of a Code Block">Choosing a working directory. Link: https://orgmode.org/list/87mssi8ht2.fsf@gmail.com --- lisp/ob-core.el | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/lisp/ob-core.el b/lisp/ob-core.el index 1de3af6ad..fd8d06c5d 100644 --- a/lisp/ob-core.el +++ b/lisp/ob-core.el @@ -840,14 +840,17 @@ (defun org-babel-execute-src-block (&optional arg info params executor-type) (dir (cdr (assq :dir params))) (mkdirp (cdr (assq :mkdirp params))) (default-directory - (cond - ((not dir) default-directory) - ((member mkdirp '("no" "nil" nil)) - (file-name-as-directory (expand-file-name dir))) - (t - (let ((d (file-name-as-directory (expand-file-name dir)))) - (make-directory d 'parents) - d)))) + (cond + ((not dir) default-directory) + ((when-let ((session (cdr (assq :session params)))) + (when (org-babel-comint-buffer-livep session) + (buffer-local-value 'default-directory (get-buffer session))))) + ((member mkdirp '("no" "nil" nil)) + (file-name-as-directory (expand-file-name dir))) + (t + (let ((d (file-name-as-directory (expand-file-name dir)))) + (make-directory d 'parents) + d)))) (cmd (intern (concat "org-babel-execute:" lang))) result exec-start-time) (unless (fboundp cmd) -- 2.43.0 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=v2-0002-ob-comint-Make-file-results-from-async-sessions-r.patch >From 8799d747ab2983f43dab117713342dbc92c6fc16 Mon Sep 17 00:00:00 2001 Message-ID: <8799d747ab2983f43dab117713342dbc92c6fc16.1707406936.git.yantar92@posteo.net> In-Reply-To: <1185e7a3569048b465b697fe173ce50a3ab7f0cc.1707406936.git.yantar92@posteo.net> References: <1185e7a3569048b465b697fe173ce50a3ab7f0cc.1707406936.git.yantar92@posteo.net> From: Jack Kamm Date: Wed, 31 Jan 2024 20:06:00 -0800 Subject: [PATCH v2 2/3] ob-comint: Make file results from async sessions respect :dir header * lisp/ob-comint.el (org-babel-comint-async-filter): Set default-directory before calling `org-babel-insert-result' https://list.orgmode.org/875xz9o4nj.fsf@localhost/T/#t --- lisp/ob-comint.el | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/lisp/ob-comint.el b/lisp/ob-comint.el index 7d258ea0e..349524701 100644 --- a/lisp/ob-comint.el +++ b/lisp/ob-comint.el @@ -224,6 +224,8 @@ (defun org-babel-comint-async-filter (string) (file-callback org-babel-comint-async-file-callback) (combined-string (concat org-babel-comint-async-dangling string)) (new-dangling combined-string) + ;; Assumes comint filter called with session buffer current + (session-dir default-directory) ;; list of UUID's matched by `org-babel-comint-async-indicator' uuid-list) (with-temp-buffer @@ -248,7 +250,8 @@ (defun org-babel-comint-async-filter (string) (let* ((info (org-babel-get-src-block-info)) (params (nth 2 info)) (result-params - (cdr (assq :result-params params)))) + (cdr (assq :result-params params))) + (default-directory session-dir)) (org-babel-insert-result (funcall file-callback (nth @@ -291,7 +294,8 @@ (defun org-babel-comint-async-filter (string) (let* ((info (org-babel-get-src-block-info)) (params (nth 2 info)) (result-params - (cdr (assq :result-params params)))) + (cdr (assq :result-params params))) + (default-directory session-dir)) (org-babel-insert-result res-str result-params info)) t)))) -- 2.43.0 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=v2-0003-org-babel-New-babel-backend-API-function-org-babe.patch >From 0d496b8fcc205810c994b32b2ab732305a4d9a26 Mon Sep 17 00:00:00 2001 Message-ID: <0d496b8fcc205810c994b32b2ab732305a4d9a26.1707406936.git.yantar92@posteo.net> In-Reply-To: <1185e7a3569048b465b697fe173ce50a3ab7f0cc.1707406936.git.yantar92@posteo.net> References: <1185e7a3569048b465b697fe173ce50a3ab7f0cc.1707406936.git.yantar92@posteo.net> From: Ihor Radchenko Date: Tue, 6 Feb 2024 15:02:48 +0100 Subject: [PATCH v2 3/3] org-babel: New babel backend API function org-babel-session-buffer: * lisp/ob-core.el (org-babel-session-buffer): New API function that return session buffer, if such buffer exists and is live. (org-babel-execute-src-block): Use `org-babel-session-buffer'. * lisp/ob-lua.el (org-babel-session-buffer:lua): * lisp/ob-python.el (org-babel-session-buffer:python): Provide API to retrieve session buffer name. (org-babel-python-initiate-session-by-key): Use `org-babel-session-buffer:python'. * etc/ORG-NEWS (Org babel backends are now expected to define an additional API function ~org-babel-session-buffer:~): Declare the API addition. Link: https://orgmode.org/list/87r0hr9f3b.fsf@localhost --- etc/ORG-NEWS | 11 +++++++++++ lisp/ob-core.el | 31 ++++++++++++++++++++++++++----- lisp/ob-lua.el | 5 +++++ lisp/ob-python.el | 8 ++++++-- 4 files changed, 48 insertions(+), 7 deletions(-) diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS index 92d363b80..7ac56702d 100644 --- a/etc/ORG-NEWS +++ b/etc/ORG-NEWS @@ -13,6 +13,17 @@ Please send Org bug reports to mailto:emacs-orgmode@gnu.org. * Version 9.7 (not released yet) ** Important announcements and breaking changes +*** Org babel backends are now expected to define an additional API function ~org-babel-session-buffer:~ + +Org babel now uses session buffer (if it exists) to retrieve +~default-directory~ environment during src block evaluation. + +By default, buffer named like session is checked. All the backends +that create sessions inside buffers named differently should provide a +function ~org-babel-session-buffer:~. The function must accept +two arguments - session name and info list (as returned by +~org-babel-get-src-block-info~); and return the session buffer name. + *** Org mode now fontifies whole table lines (including newline) according to ~org-table~ face Previously, leading indentation and trailing newline in table rows diff --git a/lisp/ob-core.el b/lisp/ob-core.el index fd8d06c5d..bfeac257b 100644 --- a/lisp/ob-core.el +++ b/lisp/ob-core.el @@ -767,8 +767,30 @@ (defun org-babel--file-desc (params result) (`(:file-desc) result) (`(:file-desc . ,(and (pred stringp) val)) val))) -(defvar *this*) ; Dynamically bound in `org-babel-execute-src-block' - ; and `org-babel-read' +(defvar *this*) +;; Dynamically bound in `org-babel-execute-src-block' +;; and `org-babel-read' + +(defun org-babel-session-buffer (&optional info) + "Return buffer name for session associated with current code block. +Return nil when no such live buffer with process exists. +When INFO is non-nil, it should be a list returned by +`org-babel-get-src-block-info'. +This function uses org-babel-session-buffer: function to +retrieve backend-specific session buffer name." + (declare-function org-babel-comint-buffer-livep "ob-comint" (buffer)) + (when-let* ((info (or info (org-babel-get-src-block-info 'no-eval))) + (lang (nth 0 info)) + (session (cdr (assq :session (nth 2 info)))) + (cmd (intern (concat "org-babel-session-buffer:" lang))) + (buffer-name + (if (fboundp cmd) + (funcall cmd session info) + ;; Use session name as buffer name by default. + session))) + (require 'ob-comint) + (when (org-babel-comint-buffer-livep buffer-name) + buffer-name))) ;;;###autoload (defun org-babel-execute-src-block (&optional arg info params executor-type) @@ -842,9 +864,8 @@ (defun org-babel-execute-src-block (&optional arg info params executor-type) (default-directory (cond ((not dir) default-directory) - ((when-let ((session (cdr (assq :session params)))) - (when (org-babel-comint-buffer-livep session) - (buffer-local-value 'default-directory (get-buffer session))))) + ((when-let ((session (org-babel-session-buffer info))) + (buffer-local-value 'default-directory (get-buffer session)))) ((member mkdirp '("no" "nil" nil)) (file-name-as-directory (expand-file-name dir))) (t diff --git a/lisp/ob-lua.el b/lisp/ob-lua.el index a2b830ca3..b241fccdc 100644 --- a/lisp/ob-lua.el +++ b/lisp/ob-lua.el @@ -184,6 +184,11 @@ (defun org-babel-lua-with-earmuffs (session) name (format "*%s*" name)))) +(defun org-babel-session-buffer:lua (session &optional _) + "Return session buffer name for SESSION." + (or (org-babel-lua-session-buffer session) + (org-babel-lua-with-earmuffs session))) + (defun org-babel-lua-without-earmuffs (session) "Remove stars around *SESSION*, leaving SESSION." (let ((name (if (stringp session) session (format "%s" session)))) diff --git a/lisp/ob-python.el b/lisp/ob-python.el index cfc0fdcb6..89cdf4c47 100644 --- a/lisp/ob-python.el +++ b/lisp/ob-python.el @@ -260,6 +260,11 @@ (defun org-babel-python-without-earmuffs (session) (substring name 1 (- (length name) 1)) name))) +(defun org-babel-session-buffer:python (session &optional _) + "Return session buffer name for SESSION." + (or (org-babel-python-session-buffer session) + (org-babel-python-with-earmuffs session))) + (defun org-babel-python--python-util-comint-end-of-output-p () "Return non-nil if the last prompt matches input prompt. Backport of `python-util-comint-end-of-output-p' to emacs28. To @@ -302,8 +307,7 @@ (defun org-babel-python-initiate-session-by-key (&optional session) initialized session." (save-window-excursion (let* ((session (if session (intern session) :default)) - (py-buffer (or (org-babel-python-session-buffer session) - (org-babel-python-with-earmuffs session))) + (py-buffer (org-babel-session-buffer:python session)) (python-shell-buffer-name (org-babel-python-without-earmuffs py-buffer)) (existing-session-p (comint-check-proc py-buffer)) -- 2.43.0 --=-=-= Content-Type: text/plain -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at --=-=-=--