From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id j8+BLxrgZl+LZAAA0tVLHw (envelope-from ) for ; Sun, 20 Sep 2020 04:52:42 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id IG+0KhrgZl8ODgAAbx9fmQ (envelope-from ) for ; Sun, 20 Sep 2020 04:52:42 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id EB2AA940238 for ; Sun, 20 Sep 2020 04:52:41 +0000 (UTC) Received: from localhost ([::1]:48812 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kJrL5-0006dB-6i for larch@yhetil.org; Sun, 20 Sep 2020 00:52:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46140) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kJrKS-0006cm-Fi for emacs-orgmode@gnu.org; Sun, 20 Sep 2020 00:52:00 -0400 Received: from mail-pg1-x544.google.com ([2607:f8b0:4864:20::544]:33326) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kJrKQ-0007x6-CC for emacs-orgmode@gnu.org; Sun, 20 Sep 2020 00:52:00 -0400 Received: by mail-pg1-x544.google.com with SMTP id o25so1096411pgm.0 for ; Sat, 19 Sep 2020 21:51:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=oJ+WtyubiqoDdznB5c0hXcPRT5xTdVP0I6vaV9kxEsw=; b=eNzMV7oBHunu47X0utR+1m/ohQ5jYmL4hNRmoAULWvh4XAcEsWA/+HfsmdRsGAzb+s DIw7TXl6kSHF5llGQMKI/15El8wMJSxOpf6Ujed7xm2RMprRoF7qfyjj1YrvjA/oqjBy ea4UUudlAtKflWamzBOaLj+L6Wd7UUY3XJkdjGAobl/EjJ2w9bRsUs6SxIlTWZU9/5Bd uu/kHDMTm3vj00gCnK4aVg3SpLXVbEmkbpoqvugIYcj/jkVTX5jNYF+5gFlssQ0l3Tue BmH03sXSBVfUS/BLCvDEid5/6Pi1zgjQjBb3SAHYylqHPgNxZIpvLkRTyz0ToxKkBTAU XAHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=oJ+WtyubiqoDdznB5c0hXcPRT5xTdVP0I6vaV9kxEsw=; b=iBlPEzsRWfIT/RsOpKMEodRTrVDp1Yd5QRHTxegOPwWPa1iXtkOi/u3cCrOkXAFjHa fL/IilvSuSigMgMXHJHE6Mt5NCWotyj6ZcFzibYDRKOejhe3GIDAUVoMDYZAlNkW5zTg kglPWJlwqqmKAaX98LsUjvb6QFmzVrXloS8J1K1vZ8zZaYSxFGricmdvn8rK4T+/MT70 zTkZnW+uhDoZjAWyI5QYLPxbU7HrYpIJ1w5m/Qa8AzbUOezwPocD3Mcp6e9PumevjOmg xspzGNG1kC9+TDnBkWdPoot5WKlDXp2gXr9Afzmu38N3fKaHXHhA77UL6sY8SBiVTgeg zVoQ== X-Gm-Message-State: AOAM531vSvDEVRwCOBnSs1jTh/x/a3QGDLSsaIlPUjr035rUQ2+KRw3O IQK3fxTU8m8W6ziuJdEVAME= X-Google-Smtp-Source: ABdhPJxF0uFRhFH1jTM/mBXfUccAjWxCKxbTCconNGHYdVKBDrdIqlPqLqlVWUd+UT2IHKlsG3sHjw== X-Received: by 2002:a62:17cd:0:b029:13c:1611:652b with SMTP id 196-20020a6217cd0000b029013c1611652bmr39489006pfx.11.1600577516432; Sat, 19 Sep 2020 21:51:56 -0700 (PDT) Received: from localhost (199-83-220-90.PUBLIC.monkeybrains.net. [199.83.220.90]) by smtp.gmail.com with ESMTPSA id a74sm8187722pfa.16.2020.09.19.21.51.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Sep 2020 21:51:54 -0700 (PDT) From: Jack Kamm To: Kyle Meyer Subject: Re: [PATCH] Expanded ob-python results handling and plotting In-Reply-To: <874kokf6zl.fsf@gmail.com> References: <87eenpfe77.fsf@gmail.com> <871rjptdje.fsf@kyleam.com> <87blisfacz.fsf@gmail.com> <874kokf6zl.fsf@gmail.com> Date: Sat, 19 Sep 2020 21:51:54 -0700 Message-ID: <87y2l51251.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=2607:f8b0:4864:20::544; envelope-from=jackkamm@gmail.com; helo=mail-pg1-x544.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: emacs-orgmode@gnu.org Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=eNzMV7oB; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Spam-Score: -1.71 X-TUID: NqC22XGzlA+D --=-=-= Content-Type: text/plain After letting it sit, I'm not sure that my patch above is a good idea anymore. While it would be useful, it also adds substantial complexity to ob-python. For now, I think I prefer to keep ob-python leaner, so am going to hold off on this. An alternative approach is to have the user handle graphics and dataframes via noweb or header arguments. I've added a couple examples on worg, demonstrating how to use noweb to insert boilerplate code for handling matplotlib figures and pandas dataframes [0,1]. Additionally, I'm attaching a small patch to make it easier to handle graphics/dataframes via the :return header argument, as an alternative to noweb. The patch includes a couple examples in ORG-NEWS illustrating this. I'll wait a week or so for comments before merging this new, more limited patch into master. [0] https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-python.html [1] worg commit 59e320ad Cheers, Jack --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-ob-python-Improvements-to-return-header-argument.patch >From 118d8b5eb817e9a21e9d84f2f942fcc841ddc51f Mon Sep 17 00:00:00 2001 From: Jack Kamm Date: Sat, 19 Sep 2020 08:44:30 -0700 Subject: [PATCH] ob-python: Improvements to :return header argument * lisp/ob-python.el (org-babel-execute:python): Allow return-val to be non-nil in sessions, and concatenate it after the expanded body. --- etc/ORG-NEWS | 53 +++++++++++++++++++++++++++++++++++++++++++++++ lisp/ob-python.el | 11 ++++++---- 2 files changed, 60 insertions(+), 4 deletions(-) diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS index 0ed626fb7..50a455ad5 100644 --- a/etc/ORG-NEWS +++ b/etc/ORG-NEWS @@ -11,6 +11,59 @@ See the end of the file for license conditions. Please send Org bug reports to mailto:emacs-orgmode@gnu.org. * Version 9.5 (not yet released) +** New features +*** =ob-python= improvements to =:return= header argument + +The =:return= header argument in =ob-python= now works for session +blocks as well as non-session blocks. Also, it now works with the +=:epilogue= header argument -- previously, setting the =:return= +header would cause the =:epilogue= to be ignored. + +This change allows more easily moving boilerplate out of the main code +block and into the header. For example, for plotting, we need to add +boilerplate to save the figure to a file and return the +filename. Instead of doing this within the code block, we can now +handle it through the header arguments as follows: + +#+BEGIN_SRC org +,#+header: :var fname="/home/jack/tmp/plot.svg" +,#+header: :epilogue plt.savefig(fname) +,#+header: :return fname +,#+begin_src python :results value file + import matplotlib, numpy + import matplotlib.pyplot as plt + fig=plt.figure(figsize=(4,2)) + x=numpy.linspace(-15,15) + plt.plot(numpy.sin(x)/x) + fig.tight_layout() +,#+end_src + +,#+RESULTS: +[[file:/home/jack/tmp/plot.svg]] +#+END_SRC + +As another example, we can use =:return= with the external [[https://pypi.org/project/tabulate/][tabulate]] +package, to convert pandas Dataframes into orgmode tables: + +#+begin_src org +,#+header: :prologue from tabulate import tabulate +,#+header: :return tabulate(table, headers=table.columns, tablefmt="orgtbl") +,#+begin_src python :results value raw :session + import pandas as pd + table = pd.DataFrame({ + "a": [1,2,3], + "b": [4,5,6] + }) +,#+end_src + +,#+RESULTS: +| | a | b | +|---+---+---| +| 0 | 1 | 4 | +| 1 | 2 | 5 | +| 2 | 3 | 6 | +#+end_src + * Version 9.4 ** Incompatible changes *** Possibly broken internal file links: please check and fix diff --git a/lisp/ob-python.el b/lisp/ob-python.el index 00a7c1a2d..785b9191b 100644 --- a/lisp/ob-python.el +++ b/lisp/ob-python.el @@ -81,13 +81,16 @@ (defun org-babel-execute:python (body params) (cdr (assq :session params)))) (result-params (cdr (assq :result-params params))) (result-type (cdr (assq :result-type params))) - (return-val (when (and (eq result-type 'value) (not session)) + (return-val (when (eq result-type 'value) (cdr (assq :return params)))) (preamble (cdr (assq :preamble params))) (full-body - (org-babel-expand-body:generic - (concat body (if return-val (format "\nreturn %s" return-val) "")) - params (org-babel-variable-assignments:python params))) + (concat + (org-babel-expand-body:generic + body params + (org-babel-variable-assignments:python params)) + (when return-val + (format (if session "\n%s" "\nreturn %s") return-val)))) (result (org-babel-python-evaluate session full-body result-type result-params preamble))) (org-babel-reassemble-table -- 2.28.0 --=-=-=--