emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* [RFC] ob-reticulate: R+Python interface from Babel
@ 2020-08-24 15:26 Jack Kamm
  2020-08-25 23:34 ` Kyle Andrews
                   ` (2 more replies)
  0 siblings, 3 replies; 8+ messages in thread
From: Jack Kamm @ 2020-08-24 15:26 UTC (permalink / raw)
  To: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 1473 bytes --]

Hi all,

Reticulate is an R package for interfacing between R and Python. It allows accessing objects in a Python session from R and vice versa. See https://rstudio.github.io/reticulate/ for more info about it.

I've written a small patch for using reticulate from org-babel. It allows creating a source block of lang "reticulate", which behaves as Python for font highlighting and editing, but is executed in an R session via reticulate.

I'm wondering whether this should go into org-mode, or whether to package this separately. I'm also curious whether this would be useful to anyone here. Any feedback is appreciated.

The main advantage of reticulate is being able to access Python objects directly from R and vice versa, without having to write them to a separate file or pass them through the ":var" header argument. For example, we could do the following:

#+begin_src reticulate :session
  import pandas as pd

  fib = [0, 1]
  for _ in range(10):
      fib.append(fib[-1] + fib[-2])

  df = pd.DataFrame({
      "i": list(range(len(fib))),
      "F_i": fib
  })
#+end_src

#+begin_src R :session :results graphics value file :file fig.png
  library(reticulate)
  with(py$df, plot(i, F_i))
#+end_src

Reticulate source blocks support both "value" and "output" results, and even supports graphics with matplotlib. It's primarily intended to be used in sessions, and the ":session" header argument should match between reticulate and R source blocks.

Cheers,
Jack


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-reticulate-Babel-source-lang-for-R-Python-reticul.patch --]
[-- Type: text/x-patch, Size: 2845 bytes --]

From 0f691a200cf088c72f93f7552d73caeafb8d588f Mon Sep 17 00:00:00 2001
From: Jack Kamm <jackkamm@gmail.com>
Date: Mon, 24 Aug 2020 08:02:17 -0700
Subject: [PATCH] ob-reticulate: Babel source lang for R+Python reticulate
 package

* lisp/ob-reticulate.el: New babel source block lang for
R's reticulate package for evaluating Python code.
* lisp/org-src.el (org-src-lang-modes): Add reticulate.
---
 lisp/ob-reticulate.el | 50 +++++++++++++++++++++++++++++++++++++++++++
 lisp/org-src.el       |  1 +
 2 files changed, 51 insertions(+)
 create mode 100644 lisp/ob-reticulate.el

diff --git a/lisp/ob-reticulate.el b/lisp/ob-reticulate.el
new file mode 100644
index 000000000..7da48681c
--- /dev/null
+++ b/lisp/ob-reticulate.el
@@ -0,0 +1,50 @@
+;;; ob-reticulate.el --- Babel Functions for reticulate -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2020 Free Software Foundation, Inc.
+
+;; Author: Jack Kamm
+;; Keywords: literate programming, reproducible research, R, statistics
+;; Homepage: https://orgmode.org
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; Org-Babel support for the R package reticulate.
+
+;;; Code:
+
+(require 'ob-R)
+(require 'ob-python)
+
+(defalias 'org-babel-edit-prep:reticulate 'org-babel-edit-prep:R)
+
+(defun org-babel-execute:reticulate (body params)
+  (let* ((tmp-src-file (org-babel-temp-file "reticulate-"))
+	 (result-type (cdr (assq :result-type params))))
+    (with-temp-file tmp-src-file (insert body))
+    (org-babel-execute:R
+     (format (concat "reticulate::py_run_string(\"%s\")"
+		     (when (equal result-type 'value) "
+reticulate::py$`__org_babel_python_final`"))
+	     (format org-babel-python--eval-ast
+		     (org-babel-process-file-name
+		      tmp-src-file 'noquote)))
+     params)))
+
+(provide 'ob-reticulate)
+
+;;; ob-reticulate.el ends here
diff --git a/lisp/org-src.el b/lisp/org-src.el
index 28733d011..1b3d83f87 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -197,6 +197,7 @@ (defcustom org-src-lang-modes
     ("dot" . fundamental)
     ("elisp" . emacs-lisp)
     ("ocaml" . tuareg)
+    ("reticulate" . python)
     ("screen" . shell-script)
     ("shell" . sh)
     ("sqlite" . sql))
-- 
2.28.0


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [RFC] ob-reticulate: R+Python interface from Babel
  2020-08-24 15:26 [RFC] ob-reticulate: R+Python interface from Babel Jack Kamm
@ 2020-08-25 23:34 ` Kyle Andrews
  2020-08-29 13:23 ` Kyle Meyer
  2020-10-12  4:15 ` Jack Kamm
  2 siblings, 0 replies; 8+ messages in thread
From: Kyle Andrews @ 2020-08-25 23:34 UTC (permalink / raw)
  To: Jack Kamm; +Cc: Emacs orgmode

[-- Attachment #1: Type: text/plain, Size: 1812 bytes --]

Hi Jack,

As a frequent reticulate user I am very excited to see this patch. I hope
others feel the same and it gets included into org mode as I cannot wait to
use it.

Best Regards,
Kyle

On Mon, Aug 24, 2020, 11:28 Jack Kamm <jackkamm@gmail.com> wrote:

> Hi all,
>
> Reticulate is an R package for interfacing between R and Python. It allows
> accessing objects in a Python session from R and vice versa. See
> https://rstudio.github.io/reticulate/ for more info about it.
>
> I've written a small patch for using reticulate from org-babel. It allows
> creating a source block of lang "reticulate", which behaves as Python for
> font highlighting and editing, but is executed in an R session via
> reticulate.
>
> I'm wondering whether this should go into org-mode, or whether to package
> this separately. I'm also curious whether this would be useful to anyone
> here. Any feedback is appreciated.
>
> The main advantage of reticulate is being able to access Python objects
> directly from R and vice versa, without having to write them to a separate
> file or pass them through the ":var" header argument. For example, we could
> do the following:
>
> #+begin_src reticulate :session
>   import pandas as pd
>
>   fib = [0, 1]
>   for _ in range(10):
>       fib.append(fib[-1] + fib[-2])
>
>   df = pd.DataFrame({
>       "i": list(range(len(fib))),
>       "F_i": fib
>   })
> #+end_src
>
> #+begin_src R :session :results graphics value file :file fig.png
>   library(reticulate)
>   with(py$df, plot(i, F_i))
> #+end_src
>
> Reticulate source blocks support both "value" and "output" results, and
> even supports graphics with matplotlib. It's primarily intended to be used
> in sessions, and the ":session" header argument should match between
> reticulate and R source blocks.
>
> Cheers,
> Jack
>
>

[-- Attachment #2: Type: text/html, Size: 2465 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC] ob-reticulate: R+Python interface from Babel
  2020-08-24 15:26 [RFC] ob-reticulate: R+Python interface from Babel Jack Kamm
  2020-08-25 23:34 ` Kyle Andrews
@ 2020-08-29 13:23 ` Kyle Meyer
  2020-09-04  7:14   ` Bastien
  2020-10-12  4:15 ` Jack Kamm
  2 siblings, 1 reply; 8+ messages in thread
From: Kyle Meyer @ 2020-08-29 13:23 UTC (permalink / raw)
  To: Jack Kamm; +Cc: emacs-orgmode

Jack Kamm writes:

> Hi all,
>
> Reticulate is an R package for interfacing between R and Python. It
> allows accessing objects in a Python session from R and vice
> versa. See https://rstudio.github.io/reticulate/ for more info about
> it.
>
> I've written a small patch for using reticulate from org-babel. It
> allows creating a source block of lang "reticulate", which behaves as
> Python for font highlighting and editing, but is executed in an R
> session via reticulate.

Neat, thanks for sharing.

> I'm wondering whether this should go into org-mode, or whether to
> package this separately. I'm also curious whether this would be useful
> to anyone here. Any feedback is appreciated.

It'd be good to hear from others, but in my view this would be fine to
add to Org proper.

I will say that in general new Babel libraries make me nervous.  My
impression is that Babel brings in a good number of bug reports, and a
new library is adding surface to that area [*] while not necessarily
adding eyes and hands.  Of course that worry doesn't apply here, as
you're already taking care of ob-python.

  [*] And that surface can be quite challenging to deal with because it
      is not just Elisp code; it brings in a whole outside language that
      other developers may have no clue about.

> +;;; Code:
> +
> +(require 'ob-R)
> +(require 'ob-python)
> +
> +(defalias 'org-babel-edit-prep:reticulate 'org-babel-edit-prep:R)
> +
> +(defun org-babel-execute:reticulate (body params)
> +  (let* ((tmp-src-file (org-babel-temp-file "reticulate-"))
> +	 (result-type (cdr (assq :result-type params))))
> +    (with-temp-file tmp-src-file (insert body))
> +    (org-babel-execute:R
> +     (format (concat "reticulate::py_run_string(\"%s\")"
> +		     (when (equal result-type 'value) "
> +reticulate::py$`__org_babel_python_final`"))
> +	     (format org-babel-python--eval-ast
> +		     (org-babel-process-file-name
> +		      tmp-src-file 'noquote)))
> +     params)))
> +
> +(provide 'ob-reticulate)

Well, that's pleasantly few lines of code :)


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC] ob-reticulate: R+Python interface from Babel
  2020-08-29 13:23 ` Kyle Meyer
@ 2020-09-04  7:14   ` Bastien
  0 siblings, 0 replies; 8+ messages in thread
From: Bastien @ 2020-09-04  7:14 UTC (permalink / raw)
  To: Kyle Meyer; +Cc: Jack Kamm, emacs-orgmode

Hi Jack and Kyle,

Kyle Meyer <kyle@kyleam.com> writes:

>> I'm wondering whether this should go into org-mode, or whether to
>> package this separately. I'm also curious whether this would be useful
>> to anyone here. Any feedback is appreciated.
>
> It'd be good to hear from others, but in my view this would be fine to
> add to Org proper.

I think this would be a good addition to Org ecosystem (thanks Jack!)
but I also feel like we need to tidy things up a bit wrt Org Babel and
have stricter rules for library inclusion.

I will bring this as a separate discussion/proposal once 9.4 is out.

Thanks!

-- 
 Bastien


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC] ob-reticulate: R+Python interface from Babel
  2020-08-24 15:26 [RFC] ob-reticulate: R+Python interface from Babel Jack Kamm
  2020-08-25 23:34 ` Kyle Andrews
  2020-08-29 13:23 ` Kyle Meyer
@ 2020-10-12  4:15 ` Jack Kamm
  2021-02-27 14:15   ` Jack Kamm
  2 siblings, 1 reply; 8+ messages in thread
From: Jack Kamm @ 2020-10-12  4:15 UTC (permalink / raw)
  To: emacs-orgmode

Hi all,

I've put ob-reticulate.el on github so it can be more easily used:
https://github.com/jackkamm/ob-reticulate

I plan to submit it to MELPA or GNU ELPA over the coming weeks as well.

Cheers,
Jack


^ permalink raw reply	[flat|nested] 8+ messages in thread

* ob-reticulate: R+Python interface from Babel
  2020-10-12  4:15 ` Jack Kamm
@ 2021-02-27 14:15   ` Jack Kamm
  2021-02-27 17:24     ` Jeremie Juste
  0 siblings, 1 reply; 8+ messages in thread
From: Jack Kamm @ 2021-02-27 14:15 UTC (permalink / raw)
  To: emacs-orgmode

Hi all,

ob-reticulate is now available on MELPA.

You can find more information here:
https://github.com/jackkamm/ob-reticulate

Cheers,
Jack


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: ob-reticulate: R+Python interface from Babel
  2021-02-27 14:15   ` Jack Kamm
@ 2021-02-27 17:24     ` Jeremie Juste
  2021-03-02 23:52       ` Jack Kamm
  0 siblings, 1 reply; 8+ messages in thread
From: Jeremie Juste @ 2021-02-27 17:24 UTC (permalink / raw)
  To: Jack Kamm; +Cc: emacs-orgmode

Hello Jack,

Many thanks for this package. It seems like a better way all together
to manipulate python output.

I admit that my python foo is decreasing more and more but

#+BEGIN_SRC python :results output :session *Python*
  import pandas as pd
  df = pd.DataFrame({"X":[1,2,3], "Y":["a","b","c"]})
  df
#+end_src

#+RESULTS:

#+BEGIN_SRC python :results value  :session *Python*
  import pandas as pd
  df = pd.DataFrame({"X":[1,2,3], "Y":["a","b","c"]})
  df
#+end_src

#+RESULTS:
:    X  Y
: 0  1  a
: 1  2  b
: 2  3  c

compared to your [1] example of reticulate is easier to work with.
https://github.com/jackkamm/ob-reticulate

Best regards,
Jeremie


On Saturday, 27 Feb 2021 at 06:15, Jack Kamm wrote:
> Hi all,
>
> ob-reticulate is now available on MELPA.
>
> You can find more information here:
> https://github.com/jackkamm/ob-reticulate
>
> Cheers,
> Jack
>

-- 
Jeremie Juste


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: ob-reticulate: R+Python interface from Babel
  2021-02-27 17:24     ` Jeremie Juste
@ 2021-03-02 23:52       ` Jack Kamm
  0 siblings, 0 replies; 8+ messages in thread
From: Jack Kamm @ 2021-03-02 23:52 UTC (permalink / raw)
  To: Jeremie Juste; +Cc: emacs-orgmode

Hi Jeremie,

> Many thanks for this package. It seems like a better way all together
> to manipulate python output.

I don't disagree, at least when it comes to handling dataframes.

This is because ob-reticulate makes the block actually executed with
ob-R.el instead of ob-python.el. R has better built-in support for
manipulating dataframes, which ob-R.el can take advantage of when
handling output.

I tried before to improve ob-python handling of dataframes, plots, and
other results, but the code got a bit messy and I decided to put it on
ice. I may return to it, some day.

Cheers,
Jack


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2021-03-02 23:53 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-08-24 15:26 [RFC] ob-reticulate: R+Python interface from Babel Jack Kamm
2020-08-25 23:34 ` Kyle Andrews
2020-08-29 13:23 ` Kyle Meyer
2020-09-04  7:14   ` Bastien
2020-10-12  4:15 ` Jack Kamm
2021-02-27 14:15   ` Jack Kamm
2021-02-27 17:24     ` Jeremie Juste
2021-03-02 23:52       ` Jack Kamm

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).