org-in-org

org-in-org

[Greg Minshall]

i have a question about org-in-org source blocks.  i volunteered to help
in an effort to provide a tutorial of using the ESS (Emacs Speaks
Statistics) package for R, in particular, from org mode.

i'd like to write my contribution as a .org file.  i'd like to include
fragments of org code, including source blocks (in R).  and, i'd like to
show various result types.  so, i'd like to be able to have the
#+RESULTS show up in the org-in-org source block as exported inside the
containing .org file.

and, i'd like to trigger all this from a makefile, using some emacs
batch script to export the containing .org file into a .html or .pdf
file.

(i *think*) what i would like to end up with is what it would like if i
had manually opened the org-in-org source blocks (C-c‌'), then went to
each (or, possibly, selected, i guess) source blocks inside *that*
(org-in-org) source block, and executed each, producing a #+RESULTS
block for each, then closed the org-in-org source block (C-c‌', again),
and then exported the containing .org file.

is this possible?  any ideas?

cheers, Greg

Re: org-in-org

[Berry, Charles via General discussions about Org-mode.]

Greg, 

See inline

> On Feb 23, 2021, at 6:24 AM, Greg Minshall <minshall@umich.edu> wrote:
> 
> i have a question about org-in-org source blocks.  i volunteered to help
> in an effort to provide a tutorial of using the ESS (Emacs Speaks
> Statistics) package for R, in particular, from org mode.
> 
> i'd like to write my contribution as a .org file.  i'd like to include
> fragments of org code, including source blocks (in R).  and, i'd like to
> show various result types.  so, i'd like to be able to have the
> #+RESULTS show up in the org-in-org source block as exported inside the
> containing .org file.

I think I get your intention. You want the visual to look like it would if the src-edit buffer was opened in emacs as org.

> 
> and, i'd like to trigger all this from a makefile, using some emacs
> batch script to export the containing .org file into a .html or .pdf
> file.
> 
> (i *think*) what i would like to end up with is what it would like if i
> had manually opened the org-in-org source blocks (C-c‌'), then went to
> each (or, possibly, selected, i guess) source blocks inside *that*
> (org-in-org) source block, and executed each, producing a #+RESULTS
> block for each, then closed the org-in-org source block (C-c‌', again),
> and then exported the containing .org file.
> 
> is this possible?  any ideas?


I have two alternative approaches:

1) Add an export engine for "org" to ox-ravel[1]. This is a trivial customization of `org-ravel-engines' to add a '("org" . "engine='org'") element.  Then add a custom language engine[2] for rmarkdown or knitr.

The actions for the makefile would be a ravel export to generate *.Rmd, *.Rnw, or *.Rhtml files followed by rmarkdown::render or knitr::knit on the generated files.

2) Define this function:

#+begin_src emacs-lisp
  (defun org-exe-org ()
    (let ((org-confirm-babel-evaluate nil))
      (org-edit-src-code)
      (org-babel-execute-buffer)
      (org-edit-src-exit)))
#+end_src

In an org buffer with one or more org src blocks containing R src blocks:

eval this

 (org-babel-map-executables nil (org-exe-org))

Then export the buffer to pdf or html.

I haven't tested this much, but for the snippet below evaluating that line and exporting to html produces two framed blocks with R src blocks and the results displayed in each:

--8<---------------cut here---------------start------------->8---
#+begin_src org
  ,#+begin_src R
    print("abc")
  ,#+end_src
#+end_src


#+begin_src org
  ,#+begin_src R
    print("def")
  ,#+end_src
#+end_src
--8<---------------cut here---------------end--------------->8---

I expect doing this in batch mode will not be a challenge.

For pdf output the result is rather plain. Maybe there is some minted idiom that would help?

HTH,
Chuck

p.s. I am an `ess-intro' member, in case it helps to move the conversation to there.


[1] https://github.com/chasberry/orgmode-accessories

[2] https://bookdown.org/yihui/rmarkdown-cookbook/custom-engine.html 

Re: org-in-org

[Greg Minshall]

Charles,

thanks.  any thing you'd like to add to the R-via-ESS/org-mode
repository, that would be great.

in general, afaik, the contents of org-in-org buffers export okay.  at
least plain ones.  would could like to have the embedded code blocks go
through some pretty-printer, but currently i don't see that as an issue.

as i mentioned to Jeremie, an issue is getting the org-in-org block to
do its export.  i don't know if creating a recursive "execute" function
for org source blocks -- which would cause them (modulo :eval, :exports,
etc.) to evaluate their source blocks -- might be useful.

cheers, Greg

Re: org-in-org

[Berry, Charles via General discussions about Org-mode.]

> On Mar 7, 2021, at 8:14 AM, Greg Minshall <minshall@umich.edu> wrote:
> 
> Charles,
> 
> thanks.  any thing you'd like to add to the R-via-ESS/org-mode
> repository, that would be great.
> 
> in general, afaik, the contents of org-in-org buffers export okay.  at
> least plain ones.  would could like to have the embedded code blocks go
> through some pretty-printer, but currently i don't see that as an issue.
> 
> as i mentioned to Jeremie, an issue is getting the org-in-org block to
> do its export.  i don't know if creating a recursive "execute" function
> for org source blocks -- which would cause them (modulo :eval, :exports,
> etc.) to evaluate their source blocks -- might be useful.
> 

Sorry, I do not know what a  `recursive "execute" function' would be/do in this context.

---

You said to Jeremie:
 
"during export of the *outer* org document, it could cause
the inner org block(s) to be executed and their results revealed."

AFAICS, the document below does exactly that upon export and can be exported using a command line idiom with proper attention to init-file issues (and let binding `org-confirm-babel-evaluate' to nil if you want to run in batch mode). 

As an example, html export will create a document that displays a `<pre class="src src-org"...> block with the R src block named `readdata-code' and the `#+RESULTS: readdata-code' block in it. 

--8<---------------cut here---------------start------------->8---
#+begin_src emacs-lisp :exports results :results none
  (org-babel-map-executables nil 
    (let ((org-confirm-babel-evaluate nil))
      (org-edit-src-code)
      (org-babel-execute-buffer)
      (org-edit-src-exit)))
#+end_src


#+NAME: readdata-code
#+BEGIN_SRC org
  ,#+NAME: readdata-code
  ,#+BEGIN_SRC R :results output

  read.csv(text="a,b,c\n1,2,3\n4,5,6",header=T) -> mydata1

  summary(mydata1)
  ,#+END_SRC

#+END_SRC
--8<---------------cut here---------------end--------------->8---


If that isn't what is needed, maybe you can provide the desired output when exporting the above sans the emacs-lisp src block.


Best,
Chuck

Re: org-in-org

[Greg Minshall]

Charles,

thanks.

ah, i apologize -- i missed the elisp content of your earlier message.
yes, that, at least for this simple case, does exactly what i was
looking for!

i guess when i used the term "recursive execute function" (i tend to
confuse "execute" and "export"), i was thinking of something like: when
i export an org file, and it runs into an org-in-org block to export,
then your code runs on that block.  the recursive part is that, when
your code is runninng on the org-in-org block, and runs into an
"org-in-org-in-org" block (that is also marked to export), it runs on
*that*.  ad, but not normally infinitum.

again, thanks, and i'm sorry for missing your elisp code.

cheers, Greg

Re: org-in-org

[Berry, Charles via General discussions about Org-mode.]

Greg,

> On Mar 7, 2021, at 11:44 PM, Greg Minshall <minshall@umich.edu> wrote:
> 
> i guess when i used the term "recursive execute function" (i tend to
> confuse "execute" and "export"), i was thinking of something like: when
> i export an org file, and it runs into an org-in-org block to export,
> then your code runs on that block.  the recursive part is that, when
> your code is runninng on the org-in-org block, and runs into an
> "org-in-org-in-org" block (that is also marked to export), it runs on
> *that*.  ad, but not normally infinitum.

I admit to being baffled by this.

If you have nested org src blocks and you recursively enter each org block using `org-edit-special' and execute the src blocks other than org lang, then exit with `org-edit-src-exit', when you complete this the org buffer will have nested src blocks and results with one or more comma escapes prepended to lines that start with `#+` or `*`. 

When you export this, you end up with a document that has the comma escapes reduced by one in those nested blocks. The appearance of these is not altogether pleasing to my eye.

Have you tried creating a document with nested org src blocks and stepping through the recursion to be sure it is something you want and really need?

If you have a particular use case, maybe it can be handled in a one-off manner.

Best,
Chuck

Re: org-in-org

[Greg Minshall]

Chuck,

> I admit to being baffled by this.
> 
> If you have nested org src blocks and you recursively enter each org
> block using `org-edit-special' and execute the src blocks other than
> org lang, then exit with `org-edit-src-exit', when you complete this
> the org buffer will have nested src blocks and results with one or
> more comma escapes prepended to lines that start with `#+` or `*`.

ah.

> When you export this, you end up with a document that has the comma
> escapes reduced by one in those nested blocks. The appearance of these
> is not altogether pleasing to my eye.

yes, i see what you mean.  though in a very limited pedagological
context, where you want to show the viewer of the exported document what
an org-in-org file looks like in an emacs .org buffer, i guess you might
want this.

for my use, your solution for org-in-org is exactly what i need.

cheers, Greg

Re: org-in-org

[Erik Iverson]

[-- Attachment #1: Type: text/plain, Size: 1492 bytes --]

Hi Greg!

I am not sure if this would be useful to your efforts, but I have an "R in
org-mode" tutorial on github:

https://github.com/erikriverson/org-mode-R-tutorial

Apparently it is now 10 years old somehow, time flies.

Best,
--Erik



On Tue, Feb 23, 2021 at 6:25 AM Greg Minshall <minshall@umich.edu> wrote:

> i have a question about org-in-org source blocks.  i volunteered to help
> in an effort to provide a tutorial of using the ESS (Emacs Speaks
> Statistics) package for R, in particular, from org mode.
>
> i'd like to write my contribution as a .org file.  i'd like to include
> fragments of org code, including source blocks (in R).  and, i'd like to
> show various result types.  so, i'd like to be able to have the
> #+RESULTS show up in the org-in-org source block as exported inside the
> containing .org file.
>
> and, i'd like to trigger all this from a makefile, using some emacs
> batch script to export the containing .org file into a .html or .pdf
> file.
>
> (i *think*) what i would like to end up with is what it would like if i
> had manually opened the org-in-org source blocks (C-c‌'), then went to
> each (or, possibly, selected, i guess) source blocks inside *that*
> (org-in-org) source block, and executed each, producing a #+RESULTS
> block for each, then closed the org-in-org source block (C-c‌', again),
> and then exported the containing .org file.
>
> is this possible?  any ideas?
>
> cheers, Greg
>
>

[-- Attachment #2: Type: text/html, Size: 2061 bytes --]

Re: org-in-org

[Greg Minshall]

Erik,

> I am not sure if this would be useful to your efforts, but I have an "R in
> org-mode" tutorial on github:

> https://github.com/erikriverson/org-mode-R-tutorial

thanks.  that gives me a model to look at in terms of structure and
content.

Greg

Re: org-in-org

[Jeremie Juste]

Hello Greg,

Many thanks for the effort. 
A possible solution might be this one 

#+NAME: readdata-code
#+BEGIN_SRC org
  ,#+NAME: readdata-code
  ,#+BEGIN_SRC R :results value silent

  read.data("datafile1.csv",sep=",",header=T)->mydata1


  ,#+END_SRC
#+END_SRC

credit goes to https://raw.githubusercontent.com/vikasrawal/orgpaper/master/orgpapers.org

Best regards,
Jeremie


On Tuesday, 23 Feb 2021 at 17:24, Greg Minshall wrote:
> i have a question about org-in-org source blocks.  i volunteered to help
> in an effort to provide a tutorial of using the ESS (Emacs Speaks
> Statistics) package for R, in particular, from org mode.
>
> i'd like to write my contribution as a .org file.  i'd like to include
> fragments of org code, including source blocks (in R).  and, i'd like to
> show various result types.  so, i'd like to be able to have the
> #+RESULTS show up in the org-in-org source block as exported inside the
> containing .org file.
>
> and, i'd like to trigger all this from a makefile, using some emacs
> batch script to export the containing .org file into a .html or .pdf
> file.
>
> (i *think*) what i would like to end up with is what it would like if i
> had manually opened the org-in-org source blocks (C-c‌'), then went to
> each (or, possibly, selected, i guess) source blocks inside *that*
> (org-in-org) source block, and executed each, producing a #+RESULTS
> block for each, then closed the org-in-org source block (C-c‌', again),
> and then exported the containing .org file.
>
> is this possible?  any ideas?
>
> cheers, Greg
>

-- 
Jeremie Juste

Re: org-in-org

[Greg Minshall]

Jeremie,

thanks for this.

> A possible solution might be this one
> #+NAME: readdata-code
> #+BEGIN_SRC org
>   ,#+NAME: readdata-code
>   ,#+BEGIN_SRC R :results value silent
> 
>   read.data("datafile1.csv",sep=3D",",header=3DT)->mydata1
> 
> 
>   ,#+END_SRC
> #+END_SRC

iiuc, this will embed the org-in-org block ("readdata-code") in the
outer org document.  and, if i open the org-in-org block, and C-c C-c to
execute it, the results will likewise be available in the outer org
document, and can be exported, etc.

this is very useful, and is the first part of what i *think* i am hoping
to find.

the second part is a way, from the outer org document, or, hopefully,
from an e-lisp batch file, to be able to cause the inner org document to
have its code blocks executed, with the results left in place.  so,
that, e.g., during export of the *outer* org document, it could cause
the inner org block(s) to be executed and their results revealed.

> credit goes to https://raw.githubusercontent.com/vikasrawal/orgpaper/master=
> /orgpapers.org

fwiw, a more recent version exists:
----
https://github.com/vikasrawal/orgpaper/blob/master/orgpapers.org
----
though i haven't in depth compared the two.

cheers, Greg