Ideas for improving ob-R (was: [FR] Org babel: mixing multiple outputs (was: Output of R code block: only text or plot but not both? And only one "result" can be output?))

[Ihor Radchenko <yantar92@posteo.net>]

"Berry, Charles" <ccberry@health.ucsd.edu> writes:

>> Thanks for sharing the project!
>> Although, I would not call going through double export, and producing
>> html output "easier time".
>> 
>
> The `easier' part is that knitr/Rmarkdown requires very little markup
> to produce nice documents in various formats (pdf, html, Word,
> ...). And dealing with R code and markup of the results of R code for
> use in documents is what that environment is attuned to, so getting a
> desired result often seemed easier to me in that environment.

So does Org markup...
What we may be missing is nicely formatted output of R blocks, where we
need something akin ob-python that knows about how to typeset various
Python types in Org - data frames, graphics, tables, etc.

> In terms of my usual workflow, the double export (with ox-ravel and an
> `org-render` helper function loaded) requires these keystrokes:
>
> `C-c C-e r m M-x org-render RET`
>
> or subsequently
>
> `C-u C-c C-e M-x M-p RET`
>
> which adds only two keystrokes. 

I understand. My point is different - what if you do _not_ need
HTML/pdf/word output and just want to work with the interactive notebook?
Then, your export workflow is a huge overkill and complication.

>> While we are here, are there any other features you find missing in Org
>> babel that are present in knitr/Rmarkdown/quarto?
>
> `:session :results output` handling in R lang src blocks can fail as
> the heuristics for finding the output of a command in the session
> buffer and removing the prompts have limited success. An ECM:
>
> #+begin_src R :session :results output
>   cat("2 > 1\n")
> #+end_src
>
> There are workarounds, of course.

Hmm. We can address this, as we already do in ob-shell.
The idea is to change the prompt string to something unique.

Or, alternatively, we can redirect output to file using `sink'.

> I recall hearing that `comint.el` would someday be replaced by a
> hardier package, so maybe if/when that happens this can be cured.

Such replacement is not in sight, unfortunately.

> A big motivation for creating `ox-ravel' was to be able to cache large
> objects. I know the `:cache` header arg helps for small objects that
> require a lot of computation, but AFAICS does not help once the object
> size gets large. The caching options [1] of knitr and friends are
> flexible and powerful enough to support the genomics work I do.

> [1] https://yihui.org/knitr/options/#cache

Org mode uses pretty much the same caching mechanism (md5 checksum for
code+block parameters) to save code block results. Except that cached
data is not persisted between Emacs sessions. It is not too hard to make
it persistent though - we have a dedicated org-persist library for such
purposes.

Patches welcome!

I guess what is not available is caching for sessions. Can also be
implemented, but it would be a slightly more complex patch.

> I mentioned above that knitr is attuned to working with R
> code/output. The handling of warnings, errors, and messages resulting
> from R code has a number of useful options under `Text Output` [2].

> [2] https://yihui.org/knitr/options/#text-output

I looked through the available options and most of them are already
supported by Org mode.

The ones that are not supported are:

warning: (TRUE; logical) Whether to preserve warnings (produced by
         warning()) in the output.

message: ...

error: ...

Fine-grained control over warning/error/message/output separation is
something that would be nice to have indeed. Not just for ob-R, but in
general.

For ob-R, it should be fairly easy to do - we can redirect stderr/stdout
output to different files and then read the results from there.

> I guess a rewrite of ob-R.el to implement such features as object
> caching and error/warning/message handling is feasible, but would
> require a lot of effort.

I believe that sufficiently motivated person familiar with R and Elisp
can implement these features within a week, working in the evenings.
But, as usual, it is a question of volunteers.

> ... And since those features (and more - like
> animation support) are already implemented in the knitr/Rmarkdown
> domain is it really worth pursuing?

Animation support is indeed tricky. Although, we do have an attempt in
https://github.com/alejandrogallo/org-inline-webkit (any volunteers to
polish that demo up to the state mergeable upstream?)

As for "already implemented", it is a question of use
case. knitr/Rmarkdown are fine as long as they fit the workflow, but Org
mode is generally more flexible - it has more stable and sophisticated
markup and can export to more formats.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>