How to intersperse commands with their output in RESULTS block?

How to intersperse commands with their output in RESULTS block?

[Diego Zamboni]

[-- Attachment #1: Type: text/plain, Size: 1050 bytes --]

Hi everyone,

tl;dr: is there a way to have ob-shell (or some similar mode) run commands
one by one and include the commands, interspersed with their output, in the
#+RESULTS block?

I would like to have examples shown with commands, followed by their
output. Something like this:

#+begin_src console
> echo "hi"
hi
> echo "bye"
bye
#+end_src

This works well for my use case - "console" blocks are understood nicely by
Pygmentize as containing both commands and output.

What I would like is to start using org-babel to run the commands and
insert their output in the export, so that I can update the examples
without having to copy-paste from the terminal. So what I would like is to
have something like this:

#+begin_src sh :exports results :wrap "src console"
echo "hi"
echo "bye"
#+end_src

And when I evaluate the block with C-c C-c, I get:

#+results:
#+begin_src console
> echo "hi"
hi
> echo "bye"
bye
#+end_src

Before I start building my own, ob-mode I was wondering if anyone knows of
any existing way to achieve this.

Thanks!
--Diego

[-- Attachment #2: Type: text/html, Size: 1627 bytes --]

Re: How to intersperse commands with their output in RESULTS block?

[Fraga, Eric]

On Wednesday,  5 Feb 2020 at 18:25, Diego Zamboni wrote:
> tl;dr: is there a way to have ob-shell (or some similar mode) run commands
> one by one and include the commands, interspersed with their output, in the
> #+RESULTS block?

You haven't said on what type of system but, if Linux, you could try
using =script= as a starting point:

#+begin_src shell :results output
  script <<EOF
  ls
  echo 'hello'
  EOF
#+end_src

You may wish to have a second shell script that massages the output in
the =typescript= file and ouputs that instead, e.g. to filter the
carriage returns.
-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.3.2-233-gc2bc48

Re: How to intersperse commands with their output in RESULTS block?

[Diego Zamboni]

[-- Attachment #1: Type: text/plain, Size: 878 bytes --]

Hi Eric,

Great idea! I hadn't considered using the =script= command, it's a great
starting point.

Thanks!
--Diego


On Thu, Feb 6, 2020 at 7:55 AM Fraga, Eric <e.fraga@ucl.ac.uk> wrote:

> On Wednesday,  5 Feb 2020 at 18:25, Diego Zamboni wrote:
> > tl;dr: is there a way to have ob-shell (or some similar mode) run
> commands
> > one by one and include the commands, interspersed with their output, in
> the
> > #+RESULTS block?
>
> You haven't said on what type of system but, if Linux, you could try
> using =script= as a starting point:
>
> #+begin_src shell :results output
>   script <<EOF
>   ls
>   echo 'hello'
>   EOF
> #+end_src
>
> You may wish to have a second shell script that massages the output in
> the =typescript= file and ouputs that instead, e.g. to filter the
> carriage returns.
> --
> : Eric S Fraga via Emacs 28.0.50, Org release_9.3.2-233-gc2bc48
>

[-- Attachment #2: Type: text/html, Size: 1325 bytes --]

Re: How to intersperse commands with their output in RESULTS block?

[Diego Zamboni]

[-- Attachment #1: Type: text/plain, Size: 2005 bytes --]

Hi everyone,

Quick follow up about this: following Eric's suggestion, I came up with the
following block, which cleans up all the cruft from the output of the
=script= command and produces a nicely formatted session transcript:

#+NAME: cleanup
#+BEGIN_SRC emacs-lisp :var data="" :results value :exports none
  (replace-regexp-in-string
   "\\$ exit\\(.\\|\n\\)*$" ""
   (replace-regexp-in-string
    "^bash-.*\\$" "$"
    (replace-regexp-in-string
     "\\(\\(.\\|\n\\)*?\\)\\$\\(.\\|\n\\)*\\'" ""
     (replace-regexp-in-string "
" "" data) nil nil 1)))
#+END_SRC

(I am not happy with the regexp nesting and repetition above, I am not an
expert yet in emacs-lisp regex facilities. Suggestions appreciated for how
to simplify it).

This produces exactly the result I need:

#+begin_src sh :exports output :results output :wrap "src console" :post
cleanup(data=*this*)
script <<EOF
echo hi
echo bye
EOF
#+end_src

#+RESULTS:
#+begin_src console
$ echo hi
hi
$ echo bye
bye
#+end_src

Thanks Eric!

--Diego


On Thu, Feb 6, 2020 at 9:45 PM Diego Zamboni <diego@zzamboni.org> wrote:

> Hi Eric,
>
> Great idea! I hadn't considered using the =script= command, it's a great
> starting point.
>
> Thanks!
> --Diego
>
>
> On Thu, Feb 6, 2020 at 7:55 AM Fraga, Eric <e.fraga@ucl.ac.uk> wrote:
>
>> On Wednesday,  5 Feb 2020 at 18:25, Diego Zamboni wrote:
>> > tl;dr: is there a way to have ob-shell (or some similar mode) run
>> commands
>> > one by one and include the commands, interspersed with their output, in
>> the
>> > #+RESULTS block?
>>
>> You haven't said on what type of system but, if Linux, you could try
>> using =script= as a starting point:
>>
>> #+begin_src shell :results output
>>   script <<EOF
>>   ls
>>   echo 'hello'
>>   EOF
>> #+end_src
>>
>> You may wish to have a second shell script that massages the output in
>> the =typescript= file and ouputs that instead, e.g. to filter the
>> carriage returns.
>> --
>> : Eric S Fraga via Emacs 28.0.50, Org release_9.3.2-233-gc2bc48
>>
>

[-- Attachment #2: Type: text/html, Size: 3313 bytes --]

Re: How to intersperse commands with their output in RESULTS block?

[Fraga, Eric]

On Friday,  7 Feb 2020 at 16:26, Diego Zamboni wrote:
> Quick follow up about this: following Eric's suggestion, I came up with the
> following block, which cleans up all the cruft from the output of the
> =script= command and produces a nicely formatted session transcript:

You might like to add "--quiet" to the =script= command to reduce some
of the cruft.  Otherwise, I'm glad you got something working!
-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.3.2-233-gc2bc48

Re: How to intersperse commands with their output in RESULTS block?

[Adam Porter]

Diego Zamboni <diego@zzamboni.org> writes:

> I came up with the following block, which cleans up all the cruft from
> the output of the =script= command and produces a nicely formatted
> session transcript:
>
> #+NAME: cleanup
> #+BEGIN_SRC emacs-lisp :var data="" :results value :exports none
>   (replace-regexp-in-string
>    "\\$ exit\\(.\\|\n\\)*$" ""
>    (replace-regexp-in-string
>     "^bash-.*\\$" "$"
>     (replace-regexp-in-string
>      "\\(\\(.\\|\n\\)*?\\)\\$\\(.\\|\n\\)*\\'" ""
>      (replace-regexp-in-string "
> " "" data) nil nil 1)))
> #+END_SRC
>
> (I am not happy with the regexp nesting and repetition above, I am not
> an expert yet in emacs-lisp regex facilities. Suggestions appreciated
> for how to simplify it).

Hi Diego,

A few suggestions:

1.  You can use `rx' to define regexps in a Lispy way, and the ELPA
package `xr' converts existing regexp strings to the rx format, which
can help in learning rx syntax.  For example:

  (xr "^bash-.*\\$" 'brief) ;;=> (seq bol "bash-" (0+ nonl) "$")

So you can use that regexp like:

  (replace-regexp-in-string (rx bol "bash-" (0+ nonl) "$") ...)

This nasty one is much easier with rx:

  (xr "\\(\\(.\\|\n\\)*?\\)\\$\\(.\\|\n\\)*\\'" 'brief)
  ;;=>
  ;; (seq (group (*? (group anything)))
  ;;      "$" (0+ (group anything)) eos)

2.  To avoid the nested calls, you can use a loop, like:

  (cl-loop for (match replace) in
           (list (list (rx foo bar) "replacement"))
           do (setf string
                    (replace-regexp-in-string match replace
                                              string)))

Re: How to intersperse commands with their output in RESULTS block?

[Diego Zamboni]

[-- Attachment #1: Type: text/plain, Size: 2561 bytes --]

Hi Adam and Eric for your further comments!

I had read a bit about the =rx= package but not used it, thanks for the
pointer to =xr=, makes it a lot easier to figure out the syntax.

One more thing I realized is that I can make the desired settings the
defaults within my document, and even automatically wrap all blocks with
the =script= command, by setting the corresponding properties:

#+property: header-args:sh+ :exports output
#+property: header-args:sh+ :results output
#+property: header-args:sh+ :wrap "src console"
#+property: header-args:sh+ :post cleanup(data=*this*)
#+property: header-args:sh+ :prologue "script <<EOF" :epilogue "EOF"

Then I can write plain =#+begin_src sh= blocks and they will produce the
result I want.

I love org and its community - thanks again!

--Diego


On Fri, Feb 7, 2020 at 5:07 PM Adam Porter <adam@alphapapa.net> wrote:

> Diego Zamboni <diego@zzamboni.org> writes:
>
> > I came up with the following block, which cleans up all the cruft from
> > the output of the =script= command and produces a nicely formatted
> > session transcript:
> >
> > #+NAME: cleanup
> > #+BEGIN_SRC emacs-lisp :var data="" :results value :exports none
> >   (replace-regexp-in-string
> >    "\\$ exit\\(.\\|\n\\)*$" ""
> >    (replace-regexp-in-string
> >     "^bash-.*\\$" "$"
> >     (replace-regexp-in-string
> >      "\\(\\(.\\|\n\\)*?\\)\\$\\(.\\|\n\\)*\\'" ""
> >      (replace-regexp-in-string "
> > " "" data) nil nil 1)))
> > #+END_SRC
> >
> > (I am not happy with the regexp nesting and repetition above, I am not
> > an expert yet in emacs-lisp regex facilities. Suggestions appreciated
> > for how to simplify it).
>
> Hi Diego,
>
> A few suggestions:
>
> 1.  You can use `rx' to define regexps in a Lispy way, and the ELPA
> package `xr' converts existing regexp strings to the rx format, which
> can help in learning rx syntax.  For example:
>
>   (xr "^bash-.*\\$" 'brief) ;;=> (seq bol "bash-" (0+ nonl) "$")
>
> So you can use that regexp like:
>
>   (replace-regexp-in-string (rx bol "bash-" (0+ nonl) "$") ...)
>
> This nasty one is much easier with rx:
>
>   (xr "\\(\\(.\\|\n\\)*?\\)\\$\\(.\\|\n\\)*\\'" 'brief)
>   ;;=>
>   ;; (seq (group (*? (group anything)))
>   ;;      "$" (0+ (group anything)) eos)
>
> 2.  To avoid the nested calls, you can use a loop, like:
>
>   (cl-loop for (match replace) in
>            (list (list (rx foo bar) "replacement"))
>            do (setf string
>                     (replace-regexp-in-string match replace
>                                               string)))
>
>
>

[-- Attachment #2: Type: text/html, Size: 3688 bytes --]

Re: How to intersperse commands with their output in RESULTS block?

[Fraga, Eric]

Excellent use of the :prologue and :epilogue header arguments!

-- 
: Professor Eric S Fraga; use plain text: http://useplaintext.email 
: https://www.ucl.ac.uk/chemical-engineering/people/prof-eric-fraga 
: PGP/GnuPG key: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D