Bug in export of call lines

Bug in export of call lines

[Thomas S. Dye]

Aloha all,

I've been dusting off a draft paper written when the new export
framework was still in contrib. Now that I've brought the file
up-to-date so that it exports asynchronously again, I'm getting an error
that wasn't there before.

This Org mode input:

  - B.01397 :: This is an unusual tool with a nearly constant thickness
               along most of its length (fig. [[fig:axes]], /a/). Most of
               the edge is missing, but one corner is intact. Too little
               remains of the edge to describe its shape, but the obtuse
               angle of the surviving corner suggests the edge was
               convex in plan view. The opposite end at the poll is also
               incomplete, so a complete longitudinal section is absent.
               The tool is made from a dark grey volcanic rock that has
               a pitted texture and feels light for its size. It is
               call_axe-length(a="B.01397") cm long,
               call_axe-x-section(a="B.01397") cm at mid-section, and
               weighs call_axe-wt(a="B.01397") g. The cutting edge
               measures call_axe-edge(a="B.01397") cm.

yields this LaTeX export:

  \begin{description}
  \item[{B.01397}] This is an unusual tool with a nearly constant thickness
  along most of its length (fig.~\ref{fig:axes}, \emph{a}). Most of
  the edge is missing, but one corner is intact. Too little
  remains of the edge to describe its shape, but the obtuse
  angle of the surviving corner suggests the edge was
  convex in plan view. The opposite end at the poll is also
  incomplete, so a complete longitudinal section is absent.
  The tool is made from a dark grey volcanic rock that has
  a pitted texture and feels light for its size. It is
  18.4
  \end{description}
  ~cm long,
               4.8~\texttimes~1.3
  ~cm at mid-section, and
               weighs 223
  ~g. The cutting edge
               measures 5.1
  ~cm.

Note the new line after each call result and the premature termination
of the {description} environment.

I'm using the latest Org mode from master.

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Bug in export of call lines

[Nicolas Goaziou]

Hello,

tsd@tsdye.com (Thomas S. Dye) writes:

> I've been dusting off a draft paper written when the new export
> framework was still in contrib. Now that I've brought the file
> up-to-date so that it exports asynchronously again, I'm getting an error
> that wasn't there before.

[...]

> Note the new line after each call result and the premature termination
> of the {description} environment.

It would help to see the source of "axe-x-section" and "axe-wt". Do they
insert spurious newline characters?

Also, you can call (org-export-execute-babel-code) to get what is really
parsed once Babel code is expanded.


Regards,

-- 
Nicolas Goaziou

Re: Bug in export of call lines

[Thomas S. Dye]

Aloha Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> tsd@tsdye.com (Thomas S. Dye) writes:
>
>> I've been dusting off a draft paper written when the new export
>> framework was still in contrib. Now that I've brought the file
>> up-to-date so that it exports asynchronously again, I'm getting an error
>> that wasn't there before.
>
> [...]
>
>> Note the new line after each call result and the premature termination
>> of the {description} environment.
>
> It would help to see the source of "axe-x-section" and "axe-wt". Do they
> insert spurious newline characters?
>
> Also, you can call (org-export-execute-babel-code) to get what is really
> parsed once Babel code is expanded.

That's a big help.  Thanks.

It appears that executing the R code block does insert a newline. This
must be the behavior that changed.

  executing R code block (axe-x-section)...
  Wrote /var/folders/fp/mgkcqpy17clb6dzh3_xgs0n80000gn/T/babel-78240Cke/ob-input-78240NlJ
  "3.7 \\texttimes 2
  "
  executing Emacs-Lisp code block...

  (results (quote "3.7 \\texttimes 2
  "))

Here is the source code:

  *** Cross-section of a particular tool, in cm
  #+name: axe-x-section
  #+header: :var t=final-axe-table
  #+header: :var a="3141"
  #+header: :results raw
  #+BEGIN_SRC R
  w <- t[t[,2]==a,16]/10
  th <- t[t[,2]==a,18]/10
  paste0(w, ' \\texttimes ', th)
  #+END_SRC

  #+results: axe-x-section
  3.9 \texttimes 2

All the best,
Tom

-- 
T.S. Dye & Colleagues, Archaeologists
735 Bishop St, Suite 315, Honolulu, HI 96813
Tel: 808-529-0866, Fax: 808-529-0884
http://www.tsdye.com

Re: Bug in export of call lines

[Eric Schulte]

tsd@tsdye.com (Thomas S. Dye) writes:

> Aloha Nicolas,
>
> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>
>> Hello,
>>
>> tsd@tsdye.com (Thomas S. Dye) writes:
>>
>>> I've been dusting off a draft paper written when the new export
>>> framework was still in contrib. Now that I've brought the file
>>> up-to-date so that it exports asynchronously again, I'm getting an error
>>> that wasn't there before.
>>
>> [...]
>>
>>> Note the new line after each call result and the premature termination
>>> of the {description} environment.
>>
>> It would help to see the source of "axe-x-section" and "axe-wt". Do they
>> insert spurious newline characters?
>>
>> Also, you can call (org-export-execute-babel-code) to get what is really
>> parsed once Babel code is expanded.
>
> That's a big help.  Thanks.
>
> It appears that executing the R code block does insert a newline. This
> must be the behavior that changed.
>
>   executing R code block (axe-x-section)...
>   Wrote /var/folders/fp/mgkcqpy17clb6dzh3_xgs0n80000gn/T/babel-78240Cke/ob-input-78240NlJ
>   "3.7 \\texttimes 2
>   "
>   executing Emacs-Lisp code block...
>
>   (results (quote "3.7 \\texttimes 2
>   "))
>
> Here is the source code:
>
>   *** Cross-section of a particular tool, in cm
>   #+name: axe-x-section
>   #+header: :var t=final-axe-table
>   #+header: :var a="3141"
>   #+header: :results raw
>   #+BEGIN_SRC R
>   w <- t[t[,2]==a,16]/10
>   th <- t[t[,2]==a,18]/10
>   paste0(w, '\\texttimes', th)
>   #+END_SRC
>   #+results: axe-x-section
>   3.9\texttimes2
>

I believe this issue of newlines returned from inline code blocks
(specifically with R) came up on the list recently, but unfortunately I
can't find the conversation.

Somehow this doesn't seem to be a problem with other languages, so I
believe the necessary fix may be R specific.

Sorry I can't be of more help,

>
> All the best,
> Tom

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: Bug in export of call lines

[Thomas S. Dye]

Eric Schulte <schulte.eric@gmail.com> writes:

> Somehow this doesn't seem to be a problem with other languages, so I
> believe the necessary fix may be R specific.

Following is a small test that appears to indicate that the bug is R
specific and only bites when the source code block has :results raw.

  #+DATE: \today
  #+LATEX_CLASS: article
  #+LATEX_CLASS_OPTIONS:
  #+LATEX_HEADER:
  #+LATEX_HEADER_EXTRA:


  * Export this subtree to LaTeX
    :PROPERTIES:
    :results:  raw
    :END:

  Will lisp-2 export call_lisp-2() with a newline?

  Will python-2 export call_python-2() with a newline?

  Will r-2 export call_r-2() with a newline?

  Will lisp-2-raw export call_lisp-2-raw() with a newline?

  Will python-2-raw export call_python-2-raw() with a newline?

  Will r-2-raw export call_r-2-raw() with a newline?

  * Similar code in three languages

  #+name: lisp-2
  #+begin_src lisp
  (+ 1 1)
  #+end_src

  #+name: python-2
  #+begin_src python
  return(1 + 1)
  #+end_src

  #+name: r-2
  #+begin_src R
  1 + 1
  #+end_src

  #+name: lisp-2-raw
  #+begin_src lisp :results raw
  (+ 1 1)
  #+end_src

  #+name: python-2-raw
  #+begin_src python :results raw
  return(1 + 1)
  #+end_src

  #+name: r-2-raw
  #+begin_src R :results raw
  1 + 1
  #+end_src

Here is the relevant portion of the LaTeX export:

  Will lisp-2 export 2 with a newline?

  Will python-2 export 2 with a newline?

  Will r-2 export 2 with a newline?

  Will lisp-2-raw export 2 with a newline?

  Will python-2-raw export 2 with a newline?

  Will r-2-raw export 2
   with a newline?
  % Emacs 24.3.1 (Org mode 8.2.5h)

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Bug in export of call lines

[Eric Schulte]

Hi Tom,

This actually is also an issue with shell code blocks.  The fix is to
customize the org-babel-inline-result-wrap variable (e.g., as follows)
instead of setting the result type to RAW.

  (setq org-babel-inline-result-wrap "%s")

Perhaps a note about this variable should be added to the description of
inline code blocks (both call and src_*) in the manual.

Best,

tsd@tsdye.com (Thomas S. Dye) writes:

> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> Somehow this doesn't seem to be a problem with other languages, so I
>> believe the necessary fix may be R specific.
>
> Following is a small test that appears to indicate that the bug is R
> specific and only bites when the source code block has :results raw.
>
>   #+DATE: \today
>   #+LATEX_CLASS: article
>   #+LATEX_CLASS_OPTIONS:
>   #+LATEX_HEADER:
>   #+LATEX_HEADER_EXTRA:
>
>
>   * Export this subtree to LaTeX
>     :PROPERTIES:
>     :results:  raw
>     :END:
>
>   Will lisp-2 export call_lisp-2() with a newline?
>
>   Will python-2 export call_python-2() with a newline?
>
>   Will r-2 export call_r-2() with a newline?
>
>   Will lisp-2-raw export call_lisp-2-raw() with a newline?
>
>   Will python-2-raw export call_python-2-raw() with a newline?
>
>   Will r-2-raw export call_r-2-raw() with a newline?
>
>   * Similar code in three languages
>
>   #+name: lisp-2
>   #+begin_src lisp
>   (+ 1 1)
>   #+end_src
>
>   #+name: python-2
>   #+begin_src python
>   return(1 + 1)
>   #+end_src
>
>   #+name: r-2
>   #+begin_src R
>   1 + 1
>   #+end_src
>
>   #+name: lisp-2-raw
>   #+begin_src lisp :results raw
>   (+ 1 1)
>   #+end_src
>
>   #+name: python-2-raw
>   #+begin_src python :results raw
>   return(1 + 1)
>   #+end_src
>
>   #+name: r-2-raw
>   #+begin_src R :results raw
>   1 + 1
>   #+end_src
>
> Here is the relevant portion of the LaTeX export:
>
>   Will lisp-2 export 2 with a newline?
>
>   Will python-2 export 2 with a newline?
>
>   Will r-2 export 2 with a newline?
>
>   Will lisp-2-raw export 2 with a newline?
>
>   Will python-2-raw export 2 with a newline?
>
>   Will r-2-raw export 2
>    with a newline?
>   % Emacs 24.3.1 (Org mode 8.2.5h)
>
> All the best,
> Tom

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: Bug in export of call lines

[Thomas S. Dye]

Hi Eric,

Eric Schulte <schulte.eric@gmail.com> writes:

> This actually is also an issue with shell code blocks.  The fix is to
> customize the org-babel-inline-result-wrap variable (e.g., as follows)
> instead of setting the result type to RAW.
>
>   (setq org-babel-inline-result-wrap "%s")
>
> Perhaps a note about this variable should be added to the description of
> inline code blocks (both call and src_*) in the manual.

AFAICT, this variable has no effect on export of calls to R source code
blocks, where the source code block is set with :results raw.

Also, AFAICT, the behavior of R source code blocks differs from others,
including shell.

With the default value of org-babel-inline-result-wrap I get this LaTeX
export:

  \section{Export this subtree raw to \LaTeX{}}
  \label{sec-1}
  Will shell-2 export 2 with a newline?

  Will r-2 export 2 with a newline?

  Will shell-2-raw export 2 with a newline?

  Will r-2-raw export 2
   with a newline?
  \section{Export this subtree without raw to \LaTeX{}}
  \label{sec-2}

  Will shell-2 export \texttt{2} with a newline?

  Will r-2 export \texttt{2} with a newline?

  Will shell-2-raw export \texttt{2} with a newline?

  \begin{verbatim}
  Will r-2-raw export 2
  \end{verbatim}
  with a newline?
  % Emacs 24.3.1 (Org mode 8.2.5h)

With org-babel-inline-result-wrap set to "%s" I get this:

  \section{Export this subtree raw to \LaTeX{}}
  \label{sec-1}
  Will shell-2 export 2 with a newline?

  Will r-2 export 2 with a newline?

  Will shell-2-raw export 2 with a newline?

  Will r-2-raw export 2
   with a newline?
  \section{Export this subtree without raw to \LaTeX{}}
  \label{sec-2}

  Will shell-2 export 2 with a newline?

  Will r-2 export 2 with a newline?

  Will shell-2-raw export 2 with a newline?

  \begin{verbatim}
  Will r-2-raw export 2
  \end{verbatim}
  with a newline?
  % Emacs 24.3.1 (Org mode 8.2.5h)

Note that there is no change in the LaTeX export of the R calls.

I think this is a bug in ob-R.el.  From my experiments, it behaves
differently than the other babel languages, including shell.

For LaTeX export, the workaround is to avoid :results raw with R source
code blocks. Although a spurious newline is inserted into the source,
this is ignored by LaTeX.

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Bug in export of call lines

[Eric Schulte]

[-- Attachment #1: Type: text/plain, Size: 709 bytes --]

tsd@tsdye.com (Thomas S. Dye) writes:

> Hi Eric,
>
> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> This actually is also an issue with shell code blocks.  The fix is to
>> customize the org-babel-inline-result-wrap variable (e.g., as follows)
>> instead of setting the result type to RAW.
>>
>>   (setq org-babel-inline-result-wrap "%s")
>>
>> Perhaps a note about this variable should be added to the description of
>> inline code blocks (both call and src_*) in the manual.
>
> AFAICT, this variable has no effect on export of calls to R source code
> blocks, where the source code block is set with :results raw.
>

Correct, however I believe the following should give the behavior you're
after no?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: good-newlines.org --]
[-- Type: text/x-org, Size: 260 bytes --]

#+Options: ^:{}

First set the inline result wrap.
#+begin_src emacs-lisp :results silent
  (setq org-babel-inline-result-wrap "%s")
#+end_src

* Desirable newline behavior

The call_r1() should not export with a newline.

#+name: r1
#+begin_src R
2
#+end_src

[-- Attachment #3: Type: text/plain, Size: 99 bytes --]


>
> Also, AFAICT, the behavior of R source code blocks differs from others,
> including shell.
>


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: bad-newlines.org --]
[-- Type: text/x-org, Size: 291 bytes --]

#+Options: ^:{}

* Undesirable newline behavior
  :PROPERTIES:
  :results:  raw
  :END:

Both of these *do* introduce undesirable newlines.  A shell code block
call_shell2() and an R code block call_R2().

#+name: shell2
#+begin_src sh
echo 2
#+end_src

#+name: R2
#+begin_src R
2
#+end_src

[-- Attachment #5: Type: text/plain, Size: 2179 bytes --]


>
> With the default value of org-babel-inline-result-wrap I get this
> LaTeX
> export:
>
>   \section{Export this subtree raw to \LaTeX{}}
>   \label{sec-1}
>   Will shell-2 export 2 with a newline?
>
>   Will r-2 export 2 with a newline?
>
>   Will shell-2-raw export 2 with a newline?
>
>   Will r-2-raw export 2
>    with a newline?
>   \section{Export this subtree without raw to \LaTeX{}}
>   \label{sec-2}
>
>   Will shell-2 export \texttt{2} with a newline?
>
>   Will r-2 export \texttt{2} with a newline?
>
>   Will shell-2-raw export \texttt{2} with a newline?
>
>   \begin{verbatim}
>   Will r-2-raw export 2
>   \end{verbatim}
>   with a newline?
>   % Emacs 24.3.1 (Org mode 8.2.5h)
>
> With org-babel-inline-result-wrap set to "%s" I get this:
>
>   \section{Export this subtree raw to \LaTeX{}}
>   \label{sec-1}
>   Will shell-2 export 2 with a newline?
>
>   Will r-2 export 2 with a newline?
>
>   Will shell-2-raw export 2 with a newline?
>
>   Will r-2-raw export 2
>    with a newline?
>   \section{Export this subtree without raw to \LaTeX{}}
>   \label{sec-2}
>
>   Will shell-2 export 2 with a newline?
>
>   Will r-2 export 2 with a newline?
>
>   Will shell-2-raw export 2 with a newline?
>
>   \begin{verbatim}
>   Will r-2-raw export 2
>   \end{verbatim}
>   with a newline?
>   % Emacs 24.3.1 (Org mode 8.2.5h)
>
> Note that there is no change in the LaTeX export of the R calls.
>
> I think this is a bug in ob-R.el.  From my experiments, it behaves
> differently than the other babel languages, including shell.
>
> For LaTeX export, the workaround is to avoid :results raw with R source
> code blocks. Although a spurious newline is inserted into the source,
> this is ignored by LaTeX.
>

I think it is allowable for raw results to include newlines.  After if R
returns a newline, the purpose of raw results is specifically to *not*
change the result.  I believe the best approach is to customize the
inline results wrap as above, and then use non-raw processed results
when inline results without newlines are desired.

Does this sound reasonable?

Best,

>
> All the best,
> Tom

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: Bug in export of call lines

[Thomas S. Dye]

Eric Schulte <schulte.eric@gmail.com> writes:

> I think it is allowable for raw results to include newlines.  After if R
> returns a newline, the purpose of raw results is specifically to *not*
> change the result.  I believe the best approach is to customize the
> inline results wrap as above, and then use non-raw processed results
> when inline results without newlines are desired.
>
> Does this sound reasonable?

I'm able to export my document correctly by removing :results raw from
the R source code blocks.

I can control wrapping of the results by making the calls :results raw,
which removes the wrap. 

I don't see any change in newline behavior by changing
org-babel-inline-result-wrap.

Unlike your shell block, I don't get an extraneous newline.

I only see unexpected behavior with R source code blocks when :results
raw is set at the source code block.

The following tables summarize what I'm seeing.

* Tables of results                                                :noexport:

With "%s", modified value of org-babel-inline-result-wrap

#+name: R-results
|                | call raw   | call not raw         |
|----------------+------------+----------------------|
| source raw     | newline    | verbatim and newline |
| source not raw | no newline | no newline           |

#+name sh-results
|                | call raw   | call not raw |
|----------------+------------+--------------|
| source raw     | no newline | no newline   |
| source not raw | no newline | no newline   |

With "=%s=", the default value of org-babel-inline-result-wrap

#+name: R-results-default
|                | call raw   | call not raw                    |
|----------------+------------+---------------------------------|
| source raw     | newline    | verbatim and newline, no texttt |
| source not raw | no newline | no newline, texttt              |

#+name sh-results-default
|                | call raw   | call not raw       |
|----------------+------------+--------------------|
| source raw     | no newline | no newline, texttt |
| source not raw | no newline | no newline, texttt |

I'll dig deeper tomorrow to see why my shell source code blocks don't
behave as yours do.  In any case, my document exports again.

Thanks,
Tom

-- 
T.S. Dye & Colleagues, Archaeologists
735 Bishop St, Suite 315, Honolulu, HI 96813
Tel: 808-529-0866, Fax: 808-529-0884
http://www.tsdye.com