* [BUG] ":sep" header argument doesn't work in ob-shell [9.6.21 ( @ /home/aragaer/.config/emacs/elpa/org-9.6.21/)]
@ 2024-03-09 20:15 Ilya Konovalov
2024-03-12 9:27 ` Ilya Konovalov
2024-03-12 14:38 ` Ihor Radchenko
0 siblings, 2 replies; 6+ messages in thread
From: Ilya Konovalov @ 2024-03-09 20:15 UTC (permalink / raw)
To: emacs-orgmode
On current master using ":sep" header argument does not work as
expected and records are always split on whitespaces. For example:
#+begin_src sh :sep "\t"
echo hello world
echo test
#+end_src
#+RESULTS:
| hello | world |
| test | |
It looks like the issue is in the org-babel-sh-evaluate function, where
at the end it passes the result to the org-babel-import-elist-from-file
function for processing. Replacing the code with the following seems to
fix the issue:
(let ((tmp-file (org-babel-temp-file "sh-"))
(sep (or (cdr (assq :sep params)) "\t")))
(with-temp-file tmp-file (insert results))
(org-babel-import-elisp-from-file tmp-file sep))
Emacs : GNU Emacs 29.2 (build 1, x86_64-pc-linux-gnu, X toolkit)
of 2024-03-02
Package: Org mode version 9.6.21 ( @ /home/aragaer/.config/emacs/elpa/org-9.6.21/)
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUG] ":sep" header argument doesn't work in ob-shell [9.6.21 ( @ /home/aragaer/.config/emacs/elpa/org-9.6.21/)]
2024-03-09 20:15 [BUG] ":sep" header argument doesn't work in ob-shell [9.6.21 ( @ /home/aragaer/.config/emacs/elpa/org-9.6.21/)] Ilya Konovalov
@ 2024-03-12 9:27 ` Ilya Konovalov
2024-03-12 10:22 ` Matt
2024-03-12 14:38 ` Ihor Radchenko
1 sibling, 1 reply; 6+ messages in thread
From: Ilya Konovalov @ 2024-03-12 9:27 UTC (permalink / raw)
To: emacs-orgmode
After some usage I found that this exact patch breaks
`org-babel-import-elisp-from-file's own separator detection. It looks
like it's better to leave it as `nil' unless specified.
Ilya Konovalov <aragaer@gmail.com> writes:
> On current master using ":sep" header argument does not work as
> expected and records are always split on whitespaces. For example:
>
> #+begin_src sh :sep "\t"
> echo hello world
> echo test
> #+end_src
>
> #+RESULTS:
> | hello | world |
> | test | |
>
> It looks like the issue is in the org-babel-sh-evaluate function, where
> at the end it passes the result to the org-babel-import-elist-from-file
> function for processing. Replacing the code with the following seems to
> fix the issue:
>
> (let ((tmp-file (org-babel-temp-file "sh-"))
> (sep (or (cdr (assq :sep params)) "\t")))
> (with-temp-file tmp-file (insert results))
> (org-babel-import-elisp-from-file tmp-file sep))
>
> Emacs : GNU Emacs 29.2 (build 1, x86_64-pc-linux-gnu, X toolkit)
> of 2024-03-02
> Package: Org mode version 9.6.21 ( @ /home/aragaer/.config/emacs/elpa/org-9.6.21/)
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUG] ":sep" header argument doesn't work in ob-shell [9.6.21 ( @ /home/aragaer/.config/emacs/elpa/org-9.6.21/)]
2024-03-12 9:27 ` Ilya Konovalov
@ 2024-03-12 10:22 ` Matt
2024-03-12 10:41 ` Ilya Konovalov
2024-03-12 14:25 ` Ihor Radchenko
0 siblings, 2 replies; 6+ messages in thread
From: Matt @ 2024-03-12 10:22 UTC (permalink / raw)
To: Ilya Konovalov; +Cc: emacs-orgmode
---- On Tue, 12 Mar 2024 10:27:46 +0100 Ilya Konovalov wrote ---
>
> After some usage I found that this exact patch breaks
> `org-babel-import-elisp-from-file's own separator detection. It looks
> like it's better to leave it as `nil' unless specified.
Thanks for taking the time to look into it. According to my notes, :sep and :separator don't work for anything beyond basic table layout. I'm not familiar with org-table and there isn't much documentation on those keywords in the manual. The :sep keyword shows up for Texinfo and LaTeX. It looks like :sep is used for export? It's not clear to me how they're supposed to work with shell block output, or if that was even a consideration at all.
They were added in https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=a60a83fbb65a368bb96ea6aaf1db20706b3e99c5 claiming "better handling of table in shell scripts" So, it looks like their inclusion was intentional.
Maybe someone has deeper insights?
--
Matt Trzcinski
Emacs Org contributor (ob-shell)
Learn more about Org mode at https://orgmode.org
Support Org development at https://liberapay.com/org-mode
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUG] ":sep" header argument doesn't work in ob-shell [9.6.21 ( @ /home/aragaer/.config/emacs/elpa/org-9.6.21/)]
2024-03-12 10:22 ` Matt
@ 2024-03-12 10:41 ` Ilya Konovalov
2024-03-12 14:25 ` Ihor Radchenko
1 sibling, 0 replies; 6+ messages in thread
From: Ilya Konovalov @ 2024-03-12 10:41 UTC (permalink / raw)
To: Matt; +Cc: emacs-orgmode
I assumed that https://org-babel.readthedocs.io/ is at least somewhat
related to org documentation. Looks like I was wrong.
The :separator parameter seems to be related to how block's :var
arguments are formatted when passing to the shell script.
In most cases separator is not passed to
`org-babel-import-elisp-from-file' -- looks like only some languages set
it to some fixed value.
Matt <matt@excalamus.com> writes:
> ---- On Tue, 12 Mar 2024 10:27:46 +0100 Ilya Konovalov wrote ---
> >
> > After some usage I found that this exact patch breaks
> > `org-babel-import-elisp-from-file's own separator detection. It looks
> > like it's better to leave it as `nil' unless specified.
>
> Thanks for taking the time to look into it. According to my notes,
> :sep and :separator don't work for anything beyond basic table layout.
> I'm not familiar with org-table and there isn't much documentation on
> those keywords in the manual. The :sep keyword shows up for Texinfo
> and LaTeX. It looks like :sep is used for export? It's not clear to me
> how they're supposed to work with shell block output, or if that was
> even a consideration at all.
>
> They were added in
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=a60a83fbb65a368bb96ea6aaf1db20706b3e99c5
> claiming "better handling of table in shell scripts" So, it looks like
> their inclusion was intentional.
>
> Maybe someone has deeper insights?
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUG] ":sep" header argument doesn't work in ob-shell [9.6.21 ( @ /home/aragaer/.config/emacs/elpa/org-9.6.21/)]
2024-03-12 10:22 ` Matt
2024-03-12 10:41 ` Ilya Konovalov
@ 2024-03-12 14:25 ` Ihor Radchenko
1 sibling, 0 replies; 6+ messages in thread
From: Ihor Radchenko @ 2024-03-12 14:25 UTC (permalink / raw)
To: Matt; +Cc: Ilya Konovalov, emacs-orgmode
Matt <matt@excalamus.com> writes:
> ... According to my notes, :sep and :separator don't work for
> anything beyond basic table layout. I'm not familiar with org-table
> and there isn't much documentation on those keywords in the manual.
> The :sep keyword shows up for Texinfo and LaTeX. It looks like :sep is
> used for export? It's not clear to me how they're supposed to work
> with shell block output, or if that was even a consideration at all.
>
> They were added in
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=a60a83fbb65a368bb96ea6aaf1db20706b3e99c5
> claiming "better handling of table in shell scripts" So, it looks like
> their inclusion was intentional.
>
> Maybe someone has deeper insights?
:sep header argument is used to format table result when writing to file:
By default, Org assumes that a table written to a file has
TAB-delimited output. You can choose a different separator with
the ‘sep’ header argument.
"16.6 Results of Evaluation" section of the manual.
It is only relevant when you have :results file and the evaluation
results are redirected to file instead of Org buffer.
:sep header argument is universal to all babel backends.
:separator argument is used in ob-sqlite and ob-shell.
However, it means different things in ob-sqlite vs. ob-shell.
In ob-sqlite, :separator argument is defining how sqlite formats its
output. AFAIU, it is only needed when sql table data contains "|" - the
default separator.
In ob-shell, :separator argument is _not_ used to format the output.
Instead, it is used to control how ob-shell passes Elisp data shell variables.
However, it is almost never used in practice (also, it is not
documented) - :separator argument only affects how deeply nested Elisp
lists are converted to bash data:
#+begin_src bash :var x='((((1 1)) 3) (4 5)) :results output :cache no :separator "+"
echo keys: ${!x[@]}
echo values: ${x[@]}
#+end_src
#+RESULTS[7de8e7c561edda2dce18a302f4d8beb8a51595df]:
: keys: 4 1+1
: values: 5 3
Normal two-dimentional tables and lists are handled without being
affected by :separator.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUG] ":sep" header argument doesn't work in ob-shell [9.6.21 ( @ /home/aragaer/.config/emacs/elpa/org-9.6.21/)]
2024-03-09 20:15 [BUG] ":sep" header argument doesn't work in ob-shell [9.6.21 ( @ /home/aragaer/.config/emacs/elpa/org-9.6.21/)] Ilya Konovalov
2024-03-12 9:27 ` Ilya Konovalov
@ 2024-03-12 14:38 ` Ihor Radchenko
1 sibling, 0 replies; 6+ messages in thread
From: Ihor Radchenko @ 2024-03-12 14:38 UTC (permalink / raw)
To: Ilya Konovalov; +Cc: emacs-orgmode
Ilya Konovalov <aragaer@gmail.com> writes:
> On current master using ":sep" header argument does not work as
> expected and records are always split on whitespaces. For example:
>
> #+begin_src sh :sep "\t"
> echo hello world
> echo test
> #+end_src
>
> #+RESULTS:
> | hello | world |
> | test | |
It looks like you are trying to achieve precise control over how
ob-shell interprets the evaluation result.
Note that you can always force :results scalar to interpret the output
as string. Otherwise, Org mode tries to guess the output format.
Since bash does not have a notion of table output, Org's guess is
arbitrary - split multiline result string by line assuming whitespace as
field separator.
So, this is not a bug.
Canceled.
To parse the result as table using custom rules, you can use :post
header argument + custom src block that parser the output string as you
need.
We might consider making ob-shell output handling smarter, as a feature
request.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-03-12 14:34 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-03-09 20:15 [BUG] ":sep" header argument doesn't work in ob-shell [9.6.21 ( @ /home/aragaer/.config/emacs/elpa/org-9.6.21/)] Ilya Konovalov
2024-03-12 9:27 ` Ilya Konovalov
2024-03-12 10:22 ` Matt
2024-03-12 10:41 ` Ilya Konovalov
2024-03-12 14:25 ` Ihor Radchenko
2024-03-12 14:38 ` Ihor Radchenko
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).