* Fix "inverted" behavior of `colnames' header argument
@ 2024-11-02 16:44 Rudolf Adamkovič
2024-11-02 18:39 ` Ihor Radchenko
0 siblings, 1 reply; 8+ messages in thread
From: Rudolf Adamkovič @ 2024-11-02 16:44 UTC (permalink / raw)
To: emacs-orgmode
When loading Org tables into Pandas data frames, the
`colnames' does exactly what it should not,
in that "yes" means "no" and vice versa:
=colnames= set to =yes=:
#+HEADER: :colnames yes
#+HEADER: :session *Python*
#+HEADER: :var INPUT = 20a0ea41-3669-482c-989e-d28486d5d400
#+BEGIN_SRC python
import pandas as pd
pd.DataFrame(INPUT)
#+END_SRC
#+RESULTS:
: 0 1
: 0 10 20
: 1 30 40
=colnames= set to =no=:
#+HEADER: :colnames no
#+HEADER: :session *Python*
#+HEADER: :var INPUT = 20a0ea41-3669-482c-989e-d28486d5d400
#+BEGIN_SRC python
import pandas as pd
pd.DataFrame(INPUT)
#+END_SRC
#+RESULTS:
: 0 1
: 0 A B
: 1 10 20
: 2 30 40
Confirmed on Stack Overflow (Sep 1, 2023):
https://stackoverflow.com/a/67091398/1306956
Rudy
--
"I love deadlines. I love the whooshing noise they make as they go by."
--- Douglas Adams, The Salmon of Doubt, 2002
Rudolf Adamkovič <rudolf@adamkovic.org> [he/him]
http://adamkovic.org
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fix "inverted" behavior of `colnames' header argument
2024-11-02 16:44 Fix "inverted" behavior of `colnames' header argument Rudolf Adamkovič
@ 2024-11-02 18:39 ` Ihor Radchenko
2024-11-03 21:24 ` Rudolf Adamkovič
0 siblings, 1 reply; 8+ messages in thread
From: Ihor Radchenko @ 2024-11-02 18:39 UTC (permalink / raw)
To: Rudolf Adamkovič; +Cc: emacs-orgmode
Rudolf Adamkovič <rudolf@adamkovic.org> writes:
> Fix "inverted" behavior of `colnames' header argument
Our convention is to use [BUG] in subject like to mark bugs.
That's what M-x org-submit-bug-report does.
> When loading Org tables into Pandas data frames, the
>
> `colnames' does exactly what it should not,
> ...
What you show looks consistent with "16.4 Environment of a Code Block"
in Org manual.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fix "inverted" behavior of `colnames' header argument
2024-11-02 18:39 ` Ihor Radchenko
@ 2024-11-03 21:24 ` Rudolf Adamkovič
2024-11-03 22:10 ` Rudolf Adamkovič
2024-11-04 20:06 ` Ihor Radchenko
0 siblings, 2 replies; 8+ messages in thread
From: Rudolf Adamkovič @ 2024-11-03 21:24 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: emacs-orgmode
Ihor Radchenko <yantar92@posteo.net> writes:
> Our convention is to use [BUG] in subject like to mark bugs. That's
> what M-x org-submit-bug-report does.
TIL ... again. :)
> What you show looks consistent with "16.4 Environment of a Code Block"
> in Org manual.
Oh, I see. Wow, that is /really/ confusing from the user's POV.
For anyone reading in the future, to get column names in Pandas, write
`:colnames no' (!) and `pd.DataFrame(VAR[1:], columns=VAR[0])'.
Using the example above, this would be:
#+HEADER: :colnames no
#+HEADER: :session *Python*
#+HEADER: :var INPUT = 20a0ea41-3669-482c-989e-d28486d5d400
#+BEGIN_SRC python
import pandas as pd
pd.DataFrame(INPUT[1:], columns=INPUT[0])
#+END_SRC
#+RESULTS:
: A B
: 0 10 20
: 1 30 40
Canceled.
Rudy
--
"Programming reliably -- must be an activity of an undeniably
mathematical nature […] You see, mathematics is about thinking, and
doing mathematics is always trying to think as well as possible."
--- Edsger W. Dijkstra, 1981
Rudolf Adamkovič <rudolf@adamkovic.org> [he/him]
http://adamkovic.org
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fix "inverted" behavior of `colnames' header argument
2024-11-03 21:24 ` Rudolf Adamkovič
@ 2024-11-03 22:10 ` Rudolf Adamkovič
2024-11-03 23:53 ` Rudolf Adamkovič
2024-11-04 20:06 ` Ihor Radchenko
1 sibling, 1 reply; 8+ messages in thread
From: Rudolf Adamkovič @ 2024-11-03 22:10 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: emacs-orgmode
Rudolf Adamkovič <rudolf@adamkovic.org> writes:
> Oh, I see. Wow, that is /really/ confusing from the user's POV.
In fact, I do not understand the manual. It says:
The default value is ‘nil’: if an input table has column names—because
the second row is a horizontal rule—then Org removes the column names,
processes the table, puts back the column names, and then writes the
table to the results block.
Three questions immediately pop up in my head:
1. Is the manual talking about the input tables, specified with `:var',
the output tables, when `:result table', or both?
2. Org "removes" the column names, then "processes" the table, and then
"puts [the column names] back". Why? And what does it mean?
3. What if the second row is not a horizontal rule? The manual says
nothing about that happens in that case, with `:colnames nil'.
Also, I tried the following with `emacs -Q':
#+NAME: 568ccf1e-f266-47ab-9402-0d380359d73c
| 10 |
|----|
| 20 |
| 30 |
With =colnames= unset:
#+HEADER: :var table = 568ccf1e-f266-47ab-9402-0d380359d73c
#+BEGIN_SRC python
return table
#+END_SRC
#+RESULTS:
| 20 |
| 30 |
With =colnames= set to =nil=:
#+HEADER: :colnames nil
#+HEADER: :var table = 568ccf1e-f266-47ab-9402-0d380359d73c
#+BEGIN_SRC python
return table
#+END_SRC
#+RESULTS:
| 10 |
|----|
| 20 |
| 30 |
Why do the two outputs differ if
The default value is ‘nil’: [...]
Thank you in advance for clarifications!
Rudy
--
"The whole science is nothing more than a refinement of everyday
thinking." --- Albert Einstein, 1879-1955
Rudolf Adamkovič <rudolf@adamkovic.org> [he/him]
http://adamkovic.org
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fix "inverted" behavior of `colnames' header argument
2024-11-03 22:10 ` Rudolf Adamkovič
@ 2024-11-03 23:53 ` Rudolf Adamkovič
2024-11-09 15:03 ` Ihor Radchenko
0 siblings, 1 reply; 8+ messages in thread
From: Rudolf Adamkovič @ 2024-11-03 23:53 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: emacs-orgmode
Rudolf Adamkovič <rudolf@adamkovic.org> writes:
> In fact, I do not understand the manual. It says: [...]
Upon further thinking, I think I now get it.
My understanding:
- =nil= ::
1. Drop the column names from the input, if tabular.
2. Run the code block, possibly producing results.
3. Re-add the column names, if any, to the results, if tabular.
N.B. Column names are row #1 if row #2 is a horizontal rule.
- =yes= ::
1. Drop the first row from the input, if tabular.
2. Run the code block, possibly producing results.
3. Re-add the row, if any, to the results, if tabular.
- no ::
Do not pre-process tabular inputs.
> Also, I tried the following with `emacs -Q':
I noticed that if I pass 'nil (with a quote) instead of nil, everything
works well. Why is that? And, no matter why that is, why do we say nil
in the manual instead of 'nil, if that is what the user must write?
Thank you!
Rudy
--
"I would prefer an intelligent hell to a stupid paradise."
--- Blaise Pascal
Rudolf Adamkovič <rudolf@adamkovic.org> [he/him]
http://adamkovic.org
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fix "inverted" behavior of `colnames' header argument
2024-11-03 21:24 ` Rudolf Adamkovič
2024-11-03 22:10 ` Rudolf Adamkovič
@ 2024-11-04 20:06 ` Ihor Radchenko
2024-11-25 15:11 ` Rudolf Adamkovič
1 sibling, 1 reply; 8+ messages in thread
From: Ihor Radchenko @ 2024-11-04 20:06 UTC (permalink / raw)
To: Rudolf Adamkovič; +Cc: emacs-orgmode
Rudolf Adamkovič <rudolf@adamkovic.org> writes:
>> Our convention is to use [BUG] in subject like to mark bugs. That's
>> what M-x org-submit-bug-report does.
>
> TIL ... again. :)
Also, see https://tracker.orgmode.org/howto
>> What you show looks consistent with "16.4 Environment of a Code Block"
>> in Org manual.
>
> Oh, I see. Wow, that is /really/ confusing from the user's POV.
Yeah. It threw me off first time I've read about column names as well.
But it is what we historically have.
We cannot change the existing behavior, but we might try to
introduce some alternative, more intuitive, parameter or introduce more
intuitive values.
If you have some good ideas how to do such thing, they would be welcome.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fix "inverted" behavior of `colnames' header argument
2024-11-03 23:53 ` Rudolf Adamkovič
@ 2024-11-09 15:03 ` Ihor Radchenko
0 siblings, 0 replies; 8+ messages in thread
From: Ihor Radchenko @ 2024-11-09 15:03 UTC (permalink / raw)
To: Rudolf Adamkovič; +Cc: emacs-orgmode
Rudolf Adamkovič <rudolf@adamkovic.org> writes:
> Upon further thinking, I think I now get it.
>
> My understanding:
>
> - =nil= ::
>
> 1. Drop the column names from the input, if tabular.
> 2. Run the code block, possibly producing results.
> 3. Re-add the column names, if any, to the results, if tabular.
>
> N.B. Column names are row #1 if row #2 is a horizontal rule.
Sounds about right. See `org-babel-disassemble-tables' and
`org-babel-reassemble-table' for what is actually being done.
I guess "auto" would have been a better name for this value.
>> Also, I tried the following with `emacs -Q':
>
> I noticed that if I pass 'nil (with a quote) instead of nil, everything
> works well. Why is that? And, no matter why that is, why do we say nil
> in the manual instead of 'nil, if that is what the user must write?
That's because Org treats :foo text as a string or number by
default. Org only parses the value as Elisp when it starts from "(",
"\\='", "\\=`" or a "[". See `org-babel-read'.
This means that :colnames nil actually means :colnames "nil".
I agree that it is confusing. It would be a good idea to treat nil and
"nil" equally. Maybe even rename the default to "auto" to avoid
confusion (in addition to nil/"nil").
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fix "inverted" behavior of `colnames' header argument
2024-11-04 20:06 ` Ihor Radchenko
@ 2024-11-25 15:11 ` Rudolf Adamkovič
0 siblings, 0 replies; 8+ messages in thread
From: Rudolf Adamkovič @ 2024-11-25 15:11 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: emacs-orgmode
Ihor Radchenko <yantar92@posteo.net> writes:
> If you have some good ideas how to do such thing, they would be
> welcome.
I think your two ideas were spot on:
> It would be a good idea to treat nil and "nil" equally. Maybe even
> rename the default to "auto" to avoid confusion (in addition to
> nil/"nil").
Rudy
--
"I love deadlines. I love the whooshing noise they make as they go by."
--- Douglas Adams, The Salmon of Doubt, 2002
Rudolf Adamkovič <rudolf@adamkovic.org> [he/him]
http://adamkovic.org
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2024-11-25 15:12 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-11-02 16:44 Fix "inverted" behavior of `colnames' header argument Rudolf Adamkovič
2024-11-02 18:39 ` Ihor Radchenko
2024-11-03 21:24 ` Rudolf Adamkovič
2024-11-03 22:10 ` Rudolf Adamkovič
2024-11-03 23:53 ` Rudolf Adamkovič
2024-11-09 15:03 ` Ihor Radchenko
2024-11-04 20:06 ` Ihor Radchenko
2024-11-25 15:11 ` Rudolf Adamkovič
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).