* noweb and :var statements
@ 2019-10-06 16:33 Ken Mankoff
2019-10-06 19:39 ` Sebastian Miele
0 siblings, 1 reply; 10+ messages in thread
From: Ken Mankoff @ 2019-10-06 16:33 UTC (permalink / raw)
To: emacs-orgmode@gnu.org
Hi Org list,
I'm having with noweb and variables. Can someone explain what I'm doing wrong? For example, if I have this table:
#+NAME: table_foo
| foo |
|-----|
| 42 |
| 100 |
And I want to import it into Python and use it, I can do that like this:
#+NAME: import
#+BEGIN_SRC python :var table=table_foo :session foo
import numpy as np
table = np.array(table).astype(np.float).flatten()
#+END_SRC
Eval of this block works, and if I tangle it, I see:
> table=[[42], [100]]
> import numpy as np
> table = np.array(table).astype(np.float).flatten()
But if I want to use that block elsewhere via noweb, it doesn't seem to work:
#+BEGIN_SRC python :results output drawer :noweb yes :session foo :tangle import_noweb.py
<<import()>>
#+END_SRC
The code runs and in a clean *foo* session I do have my table variable, but I *also* get an error. The buffer contains the text at the bottom of this message, and the tangled code in import_noweb.py is only "nil".
How can I 1) run noweb blocks with variables and 2) tangle noweb blocks with variables (i.e. tangle tables into source files).
Thanks,
-k.
table=[[42], [100]]
Python 2.7.15+ (default, Jul 9 2019, 16:51:35)
[GCC 7.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> >>> python.el: native completion setup loaded
>>>
>>> import numpy as np
table = np.array(table).astype(np.float).flatten()
open('/tmp/babel-Mb4ojd/python-GEnZeZ', 'w').write(str(_))
'org_babel_python_eoe'
>>> Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name '_' is not defined
>>> >>> >>> 'org_babel_python_eoe'
>>>
>>>
>>>
>>>
>>> 'org_babel_python_eoe'
'org_babel_python_eoe'
>>>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: noweb and :var statements
2019-10-06 16:33 noweb and :var statements Ken Mankoff
@ 2019-10-06 19:39 ` Sebastian Miele
2019-10-06 19:52 ` Sebastian Miele
2019-10-06 20:05 ` Ken Mankoff
0 siblings, 2 replies; 10+ messages in thread
From: Sebastian Miele @ 2019-10-06 19:39 UTC (permalink / raw)
To: emacs-orgmode
Hi Ken!
Ken Mankoff <mankoff@gmail.com> writes:
> I'm having with noweb and variables. Can someone explain what I'm
> doing wrong? For example, if I have this table:
>
> #+NAME: table_foo
> | foo |
> |-----|
> | 42 |
> | 100 |
>
> And I want to import it into Python and use it, I can do that like this:
>
> #+NAME: import
> #+BEGIN_SRC python :var table=table_foo :session foo
> import numpy as np
> table = np.array(table).astype(np.float).flatten()
> #+END_SRC
>
> Eval of this block works, and if I tangle it, I see:
>
>> table=[[42], [100]]
>> import numpy as np
>> table = np.array(table).astype(np.float).flatten()
>
> But if I want to use that block elsewhere via noweb, it doesn't seem to work:
>
> #+BEGIN_SRC python :results output drawer :noweb yes :session foo :tangle import_noweb.py
> <<import()>>
> #+END_SRC
>
> The code runs and in a clean *foo* session I do have my table
> variable, but I *also* get an error. The buffer contains the text at
> the bottom of this message, and the tangled code in import_noweb.py is
> only "nil".
>
> How can I 1) run noweb blocks with variables and 2) tangle noweb
> blocks with variables (i.e. tangle tables into source files).
Section 15.10 (Noweb Reference Syntax) of the manual says that e.g.
<<NAME(var=value)>> (a noweb reference with parentheses) executes the
NAMEd block with the specified binding and inserts the results of the
execution during tangling. That probably is why the table variable is
there in the session. But the results of executing <<import()>> at that
place were not what was intended.
If it were something like
#+NAME: import
#+BEGIN_SRC python :var table=table_foo :results output
import numpy as np
table = np.array(table).astype(np.float).flatten()
CODE
#+END_SRC
where CODE is replaced by some Python code that prints the wanted Python
code, it would work. (Note the omission of the :session and the addition
of :results output. See section 15.5 (Results of Evaluation) of the
manual for the reasons.)
As far as I understand the manual there may be no way to directly attain
what you are trying.
However, something like the following may suit your use case. (For the
header-args property see section 15.2 (Using Header Arguments) of the
manual.)
* A Heading
:PROPERTIES:
:header-args: :var table=table_foo
:END:
#+NAME: table_foo
| foo |
|-----|
| 42 |
| 100 |
#+NAME: import
#+BEGIN_SRC python
import numpy as np
table = np.array(table).astype(np.float).flatten()
#+END_SRC
#+BEGIN_SRC python :noweb yes :tangle import_noweb.py
<<import>>
#+END_SRC
Best wishes
Sebastian
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: noweb and :var statements
2019-10-06 19:39 ` Sebastian Miele
@ 2019-10-06 19:52 ` Sebastian Miele
2019-10-06 20:08 ` Ken Mankoff
2019-10-06 20:05 ` Ken Mankoff
1 sibling, 1 reply; 10+ messages in thread
From: Sebastian Miele @ 2019-10-06 19:52 UTC (permalink / raw)
To: emacs-orgmode
I wrote:
> [..]
>
> However, something like the following may suit your use case. (For the
> header-args property see section 15.2 (Using Header Arguments) of the
> manual.)
>
> * A Heading
> :PROPERTIES:
> :header-args: :var table=table_foo
> :END:
>
> #+NAME: table_foo
> | foo |
> |-----|
> | 42 |
> | 100 |
>
> #+NAME: import
> #+BEGIN_SRC python
> import numpy as np
> table = np.array(table).astype(np.float).flatten()
> #+END_SRC
>
> #+BEGIN_SRC python :noweb yes :tangle import_noweb.py
> <<import>>
> #+END_SRC
Just the following works too, of course:
#+NAME: table_foo
| foo |
|-----|
| 42 |
| 100 |
#+NAME: import
#+BEGIN_SRC python
import numpy as np
table = np.array(table).astype(np.float).flatten()
#+END_SRC
#+BEGIN_SRC python :noweb yes :var table=table_foo :tangle import_noweb.py
<<import>>
#+END_SRC
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: noweb and :var statements
2019-10-06 19:39 ` Sebastian Miele
2019-10-06 19:52 ` Sebastian Miele
@ 2019-10-06 20:05 ` Ken Mankoff
2019-10-06 20:55 ` Sebastian Miele
1 sibling, 1 reply; 10+ messages in thread
From: Ken Mankoff @ 2019-10-06 20:05 UTC (permalink / raw)
To: sebastian.miele; +Cc: emacs-orgmode
Hi Sebastian,
I'm not getting the results I expect from your MWE either. Perhaps I gave too much code and asked X when what I really want is Y. I think I've distilled it to this:
What is the most elegant Org way to get a table into a Python array?
I can code it directly:
#+BEGIN_SRC python
<<setup>>
print(foo)
#+END_SRC
And now I can hide <<setup>> in a section at the bottom of the document. If it looks like this, everything works:
#+NAME: setup
#+BEGIN_SRC python
foo = np.array([42,43,44])
#+END_SRC
But is there a more elegant method? Can I get the same behavior if the data I want is in an Org table rather than hard-coded directly in Python?
Ideally, I'd like to have:
#+NAME: setup
#+BEGIN_SRC python
<<setup(table="foo_data" varname="foo")>>
<<setup(table="bar_data" varname="bar")>>
#+END_SRC
And a #+NAME: setup block that takes a :var table and sticks it in the :var varname variable.
And then after calling <<setup>> be able to use variable "foo" and "bar" that are generated from column or 2D Org tables elsewhere in the document. Can I do this in Org?
Thanks,
-k.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: noweb and :var statements
2019-10-06 19:52 ` Sebastian Miele
@ 2019-10-06 20:08 ` Ken Mankoff
2019-10-06 21:08 ` Sebastian Miele
0 siblings, 1 reply; 10+ messages in thread
From: Ken Mankoff @ 2019-10-06 20:08 UTC (permalink / raw)
To: sebastian.miele; +Cc: emacs-orgmode
On 2019-10-06 at 21:52 +02, Sebastian Miele <sebastian.miele@gmail.com> wrote...
> I wrote:
>
>> [..]
>>
>> However, something like the following may suit your use case. (For the
>> header-args property see section 15.2 (Using Header Arguments) of the
>> manual.)
>>
>> * A Heading
>> :PROPERTIES:
>> :header-args: :var table=table_foo
>> :END:
>>
>> #+NAME: table_foo
>> | foo |
>> |-----|
>> | 42 |
>> | 100 |
>>
>> #+NAME: import
>> #+BEGIN_SRC python
>> import numpy as np
>> table = np.array(table).astype(np.float).flatten()
>> #+END_SRC
>>
>> #+BEGIN_SRC python :noweb yes :tangle import_noweb.py
>> <<import>>
>> #+END_SRC
>
> Just the following works too, of course:
>
> #+NAME: table_foo
> | foo |
> |-----|
> | 42 |
> | 100 |
>
> #+NAME: import
> #+BEGIN_SRC python
> import numpy as np
> table = np.array(table).astype(np.float).flatten()
> #+END_SRC
>
> #+BEGIN_SRC python :noweb yes :var table=table_foo :tangle import_noweb.py
> <<import>>
> #+END_SRC
The result of tangling this is "import_noweb.py" contains:
import numpy as np
table = np.array(table).astype(np.float).flatten()
And I'd like it to somewhere include the line:
table = [42,100]
or something similar. I don't have the table data in the tangled code.
-k.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: noweb and :var statements
2019-10-06 20:05 ` Ken Mankoff
@ 2019-10-06 20:55 ` Sebastian Miele
0 siblings, 0 replies; 10+ messages in thread
From: Sebastian Miele @ 2019-10-06 20:55 UTC (permalink / raw)
To: Ken Mankoff; +Cc: emacs-orgmode
Ken Mankoff <mankoff@gmail.com> writes:
> Hi Sebastian,
>
> I'm not getting the results I expect from your MWE either. Perhaps I
> gave too much code and asked X when what I really want is Y. I think
> I've distilled it to this:
>
> What is the most elegant Org way to get a table into a Python array?
I do not know whether it is the most elegant way, but the following
works, and probably is at least close to shortes possible way:
#+NAME: table_foo
| foo |
|-----|
| 42 |
| 100 |
#+BEGIN_SRC python :var table=table_foo :tangle table.py
#+END_SRC
After tangling, table.py contains "table=[[42], [100]]".
Did you know that you can have several source code blocks tangling to
the same file? Maybe you do not really need noweb stuff at all. If you
add a second block, e.g.
#+BEGIN_SRC python :tangle table.py
mangle_table
#+END_SRC
then, after tangling, table.py contains "table=[[42], [100]]" and
"mangle_table".
> I can code it directly:
>
> #+BEGIN_SRC python
> <<setup>>
> print(foo)
> #+END_SRC
>
>
> And now I can hide <<setup>> in a section at the bottom of the
> document. If it looks like this, everything works:
>
> #+NAME: setup
> #+BEGIN_SRC python
> foo = np.array([42,43,44])
> #+END_SRC
>
> But is there a more elegant method? Can I get the same behavior if the
> data I want is in an Org table rather than hard-coded directly in
> Python?
This use-case is covered by what I wrote above. You can put the named
Org table at the bottom of the file.
Another note: You can tangle to different files from one Org file. You
may e.g. add an additional source block:
#+BEGIN_SRC python :tangle use_table.py
include_table_py
use_table
#+END_SRC
where include_table_py must be replaced by the Python way of saying:
include the file table.py. (I do not know Python.)
> Ideally, I'd like to have:
>
> #+NAME: setup
> #+BEGIN_SRC python
> <<setup(table="foo_data" varname="foo")>>
> <<setup(table="bar_data" varname="bar")>>
> #+END_SRC
>
> And a #+NAME: setup block that takes a :var table and sticks it in the
> :var varname variable.
I strongly suspect that you do not really need noweb stuff and still
struggle to understand the very basics of tangling.
> And then after calling <<setup>> be able to use variable "foo" and
> "bar" that are generated from column or 2D Org tables elsewhere in the
> document. Can I do this in Org?
>
> Thanks,
>
> -k.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: noweb and :var statements
2019-10-06 20:08 ` Ken Mankoff
@ 2019-10-06 21:08 ` Sebastian Miele
2019-10-06 21:17 ` Sebastian Miele
2019-10-07 5:18 ` Ken Mankoff
0 siblings, 2 replies; 10+ messages in thread
From: Sebastian Miele @ 2019-10-06 21:08 UTC (permalink / raw)
To: Ken Mankoff; +Cc: emacs-orgmode
Ken Mankoff <mankoff@gmail.com> writes:
> On 2019-10-06 at 21:52 +02, Sebastian Miele <sebastian.miele@gmail.com> wrote...
>> I wrote:
>>
>>> [..]
>>>
>>> However, something like the following may suit your use case. (For the
>>> header-args property see section 15.2 (Using Header Arguments) of the
>>> manual.)
>>>
>>> * A Heading
>>> :PROPERTIES:
>>> :header-args: :var table=table_foo
>>> :END:
>>>
>>> #+NAME: table_foo
>>> | foo |
>>> |-----|
>>> | 42 |
>>> | 100 |
>>>
>>> #+NAME: import
>>> #+BEGIN_SRC python
>>> import numpy as np
>>> table = np.array(table).astype(np.float).flatten()
>>> #+END_SRC
>>>
>>> #+BEGIN_SRC python :noweb yes :tangle import_noweb.py
>>> <<import>>
>>> #+END_SRC
>>
>> Just the following works too, of course:
>>
>> #+NAME: table_foo
>> | foo |
>> |-----|
>> | 42 |
>> | 100 |
>>
>> #+NAME: import
>> #+BEGIN_SRC python
>> import numpy as np
>> table = np.array(table).astype(np.float).flatten()
>> #+END_SRC
>>
>> #+BEGIN_SRC python :noweb yes :var table=table_foo :tangle import_noweb.py
>> <<import>>
>> #+END_SRC
>
> The result of tangling this is "import_noweb.py" contains:
>
> import numpy as np
> table = np.array(table).astype(np.float).flatten()
>
> And I'd like it to somewhere include the line:
> table = [42,100]
> or something similar. I don't have the table data in the tangled code.
Strange. On my system, typing C-c C-v t (org-babel-tangle) in an Org
file with the above mentioned contents does yield a file import_noweb.py
with the following contents:
table=[[42], [100]]
import numpy as np
table = np.array(table).astype(np.float).flatten()
Please try it with emacs -Q. Maybe your config is broken.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: noweb and :var statements
2019-10-06 21:08 ` Sebastian Miele
@ 2019-10-06 21:17 ` Sebastian Miele
2019-10-07 5:18 ` Ken Mankoff
1 sibling, 0 replies; 10+ messages in thread
From: Sebastian Miele @ 2019-10-06 21:17 UTC (permalink / raw)
To: Ken Mankoff; +Cc: emacs-orgmode
I wrote:
> [..]
>
> Please try it with emacs -Q. Maybe your config is broken.
After starting emacs -Q you will have to
M-x customize-variable RET org-babel-load-languages
and add Python as a loaded language.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: noweb and :var statements
2019-10-06 21:08 ` Sebastian Miele
2019-10-06 21:17 ` Sebastian Miele
@ 2019-10-07 5:18 ` Ken Mankoff
2019-10-07 9:22 ` Sebastian Miele
1 sibling, 1 reply; 10+ messages in thread
From: Ken Mankoff @ 2019-10-07 5:18 UTC (permalink / raw)
To: sebastian.miele; +Cc: emacs-orgmode
Hi Sebastian,
Thanks for your help. I was running with "-Q" but must have been making some other mistakes. It does work.
As for your other email... I do know several tangles can go to the same file. And I may be using <<noweb>> incorrectly, but I'm using it for the following reasons:
1) I'd like to bury code that must run first but is inconsequential at the bottom of the Org file. Noweb lets me have a tangled <<setup>> at the top, and then hide the lengthy <<setup>> code elsewhere. Is this a correct use case?
2) I'd like to import 10 tables, so I thought a noweb function might be useful for code reuse.
I finally got the behavior I'm looking for. What I need to remember/understand is that <<noweb>> just pastes the body, and <<noweb()>> evaluates the function. From this, my Python code needs to generate Python code! I now have the following MWE that behaves as I want both for in-buffer C-c C-c eval of main code block and tangled results. The key bit of code is the last babel block.
Thanks for your help,
-k.
* MWE init
#+BEGIN_SRC emacs-lisp :results output
(setq org-confirm-babel-evaluate nil)
(org-babel-do-load-languages 'org-babel-load-languages '((python . t)))
(print (emacs-version))
(print (org-version nil t))
(org-babel-tangle)
#+END_SRC
* Main Project
#+NAME: main
#+BEGIN_SRC python :tangle MWE3.py :noweb yes :results output
<<setup>>
print(t42.sum())
print(t100.sum())
#+END_SRC
#+RESULTS: main
: 84.0
: 100.0
* Data Tables
#+NAME: table_42
| foo |
|-----|
| 42 |
| 42 |
#+NAME: table_100
| bar |
|-----|
| 100 |
* Setup
#+NAME: setup
#+BEGIN_SRC python :noweb yes
import numpy as np
<<import_table_to_varname(table=table_42, varname="t42")>>
<<import_table_to_varname(table=table_100, varname="t100")>>
#+END_SRC
* Table Import Code
#+NAME: import_table_to_varname
#+BEGIN_SRC python :var table=table_42 :var varname="foo" :noweb yes :results output
print(varname + "=np.array(" + str(table) + ").astype(np.float).flatten()")
#+END_SRC
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: noweb and :var statements
2019-10-07 5:18 ` Ken Mankoff
@ 2019-10-07 9:22 ` Sebastian Miele
0 siblings, 0 replies; 10+ messages in thread
From: Sebastian Miele @ 2019-10-07 9:22 UTC (permalink / raw)
To: Ken Mankoff; +Cc: emacs-orgmode
Ken Mankoff <mankoff@gmail.com> writes:
> Hi Sebastian,
>
> Thanks for your help. I was running with "-Q" but must have been
> making some other mistakes. It does work.
>
> As for your other email... I do know several tangles can go to the
> same file. And I may be using <<noweb>> incorrectly, but I'm using it
> for the following reasons:
>
> 1) I'd like to bury code that must run first but is inconsequential at
> the bottom of the Org file. Noweb lets me have a tangled <<setup>> at
> the top, and then hide the lengthy <<setup>> code elsewhere. Is this a
> correct use case?
Yes.
> 2) I'd like to import 10 tables, so I thought a noweb function might
> be useful for code reuse.
>
> I finally got the behavior I'm looking for. What I need to
> remember/understand is that <<noweb>> just pastes the body, and
> <<noweb()>> evaluates the function. From this, my Python code needs to
> generate Python code! I now have the following MWE that behaves as I
> want both for in-buffer C-c C-c eval of main code block and tangled
> results. The key bit of code is the last babel block.
A solution without having to write code-writing code is tangling to
different files (MWE3.py and setup.py). See the example below.
I read somewhere that Python has stong reflection features. It should be
possible to write Python code that, given a string and a value,
introduces a Python variable of that name and binds the value to it. If
you find out how that works, the mangle block below can be changed so
that the main block has e.g. setup.A.sum() instead of
setup.tables["A"].sum().
Final disclaimer: There may and probably are other possibilies that I do
not know or have not thought of.
* Main Project
#+NAME: main
#+BEGIN_SRC python :tangle MWE3.py :noweb yes :results output
import setup
print(setup.tables["A"].sum())
print(setup.tables["B"].sum())
#+END_SRC
* Data Tables
#+NAME: table_42
| foo |
|-----|
| 42 |
| 42 |
#+NAME: table_100
| bar |
|-----|
| 100 |
* Setup
#+BEGIN_SRC python :tangle setup.py
import numpy as np
tables={}
#+END_SRC
#+NAME: mangle
#+BEGIN_SRC python
tables[name] = np.array(table).astype(np.float).flatten()
#+END_SRC
#+BEGIN_SRC python :tangle setup.py :noweb yes :var table=table_42 :var name="A"
<<mangle>>
#+END_SRC
#+BEGIN_SRC python :tangle setup.py :noweb yes :var table=table_100 :var name="B"
<<mangle>>
#+END_SRC
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2019-10-07 9:23 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-06 16:33 noweb and :var statements Ken Mankoff
2019-10-06 19:39 ` Sebastian Miele
2019-10-06 19:52 ` Sebastian Miele
2019-10-06 20:08 ` Ken Mankoff
2019-10-06 21:08 ` Sebastian Miele
2019-10-06 21:17 ` Sebastian Miele
2019-10-07 5:18 ` Ken Mankoff
2019-10-07 9:22 ` Sebastian Miele
2019-10-06 20:05 ` Ken Mankoff
2019-10-06 20:55 ` Sebastian Miele
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).