emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed

* noweb and :var statements
@ 2019-10-06 16:33 Ken Mankoff
  2019-10-06 19:39 ` Sebastian Miele
  0 siblings, 1 reply; 10+ messages in thread
From: Ken Mankoff @ 2019-10-06 16:33 UTC (permalink / raw)
  To: emacs-orgmode@gnu.org

Hi Org list,

I'm having with noweb and variables. Can someone explain what I'm doing wrong? For example, if I have this table:

#+NAME: table_foo
| foo |
|-----|
|  42 |
| 100 |

And I want to import it into Python and use it, I can do that like this:

#+NAME: import
#+BEGIN_SRC python :var table=table_foo :session foo
import numpy as np
table = np.array(table).astype(np.float).flatten()
#+END_SRC

Eval of this block works, and if I tangle it, I see:

> table=[[42], [100]]
> import numpy as np
> table = np.array(table).astype(np.float).flatten()

But if I want to use that block elsewhere via noweb, it doesn't seem to work:

#+BEGIN_SRC python :results output drawer :noweb yes :session foo :tangle import_noweb.py
<<import()>>
#+END_SRC

The code runs and in a clean *foo* session I do have my table variable, but I *also* get an error. The buffer contains the text at the bottom of this message, and the tangled code in import_noweb.py is only "nil".

How can I 1) run noweb blocks with variables and 2) tangle noweb blocks with variables (i.e. tangle tables into source files).

Thanks,

  -k.


table=[[42], [100]]
Python 2.7.15+ (default, Jul  9 2019, 16:51:35) 
[GCC 7.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> >>> python.el: native completion setup loaded
>>> 
>>> import numpy as np
table = np.array(table).astype(np.float).flatten()

open('/tmp/babel-Mb4ojd/python-GEnZeZ', 'w').write(str(_))


'org_babel_python_eoe'
>>> Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name '_' is not defined
>>> >>> >>> 'org_babel_python_eoe'
>>> 
>>> 
>>> 
>>> 
>>> 'org_babel_python_eoe'
'org_babel_python_eoe'
>>> 

^ permalink raw reply	[flat|nested] 10+ messages in thread

• * Re: noweb and :var statements
    2019-10-06 16:33 noweb and :var statements Ken Mankoff
  @ 2019-10-06 19:39 ` Sebastian Miele
    2019-10-06 19:52   ` Sebastian Miele
    2019-10-06 20:05   ` Ken Mankoff
    0 siblings, 2 replies; 10+ messages in thread
  From: Sebastian Miele @ 2019-10-06 19:39 UTC (permalink / raw)
    To: emacs-orgmode
  
  Hi Ken!
  
  Ken Mankoff <mankoff@gmail.com> writes:
  
  > I'm having with noweb and variables. Can someone explain what I'm
  > doing wrong? For example, if I have this table:
  >
  > #+NAME: table_foo
  > | foo |
  > |-----|
  > |  42 |
  > | 100 |
  >
  > And I want to import it into Python and use it, I can do that like this:
  >
  > #+NAME: import
  > #+BEGIN_SRC python :var table=table_foo :session foo
  > import numpy as np
  > table = np.array(table).astype(np.float).flatten()
  > #+END_SRC
  >
  > Eval of this block works, and if I tangle it, I see:
  >
  >> table=[[42], [100]]
  >> import numpy as np
  >> table = np.array(table).astype(np.float).flatten()
  >
  > But if I want to use that block elsewhere via noweb, it doesn't seem to work:
  >
  > #+BEGIN_SRC python :results output drawer :noweb yes :session foo :tangle import_noweb.py
  > <<import()>>
  > #+END_SRC
  >
  > The code runs and in a clean *foo* session I do have my table
  > variable, but I *also* get an error. The buffer contains the text at
  > the bottom of this message, and the tangled code in import_noweb.py is
  > only "nil".
  >
  > How can I 1) run noweb blocks with variables and 2) tangle noweb
  > blocks with variables (i.e. tangle tables into source files).
  
  Section 15.10 (Noweb Reference Syntax) of the manual says that e.g.
  <<NAME(var=value)>> (a noweb reference with parentheses) executes the
  NAMEd block with the specified binding and inserts the results of the
  execution during tangling. That probably is why the table variable is
  there in the session. But the results of executing <<import()>> at that
  place were not what was intended.
  
  If it were something like
  
  #+NAME: import
  #+BEGIN_SRC python :var table=table_foo :results output
  import numpy as np
  table = np.array(table).astype(np.float).flatten()
  CODE
  #+END_SRC
  
  where CODE is replaced by some Python code that prints the wanted Python
  code, it would work. (Note the omission of the :session and the addition
  of :results output. See section 15.5 (Results of Evaluation) of the
  manual for the reasons.)
  
  As far as I understand the manual there may be no way to directly attain
  what you are trying.
  
  However, something like the following may suit your use case. (For the
  header-args property see section 15.2 (Using Header Arguments) of the
  manual.)
  
  * A Heading
  :PROPERTIES:
  :header-args: :var table=table_foo
  :END:
  
  #+NAME: table_foo
  | foo |
  |-----|
  |  42 |
  | 100 |
  
  #+NAME: import
  #+BEGIN_SRC python
  import numpy as np
  table = np.array(table).astype(np.float).flatten()
  #+END_SRC
  
  #+BEGIN_SRC python :noweb yes :tangle import_noweb.py
  <<import>>
  #+END_SRC
  
  Best wishes
  Sebastian
  
  ^ permalink raw reply	[flat|nested] 10+ messages in thread

• • * Re: noweb and :var statements
      2019-10-06 19:39 ` Sebastian Miele
    @ 2019-10-06 19:52   ` Sebastian Miele
      2019-10-06 20:08     ` Ken Mankoff
      2019-10-06 20:05   ` Ken Mankoff
      1 sibling, 1 reply; 10+ messages in thread
    From: Sebastian Miele @ 2019-10-06 19:52 UTC (permalink / raw)
      To: emacs-orgmode
    
    
    I wrote:
    
    > [..]
    >
    > However, something like the following may suit your use case. (For the
    > header-args property see section 15.2 (Using Header Arguments) of the
    > manual.)
    >
    > * A Heading
    > :PROPERTIES:
    > :header-args: :var table=table_foo
    > :END:
    >
    > #+NAME: table_foo
    > | foo |
    > |-----|
    > |  42 |
    > | 100 |
    >
    > #+NAME: import
    > #+BEGIN_SRC python
    > import numpy as np
    > table = np.array(table).astype(np.float).flatten()
    > #+END_SRC
    >
    > #+BEGIN_SRC python :noweb yes :tangle import_noweb.py
    > <<import>>
    > #+END_SRC
    
    Just the following works too, of course:
    
    #+NAME: table_foo
    | foo |
    |-----|
    |  42 |
    | 100 |
    
    #+NAME: import
    #+BEGIN_SRC python
    import numpy as np
    table = np.array(table).astype(np.float).flatten()
    #+END_SRC
    
    #+BEGIN_SRC python :noweb yes :var table=table_foo :tangle import_noweb.py
    <<import>>
    #+END_SRC
    
    ^ permalink raw reply	[flat|nested] 10+ messages in thread

  • • * Re: noweb and :var statements
        2019-10-06 19:52   ` Sebastian Miele
      @ 2019-10-06 20:08     ` Ken Mankoff
        2019-10-06 21:08       ` Sebastian Miele
        0 siblings, 1 reply; 10+ messages in thread
      From: Ken Mankoff @ 2019-10-06 20:08 UTC (permalink / raw)
        To: sebastian.miele; +Cc: emacs-orgmode
      
      
      On 2019-10-06 at 21:52 +02, Sebastian Miele <sebastian.miele@gmail.com> wrote...
      > I wrote:
      >
      >> [..]
      >>
      >> However, something like the following may suit your use case. (For the
      >> header-args property see section 15.2 (Using Header Arguments) of the
      >> manual.)
      >>
      >> * A Heading
      >> :PROPERTIES:
      >> :header-args: :var table=table_foo
      >> :END:
      >>
      >> #+NAME: table_foo
      >> | foo |
      >> |-----|
      >> |  42 |
      >> | 100 |
      >>
      >> #+NAME: import
      >> #+BEGIN_SRC python
      >> import numpy as np
      >> table = np.array(table).astype(np.float).flatten()
      >> #+END_SRC
      >>
      >> #+BEGIN_SRC python :noweb yes :tangle import_noweb.py
      >> <<import>>
      >> #+END_SRC
      >
      > Just the following works too, of course:
      >
      > #+NAME: table_foo
      > | foo |
      > |-----|
      > |  42 |
      > | 100 |
      >
      > #+NAME: import
      > #+BEGIN_SRC python
      > import numpy as np
      > table = np.array(table).astype(np.float).flatten()
      > #+END_SRC
      >
      > #+BEGIN_SRC python :noweb yes :var table=table_foo :tangle import_noweb.py
      > <<import>>
      > #+END_SRC
      
      The result of tangling this is "import_noweb.py" contains:
      
      import numpy as np
      table = np.array(table).astype(np.float).flatten()
      
      And I'd like it to somewhere include the line:
      table = [42,100]
      or something similar. I don't have the table data in the tangled code.
      
        -k.
      
      ^ permalink raw reply	[flat|nested] 10+ messages in thread

    • • * Re: noweb and :var statements
          2019-10-06 20:08     ` Ken Mankoff
        @ 2019-10-06 21:08       ` Sebastian Miele
          2019-10-06 21:17         ` Sebastian Miele
          2019-10-07  5:18         ` Ken Mankoff
          0 siblings, 2 replies; 10+ messages in thread
        From: Sebastian Miele @ 2019-10-06 21:08 UTC (permalink / raw)
          To: Ken Mankoff; +Cc: emacs-orgmode
        
        
        Ken Mankoff <mankoff@gmail.com> writes:
        
        > On 2019-10-06 at 21:52 +02, Sebastian Miele <sebastian.miele@gmail.com> wrote...
        >> I wrote:
        >>
        >>> [..]
        >>>
        >>> However, something like the following may suit your use case. (For the
        >>> header-args property see section 15.2 (Using Header Arguments) of the
        >>> manual.)
        >>>
        >>> * A Heading
        >>> :PROPERTIES:
        >>> :header-args: :var table=table_foo
        >>> :END:
        >>>
        >>> #+NAME: table_foo
        >>> | foo |
        >>> |-----|
        >>> |  42 |
        >>> | 100 |
        >>>
        >>> #+NAME: import
        >>> #+BEGIN_SRC python
        >>> import numpy as np
        >>> table = np.array(table).astype(np.float).flatten()
        >>> #+END_SRC
        >>>
        >>> #+BEGIN_SRC python :noweb yes :tangle import_noweb.py
        >>> <<import>>
        >>> #+END_SRC
        >>
        >> Just the following works too, of course:
        >>
        >> #+NAME: table_foo
        >> | foo |
        >> |-----|
        >> |  42 |
        >> | 100 |
        >>
        >> #+NAME: import
        >> #+BEGIN_SRC python
        >> import numpy as np
        >> table = np.array(table).astype(np.float).flatten()
        >> #+END_SRC
        >>
        >> #+BEGIN_SRC python :noweb yes :var table=table_foo :tangle import_noweb.py
        >> <<import>>
        >> #+END_SRC
        >
        > The result of tangling this is "import_noweb.py" contains:
        >
        > import numpy as np
        > table = np.array(table).astype(np.float).flatten()
        >
        > And I'd like it to somewhere include the line:
        > table = [42,100]
        > or something similar. I don't have the table data in the tangled code.
        
        Strange. On my system, typing C-c C-v t (org-babel-tangle) in an Org
        file with the above mentioned contents does yield a file import_noweb.py
        with the following contents:
        
        table=[[42], [100]]
        import numpy as np
        table = np.array(table).astype(np.float).flatten()
        
        Please try it with emacs -Q. Maybe your config is broken.
        
        ^ permalink raw reply	[flat|nested] 10+ messages in thread

      • • * Re: noweb and :var statements
            2019-10-06 21:08       ` Sebastian Miele
          @ 2019-10-06 21:17         ` Sebastian Miele
            2019-10-07  5:18         ` Ken Mankoff
            1 sibling, 0 replies; 10+ messages in thread
          From: Sebastian Miele @ 2019-10-06 21:17 UTC (permalink / raw)
            To: Ken Mankoff; +Cc: emacs-orgmode
          
          
          I wrote:
          
          > [..]
          >
          > Please try it with emacs -Q. Maybe your config is broken.
          
          After starting emacs -Q you will have to
          M-x customize-variable RET org-babel-load-languages
          and add Python as a loaded language.
          
          ^ permalink raw reply	[flat|nested] 10+ messages in thread

        • * Re: noweb and :var statements
            2019-10-06 21:08       ` Sebastian Miele
            2019-10-06 21:17         ` Sebastian Miele
          @ 2019-10-07  5:18         ` Ken Mankoff
            2019-10-07  9:22           ` Sebastian Miele
            1 sibling, 1 reply; 10+ messages in thread
          From: Ken Mankoff @ 2019-10-07  5:18 UTC (permalink / raw)
            To: sebastian.miele; +Cc: emacs-orgmode
          
          Hi Sebastian,
          
          Thanks for your help. I was running with "-Q" but must have been making some other mistakes. It does work.
          
          As for your other email... I do know several tangles can go to the same file. And I may be using <<noweb>> incorrectly, but I'm using it for the following reasons:
          
          1) I'd like to bury code that must run first but is inconsequential at the bottom of the Org file. Noweb lets me have a tangled <<setup>> at the top, and then hide the lengthy <<setup>> code elsewhere. Is this a correct use case?
          
          2) I'd like to import 10 tables, so I thought a noweb function might be useful for code reuse.
          
          I finally got the behavior I'm looking for. What I need to remember/understand is that <<noweb>> just pastes the body, and <<noweb()>> evaluates the function. From this, my Python code needs to generate Python code! I now have the following MWE that behaves as I want both for in-buffer C-c C-c eval of main code block and tangled results. The key bit of code is the last babel block.
          
          Thanks for your help,
          
            -k.
          
          
          * MWE init
          #+BEGIN_SRC emacs-lisp :results output
          (setq org-confirm-babel-evaluate nil)
          (org-babel-do-load-languages 'org-babel-load-languages '((python . t)))
          (print (emacs-version))
          (print (org-version nil t))
          (org-babel-tangle)
          #+END_SRC
          
          * Main Project
          
          #+NAME: main
          #+BEGIN_SRC python :tangle MWE3.py :noweb yes :results output
          <<setup>>
          print(t42.sum())
          print(t100.sum())
          #+END_SRC
          
          #+RESULTS: main
          : 84.0
          : 100.0
          
          * Data Tables
          #+NAME: table_42
          | foo |
          |-----|
          |  42 |
          |  42 |
          
          #+NAME: table_100
          | bar |
          |-----|
          | 100 |
          
          * Setup
          
          #+NAME: setup
          #+BEGIN_SRC python :noweb yes
          import numpy as np
          <<import_table_to_varname(table=table_42, varname="t42")>>
          <<import_table_to_varname(table=table_100, varname="t100")>>
          #+END_SRC
          
          * Table Import Code
          
          #+NAME: import_table_to_varname
          #+BEGIN_SRC python :var table=table_42 :var varname="foo" :noweb yes :results output
          print(varname + "=np.array(" + str(table) + ").astype(np.float).flatten()")
          #+END_SRC
          
          ^ permalink raw reply	[flat|nested] 10+ messages in thread

        • • * Re: noweb and :var statements
              2019-10-07  5:18         ` Ken Mankoff
            @ 2019-10-07  9:22           ` Sebastian Miele
              0 siblings, 0 replies; 10+ messages in thread
            From: Sebastian Miele @ 2019-10-07  9:22 UTC (permalink / raw)
              To: Ken Mankoff; +Cc: emacs-orgmode
            
            
            Ken Mankoff <mankoff@gmail.com> writes:
            
            > Hi Sebastian,
            >
            > Thanks for your help. I was running with "-Q" but must have been
            > making some other mistakes. It does work.
            >
            > As for your other email... I do know several tangles can go to the
            > same file. And I may be using <<noweb>> incorrectly, but I'm using it
            > for the following reasons:
            >
            > 1) I'd like to bury code that must run first but is inconsequential at
            > the bottom of the Org file. Noweb lets me have a tangled <<setup>> at
            > the top, and then hide the lengthy <<setup>> code elsewhere. Is this a
            > correct use case?
            
            Yes.
            
            > 2) I'd like to import 10 tables, so I thought a noweb function might
            > be useful for code reuse.
            >
            > I finally got the behavior I'm looking for. What I need to
            > remember/understand is that <<noweb>> just pastes the body, and
            > <<noweb()>> evaluates the function. From this, my Python code needs to
            > generate Python code! I now have the following MWE that behaves as I
            > want both for in-buffer C-c C-c eval of main code block and tangled
            > results. The key bit of code is the last babel block.
            
            A solution without having to write code-writing code is tangling to
            different files (MWE3.py and setup.py). See the example below.
            
            I read somewhere that Python has stong reflection features. It should be
            possible to write Python code that, given a string and a value,
            introduces a Python variable of that name and binds the value to it. If
            you find out how that works, the mangle block below can be changed so
            that the main block has e.g. setup.A.sum() instead of
            setup.tables["A"].sum().
            
            Final disclaimer: There may and probably are other possibilies that I do
            not know or have not thought of.
            
            * Main Project
            
            #+NAME: main
            #+BEGIN_SRC python :tangle MWE3.py :noweb yes :results output
            import setup
            print(setup.tables["A"].sum())
            print(setup.tables["B"].sum())
            #+END_SRC
            
            * Data Tables
            #+NAME: table_42
            | foo |
            |-----|
            |  42 |
            |  42 |
            
            #+NAME: table_100
            | bar |
            |-----|
            | 100 |
            
            * Setup
            
            #+BEGIN_SRC python :tangle setup.py
            import numpy as np
            tables={}
            #+END_SRC
            
            #+NAME: mangle
            #+BEGIN_SRC python
            tables[name] = np.array(table).astype(np.float).flatten()
            #+END_SRC
            
            #+BEGIN_SRC python :tangle setup.py :noweb yes :var table=table_42 :var name="A"
            <<mangle>>
            #+END_SRC
            
            #+BEGIN_SRC python :tangle setup.py :noweb yes :var table=table_100 :var name="B"
            <<mangle>>
            #+END_SRC
            
            ^ permalink raw reply	[flat|nested] 10+ messages in thread

  • * Re: noweb and :var statements
      2019-10-06 19:39 ` Sebastian Miele
      2019-10-06 19:52   ` Sebastian Miele
    @ 2019-10-06 20:05   ` Ken Mankoff
      2019-10-06 20:55     ` Sebastian Miele
      1 sibling, 1 reply; 10+ messages in thread
    From: Ken Mankoff @ 2019-10-06 20:05 UTC (permalink / raw)
      To: sebastian.miele; +Cc: emacs-orgmode
    
    Hi Sebastian,
    
    I'm not getting the results I expect from your MWE either. Perhaps I gave too much code and asked X when what I really want is Y. I think I've distilled it to this:
    
    What is the most elegant Org way to get a table into a Python array?
    
    I can code it directly:
    
    #+BEGIN_SRC python
    <<setup>>
    print(foo)
    #+END_SRC
    
    
    And now I can hide <<setup>> in a section at the bottom of the document. If it looks like this, everything works:
    
    #+NAME: setup
    #+BEGIN_SRC python
    foo = np.array([42,43,44])
    #+END_SRC
    
    But is there a more elegant method? Can I get the same behavior if the data I want is in an Org table rather than hard-coded directly in Python?
    
    
    
    
    
    Ideally, I'd like to have:
    
    #+NAME: setup
    #+BEGIN_SRC python
    <<setup(table="foo_data" varname="foo")>>
    <<setup(table="bar_data" varname="bar")>>
    #+END_SRC
    
    And a #+NAME: setup block that takes a :var table and sticks it in the :var varname variable.
    
    And then after calling <<setup>> be able to use variable "foo" and "bar" that are generated from column or 2D Org tables elsewhere in the document. Can I do this in Org?
    
    Thanks,
    
      -k.
    
    ^ permalink raw reply	[flat|nested] 10+ messages in thread

  • • * Re: noweb and :var statements
        2019-10-06 20:05   ` Ken Mankoff
      @ 2019-10-06 20:55     ` Sebastian Miele
        0 siblings, 0 replies; 10+ messages in thread
      From: Sebastian Miele @ 2019-10-06 20:55 UTC (permalink / raw)
        To: Ken Mankoff; +Cc: emacs-orgmode
      
      
      Ken Mankoff <mankoff@gmail.com> writes:
      
      > Hi Sebastian,
      >
      > I'm not getting the results I expect from your MWE either. Perhaps I
      > gave too much code and asked X when what I really want is Y. I think
      > I've distilled it to this:
      >
      > What is the most elegant Org way to get a table into a Python array?
      
      I do not know whether it is the most elegant way, but the following
      works, and probably is at least close to shortes possible way:
      
      #+NAME: table_foo
      | foo |
      |-----|
      |  42 |
      | 100 |
      
      #+BEGIN_SRC python :var table=table_foo :tangle table.py
      #+END_SRC
      
      After tangling, table.py contains "table=[[42], [100]]".
      
      Did you know that you can have several source code blocks tangling to
      the same file? Maybe you do not really need noweb stuff at all. If you
      add a second block, e.g.
      
      #+BEGIN_SRC python :tangle table.py
      mangle_table
      #+END_SRC
      
      then, after tangling, table.py contains "table=[[42], [100]]" and
      "mangle_table".
      
      
      > I can code it directly:
      >
      > #+BEGIN_SRC python
      > <<setup>>
      > print(foo)
      > #+END_SRC
      >
      >
      > And now I can hide <<setup>> in a section at the bottom of the
      > document. If it looks like this, everything works:
      >
      > #+NAME: setup
      > #+BEGIN_SRC python
      > foo = np.array([42,43,44])
      > #+END_SRC
      >
      > But is there a more elegant method? Can I get the same behavior if the
      > data I want is in an Org table rather than hard-coded directly in
      > Python?
      
      This use-case is covered by what I wrote above. You can put the named
      Org table at the bottom of the file.
      
      Another note: You can tangle to different files from one Org file. You
      may e.g. add an additional source block:
      
      #+BEGIN_SRC python :tangle use_table.py
      include_table_py
      use_table
      #+END_SRC
      
      where include_table_py must be replaced by the Python way of saying:
      include the file table.py. (I do not know Python.)
      
      > Ideally, I'd like to have:
      >
      > #+NAME: setup
      > #+BEGIN_SRC python
      > <<setup(table="foo_data" varname="foo")>>
      > <<setup(table="bar_data" varname="bar")>>
      > #+END_SRC
      >
      > And a #+NAME: setup block that takes a :var table and sticks it in the
      > :var varname variable.
      
      I strongly suspect that you do not really need noweb stuff and still
      struggle to understand the very basics of tangling.
      
      > And then after calling <<setup>> be able to use variable "foo" and
      > "bar" that are generated from column or 2D Org tables elsewhere in the
      > document. Can I do this in Org?
      >
      > Thanks,
      >
      >   -k.
      
      ^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2019-10-07  9:23 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-06 16:33 noweb and :var statements Ken Mankoff
2019-10-06 19:39 ` Sebastian Miele
2019-10-06 19:52   ` Sebastian Miele
2019-10-06 20:08     ` Ken Mankoff
2019-10-06 21:08       ` Sebastian Miele
2019-10-06 21:17         ` Sebastian Miele
2019-10-07  5:18         ` Ken Mankoff
2019-10-07  9:22           ` Sebastian Miele
2019-10-06 20:05   ` Ken Mankoff
2019-10-06 20:55     ` Sebastian Miele

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).