“Literate” python?

“Literate” python?

[Norman Walsh]

[-- Attachment #1: Type: text/plain, Size: 2571 bytes --]

Hi,

I’ve seen a couple of pointers recently to using Org mode and tangle
to write more literate Emacs configurations. I use Org+babel all the
time to write “interactive” documents, so I thought I’d try out tangle
from Org.

I didn’t want to start with something as comlicated as my Emacs
config :-) so I figured I’d kick the tires with a small python
program. That did not end well.

Consider:

#+TITLE: Python literate programming
#+OPTIONS: html-postamble:nil

It starts off as a completely standard Python3 program.

---%<------------------------------------------------------
#+BEGIN_SRC python :tangle yes :weave no
#!/usr/bin/env python3

#+END_SRC

It defines ~a~.

#+BEGIN_SRC python :tangle yes
def a():
    print("a")


#+END_SRC

And ~b~.

#+BEGIN_SRC python :tangle yes
def b():
    print("b")


#+END_SRC

Now ~c~ is a little more complicated:

#+BEGIN_SRC python :tangle yes
def c():
   print("c")
#+END_SRC

Not only does ~c~ print “c”, it calls ~a()~ and ~b()~.

#+BEGIN_SRC python :tangle yes
   b()
   a()
#+END_SRC

Finally, make it importable. Not that you’d want to.

#+BEGIN_SRC python :tangle yes
if __name__ == "__main__":
    main()
#+END_SRC
--->%------------------------------------------------------

That’s the script. It weaves into HTML more-or-less ok (there’s a
weird black box at the front of indented lines, but I can come back to
that later).

It’s a complete mess when tangled.

The extra blank lines between functions (to make pylint happy with
some PEP guideline) have disappeared. I guess I could live with that,
but the complete failure to preserve indention in the penultimate code
block is a show stopper:

#!/usr/bin/env python3

def a():
    print("a")

def b():
    print("b")

def c():
   print("c")

b()
a()

if __name__ == "__main__":
    main()

(Also, why is there an extra blank line before the incorrectly
indented block?)

Is this user error on my part somehow? I suppose I could write my own
version of tangle, though I’m not clear if the whitespace is lost in
the tangle function or in the Org mode data model.

Thoughts?

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <ndw@nwalsh.com> | We discover in ourselves what others
http://nwalsh.com/            | hide from us, and we recognize in
                              | others what we hide from
                              | ourselves.--Vauvenargues

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: “Literate” python?

[Berry, Charles]

> On Nov 29, 2019, at 9:54 AM, Norman Walsh <ndw@nwalsh.com> wrote:
> 
> Hi,
> 
> I’ve seen a couple of pointers recently to using Org mode and tangle
> to write more literate Emacs configurations. I use Org+babel all the
> time to write “interactive” documents, so I thought I’d try out tangle
> from Org.
> 
> I didn’t want to start with something as comlicated as my Emacs
> config :-) so I figured I’d kick the tires with a small python
> program. That did not end well.
> 
> Consider:
> 
> #+TITLE: Python literate programming
> #+OPTIONS: html-postamble:nil
> 
> It starts off as a completely standard Python3 program.
> 
> ---%<------------------------------------------------------
> #+BEGIN_SRC python :tangle yes :weave no
> #!/usr/bin/env python3
> 
> #+END_SRC
> 
> It defines ~a~.
> 
> #+BEGIN_SRC python :tangle yes
> def a():
>    print("a")
> 
> 
> #+END_SRC
> 
> And ~b~.
> 
> #+BEGIN_SRC python :tangle yes
> def b():
>    print("b")
> 
> 
> #+END_SRC
> 
> Now ~c~ is a little more complicated:
> 
> #+BEGIN_SRC python :tangle yes
> def c():
>   print("c")
> #+END_SRC
> 
> Not only does ~c~ print “c”, it calls ~a()~ and ~b()~.
> 
> #+BEGIN_SRC python :tangle yes
>   b()
>   a()
> #+END_SRC
> 
> Finally, make it importable. Not that you’d want to.
> 
> #+BEGIN_SRC python :tangle yes
> if __name__ == "__main__":
>    main()
> #+END_SRC
> --->%------------------------------------------------------
> 
> That’s the script. It weaves into HTML more-or-less ok (there’s a
> weird black box at the front of indented lines, but I can come back to
> that later).
> 
> It’s a complete mess when tangled.
> 
> The extra blank lines between functions (to make pylint happy with
> some PEP guideline) have disappeared. I guess I could live with that,
> but the complete failure to preserve indention in the penultimate code
> block is a show stopper:

[results of tangling deleted]

A couple of things might help.

First, use the :noweb-ref argument to mark each of the code blocks you wish to tangle.  

Say `:noweb-ref tangleyes'. Or some more evocative name.

Remove the `:tangle yes' from each of those. Then, add a code block that has only `<<tangleyes>>' in it and tangle it.

#+begin_src python :noweb yes :tangle yes
<<tangleyes>>
#+end_src

The remaining problem (as you will see) is the indentation. Fix this by adding the `-i' flag to the penultimate code block, viz.

#+BEGIN_SRC python -i :noweb-ref tangleyes
   b()
   a()
#+END_SRC

See 12.6 Literal Examples and 15.10 Noweb Reference Syntax in the manual.

HTH,

Chuck

Re: “Literate” python?

[George Mauer]

[-- Attachment #1: Type: text/plain, Size: 3797 bytes --]

I've used noweb references to actually assemble what will be tangled all at
once. See how I did it here
<https://github.com/togakangaroo/daily-programmer/tree/master/sliding-puzzle>
.

The reason why the "incorrect" block is outdented is that tangle
automatically tries to guess indentation level. (Take that as a handwavy
thing - I don't know exactly how it does that nor how configurable it is)
The solution I've found is noweb - basically you *wouldn't* tangle any of
those blocks, but would create another non-evaluated or exportable block
that has noweb references to the above blocks and how you want them
arranged. *This* is the block you tangle.

I haven't yet figured out how, but it should be possible to automatically
configure things to run autopep8 after tangling - that would take care of
indentation issues.

Also I'm pretty sure there's no :weave header arg...at least I haven't seen
it used and can't find it documented anywhere.

On Fri, Nov 29, 2019 at 12:08 PM Norman Walsh <ndw@nwalsh.com> wrote:

> Hi,
>
> I’ve seen a couple of pointers recently to using Org mode and tangle
> to write more literate Emacs configurations. I use Org+babel all the
> time to write “interactive” documents, so I thought I’d try out tangle
> from Org.
>
> I didn’t want to start with something as comlicated as my Emacs
> config :-) so I figured I’d kick the tires with a small python
> program. That did not end well.
>
> Consider:
>
> #+TITLE: Python literate programming
> #+OPTIONS: html-postamble:nil
>
> It starts off as a completely standard Python3 program.
>
> ---%<------------------------------------------------------
> #+BEGIN_SRC python :tangle yes :weave no
> #!/usr/bin/env python3
>
> #+END_SRC
>
> It defines ~a~.
>
> #+BEGIN_SRC python :tangle yes
> def a():
>     print("a")
>
>
> #+END_SRC
>
> And ~b~.
>
> #+BEGIN_SRC python :tangle yes
> def b():
>     print("b")
>
>
> #+END_SRC
>
> Now ~c~ is a little more complicated:
>
> #+BEGIN_SRC python :tangle yes
> def c():
>    print("c")
> #+END_SRC
>
> Not only does ~c~ print “c”, it calls ~a()~ and ~b()~.
>
> #+BEGIN_SRC python :tangle yes
>    b()
>    a()
> #+END_SRC
>
> Finally, make it importable. Not that you’d want to.
>
> #+BEGIN_SRC python :tangle yes
> if __name__ == "__main__":
>     main()
> #+END_SRC
> --->%------------------------------------------------------
>
> That’s the script. It weaves into HTML more-or-less ok (there’s a
> weird black box at the front of indented lines, but I can come back to
> that later).
>
> It’s a complete mess when tangled.
>
> The extra blank lines between functions (to make pylint happy with
> some PEP guideline) have disappeared. I guess I could live with that,
> but the complete failure to preserve indention in the penultimate code
> block is a show stopper:
>
> #!/usr/bin/env python3
>
> def a():
>     print("a")
>
> def b():
>     print("b")
>
> def c():
>    print("c")
>
> b()
> a()
>
> if __name__ == "__main__":
>     main()
>
> (Also, why is there an extra blank line before the incorrectly
> indented block?)
>
> Is this user error on my part somehow? I suppose I could write my own
> version of tangle, though I’m not clear if the whitespace is lost in
> the tangle function or in the Org mode data model.
>
> Thoughts?
>
>                                         Be seeing you,
>                                           norm
>
> --
> Norman Walsh <ndw@nwalsh.com> | We discover in ourselves what others
> http://nwalsh.com/            | hide from us, and we recognize in
>                               | others what we hide from
>                               | ourselves.--Vauvenargues
>

[-- Attachment #2: Type: text/html, Size: 4736 bytes --]

Re: “Literate” python?

[Norman Walsh]

[-- Attachment #1: Type: text/plain, Size: 965 bytes --]

"Berry, Charles" <ccberry@health.ucsd.edu> writes:

> A couple of things might help.
>
> First, use the :noweb-ref argument to mark each of the code blocks
> you wish to tangle.
[…]
> The remaining problem (as you will see) is the indentation. Fix this
> by adding the `-i' flag to the penultimate code block, viz.
[…]
> See 12.6 Literal Examples and 15.10 Noweb Reference Syntax in the manual.

Thank you. I had failed to locate the relevant manual sections. Those
changes did appear to resolve the issues and I’ll study the docs with
a little more care!

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <ndw@nwalsh.com> | To enjoy yourself and make others enjoy
http://nwalsh.com/            | themselves, without harming yourself or
                              | any other; that, to my mind, is the
                              | whole of ethics.--Chamfort

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: “Literate” python?

[Norman Walsh]

[-- Attachment #1: Type: text/plain, Size: 712 bytes --]

George Mauer <gmauer@gmail.com> writes:
> I've used noweb references to actually assemble what will be tangled
> all at once. See how I did it here.

Thanks.

> Also I'm pretty sure there's no :weave header arg...at least I
> haven't seen it used and can't find it documented anywhere.

No, that was just me guessing. Thanks for the pointers.

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <ndw@nwalsh.com> | A child becomes an adult when he
http://nwalsh.com/            | realizes he has a right not only to be
                              | right but also to be wrong.--Thomas
                              | Szasz

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: “Literate” python?

[Diego Zamboni]

[-- Attachment #1: Type: text/plain, Size: 4617 bytes --]

Hi Norm,

As George said, the trick in this case is to use the =:noweb= and
=:noweb-ref= headers. The change is minimal from the script you sent:

#+TITLE: Python literate programming
#+OPTIONS: html-postamble:nil

It starts off as a completely standard Python3 program.

#+BEGIN_SRC python :tangle yes :weave no
#!/usr/bin/env python3
#+END_SRC

It defines ~a~.

#+BEGIN_SRC python :tangle yes
def a():
    print("a")

#+END_SRC

And ~b~.

#+BEGIN_SRC python :tangle yes
def b():
    print("b")

#+END_SRC

Now ~c~ is a little more complicated:

#+BEGIN_SRC python :tangle yes :noweb no-export
def c():
   print("c")
   <<call-a-and-b>>
#+END_SRC

Not only does ~c~ print “c”, it calls ~a()~ and ~b()~.

#+BEGIN_SRC python :tangle no :noweb-ref call-a-and-b
   b()
   a()
#+END_SRC

Finally, make it importable. Not that you’d want to.

#+BEGIN_SRC python :tangle yes
if __name__ == "__main__":
    main()
#+END_SRC

Note the =:noweb no-export= in the block that contains =def c()=. The
=no-export= value makes it so that, on HTML export, the noweb reference is
shown as a reference instead of expanded (which is usually what you want).
The next block is given its name using the =:noweb-ref= header argument.
You could also use =#+name:= - the main difference is that =:noweb-ref=
allows you to have multiple blocks with the same name, which are
concatenated together when tangled, whereas =#+name:= only allows one block
with the same name.

If I may do a bit of self-promotion, feel free to check out my "Literate
Config" booklet, which I published just a few days ago (available for free)
and which contains some more tips for doing literate programming:
https://leanpub.com/lit-config/read

Best,
--Diego


On Fri, Nov 29, 2019 at 7:09 PM Norman Walsh <ndw@nwalsh.com> wrote:

> Hi,
>
> I’ve seen a couple of pointers recently to using Org mode and tangle
> to write more literate Emacs configurations. I use Org+babel all the
> time to write “interactive” documents, so I thought I’d try out tangle
> from Org.
>
> I didn’t want to start with something as comlicated as my Emacs
> config :-) so I figured I’d kick the tires with a small python
> program. That did not end well.
>
> Consider:
>
> #+TITLE: Python literate programming
> #+OPTIONS: html-postamble:nil
>
> It starts off as a completely standard Python3 program.
>
> ---%<------------------------------------------------------
> #+BEGIN_SRC python :tangle yes :weave no
> #!/usr/bin/env python3
>
> #+END_SRC
>
> It defines ~a~.
>
> #+BEGIN_SRC python :tangle yes
> def a():
>     print("a")
>
>
> #+END_SRC
>
> And ~b~.
>
> #+BEGIN_SRC python :tangle yes
> def b():
>     print("b")
>
>
> #+END_SRC
>
> Now ~c~ is a little more complicated:
>
> #+BEGIN_SRC python :tangle yes
> def c():
>    print("c")
> #+END_SRC
>
> Not only does ~c~ print “c”, it calls ~a()~ and ~b()~.
>
> #+BEGIN_SRC python :tangle yes
>    b()
>    a()
> #+END_SRC
>
> Finally, make it importable. Not that you’d want to.
>
> #+BEGIN_SRC python :tangle yes
> if __name__ == "__main__":
>     main()
> #+END_SRC
> --->%------------------------------------------------------
>
> That’s the script. It weaves into HTML more-or-less ok (there’s a
> weird black box at the front of indented lines, but I can come back to
> that later).
>
> It’s a complete mess when tangled.
>
> The extra blank lines between functions (to make pylint happy with
> some PEP guideline) have disappeared. I guess I could live with that,
> but the complete failure to preserve indention in the penultimate code
> block is a show stopper:
>
> #!/usr/bin/env python3
>
> def a():
>     print("a")
>
> def b():
>     print("b")
>
> def c():
>    print("c")
>
> b()
> a()
>
> if __name__ == "__main__":
>     main()
>
> (Also, why is there an extra blank line before the incorrectly
> indented block?)
>
> Is this user error on my part somehow? I suppose I could write my own
> version of tangle, though I’m not clear if the whitespace is lost in
> the tangle function or in the Org mode data model.
>
> Thoughts?
>
>                                         Be seeing you,
>                                           norm
>
> --
> Norman Walsh <ndw@nwalsh.com> | We discover in ourselves what others
> http://nwalsh.com/            | hide from us, and we recognize in
>                               | others what we hide from
>                               | ourselves.--Vauvenargues
>

[-- Attachment #2: Type: text/html, Size: 5787 bytes --]

Re: “Literate” python?

[Norman Walsh]

[-- Attachment #1: Type: text/plain, Size: 1167 bytes --]

Diego Zamboni <diego@zzamboni.org> writes:
> Hi Norm,
>
> As George said, the trick in this case is to use the =:noweb= and
> =:noweb-ref= headers. The change is minimal from the script you
> sent:

Thanks. With your help (and Barry’s and George’s), I got over the
initial hurdles. I wrote about it here:
https://so.nwalsh.com/2019/11/29/t2

> If I may do a bit of self-promotion, feel free to check out my
> "Literate Config" booklet, which I published just a few days ago
> (available for free) and which contains some more tips for doing
> literate programming: https://leanpub.com/lit-config/read

Purchased. Authors gotta get paid. :-)

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <ndw@nwalsh.com> | The First Amendment is often
http://nwalsh.com/            | inconvenient. But that is besides the
                              | point. Inconvenience does not absolve
                              | the government of its obligation to
                              | tolerate speech.--Justice Anthony
                              | Kennedy, in 91-155

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]