[BUG] ob-clojure doesn't handle namespaces properly

[BUG] ob-clojure doesn't handle namespaces properly

[Rens Oliemans]

[-- Attachment #1: Type: text/plain, Size: 582 bytes --]

Hi,

There are two bugs introduced in commit 6efb073463481ee572eb3bb1155cc3b0d6987df6;
I Cced Daniel, the commit author. It persists on main.

Since the bugs are intimately related to ob-clojure and the org mode source
blocks, I've just attached an org file that highlights the two bugs, I think
that's easiest.

I'm fairly sure that they have the same root cause: wrapping each block inside
`(binding [*out (java.io.StringWriter.)])'. I'm not too familiar with
(ob-)clojure, so I'm not sure what the best way to solve this would be, but I'm
happy to try if necessary.

Best,
Rens


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: bugs.org --]
[-- Type: text/org, Size: 2964 bytes --]

#+title: Clojure bugs
I bisected this to [[https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=6efb073463481ee572eb3bb1155cc3b0d6987df6][6efb073463481ee572eb3bb1155cc3b0d6987df6]]. I've only tested this on =cider=, I'm not sure if that makes a difference.

* Bug 1: Different namespaces in the same document don't work well
Say I have the following structure:

** Util
:PROPERTIES:
:header-args:clojure: :exports both :session *clojure-util*
:END:

It seems ~(ns example.util)~ isn't run properly: =sum= gets added to the default namespace the first time it's run. The second time it's run, it does work. In general, a block with a ~(ns)~ call needs to be run twice, it seems. To show this behaviour, I've added =:results value append= to the block below.

#+begin_src clojure :results value append
  (ns example.util)
  (defn sum [vec]
    (reduce + vec))
#+end_src

#+RESULTS:
: Please reevaluate when nREPL is connected
: #'user/sum
: #'example.util/sum

This has the result that if I execute the above block only once, other functions cannot use =util/sum=; see the code blocks below. If the REPL is restarted, it needs to be run twice again.

To show that you need to run the above block twice, see the output of the block below. I executed it once after the /first/ time the previous was run (and =sum= was defined in the =user= namespace), and once after the /second/ time.
#+begin_src clojure :results value append
  (sum (range 10))
#+end_src

#+RESULTS:
: class clojure.lang.Compiler$CompilerException
: 45

** Program
:PROPERTIES:
:header-args:clojure: :exports both :session *clojure-01*
:END:

This code block always errors:
#+begin_src clojure
  (ns example.program
    (:require [example.util :as util]))

  (util/sum (range 10))
#+end_src

#+RESULTS:
: class clojure.lang.Compiler$CompilerException

* Bug 2: When tangling, code blocks can't see previously defined vars
:PROPERTIES:
:header-args:clojure: :tangle tangled.clj :session *clojure-tangle* :results none
:END:

This does not occur with =:results output=, since in that case =ob-clojure= keeps the body unchanged. If we use the default, however, each code block is individually wrapped in a =prn= call, and all variables defined are limited to the scope of the org source block..

If we first define some function =sum=,
#+begin_src clojure
  (ns example.tangle)

  (defn sum [xs] (reduce + xs))
#+end_src

We cannot use it in later code blocks. Well, when executing this from Org mode or when exporting, things go well. However, when this file gets tangled (here to =tangled.clj=), the following line fails (/Unable to resolve symbol: sum in this context/).
#+begin_src clojure
  (sum (range 10))
#+end_src

#+RESULTS:
: 45

#+begin_src sh :results output
  clj -M tangled.clj 2>&1
#+end_src

#+RESULTS:
: #'user/sum
: Syntax error compiling at (tangled.clj:5:46).
: Unable to resolve symbol: sum in this context
: 
: Full report at:
: /tmp/clojure-14548717163049100556.edn

Re: [BUG] ob-clojure doesn't handle namespaces properly

[Ihor Radchenko]

Rens Oliemans <hallo@rensoliemans.nl> writes:

> There are two bugs introduced in commit 6efb073463481ee572eb3bb1155cc3b0d6987df6;
> I Cced Daniel, the commit author. It persists on main.
>
> Since the bugs are intimately related to ob-clojure and the org mode source
> blocks, I've just attached an org file that highlights the two bugs, I think
> that's easiest.

I cannot reproduce the problems you are showing. Although I am able to
reproduce *different* problems.

> I'm fairly sure that they have the same root cause: wrapping each block inside
> `(binding [*out (java.io.StringWriter.)])'. I'm not too familiar with
> (ob-)clojure, so I'm not sure what the best way to solve this would be, but I'm
> happy to try if necessary.

May you try reproducing from clean emacs -Q?

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] ob-clojure doesn't handle namespaces properly

[Rens Oliemans]

[-- Attachment #1: Type: text/plain, Size: 612 bytes --]

Sorry for the late reply, I took a good holiday.

Ihor Radchenko <yantar92@posteo.net> writes:

> I cannot reproduce the problems you are showing. Although I am able to
> reproduce *different* problems.

That's interesting, I can reproduce the problems with emacs -Q. Attached is the
new org file which reproduces the error with =make repro=. Included are some
settings (such as installing clojure-mode and cider). Apologies for the longish
file, if I had more time I would have made it shorter.

"Bug 2" is easiest to reproduce, it just requires a tangle and 'clj -M tangled.clj'
should then fail.

Best,
Rens


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: bugs.org --]
[-- Type: text/org, Size: 4051 bytes --]

#+title: Clojure bugs
I bisected this to [[https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=6efb073463481ee572eb3bb1155cc3b0d6987df6][6efb073463481ee572eb3bb1155cc3b0d6987df6]]. I've only
tested this on =cider=, I'm not sure if that makes a difference.

* Preface
I'm running this via =make repro=, on [[https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=405787dfbcbd4ae5e1d2227ce5aa7d4d20938388][405787dfbcbd4ae5e1d2227ce5aa7d4d20938388]].

This installs and loads =cider= and =clojure-mode= - I usually do this
via =use-package= but that makes the config for this a bit more
complicated, so please forgive any uncommon practices.

#+begin_src elisp :results silent
  (require 'ob-core)
  ;(setq org-confirm-babel-evaluate nil)
  (org-babel-do-load-languages 'org-babel-load-languages
  			     '((shell . t)
  			       (clojure . t)))

  (require 'ob-clojure)
  (setq org-babel-clojure-backend 'cider)

  (require 'package)
  (package-initialize)

  (package-refresh-contents)
  (package-install 'clojure-mode)
  (package-install 'cider)

  (add-to-list 'load-path "~/.emacs.d")

  (require 'clojure-mode)
  (require 'cider)
#+end_src

* Bug 1: Different namespaces in the same document don't work well
Say I have the following structure:

** Util
:PROPERTIES:
:header-args:clojure: :exports both :session *clojure-util*
:END:

It seems ~(ns example.util)~ isn't run properly in the first code
block: =sum= gets added to the default =user= namespace the first time
it's run. The second time it's run, it does work as expected. In
general, a block with a ~(ns)~ call needs to be run twice, it
seems?

Rather than executing the code blocks below, check the list of CALLS
below them so they're executed in the correct order.

#+name: define-sum
#+begin_src clojure
  (ns example.util)
  (defn sum [vec]
    (reduce + vec))
#+end_src

#+name: use-sum
#+begin_src clojure
  (sum (range 10))
#+end_src

*** Calls
Execute these calls one by one:

For this one, I use =cider-jack-in-clj=.
#+CALL: define-sum()

#+RESULTS:
: Please reevaluate when nREPL is connected

The call below should, imo, add =sum= to the =util= namespace, which
it doesn't.
#+CALL: define-sum()

#+RESULTS:
: #'user/sum

Therefore, the call below errors:
#+CALL: use-sum()

#+RESULTS:
: class clojure.lang.Compiler$CompilerException

When =define-sum= is run for the second time, it's added correctly.
#+CALL: define-sum()

#+RESULTS:
: #'example.util/sum

And it can be used as expected.
#+CALL: use-sum()

#+RESULTS:
: 45

** Program using Util
:PROPERTIES:
:header-args:clojure: :exports both :session *clojure-01*
:END:

This code block always errors:
#+begin_src clojure
  (ns example.program
    (:require [example.util :as util]))

  (util/sum (range 10))
#+end_src

#+RESULTS:
: class clojure.lang.Compiler$CompilerException

* Bug 2: When tangling, code blocks can't see previously defined vars
:PROPERTIES:
:header-args:clojure: :tangle tangled.clj :session *clojure-tangle* :results none
:END:

Let's create a clojure file that first defines a function =sum=:
#+begin_src clojure
  (ns example.tangle)

  (defn sum [xs] (reduce + xs))
#+end_src

And then uses it:
#+begin_src clojure
  (sum (range 10))
#+end_src

If we tangle this subtree, we get the following output:
#+begin_src elisp
  (org-babel-tangle)
  (with-temp-buffer
    (insert-file-contents "tangled.clj")
    (buffer-string))
#+end_src

#+RESULTS:
: (prn (binding [*out* (java.io.StringWriter.)](ns example.tangle)
: 
: (defn sum [xs] (reduce + xs))))
: 
: (prn (binding [*out* (java.io.StringWriter.)](sum (range 10))))

Note that each code block is wrapped in an invidiual =(prn (binding
...))= block. This means that later code blocks have a different scope
to prior code blocks. Executing this file also leads to an error:

#+begin_src sh :results output
  clj -M tangled.clj 2>&1
#+end_src

#+RESULTS:
: #'user/sum
: Syntax error compiling at (tangled.clj:5:46).
: Unable to resolve symbol: sum in this context
: 
: Full report at:
: /tmp/clojure-11215251630972501185.edn