org-ditaa woes

[Florin Boariu <florin.om@rootshell.ro>]

Hello everyone,

I am not on the mailing list, so I'm hoping that some kind soul with
moderator powers will have mercy and let my email through in a timely
manner :-) Also, please CC me in on the answer. (I'll happily
subscribe if you feel that I should, but this is likely to be my only
encounter with the Org-mode list, so it's probably bogus...)

To my problem.


What did I do
=============

I was trying to create a diagram e.g. from the following org-mode
snippet:

>  #+begin_src ditaa :file network.png
>
>       +------+
>       | moo  |
>       +------+
> 
>  #+end_src

My init.el eventually gets to parse this configuration:

>  (org-babel-do-load-languages 'org-babel-load-languages '(
>      (python . t)
>      (ditaa . t))
>  )
>
>  ;(setq org-ditaa-jar-path "/usr/bin/ditaa")
>  ;(setq org-ditaa-jar-path "/usr/share/java/ditaa.jar")
>  ;(setq org-babel-ditaa-java-cmd "/usr/bin/ditaa")
>  ;(setq org-babel-ditaa-java-cmd "flatpak-spawn --host toolbox run ditaa")

There are several attempts on setting org-ditaa-* variables, or
org-babel-ditaa-*, but all fruitless (see below).

I pressed C-c C-c in order to have a diagram generated.


What did I expect would happen
==============================

The documentation says that a "#+results" section will appear, and it
will contain a link to the ditaa document.

Obviously I'd expect to have a network.png file appearing somewhere on
my filesystem, too, or a PNG inlined into my ORG document. Or
something.


What happend instead
====================

A "#+RESULTS: ..." section with [[file:network.png]] does indeed
appear, but no network.png is to be found anywhere. Clicking on
network.png only opens a new and empty buffer named "network.png",
without any contents.

Essentially, the error messages I find in '*Messages*' so far vary,
depending on which org-ditaa / org-babel-ditaa variables I set, and
how I set them. They range from:

> org-babel-execute:ditaa: Could not find ditaa.jar at /app/share/emacs/29.1/lisp/contrib/scripts/ditaa.jar

via:

> Error: Invalid or corrupt jarfile /usr/bin/ditaa Code block evaluation complete.

to:

> Error: Unable to initialize main class org.stathissideris.ascii2image.core.CommandLineConverter
> Caused by: java.lang.NoClassDefFoundError: org/apache/commons/cli/ParseException
> Code block evaluation complete.


I'm fairly sure I know *why* this is the case, just not how to fix it.

...actually, I'm fairly sure that it can't be fixed on my part :-(

Essentially, emacs / org-babel tries to either execute "java -jar
/usr/bin/ditaa", or doesn't find any jar file, or finds one in
/usr/share/... but fails to actually make it run.

A little bit more background.

I have what would be considered an "exotic setup", although it's
pretty standard for the Linux distribution I'm using (Fedora
Silverblue):

   - My Emacs (29.1 apparently?) runs in a Flatpak

   - All other applications, including ditaa, run in a "Toolbox"
     (https://containertoolbx.org/). Think: kind of a "mutable Docker
     container". Think of it as of a chroot'ed environment where
     everything else that isn't X-windows or Gnome is installed, and
     that only shares /home/me, /var, and a few other non-relevant
     directories with the host system. In particular, it does *not*
     share /usr, /bin, /lib or anything of importance for applications.

   - ...alternatively, I can run an Emacs instance (a 28 version) in
     the same toolbox as the other applications (i.e. ditaa), but the
     results are the same.

   - ditaa within the toolbox is my "distribution standard", which is
     an executable file at /usr/bin/ditaa, and which apparently is a
     shell script with the following contents:

       #!/usr/bin/bash
       #
   
       source /usr/share/java-utils/java-functions

       MAIN_CLASS=org.stathissideris.ascii2image.core.CommandLineConverter
       BASE_JARS="ditaa commons-cli xml-commons-apis batik"

       set_classpath $BASE_JARS

       run "$@"

     This is to show that it's similar to, but still quite a bit more
     than just "java -jar ditaa.jar ...". This appears to be standard
     e.g. for all Fedora distributions (and alike). FWIF, latest Debian
     does something similar, only that /usr/bin/ditaa there is a
     binary.


The core of the problem
========================

Essentially, org-ditaa expects a jar and a java runtime to be able to
consruct a command line of the kind "java -jar /path/to/didaa.jar ..."
and expect everything to sail smoothly. And it's *very* opinionated
about that.

There's no way that I can provide that without a significant amount of
ugly hacking. What I can offer are a myriad of workarounds, but all
boil down to specifying a shell command that does what org-ditaa
expects, but doesn't begin with "java -jar ...". This is simply
because the premise -- namely that emacs, org-ditaa, ditaa.jar and
java-runtime, are in the same filesystem namespace, or the same
process namespace, is simply not true for me. This is by design --
both of the distribution I'm using, and of the whole Flatpak
ecosystem.


Where to go from here?
======================

So... yeah.

This is where I stand. What options do I have?

Besides the obvious "use another distribution" I essentially have two
options, and they both begin with "download a fresh ditaa.jar from
Sourcerforge, independent of the one distributed with your Linux, and
then...":

   1. either patch the Flatpak to actually include this, and refer to
      it from org-ditaa-jar-path, or

   2. copy it somewhere into your "toolbox" filesystem namespace and
      refer to it from org-ditaa-jar-path,

"...and then do this again every time you update your Flatpak or your
Toolbox (which is every couple of days)."

Now that I think of it, there is actually another one:

   3. Manualy download and put a ditaa.jar somewhere in
      "~/.emacs.d/..." (the home folder is persistent even in most
      sandboxes).

But Flatpak-emacs doesn't include a java runtime, so I'd still be
stuck at "java -run ...". However, this would at least enable me to
work with my toolbox-emacs -- albeit at the cost of having to manually
download a .jar file, of which a perfectly fine copy is delivered by
my distribution's package management already.

I wasn't able to find in-depth documentation on how to manipulate
command lines for org-ditaa (if there is any available, please kindly
point me in the right direction -- I looked
mainly at
https://orgmode.org/worg//org-contrib/babel/languages/ob-doc-ditaa.html
and
https://github.com/bzg/worg/blob/master/org-contrib/babel/languages/ob-doc-ditaa.org
and various stackoverflow and github bug reports.)

But in the source code of org-ditaa.el
(https://github.com/tkf/org-mode/blob/master/lisp/ob-ditaa.el) I can
see something like this on lines 87 ff:

> [...]
>        (cmd (concat "java " java " " org-ditaa-jar-option " "
>		      (shell-quote-argument
>		       (expand-file-name
>			(if eps org-ditaa-eps-jar-path org-ditaa-jar-path)))
>		      " " cmdline
>		      " " (org-babel-process-file-name in-file)
>		      " " (org-babel-process-file-name out-file)))
> [...]

I suck at LISP, but I'm guessing this means that there simply
is no way of just passing on a "/usr/bin/ditaa" command-line to
"org-ditaa", or at least an alternative Java command like "flatpak
spawn --host /usr/bin/java ...". Org-ditaa really *does* insist of
glueing it together from "java -jar ..." pieces, and is stubbornly
adamant on finding Java in the same FS namespace.

Is there a deeper reason behind this? This pretty much breaks
Flatpak, or any other sandboxing compatibility, as far as I
understand. Can it be changed? Please? :-)

How can I make it accept a command line?

Is there any "generic" way of making org-babel accept a command line,
not necessarily going through "org-ditaa", as a workaround?

Thanks & cheers,
Florin.

-- 
    "Socks come in pairs. If you put a sock on your left foot, the other
     sock of the pair instantly becomes the “right sock,” no matter where
     it is located in the universe."
                                  -- quantum entanglement explained on /.