Re: babel latex headers and image generation commands

[Matt Huszagh <huszaghmatt@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 9587 bytes --]

Bastien <bzg@gnu.org> writes:

> If you find the time to share your change as a _patch_ (not the whole
> updated Elisp function), that will allow more readers on this list to
> quickly understand what is at stake.

Ok, I've finally gotten around to taking a crack at this. The patch is
attached. Basically, it allows a lot more control when converting a
latex source block into an svg image file.

Specifically, this new method does not place any restrictions on the
latex contents (the old svg generation required the use of
\documentclass[preview]{standalone}), or on the specific generation
commands (it does require first compiling to pdf then to svg, though I
guess this could be generalized if necessary). Additionally, it allows
you to use a setup independent of the setup used for inline latex
snippets. I think this is important because snippets and latex source
blocks have different use cases. For instance, I use snippets for simple
inline math (e.g. \(x=6\)) and source blocks for complicated full-blown
latex pictures and sets of equations, etc. that are not supported by
latex fragments.

I'll present my use case as an example (see the patch to understand the
customizations).

;; this is particular to my use case, see `latex-preamble-by-backend'
(defun by-backend (blist)
  (let ((ret nil))
    (if org-export-current-backend
        (let* ((backend-name org-export-current-backend)
               (elem (assoc backend-name blist)))
          (if elem
              (setq ret (cdr elem))))
      (let ((elem (assoc t blist)))
        (setq ret (cdr elem))))
    (eval ret)))

;; custom function for preamble generation
(defun latex-preamble-by-backend (params)
  (concat "\\documentclass{"
          (cdr (assoc :class params))
          "}"
          "\\definecolor{fg}{rgb}{"
          (by-backend '((html . "0,0,0")
                        (t . (org-latex-color :foreground))))
          "}\n"
          "\\definecolor{bg}{rgb}{"
          (by-backend '((html . "1,1,1")
                        (t . (org-latex-color :background))))
          "}\n"
          "\\def\\pc{"
          (by-backend '((html . "100")
                        (t . "20")))
          "}\n"))

;; actual set customization
(setq org-babel-latex-preamble
      (lambda (params)
        (latex-preamble-by-backend params)))

Now if I define a source block:

#+header: :class math :results file link replace :file tmp/some-file.svg
#+begin_src latex :hidden
{\color{fg}
\begin{equation*}
|z| = \sqrt{x^2 + y^2}
\end{equation*}
}
#+end_src

Using my customization above, this will turn into the following latex

\documentclass{math}\definecolor{fg}{rgb}{0.819608,0.721569,0.592157}
\definecolor{bg}{rgb}{0.0235294,0.137255,0.160784}
\def\pc{20}
\begin{document}{\color{fg}
\begin{equation*}
|z| = \sqrt{x^2 + y^2}
\end{equation*}
}\end{document}

which is then turned into an svg with inkscape, though this could be
dvisvgm, or whatever you want.

Math is a custom document class I've defined. But my function also
allows the setting of other classes (which is useful because some
classes work better for tikz-like images and others work better for
math). In fact, since the preamble generation function can use any of
the source block parameters, I could make it do a bunch of other
conditional stuff. I've also configured it to set my color scheme in a
backend-dependent way. This allows me to get color schemes that match my
emacs theme when generating images for viewing within emacs and another
color scheme (in my case black foreground, white background) when
exporting to html. Note that none of this is enforced on other
people. My patch simply allows you to achieve this sort of flexibility.

I think this patch needs some discussing. I had some trouble deciding on
the best way to implement this functionality because its hard to get
anything to be thematically consistent with the current latex execute
command, which feels like a collection of loosely-related functionality
(for instance, the htlatex backend generates a completely different
latex source than the imagemagick backend, even though the outputs could
be quite similar -- svg and png). I think there's a lot of room for
improvement in the latex execute function that could make it more
general and flexible, but it's hard to do this in a way that doesn't
break existing workflows. My patch tries to be minimally invasive to
these workflows, though it will break workflows in an easily recoverable
way for anyone using htlatex for svg output.

Anyway, I'd be curious to hear thoughts and I'd be interested to discuss
options for further refactoring the latex execute function.

Matt

On Fri, Aug 28, 2020 at 11:10 PM Matt Huszagh <huszaghmatt@gmail.com> wrote:

> Bastien <bzg@gnu.org> writes:
>
> > If you find the time to share your change as a _patch_ (not the whole
> > updated Elisp function), that will allow more readers on this list to
> > quickly understand what is at stake.
>
> Ok, I've finally gotten around to taking a crack at this. The patch is
> attached. Basically, it allows a lot more control when converting a
> latex source block into an svg image file.
>
> Specifically, this new method does not place any restrictions on the
> latex contents (the old svg generation required the use of
> \documentclass[preview]{standalone}), or on the specific generation
> commands (it does require first compiling to pdf then to svg, though I
> guess this could be generalized if necessary). Additionally, it allows
> you to use a setup independent of the setup used for inline latex
> snippets. I think this is important because snippets and latex source
> blocks have different use cases. For instance, I use snippets for simple
> inline math (e.g. \(x=6\)) and source blocks for complicated full-blown
> latex pictures and sets of equations, etc. that are not supported by
> latex fragments.
>
> I'll present my use case as an example (see the patch to understand the
> customizations).
>
> ;; this is particular to my use case, see `latex-preamble-by-backend'
> (defun by-backend (blist)
>   (let ((ret nil))
>     (if org-export-current-backend
>         (let* ((backend-name org-export-current-backend)
>                (elem (assoc backend-name blist)))
>           (if elem
>               (setq ret (cdr elem))))
>       (let ((elem (assoc t blist)))
>         (setq ret (cdr elem))))
>     (eval ret)))
>
> ;; custom function for preamble generation
> (defun latex-preamble-by-backend (params)
>   (concat "\\documentclass{"
>           (cdr (assoc :class params))
>           "}"
>           "\\definecolor{fg}{rgb}{"
>           (by-backend '((html . "0,0,0")
>                         (t . (org-latex-color :foreground))))
>           "}\n"
>           "\\definecolor{bg}{rgb}{"
>           (by-backend '((html . "1,1,1")
>                         (t . (org-latex-color :background))))
>           "}\n"
>           "\\def\\pc{"
>           (by-backend '((html . "100")
>                         (t . "20")))
>           "}\n"))
>
> ;; actual set customization
> (setq org-babel-latex-preamble
>       (lambda (params)
>         (latex-preamble-by-backend params)))
>
> Now if I define a source block:
>
> #+header: :class math :results file link replace :file tmp/some-file.svg
> #+begin_src latex :hidden
> {\color{fg}
> \begin{equation*}
> |z| = \sqrt{x^2 + y^2}
> \end{equation*}
> }
> #+end_src
>
> Using my customization above, this will turn into the following latex
>
> \documentclass{math}\definecolor{fg}{rgb}{0.819608,0.721569,0.592157}
> \definecolor{bg}{rgb}{0.0235294,0.137255,0.160784}
> \def\pc{20}
> \begin{document}{\color{fg}
> \begin{equation*}
> |z| = \sqrt{x^2 + y^2}
> \end{equation*}
> }\end{document}
>
> which is then turned into an svg with inkscape, though this could be
> dvisvgm, or whatever you want.
>
> Math is a custom document class I've defined. But my function also
> allows the setting of other classes (which is useful because some
> classes work better for tikz-like images and others work better for
> math). In fact, since the preamble generation function can use any of
> the source block parameters, I could make it do a bunch of other
> conditional stuff. I've also configured it to set my color scheme in a
> backend-dependent way. This allows me to get color schemes that match my
> emacs theme when generating images for viewing within emacs and another
> color scheme (in my case black foreground, white background) when
> exporting to html. Note that none of this is enforced on other
> people. My patch simply allows you to achieve this sort of flexibility.
>
> I think this patch needs some discussing. I had some trouble deciding on
> the best way to implement this functionality because its hard to get
> anything to be thematically consistent with the current latex execute
> command, which feels like a collection of loosely-related functionality
> (for instance, the htlatex backend generates a completely different
> latex source than the imagemagick backend, even though the outputs could
> be quite similar -- svg and png). I think there's a lot of room for
> improvement in the latex execute function that could make it more
> general and flexible, but it's hard to do this in a way that doesn't
> break existing workflows. My patch tries to be minimally invasive to
> these workflows, though it will break workflows in an easily recoverable
> way for anyone using htlatex for svg output.
>
> Anyway, I'd be curious to hear thoughts and I'd be interested to discuss
> options for further refactoring the latex execute function.
>
> Matt
>
>

[-- Attachment #1.2: Type: text/html, Size: 11594 bytes --]

[-- Attachment #2: 0001-ob-latex.el-Make-latex-to-svg-compilation-very-custo.patch --]
[-- Type: application/octet-stream, Size: 2876 bytes --]

From 0834f59ac9485158c26c4b25ec40df56be2c15ca Mon Sep 17 00:00:00 2001
From: Matt Huszagh <huszaghmatt@gmail.com>
Date: Fri, 28 Aug 2020 22:26:05 -0700
Subject: [PATCH] ob-latex.el: Make latex to svg compilation very customizable

* lisp/ob-latex.el (org-babel-latex-preamble): Add latex preamble
customization.
(org-babel-latex-pdf-svg-process): Add customization for converting a
pdf to svg.
(org-babel-execute:latex): Add specific case for svg generation from
latex block.
---
 lisp/ob-latex.el | 35 +++++++++++++++++++++++++++++++++--
 1 file changed, 33 insertions(+), 2 deletions(-)

diff --git a/lisp/ob-latex.el b/lisp/ob-latex.el
index 4b343dd14..ba7bb8277 100644
--- a/lisp/ob-latex.el
+++ b/lisp/ob-latex.el
@@ -70,6 +70,23 @@
   :group 'org-babel
   :type 'string)
 
+(defcustom org-babel-latex-preamble
+  (lambda (_)
+    "\\documentclass[preview]{standalone}
+\\def\\pgfsysdriver{pgfsys-tex4ht.def}
+")
+  "Closure which evaluates at runtime to the latex preamble.  It
+takes 1 argument which is the parameters of the source block."
+  :group 'org-babel
+  :type 'function)
+
+(defcustom org-babel-latex-pdf-svg-process
+  "inkscape --pdf-poppler %f -T -l -o %O"
+  "Command used to convert a PDF file to an SVG file when
+executing a latex source block."
+  :group 'org-babel
+  :type 'string)
+
 (defcustom org-babel-latex-htlatex-packages
   '("[usenames]{color}" "{tikz}" "{color}" "{listings}" "{amsmath}")
   "Packages to use for htlatex export."
@@ -114,12 +131,26 @@ This function is called by `org-babel-execute-src-block'."
 			 (mapconcat #'identity headers "\n"))))
 	   (org-create-formula-image
             body out-file org-format-latex-options in-buffer)))
+	 ((string= "svg" extension)
+	  (with-temp-file tex-file
+		 (insert (concat (funcall org-babel-latex-preamble params)
+			 (mapconcat #'identity headers "\n")
+			 "\\begin{document}"
+			 body
+			 "\\end{document}")))
+	  (let ((tmp-pdf (org-babel-latex-tex-to-pdf tex-file)))
+                  (let* ((log-buf (get-buffer-create "*Org Babel LaTeX Output*"))
+                         (err-msg "org babel latex failed")
+                         (img-out (org-compile-file
+	                           tmp-pdf
+                                   (list org-babel-latex-pdf-svg-process)
+                                   extension err-msg log-buf)))
+                    (shell-command (format "mv %s %s" img-out out-file)))))
          ((string-suffix-p ".tikz" out-file)
 	  (when (file-exists-p out-file) (delete-file out-file))
 	  (with-temp-file out-file
 	    (insert body)))
-	 ((and (or (string= "svg" extension)
-		   (string= "html" extension))
+	 ((and (string= "html" extension)
 	       (executable-find org-babel-latex-htlatex))
 	  ;; TODO: this is a very different way of generating the
 	  ;; frame latex document than in the pdf case.  Ideally, both
-- 
2.28.0