[tip] org-publish to work with (very) large books

["Juan Manuel Macías" <maciaschain@posteo.net>]

Hi all,

- tl; dr: I describe here my workflow with org-publish to work with long
books.

—

I discovered a long time ago that `org-publish' not only works very well
for managing websites but also for working with long and complex books
with many parts, with output to LaTeX/PDF. I developed a workflow around
it that has been quite productive for me, and that I briefly describe
here in case someone finds it useful and wants to try it or modify/adapt
it to their needs. I usually use it for my typesetting work, but I think
it can also be useful for personal projects, such as doctoral theses.

First of all, each folder of my project-books has the same structure:
two subdirectories named `/org' and `/tex', for the source `org' files
and for the output `.tex' documents, respectively. And, inside the `org'
directory I include a `setup' file, an `elisp' file (for export
filters), and another `/img' directory for image files. Each `org' file
is a part of the book, and usually begins simply with the directives:

┌────
│ #+SETUPFILE: xxx.setup
│ #+INCLUDE: "elisp"
└────

`Org-publish' exports the subdocuments (body only!) as `.tex' documents
in the `/tex' folder, but they are not compiled.

What gets compiled is a master `.org' file, which is also inside the
`org' folder. I compile this master file using an asynchronous function
that calls `latexmk'. I put all the LaTeX configuration, the packages to
load, the (re)defined commands and macros, the necessary Lua code, etc.
in a `.sty' file that I load at the very beginning of the master
document. Subdocuments are loaded into this file by the LaTeX command
(\input), not by the org #+INCLUDE directive. So the master file usually
looks like this:

┌────
│ #+LaTeX_CLASS: my-custom-latex-class
│ #+LaTeX_Header: \input{my-custom-conf.sty}
│ #+SETUPFILE: xxxx.setup
│ #+INCLUDE: "elisp"
│ 
│ * Part 1
│ ** Chapter 1
│ #+LaTeX: \input{chapter1.tex}
└────

When I eval my function, `latexmk' compiles the entire book with the
`-pvc' option, which keeps the session open, and if it detects any
changes to the entire document, it recompiles and refresh the pdf view.
For example, if I edit one of the subdocuments and run
`org-publish-current-file', everything is automatically recompiled.

When I have the project folder ready, I add this to
`org-publish-project-alist' (this is an example from one of the books I
did recently):

┌────
│ ("cartas-org"
│  :base-directory "~/Git/cartas/libro/org/"
│  :base-extension "org"
│  ;; Directorio para los ficheros *.tex
│  :publishing-directory "~/Git/cartas/libro/tex/"
│  :publishing-function org-latex-publish-to-latex
│  :body-only t ;; this is important!
│  :exclude "cartas-master\\.org\\|bibliografia-cartas\\.org"
│  :recursive t)
│ 
│ ("cartas-img"
│  :base-directory "~/Git/cartas/libro/org/img/"
│  :base-extension "jpg\\|png"
│  :publishing-directory "~/Git/cartas/libro/tex/img/"
│  :recursive t
│  :publishing-function org-publish-attachment)
│ 
│ ("cartas" :components ("cartas-tex" "cartas-img"))
└────

And finally, this is the function that compiles everything (each project
usually has some local variables, like the value of ’jobname’ or the
status of the printing proofs).

Nota Bene: The reason for using async is that in some projects,
especially bilingual editions, I need to pre-compile some files first.
Under normal conditions I don't think it's necessary to use async, since
org-publish just exports everything to .tex documents (short timeout)
and then start-process-shell-command run latexmk asynchronously.

Best regards,

Juan Manuel 

┌────
│ (require 'async)
│ (require 'projectile)
│ 
│ (defun latexmk-compile-project-async ()
│   (interactive)
│   (let*
│       ((project-root (projectile-project-root))
│        (master-file (read-file-name "Compile: "
│ 				    (concat project-root "libro/org/")))
│        (master-file-tex (file-name-sans-extension
│ 			 (expand-file-name master-file)))
│        (dir-tex (file-name-directory
│ 		 (expand-file-name
│ 		  (replace-regexp-in-string "/org/" "/tex/" master-file)))))
│     ;; save the master document
│     (with-current-buffer
│ 	(find-file-noselect master-file)
│       (save-buffer))
│     (async-start
│      (lambda ()
│        (load "~/.emacs")
│        (with-current-buffer
│ 	   (find-file-noselect master-file)
│ 	 (org-show-all)
│ 	 (save-buffer)
│ 	 (org-latex-export-to-latex nil nil nil nil nil))
│        ;; remove all old auxiliary files before compiling
│        (shell-command (concat "rm -r " dir-tex (file-name-base master-file-tex) "*"))
│        (shell-command (concat "mv " master-file-tex ".tex" " " dir-tex))
│        "Document exported")
│      (lambda (resultado)
│        (message resultado)
│        (let
│ 	   ((default-directory dir-tex)
│ 	    (jobname (if (and jobname-local printing-proofs-state)
│ 			 (concat jobname-local "_" printing-proofs-state "_"
│ 				 (format-time-string "%d-%m-%y"))
│ 		       (concat (file-name-sans-extension
│ 				(file-name-nondirectory master-file-tex))
│ 			       "_"
│ 			       (format-time-string "%d-%m-%y")))))
│ 	 (start-process-shell-command
│ 	  "project"
│ 	  "*project*"
│ 	  (concat
│ 	   "latexmk"
│ 	   " -jobname="
│ 	   jobname
│ 	   " -pvc -lualatex -e '$lualatex=q/lualatex %O -shell-escape %S/' "
│ 	   (file-name-nondirectory master-file-tex)
│ 	   ".tex")))))))
└────