From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id oCcZEHZQj2I3qAAAbAwnHQ (envelope-from ) for ; Thu, 26 May 2022 12:03:34 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id 4HsrD3ZQj2IPCQEAG6o9tA (envelope-from ) for ; Thu, 26 May 2022 12:03:34 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 63A5C1791C for ; Thu, 26 May 2022 12:03:33 +0200 (CEST) Received: from localhost ([::1]:37804 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nuAL5-00052y-QW for larch@yhetil.org; Thu, 26 May 2022 06:03:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47340) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nuAIu-00052m-NJ for emacs-orgmode@gnu.org; Thu, 26 May 2022 06:01:17 -0400 Received: from mout01.posteo.de ([185.67.36.65]:49927) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nuAIm-0000Hs-7r for emacs-orgmode@gnu.org; Thu, 26 May 2022 06:01:15 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id AE0A4240026 for ; Thu, 26 May 2022 12:01:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1653559265; bh=DzueNJaISHiSCmGk5xpvUvKjevPEvaRUim/K0M8XzY0=; h=From:To:Subject:Date:From; b=IIS51OWOXp8jPHBkVgR0lHB3QJTYRgv2gSC8wY7PsAJ789XaD9yXR+syuuAxjUdOb PNT1gni6hm7j6nd9+5Xh0HzAeLoc/hQU2bDP89XS2uyaFAidyM4MFNkJ7NPOfLjevJ 4W4WWRLOSh3SBSyI6hB94i2AT8SQEnLGwjL4DBH3bIcIWZeN8PvTy1SCcNnscAaY/M IJVwTu4KknvCJfEODccAbAtG58nlh2lB4P91tT5ZWJrJuufBReDqGCv3OxHWFgCueo htBdfbKmCwnCRzqG3J0z8HtqIINNePI1pNRSDTxPrHafibfOEUO7PkCUtgBtqbsDAc iqMBrGaWrZT9w== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4L83Nd1KWnz9rxb for ; Thu, 26 May 2022 12:01:04 +0200 (CEST) From: =?utf-8?Q?Juan_Manuel_Mac=C3=ADas?= To: orgmode Subject: [tip] org-publish to work with (very) large books Date: Thu, 26 May 2022 10:01:02 +0000 Message-ID: <87y1yovcip.fsf@posteo.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=185.67.36.65; envelope-from=maciaschain@posteo.net; helo=mout01.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1653559413; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=HP963jkjdRZFtZG4kYl7GalB1e2+nQW9fIXOjq8WEdA=; b=niXcDPTlxXVDHVGJSDTRikb+QgtM60xlYEUczPglQ3+laKNs7z95kUSbZO+WdMoScJvhIM Q+oZSbv1AZQNhrSuDVgMuHzyt0XJRUQc462uBZ99Yo8hWL35cZmnifs2nw/x2u8KUyAQMl xNgdXFWI6WkjZT4gxU6d25fxliciC9aXuNavZC5vcdHkZG6PedEztFSuly8BrED99yEiwM PMEm5Rtmw3ps9dHZfGsBbpgS2LfDLS56mXBq5MdfYYSYm+P9wdWf/Y7tev+/aLWXDYVNtS jcmtIN23nQi0uSK9jjQqgURX1VcagfEDEijSX2WJoXLIQJRTObunM/ul3ZrOfw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1653559413; a=rsa-sha256; cv=none; b=rEFsreVJ9jadUulZHyDm1ULgyzxfY7fMtNTXVGAZa+sXzPZYYaDMhZmGAncP/GZzEAwJAq lYaiRPgTPiTVK36nG3HcqfsMiTMkOcuL1iSodorSA8oBheRlQtof9pdIPSFFpZTze8QWGo dHOTrP/1M0jEbmDMe7I9aOdURmZF4S8TjoavVhVzOSJxmxohL7rBOMm5UgbxSSFnI381cP jPxl60Gj8mnltB12Fmf3PB6Ngjiq8hufA9PzMVqfuWMzwCpPumusLTq53JjfO3IYReO05q EqOemvEvrNjLyj3niC+j9HjqxfU1O3w3F5dMyI3v4gJH0kXx3/uhwxmGHFjtEw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=IIS51OWO; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -9.54 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=IIS51OWO; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 63A5C1791C X-Spam-Score: -9.54 X-Migadu-Scanner: scn0.migadu.com X-TUID: shguSei3kvyw Hi all, - tl; dr: I describe here my workflow with org-publish to work with long books. =E2=80=94 I discovered a long time ago that `org-publish' not only works very well for managing websites but also for working with long and complex books with many parts, with output to LaTeX/PDF. I developed a workflow around it that has been quite productive for me, and that I briefly describe here in case someone finds it useful and wants to try it or modify/adapt it to their needs. I usually use it for my typesetting work, but I think it can also be useful for personal projects, such as doctoral theses. First of all, each folder of my project-books has the same structure: two subdirectories named `/org' and `/tex', for the source `org' files and for the output `.tex' documents, respectively. And, inside the `org' directory I include a `setup' file, an `elisp' file (for export filters), and another `/img' directory for image files. Each `org' file is a part of the book, and usually begins simply with the directives: =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80 =E2=94=82 #+SETUPFILE: xxx.setup =E2=94=82 #+INCLUDE: "elisp" =E2=94=94=E2=94=80=E2=94=80=E2=94=80=E2=94=80 `Org-publish' exports the subdocuments (body only!) as `.tex' documents in the `/tex' folder, but they are not compiled. What gets compiled is a master `.org' file, which is also inside the `org' folder. I compile this master file using an asynchronous function that calls `latexmk'. I put all the LaTeX configuration, the packages to load, the (re)defined commands and macros, the necessary Lua code, etc. in a `.sty' file that I load at the very beginning of the master document. Subdocuments are loaded into this file by the LaTeX command (\input), not by the org #+INCLUDE directive. So the master file usually looks like this: =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80 =E2=94=82 #+LaTeX_CLASS: my-custom-latex-class =E2=94=82 #+LaTeX_Header: \input{my-custom-conf.sty} =E2=94=82 #+SETUPFILE: xxxx.setup =E2=94=82 #+INCLUDE: "elisp" =E2=94=82=20 =E2=94=82 * Part 1 =E2=94=82 ** Chapter 1 =E2=94=82 #+LaTeX: \input{chapter1.tex} =E2=94=94=E2=94=80=E2=94=80=E2=94=80=E2=94=80 When I eval my function, `latexmk' compiles the entire book with the `-pvc' option, which keeps the session open, and if it detects any changes to the entire document, it recompiles and refresh the pdf view. For example, if I edit one of the subdocuments and run `org-publish-current-file', everything is automatically recompiled. When I have the project folder ready, I add this to `org-publish-project-alist' (this is an example from one of the books I did recently): =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80 =E2=94=82 ("cartas-org" =E2=94=82 :base-directory "~/Git/cartas/libro/org/" =E2=94=82 :base-extension "org" =E2=94=82 ;; Directorio para los ficheros *.tex =E2=94=82 :publishing-directory "~/Git/cartas/libro/tex/" =E2=94=82 :publishing-function org-latex-publish-to-latex =E2=94=82 :body-only t ;; this is important! =E2=94=82 :exclude "cartas-master\\.org\\|bibliografia-cartas\\.org" =E2=94=82 :recursive t) =E2=94=82=20 =E2=94=82 ("cartas-img" =E2=94=82 :base-directory "~/Git/cartas/libro/org/img/" =E2=94=82 :base-extension "jpg\\|png" =E2=94=82 :publishing-directory "~/Git/cartas/libro/tex/img/" =E2=94=82 :recursive t =E2=94=82 :publishing-function org-publish-attachment) =E2=94=82=20 =E2=94=82 ("cartas" :components ("cartas-tex" "cartas-img")) =E2=94=94=E2=94=80=E2=94=80=E2=94=80=E2=94=80 And finally, this is the function that compiles everything (each project usually has some local variables, like the value of =E2=80=99jobname=E2=80= =99 or the status of the printing proofs). Nota Bene: The reason for using async is that in some projects, especially bilingual editions, I need to pre-compile some files first. Under normal conditions I don't think it's necessary to use async, since org-publish just exports everything to .tex documents (short timeout) and then start-process-shell-command run latexmk asynchronously. Best regards, Juan Manuel=20 =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80 =E2=94=82 (require 'async) =E2=94=82 (require 'projectile) =E2=94=82=20 =E2=94=82 (defun latexmk-compile-project-async () =E2=94=82 (interactive) =E2=94=82 (let* =E2=94=82 ((project-root (projectile-project-root)) =E2=94=82 (master-file (read-file-name "Compile: " =E2=94=82 (concat project-root "libro/org/"))) =E2=94=82 (master-file-tex (file-name-sans-extension =E2=94=82 (expand-file-name master-file))) =E2=94=82 (dir-tex (file-name-directory =E2=94=82 (expand-file-name =E2=94=82 (replace-regexp-in-string "/org/" "/tex/" master-file))))) =E2=94=82 ;; save the master document =E2=94=82 (with-current-buffer =E2=94=82 (find-file-noselect master-file) =E2=94=82 (save-buffer)) =E2=94=82 (async-start =E2=94=82 (lambda () =E2=94=82 (load "~/.emacs") =E2=94=82 (with-current-buffer =E2=94=82 (find-file-noselect master-file) =E2=94=82 (org-show-all) =E2=94=82 (save-buffer) =E2=94=82 (org-latex-export-to-latex nil nil nil nil nil)) =E2=94=82 ;; remove all old auxiliary files before compiling =E2=94=82 (shell-command (concat "rm -r " dir-tex (file-name-base ma= ster-file-tex) "*")) =E2=94=82 (shell-command (concat "mv " master-file-tex ".tex" " " di= r-tex)) =E2=94=82 "Document exported") =E2=94=82 (lambda (resultado) =E2=94=82 (message resultado) =E2=94=82 (let =E2=94=82 ((default-directory dir-tex) =E2=94=82 (jobname (if (and jobname-local printing-proofs-state) =E2=94=82 (concat jobname-local "_" printing-proofs-state "_" =E2=94=82 (format-time-string "%d-%m-%y")) =E2=94=82 (concat (file-name-sans-extension =E2=94=82 (file-name-nondirectory master-file-tex)) =E2=94=82 "_" =E2=94=82 (format-time-string "%d-%m-%y"))))) =E2=94=82 (start-process-shell-command =E2=94=82 "project" =E2=94=82 "*project*" =E2=94=82 (concat =E2=94=82 "latexmk" =E2=94=82 " -jobname=3D" =E2=94=82 jobname =E2=94=82 " -pvc -lualatex -e '$lualatex=3Dq/lualatex %O -shell-escape = %S/' " =E2=94=82 (file-name-nondirectory master-file-tex) =E2=94=82 ".tex"))))))) =E2=94=94=E2=94=80=E2=94=80=E2=94=80=E2=94=80