From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id KCdQOMB6j2IjVwAAbAwnHQ (envelope-from ) for ; Thu, 26 May 2022 15:04:01 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id iLhQN8B6j2JVJAEAG6o9tA (envelope-from ) for ; Thu, 26 May 2022 15:04:00 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 495F2E5EC for ; Thu, 26 May 2022 15:04:00 +0200 (CEST) Received: from localhost ([::1]:58542 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nuD9i-0000RH-Iw for larch@yhetil.org; Thu, 26 May 2022 09:03:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52306) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nuCsz-00034a-Gy for emacs-orgmode@gnu.org; Thu, 26 May 2022 08:46:42 -0400 Received: from mailer-211-145.hitrost.net ([91.185.211.145]:32535) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nuCsw-0007tb-HL for emacs-orgmode@gnu.org; Thu, 26 May 2022 08:46:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=christianmoe.com; s=x; h=Content-Transfer-Encoding:Content-Type: MIME-Version:Message-ID:Date:In-reply-to:Subject:To:From:References:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=Uq/1PZnQoNKBHpYqwTEyIxHtn1x0I5MrIAVbET1Y/Wk=; b=Ns1+h6Lr70iIsZ1cCxO5rD2A7L 8GMDNPMOh3aYPAInvvJc8EEZAMb4AwOK+T2D1W/ss1rq8CKl9JQODFUd9V2e9abchHKXYvnS1+LHx 4iB5398j+2dHzHP1eWSuhgBkgBE3ccsv9/b2qp8hgd/1CjOGBU0t4tZxav7Gd/ipCeApqhwfFmSYw TGFiPAfa9D4vqZhnXKbu4Ik4EFT9a7CAtYxFR9/sliIZvGgAD7WVEC72gk+qr0AHCOJ/zaGMrrme8 R/e98K6uNzur9HiTtesKPGtLeVXHNw2I35nu5ZRoPEE6yv995p+G+bqa7nMhoiTDKfU3nkC2EaObv O8MiOPKQ==; Received: from 84-20-244-215.dynamic.telemach.net ([84.20.244.215] helo=Tauriel) by b1.hitrost.net with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1nuCsq-00CKZK-UV; Thu, 26 May 2022 14:46:32 +0200 References: <87y1yovcip.fsf@posteo.net> User-agent: mu4e 1.2.0; emacs 26.3 From: Christian Moe To: emacs-orgmode@gnu.org Subject: Re: [tip] org-publish to work with (very) large books In-reply-to: <87y1yovcip.fsf@posteo.net> Date: Thu, 26 May 2022 14:46:32 +0200 Message-ID: <87a6b4pil3.fsf@christianmoe.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-GeoIP: Country [IP], SI [84.20.244.215] X-Antivirus-Scanner: Clean mail though you should still use an Antivirus X-Authenticated-Id: mail@christianmoe.com Received-SPF: pass client-ip=91.185.211.145; envelope-from=mail@christianmoe.com; helo=mailer-211-145.hitrost.net X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1653570240; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=Uq/1PZnQoNKBHpYqwTEyIxHtn1x0I5MrIAVbET1Y/Wk=; b=hfJzNwontbO+/bMIemFYFe6//Euiv+YRLHdb01R5lzAcUpZjlG4/mLBvrcqANhxM+xyAd1 XXALXsD6HpMpJKwWJVWENri3ksMKROVlBPeFZ8qKNwsHjUy0ZfW8jdRnqboU6vNX2eN3rE 7Fx7oAt1hVGC+f2CPUvwH6xx117ItHINXsg+0ufN+NvPvl9E1xW/5gpFxT6+5bI7z2aD64 s3/JiGsxOXMIyO8Gj1n80GK/HTIZCOIadQ3xX4KpmVSph/gWyFjNPg/xFq4kTCX9cnMVy4 kStjVk8EWsSoR9lDyM92imDj1gVT2WjZoZtQYLX1U2ybdpEJtGH2CP8LSFJ7ZA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1653570240; a=rsa-sha256; cv=none; b=Yk2E2gRyvWEpCDQfgEh0Hh/JeCNMZAuWAG5T6bNbFFwhC1cARY82To5SgULW3pzXaNJscU 4IM9eDdY4jgOPZIYuubngT+rTDqeX6YJOLGbGdxUAPJKrm8YBFvsKJriG/JdKZ7L3ezb90 wPEA4C++8Q7gOzabo3J2i49ydyIa4+r7iXYYT725sOnZxT2uM8J8qOJfl7+np0kNoc/MOl srLQjV7+rYbO3u9r3ly4qIvq4NgZyX5KbawYGAunJqCXpBWbM/NeSBhzZGlvYvNbXnzHWc QrYEONKo/68wfNiMY0UTGxyifnJ/umBkHJFpkBxpi/+KyZVXPal7TG2drI2PCw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none ("invalid DKIM record") header.d=christianmoe.com header.s=x header.b=Ns1+h6Lr; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -3.44 Authentication-Results: aspmx1.migadu.com; dkim=none ("invalid DKIM record") header.d=christianmoe.com header.s=x header.b=Ns1+h6Lr; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 495F2E5EC X-Spam-Score: -3.44 X-Migadu-Scanner: scn0.migadu.com X-TUID: nQlz+Ed/NgUf Thanks for this, really interesting. Do I understand correctly that the main advantage of this approach (over #+INCLUDE) is the ability to continuously update preview of the whole book with latexmk -pvc even if you only re-export one chapter from Org-mode? I couldn't find the :body-only publishing option in the docs ...? Yours, Christian Juan Manuel Mac=C3=ADas writes: > Hi all, > > - tl; dr: I describe here my workflow with org-publish to work with long > books. > > =E2=80=94 > > I discovered a long time ago that `org-publish' not only works very well > for managing websites but also for working with long and complex books > with many parts, with output to LaTeX/PDF. I developed a workflow around > it that has been quite productive for me, and that I briefly describe > here in case someone finds it useful and wants to try it or modify/adapt > it to their needs. I usually use it for my typesetting work, but I think > it can also be useful for personal projects, such as doctoral theses. > > First of all, each folder of my project-books has the same structure: > two subdirectories named `/org' and `/tex', for the source `org' files > and for the output `.tex' documents, respectively. And, inside the `org' > directory I include a `setup' file, an `elisp' file (for export > filters), and another `/img' directory for image files. Each `org' file > is a part of the book, and usually begins simply with the directives: > > =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80 > =E2=94=82 #+SETUPFILE: xxx.setup > =E2=94=82 #+INCLUDE: "elisp" > =E2=94=94=E2=94=80=E2=94=80=E2=94=80=E2=94=80 > > `Org-publish' exports the subdocuments (body only!) as `.tex' documents > in the `/tex' folder, but they are not compiled. > > What gets compiled is a master `.org' file, which is also inside the > `org' folder. I compile this master file using an asynchronous function > that calls `latexmk'. I put all the LaTeX configuration, the packages to > load, the (re)defined commands and macros, the necessary Lua code, etc. > in a `.sty' file that I load at the very beginning of the master > document. Subdocuments are loaded into this file by the LaTeX command > (\input), not by the org #+INCLUDE directive. So the master file usually > looks like this: > > =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80 > =E2=94=82 #+LaTeX_CLASS: my-custom-latex-class > =E2=94=82 #+LaTeX_Header: \input{my-custom-conf.sty} > =E2=94=82 #+SETUPFILE: xxxx.setup > =E2=94=82 #+INCLUDE: "elisp" > =E2=94=82 > =E2=94=82 * Part 1 > =E2=94=82 ** Chapter 1 > =E2=94=82 #+LaTeX: \input{chapter1.tex} > =E2=94=94=E2=94=80=E2=94=80=E2=94=80=E2=94=80 > > When I eval my function, `latexmk' compiles the entire book with the > `-pvc' option, which keeps the session open, and if it detects any > changes to the entire document, it recompiles and refresh the pdf view. > For example, if I edit one of the subdocuments and run > `org-publish-current-file', everything is automatically recompiled. > > When I have the project folder ready, I add this to > `org-publish-project-alist' (this is an example from one of the books I > did recently): > > =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80 > =E2=94=82 ("cartas-org" > =E2=94=82 :base-directory "~/Git/cartas/libro/org/" > =E2=94=82 :base-extension "org" > =E2=94=82 ;; Directorio para los ficheros *.tex > =E2=94=82 :publishing-directory "~/Git/cartas/libro/tex/" > =E2=94=82 :publishing-function org-latex-publish-to-latex > =E2=94=82 :body-only t ;; this is important! > =E2=94=82 :exclude "cartas-master\\.org\\|bibliografia-cartas\\.org" > =E2=94=82 :recursive t) > =E2=94=82 > =E2=94=82 ("cartas-img" > =E2=94=82 :base-directory "~/Git/cartas/libro/org/img/" > =E2=94=82 :base-extension "jpg\\|png" > =E2=94=82 :publishing-directory "~/Git/cartas/libro/tex/img/" > =E2=94=82 :recursive t > =E2=94=82 :publishing-function org-publish-attachment) > =E2=94=82 > =E2=94=82 ("cartas" :components ("cartas-tex" "cartas-img")) > =E2=94=94=E2=94=80=E2=94=80=E2=94=80=E2=94=80 > > And finally, this is the function that compiles everything (each project > usually has some local variables, like the value of =E2=80=99jobname=E2= =80=99 or the > status of the printing proofs). > > Nota Bene: The reason for using async is that in some projects, > especially bilingual editions, I need to pre-compile some files first. > Under normal conditions I don't think it's necessary to use async, since > org-publish just exports everything to .tex documents (short timeout) > and then start-process-shell-command run latexmk asynchronously. > > Best regards, > > Juan Manuel > > =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80 > =E2=94=82 (require 'async) > =E2=94=82 (require 'projectile) > =E2=94=82 > =E2=94=82 (defun latexmk-compile-project-async () > =E2=94=82 (interactive) > =E2=94=82 (let* > =E2=94=82 ((project-root (projectile-project-root)) > =E2=94=82 (master-file (read-file-name "Compile: " > =E2=94=82 (concat project-root "libro/org/"))) > =E2=94=82 (master-file-tex (file-name-sans-extension > =E2=94=82 (expand-file-name master-file))) > =E2=94=82 (dir-tex (file-name-directory > =E2=94=82 (expand-file-name > =E2=94=82 (replace-regexp-in-string "/org/" "/tex/" master-file))))) > =E2=94=82 ;; save the master document > =E2=94=82 (with-current-buffer > =E2=94=82 (find-file-noselect master-file) > =E2=94=82 (save-buffer)) > =E2=94=82 (async-start > =E2=94=82 (lambda () > =E2=94=82 (load "~/.emacs") > =E2=94=82 (with-current-buffer > =E2=94=82 (find-file-noselect master-file) > =E2=94=82 (org-show-all) > =E2=94=82 (save-buffer) > =E2=94=82 (org-latex-export-to-latex nil nil nil nil nil)) > =E2=94=82 ;; remove all old auxiliary files before compiling > =E2=94=82 (shell-command (concat "rm -r " dir-tex (file-name-base = master-file-tex) "*")) > =E2=94=82 (shell-command (concat "mv " master-file-tex ".tex" " " = dir-tex)) > =E2=94=82 "Document exported") > =E2=94=82 (lambda (resultado) > =E2=94=82 (message resultado) > =E2=94=82 (let > =E2=94=82 ((default-directory dir-tex) > =E2=94=82 (jobname (if (and jobname-local printing-proofs-state) > =E2=94=82 (concat jobname-local "_" printing-proofs-state "_" > =E2=94=82 (format-time-string "%d-%m-%y")) > =E2=94=82 (concat (file-name-sans-extension > =E2=94=82 (file-name-nondirectory master-file-tex)) > =E2=94=82 "_" > =E2=94=82 (format-time-string "%d-%m-%y"))))) > =E2=94=82 (start-process-shell-command > =E2=94=82 "project" > =E2=94=82 "*project*" > =E2=94=82 (concat > =E2=94=82 "latexmk" > =E2=94=82 " -jobname=3D" > =E2=94=82 jobname > =E2=94=82 " -pvc -lualatex -e '$lualatex=3Dq/lualatex %O -shell-escap= e %S/' " > =E2=94=82 (file-name-nondirectory master-file-tex) > =E2=94=82 ".tex"))))))) > =E2=94=94=E2=94=80=E2=94=80=E2=94=80=E2=94=80