[RFC] [PATCH] conditional use of latex packages

[RFC] [PATCH] conditional use of latex packages

[Aaron Ecay]

Hello,

The current way that org handles LaTeX packages for export isn’t
optimal.  The org-latex-(default-)packages-alist variables define a set
of packages that are loaded always.  If a user wants to use advanced
functionality (booktabs for nicer table export, listings or minted for
nicer source code), s/he has to add the packages to these variables
manually.  And a package like longtables is imported into every
document, slowing down compilation even when it is not used.

The attached patches (specifically 1, 2, and 5) introduce a mechanism to
load certain packages only when needed.  It is possible to customize
these packages by specifying options to be passed to their \usepackage
(only inserted if needed to properly export the document), as well as
arbitrary code to place in the document’s preamble if the package is
used.

The other patches in the series (3, 4) fix the latex exporter’s handling
of tikz image files, as generated by R’s tikzDevice function.
Currently, a link to the file containing the graphics code is inserted.
The proper behavior is to \input the file; the source code therein is
compiled into a graph by LaTeX as it compiles the document.  (Tikz is a
very expensive latex package to load; the ability to load tikz only when
necessary motivated the optional packages mechanism.)

I think these patches need more testing, but I wanted to send them along
for feedback.  If it is not desired to change the status quo
wrt. packages like booktabs and minted (must be manually added), and
wrapfig and longtable (will always be used even if not needed), it would
be possible to accept only patches 1, 3, and 4.  (But obviously I think
the other patches are a marked improvement.)

With luck, the patches will be sent as replies to this message with git
send-email.

Thanks,
Aaron

PS I’ve sent away for the copyright assignment forms from the FSF...

[PATCH 1/5] ox-latex: add optional-packages machinery

[Aaron Ecay]

This code allows latex packages to be inserted into the output document
only if they are needed.  The function ‘org-latex--use-package’ is
provided for code to signal that it wants a package inserted into the
output.  The ‘org-latex-optional-packages-options-alist’ variable allows
optionally loaded packages to be customized.  It contains slots for
options to be passed to the \usepackage line, as well as arbitrary code
to be inserted into the preamble.

Code to use this mechanism is in followup patches.
---
 lisp/ox-latex.el | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 54 insertions(+), 3 deletions(-)

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 9d5b5c5..9895028 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -418,6 +418,22 @@ toc:nil option, not to those generated with #+TOC keyword."
   :group 'org-export-latex
   :type 'boolean)
 
+(defcustom org-latex-optional-package-options-alist nil
+  "An alist of options for packages which will be loaded if needed.
+
+The format is a list of triples of (PACKAGE-NAME PACKAGE-OPTIONS
+PREAMBLE-CODE).  The PACKAGE-OPTIONS will be passed as options to
+the \\usepackage command.  The PREAMBLE-CODE will be inserted in
+the preamble of the resulting document (before any LaTeX_HEADER
+lines).
+
+Either of the values may be omitted (nil)."
+  :group 'org-export-latex
+  :type '(repeat
+	  (list
+	   (string :tag "Package name")
+	   (string :tag "Package options")
+	   (string :tag "Preamble code"))))
 
 ;;;; Headline
 
@@ -1087,6 +1103,11 @@ just outside of it."
      (funcall search-refs element))
    ""))
 
+(defun org-latex--use-package (info package)
+  (let ((optional-packages (plist-get info :latex-optional-packages)))
+    (plist-put info :latex-optional-packages
+	       (add-to-list 'optional-packages package))))
+
 
 \f
 ;;; Template
@@ -1110,7 +1131,8 @@ holding export options."
 		     (if (not class-options) header
 		       (replace-regexp-in-string
 			"^[ \t]*\\\\documentclass\\(\\(\\[.*\\]\\)?\\)"
-			class-options header t nil 1)))))
+			class-options header t nil 1))))
+	       (optional-packages (plist-get info :latex-optional-packages)))
 	  (if (not document-class-string)
 	      (user-error "Unknown LaTeX class `%s'" class)
 	    (org-latex-guess-babel-language
@@ -1118,8 +1140,13 @@ holding export options."
 	      (org-splice-latex-header
 	       document-class-string
 	       org-latex-default-packages-alist
-	       org-latex-packages-alist nil
-	       (plist-get info :latex-header-extra)))
+	       (org-latex-add-optional-packages
+		optional-packages
+		org-latex-packages-alist)
+	       nil
+	       (org-latex-add-optional-package-preambles
+		optional-packages
+		(plist-get info :latex-header-extra))))
 	     info)))))
      ;; Possibly limit depth for headline numbering.
      (let ((sec-num (plist-get info :section-numbers)))
@@ -1183,6 +1210,30 @@ holding export options."
      ;; Document end.
      "\\end{document}")))
 
+(defun org-latex-add-optional-packages (optional-packages packages)
+  (if (not optional-packages)
+      packages
+    (append packages
+	    (mapcar
+	     (lambda (pkg)
+	       (list (or (cadr (assoc pkg org-latex-optional-package-options-alist)) "")
+		     pkg
+		     nil))
+	     optional-packages))))
+
+(defun org-latex-add-optional-package-preambles (optional-packages preamble)
+  (if (not optional-packages)
+      preamble
+    (concat
+     (mapconcat
+      #'identity
+      (delq nil
+	    (mapcar (lambda (pkg)
+		      (nth 2 (assoc pkg org-latex-optional-package-options-alist)))
+		    optional-packages))
+      "\n")
+     "\n" preamble)))
+
 
 \f
 ;;; Transcode Functions
-- 
1.8.1.4

[PATCH 2/5] ox-latex: convert source code and table export to use optional packages

[Aaron Ecay]

This eliminates the unconditional use of longtable in
‘org-latex-default-packages-alist,’ and also allows the use of listings,
minted, and booktabs packages to be simplified (users no longer need to
add them to ‘org-latex-packages-alist’; they are automatically imported
if needed).
---
 lisp/org.el      |  1 -
 lisp/ox-latex.el | 47 +++++++++++++++++++----------------------------
 2 files changed, 19 insertions(+), 29 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 1d83aa4..d3506d1 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -3753,7 +3753,6 @@ header, or they will be appended."
     ("T1"   "fontenc"   t)
     (""     "fixltx2e"  nil)
     (""     "graphicx"  t)
-    (""     "longtable" nil)
     (""     "float"     nil)
     (""     "wrapfig"   nil)
     (""     "soul"      t)
diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 9895028..100de1d 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -569,10 +569,8 @@ When modifying this variable, it may be useful to change
 
 (defcustom org-latex-tables-booktabs nil
   "When non-nil, display tables in a formal \"booktabs\" style.
-This option assumes that the \"booktabs\" package is properly
-loaded in the header of the document.  This value can be ignored
-locally with \":booktabs t\" and \":booktabs nil\" LaTeX
-attributes."
+This value can be ignored locally with \":booktabs t\" and
+\":booktabs nil\" LaTeX attributes."
   :group 'org-export-latex
   :type 'boolean)
 
@@ -688,29 +686,9 @@ in order to mimic default behaviour:
 
 (defcustom org-latex-listings nil
   "Non-nil means export source code using the listings package.
-This package will fontify source code, possibly even with color.
-If you want to use this, you also need to make LaTeX use the
-listings package, and if you want to have color, the color
-package.  Just add these to `org-latex-packages-alist', for
-example using customize, or with something like:
-
-  \(require 'ox-latex)
-  \(add-to-list 'org-latex-packages-alist '\(\"\" \"listings\"))
-  \(add-to-list 'org-latex-packages-alist '\(\"\" \"color\"))
-
-Alternatively,
-
-  \(setq org-latex-listings 'minted)
-
-causes source code to be exported using the minted package as
-opposed to listings.  If you want to use minted, you need to add
-the minted package to `org-latex-packages-alist', for example
-using customize, or with
-
-  \(require 'ox-latex)
-  \(add-to-list 'org-latex-packages-alist '\(\"\" \"minted\"))
-
-In addition, it is necessary to install pygments
+This package will fontify source code, possibly even with color,
+using the minted or listings LaTeX packages.  If you want to use
+minted, it is necessary to install pygments
 \(http://pygments.org), and to configure the variable
 `org-latex-pdf-process' so that the -shell-escape option is
 passed to pdflatex."
@@ -1621,6 +1599,7 @@ contextual information."
       (concat "\\verb" separator code separator))
      ;; Use minted package.
      ((eq org-latex-listings 'minted)
+      (org-latex--use-package info "minted")
       (let* ((org-lang (org-element-property :language inline-src-block))
 	     (mint-lang (or (cadr (assq (intern org-lang)
 					org-latex-minted-langs))
@@ -1633,6 +1612,8 @@ contextual information."
 		separator code separator)))
      ;; Use listings package.
      (t
+      (org-latex--use-package info "listings")
+      (org-latex--use-package info "color")
       ;; Maybe translate language's name.
       (let* ((org-lang (org-element-property :language inline-src-block))
 	     (lst-lang (or (cadr (assq (intern org-lang)
@@ -2169,6 +2150,7 @@ contextual information."
 			   custom-env))
        ;; Case 3.  Use minted package.
        ((eq org-latex-listings 'minted)
+	(org-latex--use-package info "minted")
 	(let ((float-env
 	       (when (or label caption)
 		 (format "\\begin{listing}[H]\n%%s\n%s\\end{listing}"
@@ -2208,6 +2190,8 @@ contextual information."
 	  (if float-env (format float-env body) body)))
        ;; Case 4.  Use listings package.
        (t
+	(org-latex--use-package info "listings")
+	(org-latex--use-package info "color")
 	(let ((lst-lang
 	       (or (cadr (assq (intern lang) org-latex-listings-langs)) lang))
 	      (caption-str
@@ -2436,7 +2420,14 @@ This function assumes TABLE has `org' as its `:type' property and
 	 (placement (or (plist-get attr :placement)
 			(format "[%s]" org-latex-default-figure-position)))
 	 (centerp (if (plist-member attr :center) (plist-get attr :center)
-		    org-latex-tables-centered)))
+		    org-latex-tables-centered))
+	 (booktabsp (if (plist-member attr :booktabs)
+			  (plist-get attr :booktabs)
+			org-latex-tables-booktabs)))
+    (when (equal "longtable" table-env)
+      (org-latex--use-package info "longtable"))
+    (when booktabsp
+      (org-latex--use-package info "booktabs"))
     ;; Prepare the final format string for the table.
     (cond
      ;; Longtable.
-- 
1.8.1.4

[PATCH 3/5] ob-R: change the file extension for tikz figures

[Aaron Ecay]

Because figures are included for export for a link, it is not desirable
to have R-generated tikz figures have the extension .tex.  Links to .tex
files should (and currently are) inserted as a hyperlink in the LaTeX
output.  Links to .tikz files should be \input so that they draw as
graphics.  (A following patch implements this functionality.)

This could also allow other backends (e.g. HTML) to compile .tikz files
to images for inclusion, while retaining links to .tex files as links.
---
 lisp/ob-R.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/ob-R.el b/lisp/ob-R.el
index 8db0853..9875f81 100644
--- a/lisp/ob-R.el
+++ b/lisp/ob-R.el
@@ -237,7 +237,7 @@ current code buffer."
 	 '((:bmp . "bmp")
 	   (:jpg . "jpeg")
 	   (:jpeg . "jpeg")
-	   (:tex . "tikz")
+	   (:tikz . "tikz")
 	   (:tiff . "tiff")
 	   (:png . "png")
 	   (:svg . "svg")
-- 
1.8.1.4

[PATCH 4/5] ox-latex: Treat tikz files as images

[Aaron Ecay]

Unlike other image types, they should be \input into the document –
LaTeX converts the tikz source code into the image as part of the
document’s compilation.
---
 lisp/ox-latex.el | 22 +++++++++++++---------
 1 file changed, 13 insertions(+), 9 deletions(-)

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 100de1d..8027ce7 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -505,7 +505,7 @@ order to reproduce the default set-up:
   :type 'string)
 
 (defcustom org-latex-inline-image-rules
-  '(("file" . "\\.\\(pdf\\|jpeg\\|jpg\\|png\\|ps\\|eps\\)\\'"))
+  '(("file" . "\\.\\(pdf\\|jpeg\\|jpg\\|png\\|ps\\|eps\\|tikz\\)\\'"))
   "Rules characterizing image files that can be inlined into LaTeX.
 
 A rule consists in an association whose key is the type of link
@@ -1797,6 +1797,7 @@ used as a communication channel."
 	 (path (let ((raw-path (org-element-property :path link)))
 		 (if (not (file-name-absolute-p raw-path)) raw-path
 		   (expand-file-name raw-path))))
+	 (filetype (file-name-extension path))
 	 (caption (org-latex--caption/label-string parent info))
 	 ;; Retrieve latex attributes from the element around.
 	 (attr (org-export-read-attribute :attr_latex parent))
@@ -1824,22 +1825,25 @@ used as a communication channel."
 			  ((org-string-nw-p opt) (format "[%s]" opt))
 			  ((eq float 'float) "[width=0.7\\textwidth]")
 			  ((eq float 'wrap) "[width=0.48\\textwidth]")
-			  (t "")))))
+			  (t ""))))
+	 (image-code (if (equal filetype "tikz")
+			 (format "\\input{%s}" path)
+		       (format "\\includegraphics%s{%s}" options path))))
     ;; Return proper string, depending on FLOAT.
     (case float
       (wrap (format "\\begin{wrapfigure}%s
 \\centering
-%s\\includegraphics%s{%s}
-%s\\end{wrapfigure}" placement comment-include options path caption))
+%s%s
+%s\\end{wrapfigure}" placement comment-include image-code caption))
       (multicolumn (format "\\begin{figure*}%s
 \\centering
-%s\\includegraphics%s{%s}
-%s\\end{figure*}" placement comment-include options path caption))
+%s%s
+%s\\end{figure*}" placement comment-include image-code caption))
       (figure (format "\\begin{figure}%s
 \\centering
-%s\\includegraphics%s{%s}
-%s\\end{figure}" placement comment-include options path caption))
-      (t (format "\\includegraphics%s{%s}" options path)))))
+%s%s
+%s\\end{figure}" placement comment-include image-code caption))
+      (t image-code))))
 
 (defun org-latex-link (link desc info)
   "Transcode a LINK object from Org to LaTeX.
-- 
1.8.1.4

[PATCH 5/5] ox-latex: Convert the image inclusion code to use optional packages

[Aaron Ecay]

Eliminate unconditional use of wrapfig and graphicx packages.  Also
allows loading the (large, expensive) tikz package only when tikz
graphics are present.
---
 lisp/org.el      | 2 --
 lisp/ox-latex.el | 5 +++++
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index d3506d1..4f10959 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -3752,9 +3752,7 @@ header, or they will be appended."
   '(("AUTO" "inputenc"  t)
     ("T1"   "fontenc"   t)
     (""     "fixltx2e"  nil)
-    (""     "graphicx"  t)
     (""     "float"     nil)
-    (""     "wrapfig"   nil)
     (""     "soul"      t)
     (""     "textcomp"  t)
     (""     "marvosym"  t)
diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 8027ce7..4889911 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -1829,6 +1829,11 @@ used as a communication channel."
 	 (image-code (if (equal filetype "tikz")
 			 (format "\\input{%s}" path)
 		       (format "\\includegraphics%s{%s}" options path))))
+    (if (equal filetype "tikz")
+	(org-latex--use-package info "tikz")
+      (org-latex--use-package info "graphicx"))
+    (when (eq float 'wrap)
+      (org-latex--use-package info "wrapfig"))
     ;; Return proper string, depending on FLOAT.
     (case float
       (wrap (format "\\begin{wrapfigure}%s
-- 
1.8.1.4

Re: [RFC] [PATCH] conditional use of latex packages

[Suvayu Ali]

Hello Aaron,

On Wed, Feb 20, 2013 at 11:02:21PM -0500, Aaron Ecay wrote:
> Hello,
> 
> The current way that org handles LaTeX packages for export isn’t
> optimal.  The org-latex-(default-)packages-alist variables define a set
> of packages that are loaded always.  If a user wants to use advanced
> functionality (booktabs for nicer table export, listings or minted for
> nicer source code), s/he has to add the packages to these variables
> manually.  And a package like longtables is imported into every
> document, slowing down compilation even when it is not used.

[...]

I fully sympathise with your intentions and will test the patches at the
earliest when I have some time (probably tomorrow).

Cheers,

-- 
Suvayu

Open source is the future. It sets us free.

Re: [RFC] [PATCH] conditional use of latex packages

[Nicolas Goaziou]

Hello,

Aaron Ecay <aaronecay@gmail.com> writes:

> The current way that org handles LaTeX packages for export isn’t
> optimal.  The org-latex-(default-)packages-alist variables define a set
> of packages that are loaded always.  If a user wants to use advanced
> functionality (booktabs for nicer table export, listings or minted for
> nicer source code), s/he has to add the packages to these variables
> manually.And a package like longtables is imported into every
> document, slowing down compilation even when it is not used.

I think you are misusing latex back-end configuration.

Obviously, if you need a package in every document you write, it should
go into `org-latex-packages-alist'.

If you need occasional packages, they should go into
`org-latex-classes'. Adding a new class is cheap. For example you can
create a class "article-with-tikz". It also allows to include arbitrary
code within the header.

> The attached patches (specifically 1, 2, and 5) introduce a mechanism to
> load certain packages only when needed.  It is possible to customize
> these packages by specifying options to be passed to their \usepackage
> (only inserted if needed to properly export the document), as well as
> arbitrary code to place in the document’s preamble if the package is
> used.

I don't want to take that route. Bad things can happen if you load
packages in the wrong order, or with wrong options. This is a can of
worms. That's why no package is ever loaded automatically.

Notwithstanding conditional package insertion, `org-latex-classes'
provides the same set of features.

> The other patches in the series (3, 4) fix the latex exporter’s handling
> of tikz image files, as generated by R’s tikzDevice function.
> Currently, a link to the file containing the graphics code is inserted.
> The proper behavior is to \input the file; the source code therein is
> compiled into a graph by LaTeX as it compiles the document.  (Tikz is a
> very expensive latex package to load; the ability to load tikz only when
> necessary motivated the optional packages mechanism.)

I don't mind that change. Would you mind providing it as a separate set
of patches?

Anyway, nothing can be applied before FSF registration is complete.

> I think these patches need more testing, but I wanted to send them along
> for feedback.  If it is not desired to change the status quo
> wrt. packages like booktabs and minted (must be manually added), and
> wrapfig and longtable (will always be used even if not needed), it would
> be possible to accept only patches 1, 3, and 4.  (But obviously I think
> the other patches are a marked improvement.)

I don't mind removing "longtable" from
`org-latex-default-packages-alist'. I think there's no reason for it to
be there.


Regards,

-- 
Nicolas Goaziou

Re: [RFC] [PATCH] conditional use of latex packages

[Aaron Ecay]

2013ko otsailak 21an, Nicolas Goaziou-ek idatzi zuen:
> Obviously, if you need a package in every document you write, it
> should go into `org-latex-packages-alist'.

I agree with this.

> 
> If you need occasional packages, they should go into
> `org-latex-classes'. Adding a new class is cheap. For example you can
> create a class "article-with-tikz". It also allows to include
> arbitrary code within the header.

But not with this.  It leads to combinatorial explosion: you need
article-with-tikz, article-with-biblatex,
article-with-tikz-and-biblatex, article-with-tikz-and-booktabs, ....

Apart from that, I think that the latex exporter should “know” that you
need the booktabs package to export a table with the booktabs option.
So, if I send someone an org document with a booktabs table, it will
export correctly without the recipient needing to fiddle with
‘org-latex-classes’ or ‘-packages-alist’.  And the same reasoning
applies to longtables, tikz graphics, etc.  Obviously this is not true
of arbitrarily complex LaTeX constructs, but longtables, booktabs and
tikz are all supported natively by org’s syntax.

> I don't want to take that route. Bad things can happen if you load
> packages in the wrong order, or with wrong options. This is a can of
> worms. That's why no package is ever loaded automatically.

That’s true.  I don’t think that this will replace ‘org-latex-classes’ –
for certain kinds of setup it will be necessary to explicitly spell out
the latex code.  But the hope is that the kind of packages that this is
targeted at (which implement formatting options, but don’t generally
muck with lower-level things like hyperref) ordering will not be an
issue.  That might be an optimistic assumption about the state of LaTeX
packages, though...

The patch currently does not insert the optional packages in a specific
order, but it would be possible to do so (using the
‘org-latex-optional-packages-options-alist’ variable to specify the
ordering).

> 
> I don't mind that change. Would you mind providing it as a separate
> set of patches?

If the consensus is that the optional package functionality is not
wanted, I can do so.

> I don't mind removing "longtable" from
> `org-latex-default-packages-alist'. I think there's no reason for it
> to be there.

Except that the latex exporter supports it, which is also why wrapfig is
in there, and several packages providing symbols for org-entities.  I
think that a logical goal is for ‘org-latex-default-packages-alist’ to
disappear (or be shrunk to only a couple of entries), and all packages
not explicitly requested by the user (in ‘org-latex-packages-alist’ or
in an ‘org-latex-classes’ entry) to be loaded only if actually needed by
the exporter.  (Whether or not this endpoint is reached depends, as you
point out, on whether the problem of package load order can be solved in
a satisfactory way.)

-- 
Aaron Ecay

Re: [RFC] [PATCH] conditional use of latex packages

[Nicolas Goaziou]

Aaron Ecay <aaronecay@gmail.com> writes:

> 2013ko otsailak 21an, Nicolas Goaziou-ek idatzi zuen:

>> If you need occasional packages, they should go into
>> `org-latex-classes'. Adding a new class is cheap. For example you can
>> create a class "article-with-tikz". It also allows to include
>> arbitrary code within the header.
>
> But not with this.  It leads to combinatorial explosion: you need
> article-with-tikz, article-with-biblatex,
> article-with-tikz-and-biblatex, article-with-tikz-and-booktabs, ....

I think you're nitpicking here. Tikz may be heavy to load (I don't know,
I use asymptote) but not booktabs. There's no serious reason to have
both "article-with-tikz" and "article-with-tikz-and-booktabs".

Anyway, the point of classes is not to focus on packages only, but on
whole headers. Thus, I suggest to define classes per type of document
you create. I'm quite sure you don't need 2^n templates, n being the
number of packages you use, for that.

And if you need a specific package for a specific document, there is
LATEX HEADER keyword.

> Apart from that, I think that the latex exporter should “know” that you
> need the booktabs package to export a table with the booktabs option.
> So, if I send someone an org document with a booktabs table, it will
> export correctly without the recipient needing to fiddle with
> ‘org-latex-classes’ or ‘-packages-alist’.  And the same reasoning
> applies to longtables, tikz graphics, etc.  Obviously this is not true
> of arbitrarily complex LaTeX constructs, but longtables, booktabs and
> tikz are all supported natively by org’s syntax.

Again, this goes against the rule "do not load a package automatically".

You're talking about optimizing LaTeX header (load a package only when
you need it). I think it's way out of Org's purpose. Is it really
important if one package is required even if it won't be used? Is it
worth adding another set of variables for that? I don't think so.

Don't get me wrong. There's nothing inherently wrong with your approach,
and it may even sound handy, but the truth is there's little benefit.

>> I don't mind that change. Would you mind providing it as a separate
>> set of patches?
>
> If the consensus is that the optional package functionality is not
> wanted, I can do so.

My point was that they provide distinct features, so they should be
included in different sets. But that's your call, really.

>> I don't mind removing "longtable" from
>> `org-latex-default-packages-alist'. I think there's no reason for it
>> to be there.
>
> Except that the latex exporter supports it, 

Technically, latex exporter supports every package. That doesn't mean
all of them should by included in `org-latex-default-packages-alist'.

> which is also why wrapfig is in there, and several packages providing
> symbols for org-entities.

wrapfig and entities are a different matter. You may need them without
even realizing it. So, if they were removed from the default list, that
would cripple user's experience.

On the other hand, you need longtable package only when you explicitly
write "longtable" somewhere (e.g. in an :environment property or in
`org-latex-default-table-environment'). The user knows what he's doing
in this case.

> I think that a logical goal is for ‘org-latex-default-packages-alist’
> to disappear (or be shrunk to only a couple of entries), and all
> packages not explicitly requested by the user (in
> ‘org-latex-packages-alist’ or in an ‘org-latex-classes’ entry) to be
> loaded only if actually needed by the exporter. (Whether or not this
> endpoint is reached depends, as you point out, on whether the problem
> of package load order can be solved in a satisfactory way.)

I assume this would not be made to get a header of 30 lines instead of
40, but to allow an user to use features without even thinking about the
packages they require. FWIW, I think that:

  1. it's impossible to guess every package required by an user, so he
     would ultimately have to mess with even more variables to get what
     he wants (I only suggest to tweak `org-latex-class' and, optionally
     to stuff package GCD in `org-latex-packages-alist').

  2. In general, it's a bad idea to hide LaTeX internals too much as it
     can become very hard to fix a problem when things happen in your
     back.

Your work on this is interesting, but it's a can of worms, really.


Regards,

-- 
Nicolas Goaziou

Re: [RFC] [PATCH] conditional use of latex packages

[Aaron Ecay]

Hi Nicolas,

Thank you for your comments on this patch.  I appreciate your concerns
about the ordering of packages – this was something I had not considered
fully.  I have reworked the implementation; I think it is now simpler
and more robust.  Instead of a new customization variable, I’ve added a
field to ‘org-latex-(default-)packages-alist’, which indicates whether a
package is to be loaded always, or only if needed.  This is very similar
to the mechanism that already exists to load only a subset of packages
when compiling latex snippets.  This flag is initially set to nil for
all packages, which loads them unconditionally – that is, the default
behavior is unchanged.  Users can set this flag to t for packages for
which they are sure that optional loading is appropriate.

2013ko otsailak 21an, Nicolas Goaziou-ek idatzi zuen:
[...]
> 
> My point was that they provide distinct features, so they should be
> included in different sets. But that's your call, really.

You are right about this – I could not see the forest for the trees.
I’ll send the tikz images patch separately.

>> Except that the latex exporter supports it,
> 
> Technically, latex exporter supports every package. That doesn't mean
> all of them should by included in `org-latex-default-packages-alist'.

What I meant was that the code for the latex exporter specifically
mentions those packages.  There is a very small set of latex packages
that are “blessed” in that way.

>   1. it's impossible to guess every package required by an user, so he
> would ultimately have to mess with even more variables to get what he
> wants (I only suggest to tweak `org-latex-class' and, optionally to
> stuff package GCD in `org-latex-packages-alist').

Hopefully the new version of the patch addresses this concern.

> 
>   2. In general, it's a bad idea to hide LaTeX internals too much as
> it can become very hard to fix a problem when things happen in your
> back.

Well, the LaTeX file which is the output of org’s exporter is always
available for debugging if something goes wrong.  The new patch
explicitly indicates when a package is omitted because org thinks it is
unnecessary, so hopefully it will be very obvious when something goes
wrong.  (Also, org omitting a package will only happen if the user
requests it explicitly.)

The patch should appear as a reply to this email.

-- 
Aaron Ecay

[PATCH] ox-latex: add optional-packages machinery

[Aaron Ecay]

This code allows latex packages to be inserted into the output
document only if they are needed.  The function
‘org-latex--record-package’ is provided for code to signal that it
wants a package inserted into the output.  The
‘org-latex-packages-(default-)alist’ variables are extended with an
additional field, which indicates whether each package is to be loaded
always, or only if requested by a call to ‘org-latex--record-package’.

By default, all packages are loaded unconditionally (matching the
present behavior).  The ‘minted’, ‘longtable’, ‘listings’, ‘color’,
‘graphicx’, ‘booktabs’ and ‘wrapfig’ latex packages are requested when
needed, so they may be set to load conditionally.
---
 lisp/org.el      | 88 +++++++++++++++++++++++++++++++++++---------------------
 lisp/ox-latex.el | 26 +++++++++++++++--
 2 files changed, 79 insertions(+), 35 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 1d83aa4..51be09c 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -3735,34 +3735,40 @@ header, or they will be appended."
 (defun org-set-packages-alist (var val)
   "Set the packages alist and make sure it has 3 elements per entry."
   (set var (mapcar (lambda (x)
-		     (if (and (consp x) (= (length x) 2))
-			 (list (car x) (nth 1 x) t)
+		     (if (listp x)
+			 (list (nth 0 x)
+			       (nth 1 x)
+			       (or (nth 2 x) t)
+			       (nth 3 x))
 		       x))
 		   val)))
 
 (defun org-get-packages-alist (var)
   "Get the packages alist and make sure it has 3 elements per entry."
   (mapcar (lambda (x)
-	    (if (and (consp x) (= (length x) 2))
-		(list (car x) (nth 1 x) t)
+	    (if (listp x)
+		(list (nth 0 x)
+		      (nth 1 x)
+		      (or (nth 2 x) t)
+		      (nth 3 x))
 	      x))
 	  (default-value var)))
 
 (defcustom org-latex-default-packages-alist
-  '(("AUTO" "inputenc"  t)
-    ("T1"   "fontenc"   t)
-    (""     "fixltx2e"  nil)
-    (""     "graphicx"  t)
-    (""     "longtable" nil)
-    (""     "float"     nil)
-    (""     "wrapfig"   nil)
-    (""     "soul"      t)
-    (""     "textcomp"  t)
-    (""     "marvosym"  t)
-    (""     "wasysym"   t)
-    (""     "latexsym"  t)
-    (""     "amssymb"   t)
-    (""     "hyperref"  nil)
+  '(("AUTO" "inputenc"  t   nil)
+    ("T1"   "fontenc"   t   nil)
+    (""     "fixltx2e"  nil nil)
+    (""     "graphicx"  t   nil)
+    (""     "longtable" nil nil)
+    (""     "float"     nil nil)
+    (""     "wrapfig"   nil nil)
+    (""     "soul"      t   nil)
+    (""     "textcomp"  t   nil)
+    (""     "marvosym"  t   nil)
+    (""     "wasysym"   t   nil)
+    (""     "latexsym"  t   nil)
+    (""     "amssymb"   t   nil)
+    (""     "hyperref"  nil nil)
     "\\tolerance=1000")
   "Alist of default packages to be inserted in the header.
 
@@ -3786,9 +3792,12 @@ Therefore you should not modify this variable unless you know
 what you are doing.  The one reason to change it anyway is that
 you might be loading some other package that conflicts with one
 of the default packages.  Each cell is of the format
-\( \"options\" \"package\" snippet-flag).  If SNIPPET-FLAG is t,
-the package also needs to be included when compiling LaTeX
-snippets into images for inclusion into non-LaTeX output."
+\(\"options\" \"package\" snippet-flag conditional-load).  If
+SNIPPET-FLAG is t, the package also needs to be included when
+compiling LaTeX snippets into images for inclusion into non-LaTeX
+output.  If CONDITIONAL-LOAD is t, the package will be loaded
+only if needed by the exporter (as signaled by calls to
+`org-latex--use-package'.)"
   :group 'org-latex
   :group 'org-export-latex
   :set 'org-set-packages-alist
@@ -3799,7 +3808,8 @@ snippets into images for inclusion into non-LaTeX output."
 	   (list :tag "options/package pair"
 		 (string :tag "options")
 		 (string :tag "package")
-		 (boolean :tag "Snippet"))
+		 (boolean :tag "Snippet")
+		 (boolean :tag "Conditional load"))
 	   (string :tag "A line of LaTeX"))))
 
 (defcustom org-latex-packages-alist nil
@@ -3808,11 +3818,13 @@ snippets into images for inclusion into non-LaTeX output."
 These will be inserted after `org-latex-default-packages-alist'.
 Each cell is of the format:
 
-    \(\"options\" \"package\" snippet-flag)
+    \(\"options\" \"package\" snippet-flag conditional-load)
 
 SNIPPET-FLAG, when t, indicates that this package is also needed
 when turning LaTeX snippets into images for inclusion into
-non-LaTeX output.
+non-LaTeX output.  If CONDITIONAL-LOAD is t, the package will be
+loaded only if needed by the exporter (as signaled by calls to
+`org-latex--use-package'.)
 
 Make sure that you only list packages here which:
 
@@ -3829,7 +3841,8 @@ Make sure that you only list packages here which:
 	   (list :tag "options/package pair"
 		 (string :tag "options")
 		 (string :tag "package")
-		 (boolean :tag "Snippet"))
+		 (boolean :tag "Snippet")
+		 (boolean :tag "Conditional load"))
 	   (string :tag "A line of LaTeX"))))
 
 (defgroup org-appearance nil
@@ -18246,7 +18259,7 @@ share a good deal of logic."
 		  (delete-file (concat texfilebase e))))
 	pngfile))))
 
-(defun org-splice-latex-header (tpl def-pkg pkg snippets-p &optional extra)
+(defun org-splice-latex-header (tpl def-pkg pkg snippets-p &optional extra optional-pkgs)
   "Fill a LaTeX header template TPL.
 In the template, the following place holders will be recognized:
 
@@ -18262,21 +18275,27 @@ holder is missing, the positive one (without the \"NO-\") will be
 assumed to be present at the end of the template.
 DEF-PKG and PKG are assumed to be alists of options/packagename lists.
 EXTRA is a string.
-SNIPPETS-P indicates if this is run to create snippet images for HTML."
+SNIPPETS-P indicates if this is run to create snippet images for HTML.
+OPTIONAL-PKGS is the list of packages that the exporter used for this
+document."
   (let (rpl (end ""))
     (if (string-match "^[ \t]*\\[\\(NO-\\)?DEFAULT-PACKAGES\\][ \t]*\n?" tpl)
 	(setq rpl (if (or (match-end 1) (not def-pkg))
-		      "" (org-latex-packages-to-string def-pkg snippets-p t))
+		      "" (org-latex-packages-to-string
+			  def-pkg snippets-p t optional-pkgs))
 	      tpl (replace-match rpl t t tpl))
-      (if def-pkg (setq end (org-latex-packages-to-string def-pkg snippets-p))))
+      (if def-pkg (setq end (org-latex-packages-to-string
+			     def-pkg snippets-p nil optional-pkgs))))
 
     (if (string-match "\\[\\(NO-\\)?PACKAGES\\][ \t]*\n?" tpl)
 	(setq rpl (if (or (match-end 1) (not pkg))
-		      "" (org-latex-packages-to-string pkg snippets-p t))
+		      "" (org-latex-packages-to-string
+			  pkg snippets-p t optional-pkgs))
 	      tpl (replace-match rpl t t tpl))
       (if pkg (setq end
 		    (concat end "\n"
-			    (org-latex-packages-to-string pkg snippets-p)))))
+			    (org-latex-packages-to-string
+			     pkg snippets-p nil optional-pkgs)))))
 
     (if (string-match "\\[\\(NO-\\)?EXTRA\\][ \t]*\n?" tpl)
 	(setq rpl (if (or (match-end 1) (not extra))
@@ -18289,13 +18308,16 @@ SNIPPETS-P indicates if this is run to create snippet images for HTML."
 	(concat tpl "\n" end)
       tpl)))
 
-(defun org-latex-packages-to-string (pkg &optional snippets-p newline)
+(defun org-latex-packages-to-string (pkg &optional snippets-p newline optional-pkgs)
   "Turn an alist of packages into a string with the \\usepackage macros."
   (setq pkg (mapconcat (lambda(p)
 			 (cond
 			  ((stringp p) p)
 			  ((and snippets-p (>= (length p) 3) (not (nth 2 p)))
-			   (format "%% Package %s omitted" (cadr p)))
+			   (format "%% Package %s omitted (for snippet)" (cadr p)))
+			  ((and (>= (length p) 4) (nth 3 p)
+				(not (member (cadr p) optional-pkgs)))
+			   (format "%% Package %s omitted (not needed)" (cadr p)))
 			  ((equal "" (car p))
 			   (format "\\usepackage{%s}" (cadr p)))
 			  (t
diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 9d5b5c5..0869a40 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -1087,6 +1087,11 @@ just outside of it."
      (funcall search-refs element))
    ""))
 
+(defun org-latex--record-package (info package)
+  (let ((optional-packages (plist-get info :latex-optional-packages)))
+    (plist-put info :latex-optional-packages
+	       (add-to-list 'optional-packages package))))
+
 
 \f
 ;;; Template
@@ -1119,7 +1124,8 @@ holding export options."
 	       document-class-string
 	       org-latex-default-packages-alist
 	       org-latex-packages-alist nil
-	       (plist-get info :latex-header-extra)))
+	       (plist-get info :latex-header-extra)
+	       (plist-get info :latex-optional-packages)))
 	     info)))))
      ;; Possibly limit depth for headline numbering.
      (let ((sec-num (plist-get info :section-numbers)))
@@ -1570,6 +1576,7 @@ contextual information."
       (concat "\\verb" separator code separator))
      ;; Use minted package.
      ((eq org-latex-listings 'minted)
+      (org-latex--record-package info "minted")
       (let* ((org-lang (org-element-property :language inline-src-block))
 	     (mint-lang (or (cadr (assq (intern org-lang)
 					org-latex-minted-langs))
@@ -1582,6 +1589,8 @@ contextual information."
 		separator code separator)))
      ;; Use listings package.
      (t
+      (org-latex--record-package info "listings")
+      (org-latex--record-package info "color")
       ;; Maybe translate language's name.
       (let* ((org-lang (org-element-property :language inline-src-block))
 	     (lst-lang (or (cadr (assq (intern org-lang)
@@ -1793,6 +1802,9 @@ used as a communication channel."
 			  ((eq float 'float) "[width=0.7\\textwidth]")
 			  ((eq float 'wrap) "[width=0.48\\textwidth]")
 			  (t "")))))
+    (org-latex--record-package info "graphicx")
+    (when (eq float 'wrap)
+      (org-latex--record-package info "wrapfig"))
     ;; Return proper string, depending on FLOAT.
     (case float
       (wrap (format "\\begin{wrapfigure}%s
@@ -2118,6 +2130,7 @@ contextual information."
 			   custom-env))
        ;; Case 3.  Use minted package.
        ((eq org-latex-listings 'minted)
+	(org-latex--record-package info "minted")
 	(let ((float-env
 	       (when (or label caption)
 		 (format "\\begin{listing}[H]\n%%s\n%s\\end{listing}"
@@ -2157,6 +2170,8 @@ contextual information."
 	  (if float-env (format float-env body) body)))
        ;; Case 4.  Use listings package.
        (t
+	(org-latex--record-package info "listings")
+	(org-latex--record-package info "color")
 	(let ((lst-lang
 	       (or (cadr (assq (intern lang) org-latex-listings-langs)) lang))
 	      (caption-str
@@ -2385,7 +2400,14 @@ This function assumes TABLE has `org' as its `:type' property and
 	 (placement (or (plist-get attr :placement)
 			(format "[%s]" org-latex-default-figure-position)))
 	 (centerp (if (plist-member attr :center) (plist-get attr :center)
-		    org-latex-tables-centered)))
+		    org-latex-tables-centered))
+	 (booktabsp (if (plist-member attr :booktabs)
+			  (plist-get attr :booktabs)
+			org-latex-tables-booktabs)))
+    (when (equal "longtable" table-env)
+      (org-latex--record-package info "longtable"))
+    (when booktabsp
+      (org-latex--record-package info "booktabs"))
     ;; Prepare the final format string for the table.
     (cond
      ;; Longtable.
-- 
1.8.1.4