[patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Juan Manuel Macías]

[-- Attachment #1: Type: text/plain, Size: 976 bytes --]

Rationale for this patch: in certain cases, in a LaTeX document, it is
necessary to add code before the class declaration (\documentclass...).
For example, commands like \PassOptionsToPackage, \DocumentMetadata{ },
etc.; or certain packages that should be loaded first using
\RequirePackage{}; and other miscellaneous cases[1]. I think that by
defining a new keyword `latex_pre_header', which behaves the same as
latex_header but concatenated before the class, these situations can be
resolved from Org.

[1] A longer example to export to a pdf that has pdf-x compliance, with
the pdfx package:

\providecommand{\pdfxopts}{x-1a}
\begin{filecontents*}{\jobname.xmpdata}
  \Title{Some Title}
  \Author{Author}
  \Language{es-ES}
  \Keywords{keywords}
  \Publisher{publisher}
\end{filecontents*}
\documentclass{...


-- 
Juan Manuel Macías

https://juanmanuelmacias.com

https://lunotipia.juanmanuelmacias.com

https://gnutas.juanmanuelmacias.com

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-lisp-ox-latex.el-latex-Add-LATEX_PRE_HEADER-keyword.patch --]
[-- Type: text/x-patch, Size: 1944 bytes --]

From ac6b743a4489f7bc8ab1cdae7ffd3b36e5f3c1d2 Mon Sep 17 00:00:00 2001
From: Juan Manuel Macias <maciaschain@posteo.net>
Date: Sat, 16 Sep 2023 19:22:39 +0200
Subject: [PATCH] lisp/ox-latex.el (latex): Add `LATEX_PRE_HEADER' keyword

* (org-latex-make-preamble): In some cases it is necessary to add code
before the `\documentclass' line. `LATEX_PRE_HEADER' behaves the same as
`LATEX_HEADER', except that it is concatenated before the class.
---
 lisp/ox-latex.el | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 14105c7cc..5e97b8b3d 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -124,6 +124,7 @@
     (:latex-class-options "LATEX_CLASS_OPTIONS" nil nil t)
     (:latex-header "LATEX_HEADER" nil nil newline)
     (:latex-header-extra "LATEX_HEADER_EXTRA" nil nil newline)
+    (:latex-pre-header "LATEX_PRE_HEADER" nil nil newline)
     (:description "DESCRIPTION" nil nil parse)
     (:keywords "KEYWORDS" nil nil parse)
     (:subtitle "SUBTITLE" nil nil parse)
@@ -1984,13 +1985,18 @@ specified in `org-latex-default-packages-alist' or
 		       (replace-regexp-in-string
 			"^[ \t]*\\\\documentclass\\(\\(\\[[^]]*\\]\\)?\\)"
 			class-options header t nil 1))))
-	      (user-error "Unknown LaTeX class `%s'" class))))
+	      (user-error "Unknown LaTeX class `%s'" class)))
+         (pre-header (mapconcat
+		      #'org-element-normalize-string
+		      (list (plist-get info :latex-pre-header) ""))))
     (org-latex-guess-polyglossia-language
      (org-latex-guess-babel-language
       (org-latex-guess-inputenc
        (org-element-normalize-string
 	(org-splice-latex-header
-	 class-template
+         (if pre-header
+	     (format "%s\n%s" pre-header class-template)
+	   class-template)
 	 (org-latex--remove-packages org-latex-default-packages-alist info)
 	 (org-latex--remove-packages org-latex-packages-alist info)
 	 snippet?
-- 
2.42.0

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Ihor Radchenko]

Juan Manuel Macías <maciaschain@posteo.net> writes:

> Rationale for this patch: in certain cases, in a LaTeX document, it is
> necessary to add code before the class declaration (\documentclass...).
> For example, commands like \PassOptionsToPackage, \DocumentMetadata{ },
> etc.; or certain packages that should be loaded first using
> \RequirePackage{}; and other miscellaneous cases[1]. I think that by
> defining a new keyword `latex_pre_header', which behaves the same as
> latex_header but concatenated before the class, these situations can be
> resolved from Org.

Thanks for the patch!
It looks reasonable.

> https://gnutas.juanmanuelmacias.com
> From ac6b743a4489f7bc8ab1cdae7ffd3b36e5f3c1d2 Mon Sep 17 00:00:00 2001
> From: Juan Manuel Macias <maciaschain@posteo.net>
> Date: Sat, 16 Sep 2023 19:22:39 +0200
> Subject: [PATCH] lisp/ox-latex.el (latex): Add `LATEX_PRE_HEADER' keyword
>
> * (org-latex-make-preamble): In some cases it is necessary to add code
> before the `\documentclass' line. `LATEX_PRE_HEADER' behaves the same as
> `LATEX_HEADER', except that it is concatenated before the class.

I think that you do not need to `quote' LATEX_PRE_HEADER - it is not a
lisp symbol.

Also, the new keyword should be documented in the manual.

> -	      (user-error "Unknown LaTeX class `%s'" class))))
> +	      (user-error "Unknown LaTeX class `%s'" class)))
> +         (pre-header (mapconcat
> +		      #'org-element-normalize-string
> +		      (list (plist-get info :latex-pre-header) ""))))

mapconcat is redundant here.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 1373 bytes --]

Hi Juan,

I’d rather not have this merged at this stage, as I think it may be redundant
once my conditional/generated export work makes its way into Org (next few
months?).

Juan Manuel Macías <maciaschain@posteo.net> writes:

> Rationale for this patch: in certain cases, in a LaTeX document, it is
> necessary to add code before the class declaration (\documentclass…).
> For example, commands like \PassOptionsToPackage, \DocumentMetadata{ },
> etc.; or certain packages that should be loaded first using
> \RequirePackage{}; and other miscellaneous cases[1]. I think that by
> defining a new keyword `latex_pre_header’, which behaves the same as
> latex_header but concatenated before the class, these situations can be
> resolved from Org.
>
> [1] A longer example to export to a pdf that has pdf-x compliance, with
> the pdfx package:
>
> \providecommand{\pdfxopts}{x-1a}
> \begin{filecontents*}{\jobname.xmpdata}
>   \Title{Some Title}
>   \Author{Author}
>   \Language{es-ES}
>   \Keywords{keywords}
>   \Publisher{publisher}
> \end{filecontents*}
> \documentclass{…

All the best,
Timothy

-- 
Timothy (‘tecosaur’/‘TEC’), Org mode contributor.
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/tec>.

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Ihor Radchenko]

Timothy <orgmode@tec.tecosaur.net> writes:

> I’d rather not have this merged at this stage, as I think it may be redundant
> once my conditional/generated export work makes its way into Org (next few
> months?).

Does it mean that conditional export will also support in-file keywords?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Max Nikulin]

On 17/09/2023 01:00, Juan Manuel Macías wrote:
> 
> [1] A longer example to export to a pdf that has pdf-x compliance, with
> the pdfx package:
> 
> \providecommand{\pdfxopts}{x-1a}
> \begin{filecontents*}{\jobname.xmpdata}
>    \Title{Some Title}
>    \Author{Author}
>    \Language{es-ES}
>    \Keywords{keywords}
>    \Publisher{publisher}
> \end{filecontents*}
> \documentclass{...

Perhaps you have in mind other cases, but I am unsure concerning this 
one. However I have tried only setting metadata for hyperref only, so I 
may miss some important aspects.

First of all, I was unaware of .xmpdata and, trying to get more info, I 
have found hyperxmp that declares to reuse data from \hypersetup without 
extra files and without the requirement to add settings before 
\documentclass.

Do you need to add commands before \documentclass because you are using 
custom classes that loads a lot of packages, so there is no chance to 
specify various options after \documentclass, but before specific 
packages are loaded?

I do not think the lines above should be used directly in 
LATEX_PRE_HEADER. They should be generated from #+AUTHOR:, #+TITLE:, 
#+LANGUAGE: keywords. Have you considered the option to generate 
.xmpdata files directly from Org instead of delegating the task to LaTeX?

Another idea is to try export filter to insert content of a source block 
to the beginning of the export buffer. Unsure if it is convenient.

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Juan Manuel Macías]

Max Nikulin writes:

> On 17/09/2023 01:00, Juan Manuel Macías wrote:
>> 
>> [1] A longer example to export to a pdf that has pdf-x compliance, with
>> the pdfx package:
>> 
>> \providecommand{\pdfxopts}{x-1a}
>> \begin{filecontents*}{\jobname.xmpdata}
>>    \Title{Some Title}
>>    \Author{Author}
>>    \Language{es-ES}
>>    \Keywords{keywords}
>>    \Publisher{publisher}
>> \end{filecontents*}
>> \documentclass{...
>
> Perhaps you have in mind other cases, but I am unsure concerning this 
> one. However I have tried only setting metadata for hyperref only, so I 
> may miss some important aspects.
>
> First of all, I was unaware of .xmpdata and, trying to get more info, I 
> have found hyperxmp that declares to reuse data from \hypersetup without 
> extra files and without the requirement to add settings before 
> \documentclass.

According to the pdfx package documentation
(https://www.ctan.org/pkg/pdfx), p. 6 (at the bottom of the page)/7:

---
Warning: The \jobname.xmpdata file may be included in the main document
source, within a {filecontents*} environment, provided this comes before
the \documentclass command, as follows[...]
---

> Do you need to add commands before \documentclass because you are using 
> custom classes that loads a lot of packages, so there is no chance to 
> specify various options after \documentclass, but before specific 
> packages are loaded?

Apart from the previous case, there are other varied cases (as I
mentioned) in which it is necessary to put code before documentclass.
For example, the newpax package, to preserve the internal links of a pdf
included in the document, requires putting commands like
\DocumentMetadata{ } before documentclass.

-- 
Juan Manuel Macías

https://juanmanuelmacias.com

https://lunotipia.juanmanuelmacias.com

https://gnutas.juanmanuelmacias.com

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Max Nikulin]

On 25/09/2023 01:42, Juan Manuel Macías wrote:
> According to the pdfx package documentation
> (https://www.ctan.org/pkg/pdfx), p. 6 (at the bottom of the page)/7:
> ---
> Warning: The \jobname.xmpdata file may be included in the main document
> source, within a {filecontents*} environment, provided this comes before
> the \documentclass command, as follows[...]
> ---

Does it mean that with hyperxmp you can not generate documents 
satisfying some requirements and you need namely pdfx? Is there a reason 
that .xmpdata files must be generated namely by a LaTeX engine and 
writing them by elisp code is unacceptable?

> For example, the newpax package, to preserve the internal links of a pdf
> included in the document, requires putting commands like
> \DocumentMetadata{ } before documentclass.

If I understand it correctly, \DocumentMetadata is a switch to enable 
"new" LaTeX like it was for \documentclass vs. \documentstyle during 
transition to LaTeX2e from LaTeX 2.09. If so, Org should have native 
support of \DocumentMetadata, not LATEX_PRE_HEADER or something similar. 
I have not looked into Timothy's branch, so I am unaware if it is addressed.

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Juan Manuel Macías]

Max Nikulin writes:

> On 25/09/2023 01:42, Juan Manuel Macías wrote:
>> According to the pdfx package documentation
>> (https://www.ctan.org/pkg/pdfx), p. 6 (at the bottom of the page)/7:
>> ---
>> Warning: The \jobname.xmpdata file may be included in the main document
>> source, within a {filecontents*} environment, provided this comes before
>> the \documentclass command, as follows[...]
>> ---
>
> Does it mean that with hyperxmp you can not generate documents
> satisfying some requirements and you need namely pdfx? Is there a
> reason that .xmpdata files must be generated namely by a LaTeX engine
> and writing them by elisp code is unacceptable?

In the hyperxmp manual a comparison is made with the pdfx package and
other similar packages. Anyway, this topic deviates drastically from the
original topic of the thread. There will be users who prefer to use
hyperxmp package and users who prefer pdfx. Hyperxmp is probably more
recommended, but I don't know either one deeply, so I couldn't say. In
any case: I think the central issue is to provide support for inserting
arbitrary code before \documentclass. The pdfx example is just one valid
example (ideally Org should not put obstacles to the use of any
package). As I said, there are more cases. For example, a few packages
(I don't remember their names) are recommended to be loaded first using
\RequirePackage. And there is also the case of the \PassOptionsToPackage
command. A good practice in LaTeX documents is usually to put this
command at the very beginning of the document, to avoid messages such as
"option class for xx package error". It is placed at the very beginning
because there may be classes that already require the package involved
in the error. And one more example: a new package for LuaLaTeX was
recently uploaded to CTAN, which allows the use of shebangs in LaTeX
documents. In short, there are many heterogeneous cases, and it is the
very heterogeneity of these cases that leads to a solution similar to
LaTeX_header, IMHO. Actually, any code (except \usepackage) would be
possible before \cocumentclass.

>> For example, the newpax package, to preserve the internal links of a pdf
>> included in the document, requires putting commands like
>> \DocumentMetadata{ } before documentclass.
>
> If I understand it correctly, \DocumentMetadata is a switch to enable
> "new" LaTeX like it was for \documentclass vs. \documentstyle during
> transition to LaTeX2e from LaTeX 2.09.

https://tex.stackexchange.com/questions/686898/what-is-documentmetadata-what-key-value-pairs-does-it-take-and-when-should-i-u

(As for the transition from LaTeX2e to LaTeX 3, it seems that it is not
going to be as defined as the transition we witnessed in the leap from
LaTeX 2.09. In reality it seems that there is not going to be such a
leap but rather a "coexistence" of both worlds:
https://www.latex-project.org/publications/2020-FMi-TUB-tb128mitt-quovadis.pdf)

> If so, Org should have native support of \DocumentMetadata, not
> LATEX_PRE_HEADER or something similar.

If it were the only case of code before \documentclass, I would agree.
But, as I have already said above, the diversity of use cases makes the
implementation of /ad hoc/ solutions unviable, in my opinion.

--
Juan Manuel Macías

https://juanmanuelmacias.com

https://lunotipia.juanmanuelmacias.com

https://gnutas.juanmanuelmacias.com

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Thomas S. Dye]

Aloha all,

Juan Manuel Macías <maciaschain@posteo.net> writes:
>
>> If so, Org should have native support of \DocumentMetadata, not
>> LATEX_PRE_HEADER or something similar.
>
> If it were the only case of code before \documentclass, I would 
> agree.
> But, as I have already said above, the diversity of use cases 
> makes the
> implementation of /ad hoc/ solutions unviable, in my opinion.

FWIW, I agree with Juan Manuel here and would welcome a 
straightforward way to insert material before the header.

All the best,
Tom

-- 
Thomas S. Dye
https://tsdye.online/tsdye

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Max Nikulin]

On 26/09/2023 04:57, Thomas S. Dye wrote:
> Aloha all,
> 
> Juan Manuel Macías <maciaschain@posteo.net> writes:
>>
>>> If so, Org should have native support of \DocumentMetadata, not
>>> LATEX_PRE_HEADER or something similar.
>>
>> If it were the only case of code before \documentclass, I would agree.
>> But, as I have already said above, the diversity of use cases makes the
>> implementation of /ad hoc/ solutions unviable, in my opinion.
> 
> FWIW, I agree with Juan Manuel here and would welcome a straightforward 
> way to insert material before the header.

I just have checked that a dirty hack with a few lines of code for 
`org-export-filter-final-output-functions' allows to insert arbitrary 
text to the beginning of export result. Perhaps a more elegant solution 
exists, but I admit it is not a straightforward way. At least it is 
possible.

I do not mind that generation of preamble should be more flexible, but I 
consider LATEX_PRE_HEADER as an ad hoc solution, so I am trying to find 
a better variant. That is why I asked for details concerning particular 
use cases.

I remember recipes like "put \usepackage{cmap} immediately after 
\documentclass" (nowadays this particular one should not be necessary). 
So I would prefer to avoid keywords per each chunk of preamble code.

\begin{filecontents*} from the original post is not convincing. 
\DocumentMetadata perhaps should be supported out of the box.

I would consider some kind of templates that use predefined fragments

#+LATEX_REPLACE_TEMPLATE: :preamble mypreamble
#+name: mypreamble
#+begin_src latex :exports none :noweb yes
\providecommand{\pdfxopts}{x-1a}
<<ox-latex-template-preamble>>
#+end_src

with ability to use fine grain snippets instead

#+LATEX_REPLACE_TEMPLATE: :preamble detailedpreamble
#+name: detailedpreamble
#+begin_src latex :exports none :noweb yes
<<ox-latex-template-DocumentMetadata>>
\PassOptionsToPackage...
<<ox-latex-template-documentclass>>
\usepackage{cmap}
\usepackage[english,<<ox-template-language>>]{babel}
<<ox-latex-template-usepackage>>
#+end_src


I hope, something similar may be made more readable than series of 
LATEX_HEADER & Co lines.

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Juan Manuel Macías]

Max Nikulin writes:

> I just have checked that a dirty hack with a few lines of code for
> `org-export-filter-final-output-functions' allows to insert arbitrary
> text to the beginning of export result. Perhaps a more elegant solution
> exists, but I admit it is not a straightforward way. At least it is
> possible.

And it is also possible by simply defining a new class (in Org
terminology, not LaTeX). My usual class is an almost empty one, without
documentclass and without packages, because I don't want org to write
the preamble for me. But I think this is not the point.

> I do not mind that generation of preamble should be more flexible, but I
> consider LATEX_PRE_HEADER as an ad hoc solution, so I am trying to find
> a better variant. That is why I asked for details concerning particular
> use cases.

IMHO, LaTeX_pre_header is obvious and transparent to the user: a version
of LaTeX_header that behaves exactly the same, except that it exports
the lines before \documentclass. It is simple and requires little code
to implement, and it is used for all cases of code before the
declaration of the class, since its syntax is consistent with
LaTeX_header. And its use does not require further explanation. You can
even play with constructions like:

#+NAME: pre
#+begin_src latex :exports none
(some stuff)
#+end_src

#+begin_src latex :noweb yes :results raw
,#+LaTeX_pre_header: <<pre>>
#+end_src

> I remember recipes like "put \usepackage{cmap} immediately after
> \documentclass" (nowadays this particular one should not be necessary).
> So I would prefer to avoid keywords per each chunk of preamble code.

I think that this would not be the case. This is not just any part of
the preamble, but a fairly definite part. Broadly speaking, LaTeX_header
would take care of what is after \documenclass (the axis of a LaTeX
document); LaTeX_pre_header would do it of anything that may have come
before. And both provide a less constricted preamble for advanced use of
LaTeX. Frankly, I can't think of a simpler solution.

> \begin{filecontents*} from the original post is not convincing.

Are you not convinced by some instructions that are included in the
official documentation of a LaTeX package (pdfx)?

https://i.imgur.com/NdLWmwc.png

The thing is that here it is not a question of whether something can be
done in this way or in another better way. This is how a given package
recommends doing it. If the user wants to use that specific package, she/he
will have to follow these instructions. It's more. I am thinking, for
example, of the case in which the user has to obtain a * tex file, not a
PDF, because she/he is collaborating with more people who do not use
Org, but do use that code in the * tex document.

--
Juan Manuel Macías

https://juanmanuelmacias.com

https://lunotipia.juanmanuelmacias.com

https://gnutas.juanmanuelmacias.com

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Max Nikulin]

On 27/09/2023 02:12, Juan Manuel Macías wrote:
> Max Nikulin writes:
>> I remember recipes like "put \usepackage{cmap} immediately after
>> \documentclass" (nowadays this particular one should not be necessary).
>> So I would prefer to avoid keywords per each chunk of preamble code.
> 
> I think that this would not be the case. This is not just any part of
> the preamble, but a fairly definite part. Broadly speaking, LaTeX_header
> would take care of what is after \documenclass (the axis of a LaTeX
> document); LaTeX_pre_header would do it of anything that may have come
> before. And both provide a less constricted preamble for advanced use of
> LaTeX. Frankly, I can't think of a simpler solution.

LaTeX code may be inserted
- before \DocumentMetadata
- between \DocumentMetadata and \documentclass
- between \documentclass and preamble added by Org
- between Org preamble and \begin{document}

Since ordering is important, I would prefer to assemble preamble from 
predefined fragments (or replace particular ones) to putting code into 
these (and probably more) slots.

>> \begin{filecontents*} from the original post is not convincing.
> 
> Are you not convinced by some instructions that are included in the
> official documentation of a LaTeX package (pdfx)?

More I read about .xmpdata, more it looks similar to an ugly kludge from 
my point of view. The following phrases are hardly consistent:
- "Including the metadata with the LATEX source is very convenient."
- "remember to remove the existing copy of \jobname.xmpdata file before 
the next processing run".
(A side note: "overwrite" option of filecontents looks promising.)

So the goal is an XML document embedded into PDF. I admit, Org can not 
provide it directly since e.g. PDF compliance level is responsibility of 
a PDF engine.

However the intermediate .xmpdata file may be provided independently of 
its .tex counterpart accordingly to docs. This file contains metadata 
and in Org metadata are managed through keywords. Significant fraction 
of metadata may be shared with HTML or MarkDown, so keywords should be 
considered as primary data. Writing this file from Org (e.g. by a babel 
source code block) should avoid issues with LaTeX input encodings 
described in the docs. Thus generation of .xmpdata during exporting of 
an .org file allows to keep metadata consistent.

Currently I do not see a way to get values of INFO argument of 
org-export functions from source code blocks making access to metadata 
tricky. This should be addressed somehow. I would consider specifying 
metadata in a way similar to org-babel header arguments to allow more 
fields than currently supported by Org. It should facilitate generation 
of \DocumentMedata, .xmpdada, etc.

Ability to overwrite fragments of preamble should be supported, but only 
as last resort. Specifying \DocumentMetadata or .xmpdata contents 
literally should be avoided to prevent discrepancy with metadata keywords.

I consider \begin{filecontents*}{\jobname.xmpdata} as an acceptable (but 
perhaps fragile) hack for LaTeX-first document. During export of Org 
file, this file should be written directly with values from Org metadata.

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Juan Manuel Macías]

Max Nikulin writes:

> LaTeX code may be inserted
> - before \DocumentMetadata
> - between \DocumentMetadata and \documentclass
> - between \documentclass and preamble added by Org
> - between Org preamble and \begin{document}

The first two cases can be solved perfectly with LaTeX_pre_header,
without having to complicate your life further.

> Since ordering is important [...]

Starting with \documentclass, the order is important in certain cases
and not so important in others. And, anyway, you can always use a hook
like \AtBegin... \AtEnd..., etc.

And if users want to have complete control over the preamble (that is,
everything before \begin{document}), they can always define a new class
(in Org terminology) or write a preamble or a sty file in a separate Org
document using org-babel and literary programming. In a job I'm doing
now my preamble has 1736 lines. Writing it with org-babel is quite
comfortable. In short, I think (IMHO) Org already has enough resources
for those situations.

> I consider \begin{filecontents*}{\jobname.xmpdata} as an acceptable
> (but perhaps fragile) hack for LaTeX-first document. During export of
> Org file, this file should be written directly with values from Org
> metadata.

I think I should insist on what I said in my previous message, with a
copy/paste:

The thing is that here it is not a question of whether something can be
done in this way or in another better way. This is how a given package
recommends doing it. If the user wants to use that specific package, she/he
will have to follow these instructions. It's more. I am thinking, for
example, of the case in which the user has to obtain a * tex file, not a
PDF, because she/he is collaborating with more people who do not use
Org, but do use that code in the * tex document.

Apart from that, since any code (except \usepackage) is allowed before
\documentclass, I think that the user should have the possibility (and
the freedom) to enter in their *tex documents whatever they want in that
place, including comments, luacode, shebangs, etc.

-- 
Juan Manuel Macías

https://juanmanuelmacias.com

https://lunotipia.juanmanuelmacias.com

https://gnutas.juanmanuelmacias.com

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[AW]

[-- Attachment #1: Type: text/plain, Size: 1243 bytes --]

Am Donnerstag, 28. September 2023, 12:07:41 CEST schrieb Max Nikulin:
> More I read about .xmpdata, more it looks similar to an ugly kludge from
> my point of view.

Exporting from orgmode to LaTeX needs a high level approach: don't do 
complicated things, just use the appropriate LaTeX API. 

Unfortunately adding XMP metadata is the largest construction site in the 
LaTeX world: https://tug.org/TUGboat/tb43-3/tb135fischer-xmp.pdf 

The LaTeX Project Team developed a new pdf management, see here: https://
ctan.org/pkg/pdfmanagement-testphase : 

"The new PDF management code offers backend-independent interfaces to central 
PDF dictionaries, tools to create annotations, form Xobjects, to embed files, 
and to handle PDF standards."

The most important command is `\DocumentMetadata{}` and it needs to be placed 
before `\ḑocumentclass{}`

The idea to have a LATEX_PRE_HEADER to insert `\DocumentMetadata{}` is exactly 
what you need right now, if you export from orgmode to current LaTeX. With 
`\DocumentMetadata{}` you can add most of the necessary xmp data -- and I 
write most, because I'm using it on a daily basis, but haven't checked if 
really everything is included yet. 

-- 
Regards,
Alexander 

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Max Nikulin]

On 28/09/2023 19:31, Juan Manuel Macías wrote:
> I think I should insist on what I said in my previous message, with a
> copy/paste:
> 
> The thing is that here it is not a question of whether something can be
> done in this way or in another better way. This is how a given package
> recommends doing it. If the user wants to use that specific package, she/he
> will have to follow these instructions.

My reading of these instructions: users have 2 options, they either 
provide .xmpdata files directly or they ask LaTeX to generate it using 
filecontents* and take responsibility to remove the file when they 
change metadata.

I do not see any words discouraging the former way, while there are 
warnings concerning pitfalls with the latter approach that still may be 
convenient in some cases.

> It's more. I am thinking, for
> example, of the case in which the user has to obtain a * tex file, not a
> PDF, because she/he is collaborating with more people who do not use
> Org, but do use that code in the * tex document.

Perhaps I am wrong in my assumption that in particular case of PDF-X 
compliant documents, it is unlikely that it is just a single .tex 
document. If there are more files: graphics, custom packages, included 
.tex file then an additional separate .xmpdata file is not an issue.

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Max Nikulin]

On 28/09/2023 22:36, AW wrote:
> 
> The idea to have a LATEX_PRE_HEADER to insert `\DocumentMetadata{}` is exactly
> what you need right now, if you export from orgmode to current LaTeX. With
> `\DocumentMetadata{}` you can add most of the necessary xmp data -- and I
> write most, because I'm using it on a daily basis, but haven't checked if
> really everything is included yet.

The question is if Juan Manuel can use \DocumentMetadata instead of the 
pdfx package in his workflow.

However there is a common part of \DocumentMetadata and .xmpdata usage. 
Currently Org mode allows to specify a very limited subset of metadata 
through keywords

#+language: en
#+author: Me
#+title: Statement

The advantage is that the same data are used by all export backends: 
LaTeX, HTML, etc. Specifying \DocumentMetadata literally without 
substitutions from Org keywords may cause diverged values. That is why I 
proposed to consider some kind of templates.

Another issue is extended metadata. I would consider an approach similar 
to header arguments for source code blocks:

#+property: header-args :eval never-export
for all languages
#+property: header-args:elisp :exports both :eval yes
overrides for emacs-lisp

Perhaps it is not reasonable to require to specify all metadata through 
Org keywords and it may be more convenient to combine raw LaTeX code and 
fragments generated by Org

#+name: preamble
#+begin_src latex :exports none :noweb yes
\DocumentMetadata{
   pdfstandard = a-2b
   <<ox-latex-template-document-metadata-entries>>
}
<<ox-latex-template-documentclass>>
<<ox-latex-template-usepackage>>
#+end_src

Access to keywords values during evaluation of source code blocks have 
to be implemented however.

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Max Nikulin]

On 28/09/2023 19:31, Juan Manuel Macías wrote:
> Starting with \documentclass, the order is important in certain cases
> and not so important in others. And, anyway, you can always use a hook
> like \AtBegin... \AtEnd..., etc.

I am judging from my past experience. Besides \usepackage{cmap} that was 
necessary to be put immediately after \documentclass, there were issues 
with order of \usepackage for hyperref and some other packages.

Custom commands may require some attention as well. There is an example 
of reusing year in .xmpdata and document title in the pdfx docs. Such 
cases may be more subtle. In the case of .xmpdata command must be 
defined at the moment when the file is loaded, so it is not important 
either command definition or \begin{filecontents*} placed first 
(.xmpdata may be provided directly as well). In the case of 
\DocumentMetadata it is not obvious if commands may defined later.

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Juan Manuel Macías]

Max Nikulin writes:

> On 28/09/2023 22:36, AW wrote:
>> 
>> The idea to have a LATEX_PRE_HEADER to insert `\DocumentMetadata{}` is exactly
>> what you need right now, if you export from orgmode to current LaTeX. With
>> `\DocumentMetadata{}` you can add most of the necessary xmp data -- and I
>> write most, because I'm using it on a daily basis, but haven't checked if
>> really everything is included yet.
>
> The question is if Juan Manuel can use \DocumentMetadata instead of the 
> pdfx package in his workflow.

I'm afraid that with the pdfx example that I put in the first post I
have caused a little confusion. pdfx is not part of my workflow. I only
used it once a long time ago. I've cited the code from that package here
as another example of arbitrary code that can be placed before
\documentclass. As I said, there are many more examples and possible
cases: \DocumentMetadata and \PassOptionsToPackage are two typical
cases. But, as I have already mentioned, /almost/ any LaTeX code can be
accommodated before class declaration. Even simple comments. Why not
think about a hypothetical case in which a user needs to create from Org
a *.tex file that contains comments before \documentclass? With a
LaTeX_pre_header keyword that would be possible.

-- 
Juan Manuel Macías

https://juanmanuelmacias.com

https://lunotipia.juanmanuelmacias.com

https://gnutas.juanmanuelmacias.com

Re: [patch] ox-latex.el: Add `LATEX_PRE_HEADER' keyword

[Max Nikulin]

On 27/09/2023 02:12, Juan Manuel Macías wrote:
> Max Nikulin writes:
> 
>> I just have checked that a dirty hack with a few lines of code for
>> `org-export-filter-final-output-functions' allows to insert arbitrary
>> text to the beginning of export result. Perhaps a more elegant solution
>> exists, but I admit it is not a straightforward way. At least it is
>> possible.
> 
> And it is also possible by simply defining a new class (in Org
> terminology, not LaTeX). My usual class is an almost empty one, without
> documentclass and without packages, because I don't want org to write
> the preamble for me. But I think this is not the point.

LATEX_PRE_HEADER you proposed is a document-local setting while 
`org-latex-classes' is shared by all Org documents. In that hack I used 
a buffer-local variable to achieve document-specific fragment.

In the specific cases of \DocumentMetadata and .xmpdata, 
LATEX_PRE_HEADER needs to be passed through `format-spec' with result of 
`org-latex--format-spec' call to get notion of Org metadata. Whether it 
is necessary for \PassOptionsToPackage depends on particular cases. If 
such options do not vary across documents then certainly 
`org-latex-classes' becomes viable.

I do not like "[NO-PACKAGES]" and similar keywords to suppress adding of 
some code snippets. That is why I believe it is better to seek for 
another approach to generate LaTeX preamble. I have not had a look into 
Timothy's code yet, so I can say nothing concerning its flexibility in 
respect to substituting metadata into code before \documentclass.