suggestion to change default org-latex-pdf-process to latexmk

suggestion to change default org-latex-pdf-process to latexmk

[Bruce D'Arcus]

While testing org-cite a few weeks ago, I noticed by default bibtex
won't run with natbib export, or biblatex with biblatex export.

Since org has not had native citations, this is not surprising.

But with org-cite coming soon, this will change.

At the time I suggested changing default to use latexmk, which will
handle both bibtex and biblatex runs.

https://lists.gnu.org/archive/html/emacs-orgmode/2021-05/msg00486.html

So what do LaTeX users think about changing the default for
"org-latex-pdf-process" to "latexmk"?

Is there any reason not to do this?

Bruce

Re: suggestion to change default org-latex-pdf-process to latexmk

[Juan Manuel Macías]

Hi Bruce,

Bruce D'Arcus writes:

> So what do LaTeX users think about changing the default for
> "org-latex-pdf-process" to "latexmk"?

It sounds like a good idea to me. In fact I have `org-latex-pdf-process'
set to latexmk in my ~ /.emacs (with the option to always compile with
LuaTeX).

Best regards,

Juan Manuel 

Re: suggestion to change default org-latex-pdf-process to latexmk

[Timothy]

Hi Bruce,

In case it's of interest, I've been quite happy with this:
https://tecosaur.github.io/emacs-config/config.html#compiling

Bruce D'Arcus <bdarcus@gmail.com> writes:

> While testing org-cite a few weeks ago, I noticed by default bibtex
> won't run with natbib export, or biblatex with biblatex export.
>
> Since org has not had native citations, this is not surprising.
>
> But with org-cite coming soon, this will change.
>
> At the time I suggested changing default to use latexmk, which will
> handle both bibtex and biblatex runs.
>
> https://lists.gnu.org/archive/html/emacs-orgmode/2021-05/msg00486.html
>
> So what do LaTeX users think about changing the default for
> "org-latex-pdf-process" to "latexmk"?
>
> Is there any reason not to do this?
>
> Bruce

--
Timoty

Re: suggestion to change default org-latex-pdf-process to latexmk

[Bruce D'Arcus]

[-- Attachment #1: Type: text/plain, Size: 973 bytes --]

Thanks Timothy.

What's your view on modifying current default?

On Tue, Jun 1, 2021, 12:36 PM Timothy <tecosaur@gmail.com> wrote:

>
> Hi Bruce,
>
> In case it's of interest, I've been quite happy with this:
> https://tecosaur.github.io/emacs-config/config.html#compiling
>
> Bruce D'Arcus <bdarcus@gmail.com> writes:
>
> > While testing org-cite a few weeks ago, I noticed by default bibtex
> > won't run with natbib export, or biblatex with biblatex export.
> >
> > Since org has not had native citations, this is not surprising.
> >
> > But with org-cite coming soon, this will change.
> >
> > At the time I suggested changing default to use latexmk, which will
> > handle both bibtex and biblatex runs.
> >
> > https://lists.gnu.org/archive/html/emacs-orgmode/2021-05/msg00486.html
> >
> > So what do LaTeX users think about changing the default for
> > "org-latex-pdf-process" to "latexmk"?
> >
> > Is there any reason not to do this?
> >
> > Bruce
>
> --
> Timoty
>

[-- Attachment #2: Type: text/html, Size: 1790 bytes --]

Re: suggestion to change default org-latex-pdf-process to latexmk

[Timothy]

Bruce D'Arcus <bdarcus@gmail.com> writes:

> Thanks Timothy.
>
> What's your view on modifying current default?

I think latexmk is a pretty reasonable expectation, and your
justification that it's more important seems sound to me.

Using the value seen in my config, I've found that at the least my
version of latexmk interprets the arguments such that
`org-latex-compilers' still works as expected (with xelatex, lualatex,
etc.). Perhaps this would make sense as a new default if it works well
for others too (without the -shell-escape bit though).

> On Tue, Jun 1, 2021, 12:36 PM Timothy <tecosaur@gmail.com> wrote:
>
>>
>> Hi Bruce,
>>
>> In case it's of interest, I've been quite happy with this:
>> https://tecosaur.github.io/emacs-config/config.html#compiling
>>
>> Bruce D'Arcus <bdarcus@gmail.com> writes:
>>
>> > While testing org-cite a few weeks ago, I noticed by default bibtex
>> > won't run with natbib export, or biblatex with biblatex export.
>> >
>> > Since org has not had native citations, this is not surprising.
>> >
>> > But with org-cite coming soon, this will change.
>> >
>> > At the time I suggested changing default to use latexmk, which will
>> > handle both bibtex and biblatex runs.
>> >
>> > https://lists.gnu.org/archive/html/emacs-orgmode/2021-05/msg00486.html
>> >
>> > So what do LaTeX users think about changing the default for
>> > "org-latex-pdf-process" to "latexmk"?
>> >
>> > Is there any reason not to do this?
>> >
>> > Bruce
>>
>> --
>> Timothy
>>

--
Timothy

Re: suggestion to change default org-latex-pdf-process to latexmk

[Bruce D'Arcus]

On Tue, Jun 1, 2021 at 1:03 PM Timothy <tecosaur@gmail.com> wrote:
>
>
> Bruce D'Arcus <bdarcus@gmail.com> writes:
>
> > Thanks Timothy.
> >
> > What's your view on modifying current default?
>
> I think latexmk is a pretty reasonable expectation, and your
> justification that it's more important seems sound to me.
>
> Using the value seen in my config, I've found that at the least my
> version of latexmk interprets the arguments such that
> `org-latex-compilers' still works as expected (with xelatex, lualatex,
> etc.). Perhaps this would make sense as a new default if it works well
> for others too (without the -shell-escape bit though).

Here's the current command for the "latexmk" option:

"latexmk -g -pdf -pdflatex=\"%latex\" -outdir=%o %f"

... and here's what you have, minus shell-escape.

"latexmk -f -pdf -%latex -interaction=nonstopmode -output-directory=%o %f"

Do they not both account for the engine (luatex, xetex, etc.)?

In any case, whatever the command is, clearly it should.

Bruce

Re: suggestion to change default org-latex-pdf-process to latexmk

[Tim Cross]

"Bruce D'Arcus" <bdarcus@gmail.com> writes:

> While testing org-cite a few weeks ago, I noticed by default bibtex
> won't run with natbib export, or biblatex with biblatex export.
>
> Since org has not had native citations, this is not surprising.
>
> But with org-cite coming soon, this will change.
>
> At the time I suggested changing default to use latexmk, which will
> handle both bibtex and biblatex runs.
>
> https://lists.gnu.org/archive/html/emacs-orgmode/2021-05/msg00486.html
>
> So what do LaTeX users think about changing the default for
> "org-latex-pdf-process" to "latexmk"?
>
> Is there any reason not to do this?
>

One reason is that latexmk is not installed on some systems. This means
it will require users to install an additional package not previously
required. It is also a perl script, so will need perl installed - not
sure what sort of overhead this might represent for Windows and Mac
users? Does latexmk work on all the platforms Emacs+org is supported on?

It isn't hard for users who want to use latexmk to set the value and as
it will break the installation for many existing users, I think we need
to be careful about changing this default. For many users who don't use
bibtex, this change could mean they have to install additional
software with little other observable benefit or change the default back
to what it is. 

If we do make this change, it will be necessary to update any docs to
add details about the additional dependncy and how to resolve it (for
all supported platforms). 


-- 
Tim Cross

Re: suggestion to change default org-latex-pdf-process to latexmk

[Scott Randby]

On 6/1/21 8:00 PM, Tim Cross wrote:
> 
> "Bruce D'Arcus" <bdarcus@gmail.com> writes:
> 
>> While testing org-cite a few weeks ago, I noticed by default bibtex
>> won't run with natbib export, or biblatex with biblatex export.
>>
>> Since org has not had native citations, this is not surprising.
>>
>> But with org-cite coming soon, this will change.
>>
>> At the time I suggested changing default to use latexmk, which will
>> handle both bibtex and biblatex runs.
>>
>> https://lists.gnu.org/archive/html/emacs-orgmode/2021-05/msg00486.html
>>
>> So what do LaTeX users think about changing the default for
>> "org-latex-pdf-process" to "latexmk"?
>>
>> Is there any reason not to do this?
>>
> 
> One reason is that latexmk is not installed on some systems. This means
> it will require users to install an additional package not previously
> required. It is also a perl script, so will need perl installed - not
> sure what sort of overhead this might represent for Windows and Mac
> users? Does latexmk work on all the platforms Emacs+org is supported on?
> 
> It isn't hard for users who want to use latexmk to set the value and as
> it will break the installation for many existing users, I think we need
> to be careful about changing this default. For many users who don't use
> bibtex, this change could mean they have to install additional
> software with little other observable benefit or change the default back
> to what it is.
> 
> If we do make this change, it will be necessary to update any docs to
> add details about the additional dependncy and how to resolve it (for
> all supported platforms).
> 
> 

As a constant user of Org's LaTeX export capability, this change would break my ability to export to PDF, and it would be difficult for me to fix. I don't support this change.

Scott Randby

Re: suggestion to change default org-latex-pdf-process to latexmk

[Nick Dokos]

"Bruce D'Arcus" <bdarcus@gmail.com> writes:

> While testing org-cite a few weeks ago, I noticed by default bibtex
> won't run with natbib export, or biblatex with biblatex export.
>
> Since org has not had native citations, this is not surprising.
>
> But with org-cite coming soon, this will change.
>
> At the time I suggested changing default to use latexmk, which will
> handle both bibtex and biblatex runs.
>
> https://lists.gnu.org/archive/html/emacs-orgmode/2021-05/msg00486.html
>
> So what do LaTeX users think about changing the default for
> "org-latex-pdf-process" to "latexmk"?
>
> Is there any reason not to do this?
>

The only reason I can think of is that it might not be available by
default, so it will be one more thing for a noob to install before Org mode
starts "working".

-- 
Nick

"There are only two hard problems in computer science: cache
invalidation, naming things, and off-by-one errors." -Martin Fowler

Re: suggestion to change default org-latex-pdf-process to latexmk

[Bruce D'Arcus]

On Tue, Jun 1, 2021 at 9:09 PM Scott Randby <srandby@gmail.com> wrote:

> As a constant user of Org's LaTeX export capability, this change would break my ability to export to PDF, and it would be difficult for me to fix. I don't support this change.

Can you explain this, Scott? Why would anything "break'?

And for those that think this is a bad idea, suggested alternatives?

With the status quo; new org-cite users will run latex/pdf output
process and will be surprised by the results, as there will be no
citations or bibliography.

I suppose the easy solution is a prominent note in the documentation.

Bruce

Re: suggestion to change default org-latex-pdf-process to latexmk

[Tim Cross]

"Bruce D'Arcus" <bdarcus@gmail.com> writes:

> On Tue, Jun 1, 2021 at 9:09 PM Scott Randby <srandby@gmail.com> wrote:
>
>> As a constant user of Org's LaTeX export capability, this change would break
>> my ability to export to PDF, and it would be difficult for me to fix. I don't
>> support this change.
>
> Can you explain this, Scott? Why would anything "break'?
>
> And for those that think this is a bad idea, suggested alternatives?
>
> With the status quo; new org-cite users will run latex/pdf output
> process and will be surprised by the results, as there will be no
> citations or bibliography.
>
> I suppose the easy solution is a prominent note in the documentation.
>

I'm guessing Scott is referring to the latexmk not being part of a
'standard' texlive installation and if not installed, pdf generation
would fail.

Yes, document the requirement would be my suggestion. There are a
number of 'extensions' to basic org functionality which require
additional prerequisites. For users of org-cite, one solution would be
to document latexmk as a prerequisite.

Another alternative would be to investigate if there is a better default
for org-latex-pdf-program which does not require installation of
latexmk or perhaps make the default more 'dynamic' - have it check for
present of latexmk and if found, use it, otherwise use existing value.

Big question IMO still needing to be answered is what is the situation
with installation of latexmk on non GNU Linux based platforms. For
example, how easy is it to install latexmk on a windows system? If
installation of latexmk is trivial on all supported platforms, there is
a higher chance of being able to make it the defualt. On the other hand,
if you need to jump through a lot of hoops to get it running, making it
the default is less likely. 

-- 
Tim Cross

Re: suggestion to change default org-latex-pdf-process to latexmk

[Vladimir Lomov]

[-- Attachment #1: Type: text/plain, Size: 1728 bytes --]

Hello,
** Tim Cross <theophilusx@gmail.com> [2021-06-02 12:56:59 +1000]:

> "Bruce D'Arcus" <bdarcus@gmail.com> writes:

>> On Tue, Jun 1, 2021 at 9:09 PM Scott Randby <srandby@gmail.com> wrote:
>>
>>> As a constant user of Org's LaTeX export capability, this change would break
>>> my ability to export to PDF, and it would be difficult for me to fix. I don't
>>> support this change.
>>
>> Can you explain this, Scott? Why would anything "break'?
>>
>> And for those that think this is a bad idea, suggested alternatives?
>>
>> With the status quo; new org-cite users will run latex/pdf output
>> process and will be surprised by the results, as there will be no
>> citations or bibliography.
>>
>> I suppose the easy solution is a prominent note in the documentation.
>>

> I'm guessing Scott is referring to the latexmk not being part of a
> 'standard' texlive installation and if not installed, pdf generation
> would fail.

If you are talking about Linux then there is TeX Live distribution (TUG TeX
Live) and texlive packages. latexmk availability depends on how you install
TUG TeX Live (full scheme or not, I didn't investigate this I prefer to use
full always) but if you are referring to texlive packages (Linux distribution
packages) then the latexmk could be available as distribution package.

It could be a problem either on Windows (suppose one could use MikTeX but
Emacs on Windows is a bit a pain, IMHO) or on Mac but if one uses TUG TeX Live
I'm sure there would be latexmk.

Of course, it would be best if there will be a note in documentation.

[...]

---
WBR, Vladimir Lomov

-- 
I can't decide whether to commit suicide or go bowling.
		-- Florence Henderson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: suggestion to change default org-latex-pdf-process to latexmk

[Stefan Nobis]

"Bruce D'Arcus" <bdarcus@gmail.com> writes:

> Here's the current command for the "latexmk" option:

> "latexmk -g -pdf -pdflatex=\"%latex\" -outdir=%o %f"

> ... and here's what you have, minus shell-escape.

> "latexmk -f -pdf -%latex -interaction=nonstopmode -output-directory=%o %f"

The option "-interaction=nonstopmode" forbids user interaction (like
asking for a missing file). Without this explicit option latexmk uses
"-interaction=batchmode" by default - in this case some diagnotic
messages to the terminal (not log file) are suppressed. Thus the
default option from latexmk might suffice.

Regarding "-f" versus "-g" I personally would slightly prefer "-f" for
performance reasons. With "-g" the document will always be newly
generated, even if nothing changed. This is only useful if the latexmk
config (or some environment variable) has changed. Other changes
should be detectable by latexmk, so "-f" should suffice.

The options "-output-directory" and "-outdir" are synonyms.

Regarding the choise of engine: "-pdflatex=\"%latex\"" explicitly sets
the command (including options) to be called by latexmk, without
letting latexmk know that the engine changed. Using "-%latex" (e.g.
the option "-lualatex" or "-xelatex") lets latexmk know that a
different engine should be used.

This matters at least for xelatex, because latexmk can utilize some
optimizations if it knows that xetex is used. In this case the first
runs only produce .xdv files and only the last run will produce the
full pdf to save time (see also option "-pdfxe" which is triggered by
"-xelatex").

Therefore I would tend to use this call:

    "latexmk -f -pdf -%latex -outdir=%o %f"

All other aspects of latexmk should be configured via config file
(IMHO).

> In any case, whatever the command is, clearly it should.

Yes, I would also recommend latexmk (it may also speed up PDF
generation, because in quite some cases a single or two passes suffice
and latexmk is quite good in detecting these cases).

-- 
Until the next mail...,
Stefan.

Re: suggestion to change default org-latex-pdf-process to latexmk

[Stefan Nobis]

Tim Cross <theophilusx@gmail.com> writes:

> One reason is that latexmk is not installed on some systems.

Just after starting to write that this is false nowadays I realized
you are right. :)

See: https://mg.readthedocs.io/latexmk.html

For macOS latexmk is distributed with the default TeX installation
MacTeX for quite some years (IIRC at least since 2012). And it is
working OOTB.

As far as I know the default TeX installation for Windows is MikTeX
(is this still true?), which also includes latexmk but lacks the Perl
part (therefore Perl needs to be installed manually). But there is
TeXLive for Windows and as far as I understand in this case Perl is
also installed and latexmk works OOTB.

A manual installation of TeXLive for Linux should also install latexmk
(and Perl should also be available on next to every Linux box). Only
some distributions bundle latexmk in a separate package - that should
be easy to install (but breaks PDF creation if forgotten).

Therefore: latexmk is available on all plattforms and in most cases it
is already installed with the TeX system or easily installable. But
not in all cases does it work OOTB and requires more work like
installing Perl on Windows.

In my opinion its worth to depend on this tool as it makes handling
LaTeX documents much easier. On the other hand it may raise the bar
for some users just to high. Hard to say.

An alternative may be to use latexmk only if citations are found (new
feature, new dependencies). Or a wrapper that checks whether latexmk
is available and works (e.g. trying to call "latexmk --version") and
falls back to the old routine of manually running the engine and
bibtex/biber if necessary.

-- 
Until the next mail...,
Stefan.

Re: suggestion to change default org-latex-pdf-process to latexmk

[Loris Bennett]

Nick Dokos <ndokos@gmail.com> writes:

> "Bruce D'Arcus" <bdarcus@gmail.com> writes:
>
>> While testing org-cite a few weeks ago, I noticed by default bibtex
>> won't run with natbib export, or biblatex with biblatex export.
>>
>> Since org has not had native citations, this is not surprising.
>>
>> But with org-cite coming soon, this will change.
>>
>> At the time I suggested changing default to use latexmk, which will
>> handle both bibtex and biblatex runs.
>>
>> https://lists.gnu.org/archive/html/emacs-orgmode/2021-05/msg00486.html
>>
>> So what do LaTeX users think about changing the default for
>> "org-latex-pdf-process" to "latexmk"?
>>
>> Is there any reason not to do this?
>>
>
> The only reason I can think of is that it might not be available by
> default, so it will be one more thing for a noob to install before Org mode
> starts "working".

Of course, LaTeX itself, with or without latexmk, is something the noob
will have to install.  So the prerequisite could be "LaTeX with
latexmk".  With regards to Windows, Cygwin has latexmk in the
'texlive-collection-binextra' package and MikTeX has a 'latexmf'
package.  On Debian latexmf is also its own package and thus has to be
installed explicitly.

Personally I use Debian and have "org-latex-pdf-process" set to use
latexmk, so the change in the default wouldn't affect me.

Cheers,

Loris

-- 
This signature is currently under construction.

Re: suggestion to change default org-latex-pdf-process to latexmk

[Tim Cross]

Stefan Nobis <stefan-ml@snobis.de> writes:

> Tim Cross <theophilusx@gmail.com> writes:
>
>> One reason is that latexmk is not installed on some systems.
>
> Just after starting to write that this is false nowadays I realized
> you are right. :)
>
> See: https://mg.readthedocs.io/latexmk.html
>
> For macOS latexmk is distributed with the default TeX installation
> MacTeX for quite some years (IIRC at least since 2012). And it is
> working OOTB.
>
> As far as I know the default TeX installation for Windows is MikTeX
> (is this still true?), which also includes latexmk but lacks the Perl
> part (therefore Perl needs to be installed manually). But there is
> TeXLive for Windows and as far as I understand in this case Perl is
> also installed and latexmk works OOTB.
>
> A manual installation of TeXLive for Linux should also install latexmk
> (and Perl should also be available on next to every Linux box). Only
> some distributions bundle latexmk in a separate package - that should
> be easy to install (but breaks PDF creation if forgotten).
>
> Therefore: latexmk is available on all plattforms and in most cases it
> is already installed with the TeX system or easily installable. But
> not in all cases does it work OOTB and requires more work like
> installing Perl on Windows.
>
> In my opinion its worth to depend on this tool as it makes handling
> LaTeX documents much easier. On the other hand it may raise the bar
> for some users just to high. Hard to say.
>
> An alternative may be to use latexmk only if citations are found (new
> feature, new dependencies). Or a wrapper that checks whether latexmk
> is available and works (e.g. trying to call "latexmk --version") and
> falls back to the old routine of manually running the engine and
> bibtex/biber if necessary.

The more I think about it, I think the best solution would be to update
the code which sets the default and have it check for latexmk. If it is
found, set it as the default and if it is not found, set the existing
default. This would have minimum impact on users and allow those who
have installed latexmk to get the benefits while not forcing those who
don't need it to install it.


-- 
Tim Cross

Re: suggestion to change default org-latex-pdf-process to latexmk

[Juan Manuel Macías]

Stefan Nobis writes:

> An alternative may be to use latexmk only if citations are found (new
> feature, new dependencies). Or a wrapper that checks whether latexmk
> is available and works (e.g. trying to call "latexmk --version") and
> falls back to the old routine of manually running the engine and
> bibtex/biber if necessary.

Another possibility would be to add to the documentation all the
instructions needed on how to properly configure `org-latex-pdf-process'
with latexmk, for those users who wish to use bibtex or biblatex.
Although latexmk is also useful for many packages that need multiple
compilations, indexes, etc.

Best regards,

Juan Manuel 

Re: suggestion to change default org-latex-pdf-process to latexmk

[Bruce D'Arcus]

On Wed, Jun 2, 2021 at 4:28 AM Tim Cross <theophilusx@gmail.com> wrote:

> The more I think about it, I think the best solution would be to update
> the code which sets the default and have it check for latexmk. If it is
> found, set it as the default and if it is not found, set the existing
> default. This would have minimum impact on users and allow those who
> have installed latexmk to get the benefits while not forcing those who
> don't need it to install it.

If this were possible, I agree it would be the best approach.

Bruce

Re: suggestion to change default org-latex-pdf-process to latexmk

[Colin Baxter]

>>>>> Bruce D'Arcus <bdarcus@gmail.com> writes:

    > On Wed, Jun 2, 2021 at 4:28 AM Tim Cross <theophilusx@gmail.com> wrote:
    >> The more I think about it, I think the best solution would be to
    >> update the code which sets the default and have it check for
    >> latexmk. If it is found, set it as the default and if it is not
    >> found, set the existing default. This would have minimum impact
    >> on users and allow those who have installed latexmk to get the
    >> benefits while not forcing those who don't need it to install it.

    > If this were possible, I agree it would be the best approach.

    > Bruce

What effect is this likely to have on people who have specified latexmk?
For example, I have

#+begin_src elisp
(setq org-latex-pdf-process '("latexmk -outdir=%o -f %f"))
(setq org-latex-compiler "pdflatex")
#+end_src

I assume any change will have no effect, but is this correct?

Best wishes,

Colin.

Re: suggestion to change default org-latex-pdf-process to latexmk

[Scott Randby]

On 6/2/21 1:33 AM, Stefan Nobis wrote:
> "Bruce D'Arcus" <bdarcus@gmail.com> writes:
> 
>> Here's the current command for the "latexmk" option:
> 
>> "latexmk -g -pdf -pdflatex=\"%latex\" -outdir=%o %f"
> 
>> ... and here's what you have, minus shell-escape.
> 
>> "latexmk -f -pdf -%latex -interaction=nonstopmode -output-directory=%o %f"
> 
> The option "-interaction=nonstopmode" forbids user interaction (like
> asking for a missing file). Without this explicit option latexmk uses
> "-interaction=batchmode" by default - in this case some diagnotic
> messages to the terminal (not log file) are suppressed. Thus the
> default option from latexmk might suffice.
> 
> Regarding "-f" versus "-g" I personally would slightly prefer "-f" for
> performance reasons. With "-g" the document will always be newly
> generated, even if nothing changed. This is only useful if the latexmk
> config (or some environment variable) has changed. Other changes
> should be detectable by latexmk, so "-f" should suffice.
> 
> The options "-output-directory" and "-outdir" are synonyms.
> 
> Regarding the choise of engine: "-pdflatex=\"%latex\"" explicitly sets
> the command (including options) to be called by latexmk, without
> letting latexmk know that the engine changed. Using "-%latex" (e.g.
> the option "-lualatex" or "-xelatex") lets latexmk know that a
> different engine should be used.
> 
> This matters at least for xelatex, because latexmk can utilize some
> optimizations if it knows that xetex is used. In this case the first
> runs only produce .xdv files and only the last run will produce the
> full pdf to save time (see also option "-pdfxe" which is triggered by
> "-xelatex").
> 
> Therefore I would tend to use this call:
> 
>      "latexmk -f -pdf -%latex -outdir=%o %f"
> 
> All other aspects of latexmk should be configured via config file
> (IMHO).
> 
>> In any case, whatever the command is, clearly it should.
> 

I'm no longer sure that a change would break my ability to export to PDF. I'm using Linux with TeX Live, and latexmk is available. I'm willing to try changing org-latex-pdf-process to one of the above suggestions, but I have some questions. I've never used latexmk, and I really depend on Org to set things up so that they just work out of the box. It took me a long time to figure out how to get Emacs to find TeX so that previewing LaTeX fragments works (the solution is not in the Org manual).

(1) Which one of the above suggestions should I use? I don't want interactions when I'm exporting. I don't want to have to mess with a configuration file either.

(2) How many shell commands will org-latex-pdf-process need? Right now there are three. Replace those three with just one latexmk?

Scott Randby

Re: suggestion to change default org-latex-pdf-process to latexmk

[Scott Randby]

On 6/2/21 8:07 AM, Bruce D'Arcus wrote:
> On Wed, Jun 2, 2021 at 4:28 AM Tim Cross <theophilusx@gmail.com> wrote:
> 
>> The more I think about it, I think the best solution would be to update
>> the code which sets the default and have it check for latexmk. If it is
>> found, set it as the default and if it is not found, set the existing
>> default. This would have minimum impact on users and allow those who
>> have installed latexmk to get the benefits while not forcing those who
>> don't need it to install it.
> 
> If this were possible, I agree it would be the best approach.

Perhaps this is a stupid question, but what about the case in which TeX is installed after Emacs is installed? I do this sometimes.

Scott Randby

Re: suggestion to change default org-latex-pdf-process to latexmk

[Stefan Nobis]

Scott Randby <srandby@gmail.com> writes:

> (1) Which one of the above suggestions should I use? I don't want
> interactions when I'm exporting. I don't want to have to mess with a
> configuration file either.

> (2) How many shell commands will org-latex-pdf-process need? Right
> now there are three. Replace those three with just one latexmk?

I have

--8<---------------cut here---------------start------------->8---
(setq org-latex-pdf-process '("latexmk -f -pdf -%latex -outdir=%o %f"))
--8<---------------cut here---------------end--------------->8---

in my Emacs configuration. The nice thing with latexmk is, that a
single call will suffice. It reads the generated log and checks
whether multiple calls to the LaTeX engine are necessary (after each
run it check whether another run is need, up-to the configurable
maximum of 5 runs). Therefore in some cases (depending on used LaTeX
packages and commands) latexmk may even produce the PDF faster, if
only one or two passes are necessary. But sometimes LaTeX thinks it
may need more passes (e.g. having undefined references, unstable
output due to dynamically generated things at page breaks for example
with cleveref and varioref where floats/references are put on one page
in one run and another page in the following run) - and in these cases
creating the PDF may take a bit longer due to the maximum of 5
(instead of only 3) runs.

Hope that helps a little bit.

-- 
Until the next mail...,
Stefan.

Re: suggestion to change default org-latex-pdf-process to latexmk

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1047 bytes --]


Scott Randby <srandby@gmail.com> writes:

> On 6/2/21 8:07 AM, Bruce D'Arcus wrote:
>> On Wed, Jun 2, 2021 at 4:28 AM Tim Cross <theophilusx@gmail.com> wrote:
>> 
>>> The more I think about it, I think the best solution would be to update
>>> the code which sets the default and have it check for latexmk. If it is
>>> found, set it as the default and if it is not found, set the existing
>>> default. This would have minimum impact on users and allow those who
>>> have installed latexmk to get the benefits while not forcing those who
>>> don't need it to install it.
>> If this were possible, I agree it would be the best approach.
>
> Perhaps this is a stupid question, but what about the case in which TeX is installed after Emacs is installed? I do this sometimes.

Is there a reason not to have the default check for availability when
run? It should be fast and would progressively improve (one step) when
the user installs latexmk.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: suggestion to change default org-latex-pdf-process to latexmk

[Colin Baxter]

I'm beginning to think this change of the default should not
happen. From this thread it seems to be that the chances of it breaking
existing work flow is high.

I really don't understand why the changed is needed.

Best wishes,

Re: suggestion to change default org-latex-pdf-process to latexmk

[Robert Pluim]

>>>>> On Wed, 02 Jun 2021 17:29:49 +0100, Colin Baxter <m43cap@yandex.com> said:

    Colin> I'm beginning to think this change of the default should not
    Colin> happen. From this thread it seems to be that the chances of it breaking
    Colin> existing work flow is high.

    Colin> I really don't understand why the changed is needed.

It sounds like a better idea is to update the defcustom to offer
latexmk as an option, with copious documentation as to when it
could/should be used.

Robert
-- 

Re: suggestion to change default org-latex-pdf-process to latexmk

[Bruce D'Arcus]

On Wed, Jun 2, 2021 at 12:30 PM Colin Baxter <m43cap@yandex.com> wrote:
>
>
> I'm beginning to think this change of the default should not
> happen. From this thread it seems to be that the chances of it breaking
> existing work flow is high.

I don't see how that follows. Can you clarify?

> I really don't understand why the changed is needed.

I've stated it twice, I believe, but it's now a longer thread, so:

Org is set to get native citation support.

Once that's merged, people who use this feature will be disappointed
to know that when using the natbib or biblatex export processors,
their citations and bibliographies will not render.

As in, from their perspective, it will be broken upfront.

Bruce

Re: suggestion to change default org-latex-pdf-process to latexmk

[Stefan Nobis]

Colin Baxter <m43cap@yandex.com> writes:

> I really don't understand why the changed is needed.

The upcoming citation support needs a reliable way to run the required
bibliography processor (e.g. biber or bibtex). Therefore the current
default setting that only runs latex three times does not suffice.

On the other hand running biber unconditionally will result in errors
if no citation and bibliography has been used in the document.

So we need a more sophisticated method to run the LaTeX engine and
associated tools. One way would be to use something that tries to
do exactly this: Depending on the LaTeX document and/or the log file
decide what's left to do to get a properly set result. With latexmk it
is also easier to handle generation of index or glossaries files etc.

Or we try to re-build something like this in Elisp (which would be
nice but quite a lot of work; even if we utilize the lessons learned
from latexmk).

-- 
Until the next mail...,
Stefan.

Re: suggestion to change default org-latex-pdf-process to latexmk

[Scott Randby]

On 6/2/21 12:07 PM, Stefan Nobis wrote> I have
> 
> --8<---------------cut here---------------start------------->8---
> (setq org-latex-pdf-process '("latexmk -f -pdf -%latex -outdir=%o %f"))
> --8<---------------cut here---------------end--------------->8---
> 

I could not get that setting to work. The only setting that worked for me was this one:

--8<---------------cut here---------------start------------->8---
(customize-set-variable 'org-latex-pdf-process '("latexmk -f -pdf -%latex -interaction=nonstopmode -output-directory=%o %f"))
--8<---------------cut here---------------end--------------->8---

I've added the above code to my configuration file.

Scott

Re: suggestion to change default org-latex-pdf-process to latexmk

[Greg Minshall]

Bruce,

> > I really don't understand why the changed is needed.
> 
> I've stated it twice, I believe, but it's now a longer thread, so:
> 
> Org is set to get native citation support.
> 
> Once that's merged, people who use this feature will be disappointed
> to know that when using the natbib or biblatex export processors,
> their citations and bibliographies will not render.
> 
> As in, from their perspective, it will be broken upfront.

thanks.

one important thing will be the manual section describing the new
capability -- it should highlight the desirability of using latexmk ("or
some process with similar behavior"), and describe the results one will
get without it.  presumably, people will initially need to read this
section to figure out the syntax, etc.?

i worry about breaking people's currently working setups.

it seems there are N classes of users:

1.  not using citation engine.
2.  using citation engine
2a. ... and using default latex-to-pdf processing
2b. ... having customized that processing
2bi. ... in a way that works with new citation engine
2bii. .. in a way that does *not* work with new citation engine

class 1 could likely benefit from latexmk, but it's optional.

class 2a is probably the main class that would find it really helpful to
have an automated latexmk.

from *our* (the org-mode provider) point of view, i guess the 2b
subclasses are indistinguishable.  so, if the user finds herself in
2bii, she will have to grep the manual or the web.  again, it would be
good if information in the manual stuck out.  maybe a "TROUBLESHOOTING"
sub-section for the new feature?

i wonder if we could detect 2a, and offer them the customization dialog?
(one would want the user to be able to click the "do not show me this
again" button, or equivalent.)

packaging.  :)

cheers, Greg

Re: suggestion to change default org-latex-pdf-process to latexmk

[Tim Cross]

Colin Baxter <m43cap@yandex.com> writes:

>>>>>> Bruce D'Arcus <bdarcus@gmail.com> writes:
>
>     > On Wed, Jun 2, 2021 at 4:28 AM Tim Cross <theophilusx@gmail.com> wrote:
>     >> The more I think about it, I think the best solution would be to
>     >> update the code which sets the default and have it check for
>     >> latexmk. If it is found, set it as the default and if it is not
>     >> found, set the existing default. This would have minimum impact
>     >> on users and allow those who have installed latexmk to get the
>     >> benefits while not forcing those who don't need it to install it.
>
>     > If this were possible, I agree it would be the best approach.
>
>     > Bruce
>
> What effect is this likely to have on people who have specified latexmk?
> For example, I have
>
> #+begin_src elisp
> (setq org-latex-pdf-process '("latexmk -outdir=%o -f %f"))
> (setq org-latex-compiler "pdflatex")
> #+end_src
>
> I assume any change will have no effect, but is this correct?
>

Correct. Changing any default setting has no impact on anyone who sets
the value to a non-default value.

If the default is changed to be latexmk if latexmk is installed and set
it to the existing default if not will only impact users who have
latexmk installed, but have not configured org-latex-pdf-process to use
it. For those who do not have it installed, it would have no effect.

If on the other hand we just change the default to be latexmk, it will
break org for anyone who does not have latexmk installed and who have
not set org-latex-pdf-program until they install latexmk. For new users,
it means needing to install latexmk (an optional package on some systems
and already installed on others).

As you can call elisp to initialise a variable, it would not be
difficult to add initialising code for org-latex-pdf-program which first
checks to see if latexmk is in the exec patch and if it is, set
org-latex-pdf-program accordingly and if it is not, set the current
default. I think this would have the least impact on all users.



-- 
Tim Cross

Re: suggestion to change default org-latex-pdf-process to latexmk

[Tim Cross]

Scott Randby <srandby@gmail.com> writes:

> On 6/2/21 8:07 AM, Bruce D'Arcus wrote:
>> On Wed, Jun 2, 2021 at 4:28 AM Tim Cross <theophilusx@gmail.com> wrote:
>> 
>>> The more I think about it, I think the best solution would be to update
>>> the code which sets the default and have it check for latexmk. If it is
>>> found, set it as the default and if it is not found, set the existing
>>> default. This would have minimum impact on users and allow those who
>>> have installed latexmk to get the benefits while not forcing those who
>>> don't need it to install it.
>> If this were possible, I agree it would be the best approach.
>
> Perhaps this is a stupid question, but what about the case in which TeX is
> installed after Emacs is installed? I do this sometimes.
>

No impact. The code is initialised each time you load org. If you start
Emacs and then install TeX/LaTeX (including latexmk), you would have the existing default
until the next time you start Emacs, at which time the default will be
set to latexmk. 

This 'pattern' of setting a value based on code which selects an
appropriate value based on what is on the system is already used
extensively in Emacs. 

-- 
Tim Cross

Re: suggestion to change default org-latex-pdf-process to latexmk

[Tim Cross]

"Dr. Arne Babenhauserheide" <arne_bab@web.de> writes:

> [[PGP Signed Part:Undecided]]
>
> Scott Randby <srandby@gmail.com> writes:
>
>> On 6/2/21 8:07 AM, Bruce D'Arcus wrote:
>>> On Wed, Jun 2, 2021 at 4:28 AM Tim Cross <theophilusx@gmail.com> wrote:
>>> 
>>>> The more I think about it, I think the best solution would be to update
>>>> the code which sets the default and have it check for latexmk. If it is
>>>> found, set it as the default and if it is not found, set the existing
>>>> default. This would have minimum impact on users and allow those who
>>>> have installed latexmk to get the benefits while not forcing those who
>>>> don't need it to install it.
>>> If this were possible, I agree it would be the best approach.
>>
>> Perhaps this is a stupid question, but what about the case in which TeX is
>> installed after Emacs is installed? I do this sometimes.
>
> Is there a reason not to have the default check for availability when
> run? It should be fast and would progressively improve (one step) when
> the user installs latexmk.
>

Yes, because some people want to set specific programs. If you have code
which checks for things at the point of being called, this becomes more
complicated than necessary. If on the other hand you just check when
first initialising, users who want to can override with a simple setq.
It is also more efficient as the check is only done once at load time
rather than every time you export to pdf.

It should also be noted that the custom framework has good support for
doing it this way. You can even add code which will check values entered
by the user to verify they exist (for example and assuming they use
custom to set a new value and not just a setq in their init file). 

-- 
Tim Cross

Re: suggestion to change default org-latex-pdf-process to latexmk

[Tim Cross]

Greg Minshall <minshall@umich.edu> writes:

> Bruce,
>
>> > I really don't understand why the changed is needed.
>> 
>> I've stated it twice, I believe, but it's now a longer thread, so:
>> 
>> Org is set to get native citation support.
>> 
>> Once that's merged, people who use this feature will be disappointed
>> to know that when using the natbib or biblatex export processors,
>> their citations and bibliographies will not render.
>> 
>> As in, from their perspective, it will be broken upfront.
>
> thanks.
>
> one important thing will be the manual section describing the new
> capability -- it should highlight the desirability of using latexmk ("or
> some process with similar behavior"), and describe the results one will
> get without it.  presumably, people will initially need to read this
> section to figure out the syntax, etc.?
>
> i worry about breaking people's currently working setups.
>
> it seems there are N classes of users:
>
> 1.  not using citation engine.
> 2.  using citation engine
> 2a. ... and using default latex-to-pdf processing
> 2b. ... having customized that processing
> 2bi. ... in a way that works with new citation engine
> 2bii. .. in a way that does *not* work with new citation engine
>
> class 1 could likely benefit from latexmk, but it's optional.
>
> class 2a is probably the main class that would find it really helpful to
> have an automated latexmk.
>
> from *our* (the org-mode provider) point of view, i guess the 2b
> subclasses are indistinguishable.  so, if the user finds herself in
> 2bii, she will have to grep the manual or the web.  again, it would be
> good if information in the manual stuck out.  maybe a "TROUBLESHOOTING"
> sub-section for the new feature?
>
> i wonder if we could detect 2a, and offer them the customization dialog?
> (one would want the user to be able to click the "do not show me this
> again" button, or equivalent.)
>
> packaging.  :)
>
> cheers, Greg

My suggestion is to do something like


(defvar org-latex-pdf-process (if (executable-find "latexmk")
                                 '("latexmk -f -pdf -%latex -interaction=nonstopmode -output-directory=%o %f")
                               '("%latex -interaction nonstopmode -output-directory %o %f"
                                 "%latex -interaction nonstopmode -output-directory %o %f"
                                 "%latex -interaction nonstopmode -output-directory %o %f")))

This is a simplistic example. It would need some refinement to work
correctly with custom, but the principal is the same. With the above,
the default would be latexmk if latexmk is installed and on the
exec-path, otherwise it will be the default latex.

We can make this as sophisticated as we want. For example, we could also
check to verify the 'default' latex is available and issue a warning to
the user if it isn't or issue a warning if none of the options are
available etc.

I believe something like the above would have the least impact as

1. It would have no impact on anyone who already sets a preference for
org-latex-pdf-process
2. Would provide the advantages of latexmk for those who already have it
installed or once they install it (assuming no preference has been set
by the user)
3. Maintain the status quo for existing users who don't have latexmk
installed, avoiding breakage of their current setup.

Note that this is assuming that latexmk has no other 'side effects' not
raised in this thread. I don't use latexmk, so don't know precisely what
impact it would have, but based on posts from others, it would appear to
be fine. 
-- 
Tim Cross

Re: suggestion to change default org-latex-pdf-process to latexmk

[Colin Baxter]

>>>>> Stefan Nobis <stefan-ml@snobis.de> writes:

    > Colin Baxter <m43cap@yandex.com> writes:
    >> I really don't understand why the changed is needed.

    > The upcoming citation support needs a reliable way to run the
    > required bibliography processor (e.g. biber or bibtex). Therefore
    > the current default setting that only runs latex three times does
    > not suffice.

    > On the other hand running biber unconditionally will result in
    > errors if no citation and bibliography has been used in the
    > document.

    > So we need a more sophisticated method to run the LaTeX engine and
    > associated tools. One way would be to use something that tries to
    > do exactly this: Depending on the LaTeX document and/or the log
    > file decide what's left to do to get a properly set result. With
    > latexmk it is also easier to handle generation of index or
    > glossaries files etc.

    > Or we try to re-build something like this in Elisp (which would be
    > nice but quite a lot of work; even if we utilize the lessons
    > learned from latexmk).

Thank you for this useful explanation. I should have read the whole
thread, but I'm lazy.

I'm old school and use bibtex, and, before I adopted latexmk as a
solution, I remember setting org-latex-pdf-process to run multiple
times (5+) to cope with my setup of multiple bib files and multiple
layers of cross-referencing and citing within the bib files. It didn't
always work so I understand fully the motive behind using latexmk. 

Thanks again - and to others for explaining the thinking.

Best wishes, Colin.

Re: suggestion to change default org-latex-pdf-process to latexmk

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1078 bytes --]


Tim Cross <theophilusx@gmail.com> writes:

> Scott Randby <srandby@gmail.com> writes:
>
>> On 6/2/21 8:07 AM, Bruce D'Arcus wrote:
>>> On Wed, Jun 2, 2021 at 4:28 AM Tim Cross <theophilusx@gmail.com> wrote:
>>> 
>>>> The more I think about it, I think the best solution would be to update
>>>> the code which sets the default and have it check for latexmk. If it is
>>>> found, set it as the default and if it is not found, set the existing
>>>> default. This would have minimum impact on users and allow those who
>>>> have installed latexmk to get the benefits while not forcing those who
>>>> don't need it to install it.
>>> If this were possible, I agree it would be the best approach.
>>
>> Perhaps this is a stupid question, but what about the case in which TeX is
>> installed after Emacs is installed? I do this sometimes.
>>
>
> No impact. The code is initialised each time you load org.

This is the point I missed. Sounds like a good solution to me then.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]