[patch] ox.latex.el: Add missing character warnings

[patch] ox.latex.el: Add missing character warnings

[Juan Manuel Macías]

[-- Attachment #1: Type: text/plain, Size: 611 bytes --]


Rationale for the attached patch: It seems that a common problem that
users have with exporting to LaTeX is unicode characters that cannot be
represented in pdfLaTeX or LuaLaTeX/XelaTeX. In the Unicode TeX engines
the warning is insidious, since the missing character warning is not
preceded by a 'warning' string.

Naturally, the added Org warnings do not solve the problem, but at least
they give some clues on how to properly adjust the document.

Best regards,

Juan Manuel 

-- 
Juan Manuel Macías -- Composición tipográfica, tratamiento de datos, diseño editorial y ortotipografía


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-lisp-ox-latex.el-Add-missing-character-warnings.patch --]
[-- Type: text/x-patch, Size: 1247 bytes --]

From 03c4c94c22f720e38f1ffb180aaa8a23abd90406 Mon Sep 17 00:00:00 2001
From: Juan Manuel Macias <maciaschain@posteo.net>
Date: Mon, 12 Feb 2024 16:10:28 +0100
Subject: [PATCH] lisp/ox-latex.el: Add missing character warnings

* (org-latex-known-warnings): Two missing character warnings are
added: one for LuaLaTeX/XelaTeX and another for pdfLaTeX.
---
 lisp/ox-latex.el | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index cfa2b8178..80d992160 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -1511,6 +1511,8 @@ logfiles to remove, set `org-latex-logfiles-extensions'."
     ("Underfull \\hbox" . "[underfull hbox]")
     ("Overfull \\hbox" . "[overfull hbox]")
     ("Citation.*?undefined" . "[undefined citation]")
+    ("LaTeX Error: Unicode character" . "[character(s) not set up for use with pdflatex. You can run lualatex or xelatex instead]")
+    ("Missing character: There is no" . "[Missing character(s): please load an appropriate font with the fontspec package]")
     ("Undefined control sequence" . "[undefined control sequence]"))
   "Alist of regular expressions and associated messages for the user.
 The regular expressions are used to find possible warnings in the
-- 
2.43.1

Re: [patch] ox.latex.el: Add missing character warnings

[Juan Manuel Macías]

[-- Attachment #1: Type: text/plain, Size: 615 bytes --]

Sorry, the previous patch was incomplete. The attached patch is correct.

Best regards,

Juan Manuel 

Juan Manuel Macías writes:

> Rationale for the attached patch: It seems that a common problem that
> users have with exporting to LaTeX is unicode characters that cannot be
> represented in pdfLaTeX or LuaLaTeX/XelaTeX. In the Unicode TeX engines
> the warning is insidious, since the missing character warning is not
> preceded by a 'warning' string.
>
> Naturally, the added Org warnings do not solve the problem, but at least
> they give some clues on how to properly adjust the document.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-lisp-ox-latex.el-Add-missing-character-warnings.patch --]
[-- Type: text/x-patch, Size: 1825 bytes --]

From 19fb7b81d6ce3a657c86497707f590063b27683c Mon Sep 17 00:00:00 2001
From: Juan Manuel Macias <maciaschain@posteo.net>
Date: Mon, 12 Feb 2024 16:10:28 +0100
Subject: [PATCH] lisp/ox-latex.el: Add missing character warnings

* (org-latex-known-warnings): Two missing character warnings are
added: one for LuaLaTeX/XelaTeX and another for pdfLaTeX.
---
 lisp/ox-latex.el | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index cfa2b8178..da4792c04 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -1511,6 +1511,8 @@ logfiles to remove, set `org-latex-logfiles-extensions'."
     ("Underfull \\hbox" . "[underfull hbox]")
     ("Overfull \\hbox" . "[overfull hbox]")
     ("Citation.*?undefined" . "[undefined citation]")
+    ("LaTeX Error: Unicode character" . "[unicode character(s) not set up for use with pdflatex. You can run lualatex or xelatex instead]")
+    ("Missing character: There is no" . "[Missing character(s): please load an appropriate font with the fontspec package]")
     ("Undefined control sequence" . "[undefined control sequence]"))
   "Alist of regular expressions and associated messages for the user.
 The regular expressions are used to find possible warnings in the
@@ -4435,7 +4437,11 @@ encountered or nil if there was none."
     (save-excursion
       (goto-char (point-max))
       (when (re-search-backward "^[ \t]*This is .*?TeX.*?Version" nil t)
-	(if (re-search-forward "^!" nil t) 'error
+	(if (and
+	     (re-search-forward "^!\\(.+\\)" nil t)
+             ;; This error is passed as missing character warning
+             (not (string-match-p "Unicode character" (match-string 1))))
+            'error
 	  (let ((case-fold-search t)
 		(warnings ""))
 	    (dolist (warning org-latex-known-warnings)
-- 
2.43.1

Re: [patch] ox.latex.el: Add missing character warnings

[Ihor Radchenko]

Juan Manuel Macías <maciaschain@posteo.net> writes:

> Sorry, the previous patch was incomplete. The attached patch is correct.

When I try the patch with a simple file like

Hello. 你好。

I do not see any warnings or errors indicated.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [patch] ox.latex.el: Add missing character warnings

[Juan Manuel Macías]

Ihor Radchenko writes:

> Juan Manuel Macías <maciaschain@posteo.net> writes:
>
>> Sorry, the previous patch was incomplete. The attached patch is correct.
>
> When I try the patch with a simple file like
>
> Hello. 你好。
>
> I do not see any warnings or errors indicated.

How weird... And don't they at least appear in the *Messages* buffer?
With your example, they appear to me with pdfLaTeX, lualatex and XelaTeX
(I have used the default value of org-latex-pdf-process):

https://i.imgur.com/OPOpH5c.png

Best regards,

Juan Manuel 

Re: [patch] ox.latex.el: Add missing character warnings

[Ihor Radchenko]

Juan Manuel Macías <maciaschain@posteo.net> writes:

>> When I try the patch with a simple file like
>>
>> Hello. 你好。
>>
>> I do not see any warnings or errors indicated.
>
> How weird... And don't they at least appear in the *Messages* buffer?
> With your example, they appear to me with pdfLaTeX, lualatex and XelaTeX
> (I have used the default value of org-latex-pdf-process):

I did

1. Install your patch on top of the latest main
2. make repro
3. Open /tmp/1.org and enter
Hello. 你好.
4. C-c C-e l o

In *Messages* I see
For information about GNU Emacs and the GNU system, type C-h C-a.
Org mode version 9.7-pre (release_9.6.18-1196-g8bd0dd @ /home/yantar92/Git/org-mode/lisp/)
(New file)
Making completion list... [3 times]
Loading quail/PY (native compiled elisp)...done
Saving file /tmp/1.org...
Wrote /tmp/1.org
Wrote /tmp/1.tex
Processing LaTeX file 1.tex...
PDF file produced.
Running xdg-open /tmp/1.pdf...done

My `org-latex-pdf-process' is
("latexmk -f -pdf -%latex -interaction=nonstopmode -output-directory=%o %f")
since I have latexmk installed.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [patch] ox.latex.el: Add missing character warnings

[Juan Manuel Macías]

Ihor Radchenko writes:

> Juan Manuel Macías <maciaschain@posteo.net> writes:
>
>>> When I try the patch with a simple file like
>>>
>>> Hello. 你好。
>>>
>>> I do not see any warnings or errors indicated.
>>
>> How weird... And don't they at least appear in the *Messages* buffer?
>> With your example, they appear to me with pdfLaTeX, lualatex and XelaTeX
>> (I have used the default value of org-latex-pdf-process):
>
> I did
>
> 1. Install your patch on top of the latest main
> 2. make repro
> 3. Open /tmp/1.org and enter
> Hello. 你好.
> 4. C-c C-e l o
>
> In *Messages* I see
> For information about GNU Emacs and the GNU system, type C-h C-a.
> Org mode version 9.7-pre (release_9.6.18-1196-g8bd0dd @ /home/yantar92/Git/org-mode/lisp/)
> (New file)
> Making completion list... [3 times]
> Loading quail/PY (native compiled elisp)...done
> Saving file /tmp/1.org...
> Wrote /tmp/1.org
> Wrote /tmp/1.tex
> Processing LaTeX file 1.tex...
> PDF file produced.
> Running xdg-open /tmp/1.pdf...done
>
> My `org-latex-pdf-process' is
> ("latexmk -f -pdf -%latex -interaction=nonstopmode -output-directory=%o %f")
> since I have latexmk installed.

I have done the same, on a clean Emacs init. Warning appears.

And in *Messages*:

PDF file produced with warnings: [unicode character(s) not set up for use with pdflatex. You can run lualatex or xelatex instead]

In *Org PDF LaTeX Output*:

! LaTeX Error: Unicode character 你 (U+4F60)
               not set up for use with LaTeX.

(this error is the one that passes as a warning in my patch when
compiled with pdfLaTeX).

Re: [patch] ox.latex.el: Add missing character warnings

[Ihor Radchenko]

Juan Manuel Macías <maciaschain@posteo.net> writes:

>>>> When I try the patch with a simple file like
>>>>
>>>> Hello. 你好。
>>>>
>>>> I do not see any warnings or errors indicated.
>>
>> I did
>> ...
>
> I have done the same, on a clean Emacs init. Warning appears.
>
> And in *Messages*:
>
> PDF file produced with warnings: [unicode character(s) not set up for use with pdflatex. You can run lualatex or xelatex instead]
>
> In *Org PDF LaTeX Output*:
>
> ! LaTeX Error: Unicode character 你 (U+4F60)
>                not set up for use with LaTeX.
>
> (this error is the one that passes as a warning in my patch when
> compiled with pdfLaTeX).

Probably something to do with my Texlive technically having Chinese
support.

I am getting

! Package inputenc Error: Unicode character 你 (U+4F60)
(inputenc)                not set up for use with LaTeX.

See the inputenc package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                              
                                                  
l.28 Hello. 你
               好.

! Package inputenc Error: Unicode character 好 (U+597D)
(inputenc)                not set up for use with LaTeX.

See the inputenc package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                              
                                                  
l.28 Hello. 你好


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [patch] ox.latex.el: Add missing character warnings

[Juan Manuel Macías]

[-- Attachment #1: Type: text/plain, Size: 1089 bytes --]

Ihor Radchenko writes:

> Probably something to do with my Texlive technically having Chinese
> support.
>
> I am getting
>
> ! Package inputenc Error: Unicode character 你 (U+4F60)
> (inputenc)                not set up for use with LaTeX.
>
> See the inputenc package documentation for explanation.
> Type  H <return>  for immediate help.
>  ...                                              
>                                                   
> l.28 Hello. 你
>                好.
>
> ! Package inputenc Error: Unicode character 好 (U+597D)
> (inputenc)                not set up for use with LaTeX.
>
> See the inputenc package documentation for explanation.
> Type  H <return>  for immediate help.
>  ...                                              
>                                                   
> l.28 Hello. 你好

I have fixed the org-latex-known-warnings regexp in the attached patch.
I think it should work fine now...

-- 
Juan Manuel Macías -- Composición tipográfica, tratamiento de datos, diseño editorial y ortotipografía


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-lisp-ox-latex.el-Add-missing-character-warnings.patch --]
[-- Type: text/x-patch, Size: 1816 bytes --]

From 742d67f2e8d4f1896d89f7543948facd65687ffe Mon Sep 17 00:00:00 2001
From: Juan Manuel Macias <maciaschain@posteo.net>
Date: Tue, 13 Feb 2024 16:56:23 +0100
Subject: [PATCH] lisp/ox-latex.el: Add missing character warnings

* (org-latex-known-warnings): Two missing character warnings are
added: one for LuaLaTeX/XelaTeX and another for pdfLaTeX.
---
 lisp/ox-latex.el | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index cfa2b8178..2fdc2afe8 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -1511,6 +1511,8 @@ logfiles to remove, set `org-latex-logfiles-extensions'."
     ("Underfull \\hbox" . "[underfull hbox]")
     ("Overfull \\hbox" . "[overfull hbox]")
     ("Citation.*?undefined" . "[undefined citation]")
+    ("^!.+Unicode character" . "[unicode character(s) not set up for use with pdflatex. You can run lualatex or xelatex instead]")
+    ("Missing character: There is no" . "[Missing character(s): please load an appropriate font with the fontspec package]")
     ("Undefined control sequence" . "[undefined control sequence]"))
   "Alist of regular expressions and associated messages for the user.
 The regular expressions are used to find possible warnings in the
@@ -4435,7 +4437,11 @@ encountered or nil if there was none."
     (save-excursion
       (goto-char (point-max))
       (when (re-search-backward "^[ \t]*This is .*?TeX.*?Version" nil t)
-	(if (re-search-forward "^!" nil t) 'error
+	(if (and
+	     (re-search-forward "^!\\(.+\\)" nil t)
+             ;; This error is passed as missing character warning
+             (not (string-match-p "Unicode character" (match-string 1))))
+            'error
 	  (let ((case-fold-search t)
 		(warnings ""))
 	    (dolist (warning org-latex-known-warnings)
-- 
2.43.1

Re: [patch] ox.latex.el: Add missing character warnings

[Ihor Radchenko]

Juan Manuel Macías <maciaschain@posteo.net> writes:

> I have fixed the org-latex-known-warnings regexp in the attached patch.
> I think it should work fine now...

Thanks!
Applied, onto main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=9eec4af62

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>