LaTeX > PDF blocked by extender chars in filename

LaTeX > PDF blocked by extender chars in filename

[Eduardo Mercovich]

Dear all.

I hope this is the right place to share this. If not, please 
forgive me for the bothering and point me to the right place.

When a referenced file has an extended character in it's filename, 
the export to pdf function ends without export nor any informative 
error.

I found it by chance... I do most of my work in spanish but I tend 
not to use spaces or extended chars (á, é, í, ó, ú, ñ at least) in 
filenames to simplify potential issues with the OS. However, today 
a "ñ" slipped by and suddently I couldn't export the document to 
pdf. I found no other information in the messages buffer than 
buffername".pdf wasn't produced". It took me some time to bisect 
the origin and finally got it. 

The file > img/ProcesoDeDiseñoDeInteraccion.pdf

was translated/inserted on the org buffer as > 
img/ProcesoDeDise%C3%B1oDeInteraccion.pdf

Is this how it's supposed to be?

Some relevant info:

+ OS: Debian 8.9
+ Emacs: GNU Emacs 24.5.1 of 2016-03-19 on trouble, modified by 
Debian
+ Org-mode: Org-mode version 8.3.4 (8.3.4-88-g792bb9-elpaplus)

locale says:
--8<---------------cut here---------------start------------->8---
LANG=es_AR.UTF-8
LANGUAGE=es_AR:es
LC_CTYPE="es_AR.UTF-8"
LC_NUMERIC="es_AR.UTF-8"
LC_TIME="es_AR.UTF-8"
LC_COLLATE="es_AR.UTF-8"
LC_MONETARY="es_AR.UTF-8"
LC_MESSAGES="es_AR.UTF-8"
LC_PAPER="es_AR.UTF-8"
LC_NAME="es_AR.UTF-8"
LC_ADDRESS="es_AR.UTF-8"
LC_TELEPHONE="es_AR.UTF-8"
LC_MEASUREMENT="es_AR.UTF-8"
LC_IDENTIFICATION="es_AR.UTF-8"
LC_ALL=
--8<---------------cut here---------------end--------------->8---

Is it any other test or info that could be useful? I'd be glad to 
help. 

Thanks a lot, as always, for such a worderful tool and community. 
:)

Best...


-- 
eduardo mercovich

 Donde se cruzan tus talentos 
 con las necesidades del mundo, 
 ahí está tu vocación. 
 (Anónimo)

Re: LaTeX > PDF blocked by extender chars in filename

[Nicolas Goaziou]

Hello,

Eduardo Mercovich <eduardo@mercovich.net> writes:

> The file > img/ProcesoDeDiseñoDeInteraccion.pdf
>
> was translated/inserted on the org buffer as > 
> img/ProcesoDeDise%C3%B1oDeInteraccion.pdf

What command did you use to "translate/insert" the filename?
>
> + Org-mode: Org-mode version 8.3.4 (8.3.4-88-g792bb9-elpaplus)

This release is old. Could you update to a more recent one?

Regards,

-- 
Nicolas Goaziou

Re: LaTeX > PDF blocked by extender chars in filename

[Eduardo Mercovich]

Hola Nicolas.

>> The file > img/ProcesoDeDiseñoDeInteraccion.pdf
>> was translated/inserted on the org buffer as > 
>> img/ProcesoDeDise%C3%B1oDeInteraccion.pdf

> What command did you use to "translate/insert" the filename?

C-c C-l with keyboard and tab completion.

>> + Org-mode: Org-mode version 8.3.4 (8.3.4-88-g792bb9-elpaplus)

> This release is old. Could you update to a more recent one?

Sure. Upgraded from the package manager and now I have 
org-20170821 and org-plus-contrib-20170821. Both from elpa. Would 
it be better from the org repository?

org-version gives: Org mode version 9.0.9 
(9.0.9-88-g251f88-elpaplus [...])


Repeating the same experiment now results in: 

1. Org inserts the same string 
(file:img/ProcesoDeDise%C3%B1oDeInteraccion.pdf)

2. But export is not blocked! :)

3. The linked file is not present on the exported pdf (may not be 
an org issue, but about other latex to pdf component)... 

I hope this is useful.
How do I proceed from here? Any other test or info to report? 

Thanks a lot for your hard work on Org. :)


-- 
eduardo mercovich

 Donde se cruzan tus talentos 
 con las necesidades del mundo, 
 ahí está tu vocación. 
 (Anónimo)

Re: LaTeX > PDF blocked by extender chars in filename

[Thomas S. Dye]

Aloha Eduardo,

In my experience, having both org and org-plus-contrib installed is a
problem.  You might want to settle on the one or the other, then see if
your problem persists.

hth,
Tom

Eduardo Mercovich writes:

> Hola Nicolas.
>
>>> The file > img/ProcesoDeDiseñoDeInteraccion.pdf
>>> was translated/inserted on the org buffer as >
>>> img/ProcesoDeDise%C3%B1oDeInteraccion.pdf
>
>> What command did you use to "translate/insert" the filename?
>
> C-c C-l with keyboard and tab completion.
>
>>> + Org-mode: Org-mode version 8.3.4 (8.3.4-88-g792bb9-elpaplus)
>
>> This release is old. Could you update to a more recent one?
>
> Sure. Upgraded from the package manager and now I have
> org-20170821 and org-plus-contrib-20170821. Both from elpa. Would
> it be better from the org repository?
>
> org-version gives: Org mode version 9.0.9
> (9.0.9-88-g251f88-elpaplus [...])
>
>
> Repeating the same experiment now results in:
>
> 1. Org inserts the same string
> (file:img/ProcesoDeDise%C3%B1oDeInteraccion.pdf)
>
> 2. But export is not blocked! :)
>
> 3. The linked file is not present on the exported pdf (may not be
> an org issue, but about other latex to pdf component)...
>
> I hope this is useful.
> How do I proceed from here? Any other test or info to report?
>
> Thanks a lot for your hard work on Org. :)


--
Thomas S. Dye
http://www.tsdye.com

Re: LaTeX > PDF blocked by extender chars in filename

[Eduardo Mercovich]

Hi Thomas.

> In my experience, having both org and org-plus-contrib installed 
> is a
> problem.  You might want to settle on the one or the other, then 
> see if
> your problem persists.

I didn't knew that, thanks a lot for the recommendation. :)
Based on my (lack of) use of the contributions I removed 
org-plus-contrib and left only org.

The result is still the previously reported: the export is not 
blocked but the embedded pdf isn't included in the exported file 
neither.

What I noted (probably nothing to do with the present report) is 
that if I add the link to the file in a line of it's own, the 
following paragraph appears centered.

Going back to the accented file issue, is this filename 
"translation" supposed to happen like this in org? Is the export 
block happening in the org part, or the latex part?
Is any other test or info I could gather to help identify the 
source of this problem? 

Best...


-- 
eduardo mercovich

 Donde se cruzan tus talentos 
 con las necesidades del mundo, 
 ahí está tu vocación. 
 (Anónimo)

Re: LaTeX > PDF blocked by extender chars in filename

[Nicolas Goaziou]

Hello,

Eduardo Mercovich <eduardo@mercovich.net> writes:

> Repeating the same experiment now results in: 
>
> 1. Org inserts the same string
> (file:img/ProcesoDeDise%C3%B1oDeInteraccion.pdf)
>
> 2. But export is not blocked! :)

Great.

> 3. The linked file is not present on the exported pdf (may not be an
> org issue, but about other latex to pdf component)...

Could you show the produced ".tex" counterpart?

Regards,

-- 
Nicolas Goaziou                                                0x80A93738

Re: LaTeX > PDF blocked by extender chars in filename

[Eduardo Mercovich]

Hello Nicolas.

>> Repeating the same experiment now results in: 
>> 1. Org inserts the same string
>> (file:img/ProcesoDeDise%C3%B1oDeInteraccion.pdf)
>> 2. But export is not blocked! :)

> Great.

>> 3. The linked file is not present on the exported pdf (may not 
>> be an
>> org issue, but about other latex to pdf component)...

> Could you show the produced ".tex" counterpart?

Totally. Just in case, I made a specific org file to test this.

The relevant part of the source org says:
--8<---------------cut here---------------start------------->8---
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas 
a nisl ut dui placerat mattis sit amet ut justo. Phasellus 
faucibus cursus aliquet. Sed blandit mattis gravida. Maecenas 
tincidunt purus blandit sapien congue, vel molestie lacus tempor. 
Donec ut magna auctor, cursus augue id, hendrerit neque. Donec 
viverra elit venenatis ullamcorper malesuada. Etiam at tellus 
maximus, aliquam neque ut, bibendum justo. Aliquam sem nisl, 
tempus vitae placerat at, bibendum a erat. Donec tincidunt auctor 
volutpat. 

[[file:img/ProcesoDeDise%C3%B1oDeInteraccion.pdf]]

Praesent iaculis nisi interdum justo placerat, eu gravida nibh 
posuere. Fusce scelerisque, nisi ut fermentum aliquet, lorem arcu 
finibus arcu, fringilla consectetur erat purus id magna. Etiam vel 
neque id risus volutpat lacinia. Nam non lorem lectus. Ut neque 
ex, sagittis sit amet porta vitae, commodo nec urna. Integer non 
ipsum at dolor eleifend hendrerit in a mi. Vivamus pellentesque 
interdum consectetur. 
--8<---------------cut here---------------end--------------->8---

The tex file (only the relevant part) says:

--8<---------------cut here---------------start------------->8---
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas 
a nisl ut dui placerat mattis sit amet ut justo. Phasellus 
faucibus cursus aliquet. Sed blandit mattis gravida. Maecenas 
tincidunt purus blandit sapien congue, vel molestie lacus tempor. 
Donec ut magna auctor, cursus augue id, hendrerit neque. Donec 
viverra elit venenatis ullamcorper malesuada. Etiam at tellus 
maximus, aliquam neque ut, bibendum justo. Aliquam sem nisl, 
tempus vitae placerat at, bibendum a erat. Donec tincidunt auctor 
volutpat. 

\begin{center}
\includegraphics[width=.9\linewidth]{img/ProcesoDeDise%C3%B1oDeInteraccion.pdf}
\end{center}

Praesent iaculis nisi interdum justo placerat, eu gravida nibh 
posuere. Fusce scelerisque, nisi ut fermentum aliquet, lorem arcu 
finibus arcu, fringilla consectetur erat purus id magna. Etiam vel 
neque id risus volutpat lacinia. Nam non lorem lectus. Ut neque 
ex, sagittis sit amet porta vitae, commodo nec urna. Integer non 
ipsum at dolor eleifend hendrerit in a mi. Vivamus pellentesque 
interdum consectetur.  --8<---------------cut 
here---------------end--------------->8---

The 2nd paragraph, right after the "\end{center}", is centered, 
don't know why.

So, it gets exported to the tex file. But I don't know if that 
"translation" breaks something...

Running "pdflatex TestDeLinkConCaracteresExtendidos.tex" gives 
errors but produces the pdf just as from inside Orgmode.

Now, replacing the specific chars directly in the tex file 
"/ProcesoDeDise%C3%B1oDeInteraccion.pdf" by 
"/ProcesoDeDiseñoDeInteraccion.pdf" (as it is the filename) and 
running again pdflatex gives a not found error (Package pdftex.def 
Error: File `img/ProcesoDeDise�oDeInteraccion.pdf' not found) but 
makes the pdf with a rectangle in place of the pdf diagram. So 
definitively there is something with the filename 
encoding/translation here (sorry if those words are used without 
precision, I'm far outside my domain). 

How is it supposed to be referenced a file with extended chars in 
the filename? Is the issue then with the latex to pdf part, or do 
we have to write such filenames differently?

Thanks a lot for your help and attention to this issue... :)

-- 
eduardo mercovich

 Donde se cruzan tus talentos 
 con las necesidades del mundo, 
 ahí está tu vocación. 
 (Anónimo)

Re: LaTeX > PDF blocked by extender chars in filename

[Adonay Felipe Nogueira]

This somehow happens to be problematic to me too, if for some reason the
inclusion of the file isn't made by Org mode itself.

For example, we know that "#+INCLUDE:" keywords at the start of the line
are Org mode specific includes, and you can place the file references in
the first argument with the literal characters (that is: no need to do
percent encoding/escaping).

However, hyperlinks (that is: those between "[[" and "]]") demand
percent encoding/escaping. But, it seems that the Org-to-LaTeX exporter
isn't translating the hyperlinks to something LaTeX understands (LaTeX
expects literal characters, and "%" is the start of a comment).

Also, as a final note, the default grffile inclusion in the Org-to-LaTeX
(and to PDF also) doesn't include the necessary options to make LaTeX
accept spaces and accents in file names. That's OK for compatibility
reasons, and if you do want to force it to accept such special
characters, use the grffilesetup LaTeX command with the proper grffile
options.

Personally, I like to go the safest route: remove special characters
from file names whenever I don't need them. I generally replace spaces
with underscores, and leave letters without accent. This also avoids
having to deal with the broken percent decoding/unescaping when doing
Org-to-LaTeX exports.


Hope this helps! :)

-- 
- https://libreplanet.org/wiki/User:Adfeno
- Palestrante e consultor sobre /software/ livre (não confundir com
  gratis).
- "WhatsApp"? Ele não é livre. Por favor, use o GNU Ring ou o Tox.
- Contato: https://libreplanet.org/wiki/User:Adfeno#vCard
- Arquivos comuns aceitos (apenas sem DRM): Corel Draw, Microsoft
  Office, MP3, MP4, WMA, WMV.
- Arquivos comuns aceitos e enviados: CSV, GNU Dia, GNU Emacs Org, GNU
  GIMP, Inkscape SVG, JPG, LibreOffice (padrão ODF), OGG, OPUS, PDF
  (apenas sem DRM), PNG, TXT, WEBM.

Re: LaTeX > PDF blocked by extender chars in filename

[Eduardo Mercovich]

Dear Felipe.

> This somehow happens to be problematic to me too [...] However, 
> hyperlinks (that is: those between "[[" and "]]") demand percent 
> encoding/escaping. But, it seems that the Org-to-LaTeX exporter 
> isn't translating the hyperlinks to something LaTeX understands 
> (LaTeX expects literal characters, and "%" is the start of a 
> comment).

Great finding, this seems like a possible culprit...

> Also, as a final note, the default grffile inclusion in the 
> Org-to-LaTeX (and to PDF also) doesn't include the necessary 
> options to make LaTeX accept spaces and accents in file names. 
> That's OK for compatibility reasons, and if you do want to force 
> it to accept such special characters, use the grffilesetup LaTeX 
> command with the proper grffile options.

I didn't knew about grffile, thanks. 
So, does this means that to support extended ascii chars we need 
to include this package? 

> Personally, I like to go the safest route: remove special 
> characters from file names whenever I don't need them. I 
> generally replace spaces with underscores, and leave letters 
> without accent. This also avoids having to deal with the broken 
> percent decoding/unescaping when doing Org-to-LaTeX exports.

I also do that usually. I found this case because a special char 
(ñ) escaped me. ;)

However, given the widespread use of extended ascii chars in many 
languages around the world (and the excellent support that emacs 
in particular and the free/libre software movement in general are 
proud to have for them) it would be good to have something as this 
working without glitches. Also, it may be a showstopper for 
newbies since it may take quite a while to find what is happening. 
After all, if there is no problem with my OS, why would it be an 
issue with emacs?

Except programming that I don't know how to do, I would gladly do 
what it takes to help with this. :)

Thanks Nicolas and Felipe for your attention... :D

-- 
eduardo mercovich

 Donde se cruzan tus talentos 
 con las necesidades del mundo, 
 ahí está tu vocación. 
 (Anónimo)

Re: LaTeX > PDF blocked by extender chars in filename

[Adonay Felipe Nogueira]

The grffile package is already included, in:

[[help:org-latex-default-packages-alist]]

The text above is an Org hyperlink describing the
"org-latex-default-packages-alist" variable.

However, grffile, as it is included, doesn't have any options (this is
indicated with the empty quote-unquote next to the "grffile").

So, for now, in your LaTeX document, do:

#+LATEX_HEADER: grffilesetup{}

... and insert the options for grffile between "{" and "}". For example:

#+LATEX_HEADER: grffilesetup{filenameencoding = utf8, space}

I'm also not a programmer, but I did notice this odd behavior. :)

Re: LaTeX > PDF blocked by extender chars in filename

[Eduardo Mercovich]

Aló Felipe.

> The grffile package is already included, in: 
> [[help:org-latex-default-packages-alist]] The text above is an 
> Org hyperlink describing the "org-latex-default-packages-alist" 
> variable.

Beautiful, loved the link inclusion in the mail, thanks. :)

> However, grffile, as it is included, doesn't have any options 
> (this is indicated with the empty quote-unquote next to the 
> "grffile").

> So, for now, in your LaTeX document, do:
> #+LATEX_HEADER: grffilesetup{}
> ... and insert the options for grffile between "{" and "}". For 
> example:
> #+LATEX_HEADER: grffilesetup{filenameencoding = utf8, space}

It seems that you found the origin... :)

(I had to add a backslash before grffilesetup, as in 
"#+LATEX_HEADER: \grffilesetup{filenameencoding = utf8, space}")

I added this to the file options and tried the experiment again.

+ With the defaul tex file export 
"{img/ProcesoDeDise%C3%B1oDeInteraccion.pdf}" there is no diagram 
in the pdf. 

+ With a small change in the reference 
"{img/ProcesoDeDiseñoDeInteraccion.pdf}" now the pdf includes the 
diagram as it's supposed to be. 

> I'm also not a programmer, but I did notice this odd behavior. 
> :)

My friend, it seems you can see far and clear. :D

Rounding up:
+ if we include the grffilesetup options "{filenameencoding = 
utf8, space}", and
+ insert the filename as it is,
+ the pdf is exported and the referenced file is correctly 
included.

Nicolas, does this means we should modify these little things in 
Org? 


-- 
eduardo mercovich

 Donde se cruzan tus talentos 
 con las necesidades del mundo, 
 ahí está tu vocación. 
 (Anónimo)

Re: LaTeX > PDF blocked by extender chars in filename

[Nicolas Goaziou]

Hello,

Eduardo Mercovich <eduardo@mercovich.net> writes:

> Rounding up:
> + if we include the grffilesetup options "{filenameencoding = utf8,
> space}", and
> + insert the filename as it is,
> + the pdf is exported and the referenced file is correctly included.
>
> Nicolas, does this means we should modify these little things in Org?

Unfortunately these are not "little things". 

First, I assume "filenameencoding" is not necessarily utf8, so it cannot
be a default value.

More importantly, there is an ongoing issue with link encoding, which is
debated in another (moribund) thread. IOW, "insert the filename as it
is" is not easy, because Org needs to encode file names, but doesn't
know for sure when a file name has been encoded.


Regards,

-- 
Nicolas Goaziou

Re: LaTeX > PDF blocked by extender chars in filename

[Eduardo Mercovich]

Hello Nicolas.

[...]
>> Nicolas, does this means we should modify these little things 
>> in Org?

> Unfortunately these are not "little things". 

Sorry, it was my ignorance of the infrastructure on which this 
depends...

> First, I assume "filenameencoding" is not necessarily utf8, so 
> it cannot be a default value.

Ah, super clear.

> More importantly, there is an ongoing issue with link encoding, 
> which is debated in another (moribund) thread. IOW, "insert the 
> filename as it is" is not easy, because Org needs to encode file 
> names, but doesn't know for sure when a file name has been 
> encoded.

Ok, got it. So, what should we do?
Here are some possible things to discuss... 

+ One thing that may be relatively simple to do (I won't assume 
"little things" anymore) is to warn in the documentation that for 
the moment, it's preferable to avoid extended chars in filenames.

+ Other point: is it logical to warn the User in the moment when 
the link is created, that we found extended chars and that may 
complicate things on the export? (I wouldn't recomend to change 
the filename even if from the HCI POV seems logical, since we 
don't know the origin of this filename, among other reasons, and 
could even be re-writed).

+ Does it make sense to check the files in the moment we write the 
latex file, and change the filename/s in that instant? Or at least 
to warn the User then?  

+ Farther in the future is that decision about to encode or not 
the links, in which I can't emit any opinion since it is far from 
my domain.

+ Other possibilities? 

-- 
eduardo mercovich

 Donde se cruzan tus talentos 
 con las necesidades del mundo, 
 ahí está tu vocación. 
 (Anónimo)

Re: LaTeX > PDF blocked by extender chars in filename

[Rasmus]

Hi Adonay,

Adonay Felipe Nogueira <adfeno@hyperbola.info> writes:

> Also, as a final note, the default grffile inclusion in the Org-to-LaTeX
> (and to PDF also) doesn't include the necessary options to make LaTeX
> accept spaces and accents in file names. That's OK for compatibility
> reasons, and if you do want to force it to accept such special
> characters, use the grffilesetup LaTeX command with the proper grffile
> options.

Including the "space" option doesn’t seem to make a difference on my
GNU/Linux system with TeXLive 2017.  The grffile manual suggests that
space is conditionally loaded anyway:

    Therefore option space is only enabled by default, if the supported
    pdfTEX in PDF mode is detected or XƎTEX is running.

As for other options like "extendedchars" I am not sure it’s straight
forward to load by default.

Rasmus

-- 
9000!

Re: LaTeX > PDF blocked by extender chars in filename

[Adonay Felipe Nogueira]

Oh... Are you generating .dvi or .pdf?

Commonly, .dvi is generated by running: `latex [File path.]' or `latexmk
[File path.]'.

And, .pdf is commonly generated by running: `pdflatex [File path.]' or
`latexmk -pdf [File path.]'.

If generating .dvi, the "space" option won't work.

Re: LaTeX > PDF blocked by extender chars in filename

[Adonay Felipe Nogueira]

I forgot to mention...

To tell Emacs Org mode that you want a .pdf, customize the
"org-latex-pdf-process" variable.

To do so, do, visit: [[help:org-latex-pdf-process]] and click the
"customize" link. Near the name, you will see a "Value Menu" buttom, it
has latexmk there.

Re: LaTeX > PDF blocked by extender chars in filename

[Eduardo Mercovich]

Aló Felipe.

> To tell Emacs Org mode that you want a .pdf, customize the 
> "org-latex-pdf-process" variable.

I usually do it from the export menu: C-c C-e l p (export, latex, 
pdf). 

> To do so, do, visit: [[help:org-latex-pdf-process]] and click 
> the "customize" link. Near the name, you will see a "Value Menu" 
> buttom, it has latexmk there.

I still never customized that export, thanks a lot for the 
information (emacs and orgmode depth and power never cease to 
amaze me).
I will investigate the difference between the standard 3 runs of 
latex and latexmk. Of course, any experience about this will be 
gratefully enjoyed. :)

Best regards...


-- 
eduardo mercovich

 Donde se cruzan tus talentos 
 con las necesidades del mundo, 
 ahí está tu vocación. 
 (Anónimo)

Re: LaTeX > PDF blocked by extender chars in filename

[Adonay Felipe Nogueira]

You're welcome! :)

Eduardo Mercovich <eduardo@mercovich.net> writes:

> Aló Felipe.
>
>
> I usually do it from the export menu: C-c C-e l p (export, latex,
> pdf). 
>
>
> I still never customized that export, thanks a lot for the information
> (emacs and orgmode depth and power never cease to amaze me).
> I will investigate the difference between the standard 3 runs of latex
> and latexmk. Of course, any experience about this will be gratefully
> enjoyed. :)
>
> Best regards...