emacs-orgmode@gnu.org archives
help / color / mirror / code / Atom feed
* [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX
@ 2022-10-15 21:35 Juan Manuel Macías
2022-10-16  3:24  Ihor Radchenko
2022-10-16 15:04  Max Nikulin
0 siblings, 2 replies; 53+ messages in thread
From: Juan Manuel Macías @ 2022-10-15 21:35 UTC (permalink / raw)
To: orgmode

The recent addition of the org-latex-line-break-safe' constant makes it
impossible to compile verse blocks. The reason is that now a \\ is
inserted between each stanza (aka each paragraph), instead of the
\vspace{1em}' as before. That's wrong, as this compile error message
says:

---------
A \newline or \\ command appears between paragraphs, where it makes no
sense. If you're trying to leave a blank line'', use a \vspace
command.
---------

You can test yourself with this example:

#+begin_verse
lorem ipsum dolor
lorem ipsum dolor

lorem ipsum dolor
lorem ipsum dolor

lorem ipsum dolor
lorem ipsum dolor
#+end_verse

On the other hand, I have a few reservations about the usefulness of
org-latex-line-break-safe. To begin with, it is a particular solution
applied in a general way. This can have unexpected consequences, as has
happened in verse blocks. It's like putting your whole body in a
cast because of a broken finger.

I think if the reason for this addition is to prevent problems with
square brackets, in certain contexts, why not just apply a point
solution? For example, putting {[}, which is a common LaTeX way to
protect a square bracket. Use a macro, for example, or even
define a new entity, something like:

(setq org-entities-user
'(("lbrack" "{[}" nil "[" "[" "[" "[")
("rbrack" "{]}" nil "]" "]" "]" "]")))

And finally, I think that applying a general solution to this problem is
something that should be done(IMHO) in LaTeX and not in Org, for
example, with some new package that would protect certain signs or

Best regards,

Juan Manuel

--
--
------------------------------------------------------
Juan Manuel Macías

https://juanmanuelmacias.com

https://lunotipia.juanmanuelmacias.com

https://gnutas.juanmanuelmacias.com

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX
2022-10-15 21:35 [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX Juan Manuel Macías
@ 2022-10-16  3:24  Ihor Radchenko
2022-10-16 12:08    Juan Manuel Macías
2022-10-16 15:04  Max Nikulin
To: Juan Manuel Macías; +Cc: orgmode

Juan Manuel Macías <maciaschain@posteo.net> writes:

> The recent addition of the org-latex-line-break-safe' constant makes it
> impossible to compile verse blocks. The reason is that now a \\ is
> inserted between each stanza (aka each paragraph), instead of the
> \vspace{1em}' as before. That's wrong, as this compile error message
> says:

Thanks for reporting!
Fixed on main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=5fa66c7ffc4efbea11357fe0d05bc570855cfa91

This was an omission when replacing "\\" instances with safer version in
ox-latex.el.

> On the other hand, I have a few reservations about the usefulness of
> org-latex-line-break-safe. To begin with, it is a particular solution
> applied in a general way. This can have unexpected consequences, as has
> happened in verse blocks. It's like putting your whole body in a
> cast because of a broken finger.

The idea is to use "\\\someescapeLaTeXcommand" rather than "\\". The
latter can take arguments unexpectedly. See
https://orgmode.org/list/8735c0kpb9.fsf@localhost

> I think if the reason for this addition is to prevent problems with
> square brackets, in certain contexts, why not just apply a point
> solution? For example, putting {[}, which is a common LaTeX way to
> protect a square bracket.

Because people export tables not only to LaTeX, but also to other
formats. {[} makes no sense then.

> Use a macro, for example, or even
> define a new entity, something like:
>
> (setq org-entities-user
>       '(("lbrack" "{[}" nil "[" "[" "[" "[")
> 	("rbrack" "{]}" nil "]" "]" "]" "]")))

It will be unclear when to use such entity. Note that it is not obvious
at all that LaTeX export specifically will stumble on

| [1, 2] |
| [3, 4] |

without knowing LaTeX internals and checking the exported LaTeX
intermediate source closely.

We already ask users to deal with Org parser oddities. If we also demand
knowing LaTeX oddities, it will be too much. Better modify parser to
export valid Org syntax into valid LaTeX.

> And finally, I think that applying a general solution to this problem is
> something that should be done(IMHO) in LaTeX and not in Org, for
> example, with some new package that would protect certain signs or
> through ad hoc LaTeX/Lua code.

Either way, we will need to use the new package in ox-latex.el. And our
current approach is somewhat better because users can still use direct
LaTeX @@latex:\\@@ if they want to get "\\" exactly.

--
Org mode contributor,
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX
2022-10-16  3:24  Ihor Radchenko
@ 2022-10-16 12:08    Juan Manuel Macías
0 siblings, 0 replies; 53+ messages in thread
From: Juan Manuel Macías @ 2022-10-16 12:08 UTC (permalink / raw)

>> And finally, I think that applying a general solution to this problem is
>> something that should be done(IMHO) in LaTeX and not in Org, for
>> example, with some new package that would protect certain signs or
>> through ad hoc LaTeX/Lua code.
>
> Either way, we will need to use the new package in ox-latex.el. And our
> current approach is somewhat better because users can still use direct
> LaTeX @@latex:\\@@ if they want to get "\\" exactly.

I'm afraid we differ completely on what Org's priorities are at this
point. The way I see it, Org shouldn't try to solve LaTeX problems,
because Org isn't proficient in LaTeX. This is a known issue in LaTeX,
where a leading bracket can be confused with the optional argument of a
preceding macro (such as \\ or \linebreak). And it doesn't just happen
in this context. It can also happen with environments that have optional
arguments, although the case of \\ is the most common and the case of
environments depends on how the environment is defined. For this
problem, LaTeX has been recommending various *specific* solutions
(because the probability of it appearing is very low, luckily), such as
enclosing the brackets in two {} or putting a \relax before .

In this case Org intends to solve the LaTeX problem, and it does it, in
my opinion, poorly, by generalizing a particular solution for what is
even an extremely rare problem. This is the typical case that can be
easily solved by the user with a macro or an export filter, as I have
been doing for a long time, and at no time have I needed anything else.

Org must worry about exporting well and offering the necessary tools for
that to happen, as it has been doing until now. And the duty of the
users is to know reasonably the language to which they export, to the
extent of what they want to obtain when they export. Org should not be a
substitute for LaTeX: whoever wants to get special things in LaTeX, must
know LaTeX. It's that easy. If Org is going to start trying to solve at
its own risk all the *specific* problems that can appear in the typical
compilation of a LaTeX document, then I prefer to stop using Org and use
pure LaTeX from the beginning.

And as for the ox-latex.el file, in its current state it is completely
unusable for me. At least the user could be offered the possibility of
choosing.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX
2022-10-15 21:35 [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX Juan Manuel Macías
2022-10-16  3:24  Ihor Radchenko
@ 2022-10-16 15:04  Max Nikulin
2022-10-16 16:33    Verse block and separations (was: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX) Juan Manuel Macías
2022-10-16 17:14    Line breaks and brackets in LaTeX export (was: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX) Juan Manuel Macías
1 sibling, 2 replies; 53+ messages in thread
From: Max Nikulin @ 2022-10-16 15:04 UTC (permalink / raw)
To: emacs-orgmode

On 16/10/2022 04:35, Juan Manuel Macías wrote:
>
> To begin with, it is a particular solution
> applied in a general way. This can have unexpected consequences, as has
> happened in verse blocks.

In my opinion Org has to apply a general solution merely because a table
may be result of evaluation of an org-babel code block. I may be wrong,
but I suspect that some users do not care what is the intermediate
format when they need a PDF file.

> And finally, I think that applying a general solution to this problem is
> something that should be done(IMHO) in LaTeX and not in Org,

They may be just text of a table cell or contents of an export snippet.
In a LaTeX file they are just square brackets and there is no way to
distinguish them.

I believe, it is a design flaw in LaTeX that the \\ command does not
have a counterpart with no optional * and [] arguments. For humans \\ is
enough, but it is fragile when LaTeX is used as an intermediate export
format. A part of the problem is that all 3rd party packages should be
adapted for the robust \\ sibling. Likely it may be solved by additional
pass in the exporter code but it will significantly increase its
complexity. I have no idea what is appropriate place to discuss such
issue with LaTeX developers.

The following is irrelevant to the recent changes. I have tried

---- >8 ----
text
#+begin_verse

a b
c d

e f
g h
#+end_verse
---- 8< ----

With the fix Ihor committed today I have got

---- >8 ----
text
\begin{verse}
\vspace*{1em}
a b\\\empty
c d\\\empty
\vspace*{1em}
e f\\\empty
g h\\\empty
\end{verse}
---- 8< ----

My expectation was
---- >8 ----
\begin{verse}
a b\\\empty
c d

e f\\\empty
g h
\end{verse}
---- 8< ----

I am surprised that \vspace is added instead of empty line between
stanzas. Is there a reason to override LaTeX defaults? If such reason
exists would not it better to add

\parskip=1em plus 0.5em minus 0.25em\relax

before first verse line? Is hard coded rigid vertical space acceptable
when high quality output is desired? \vspace before first line looks
like a bug. Frankly speaking, nested calls of replace-regexp-in-string'
makes the code hard to read.

P.S. I have not found exact citation, but I noticed a mention: Lamport
expected that verse environment would get negative feedback from poets.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Verse block and separations (was: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX)
2022-10-16 15:04  Max Nikulin
@ 2022-10-16 16:33    Juan Manuel Macías
2022-10-17  8:54      Ihor Radchenko
(2 more replies)
2022-10-16 17:14    Line breaks and brackets in LaTeX export (was: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX) Juan Manuel Macías
1 sibling, 3 replies; 53+ messages in thread
From: Juan Manuel Macías @ 2022-10-16 16:33 UTC (permalink / raw)
To: Max Nikulin; +Cc: emacs-orgmode

Max Nikulin writes:

> The following is irrelevant to the recent changes. I have tried
>
> ---- >8 ----
> text
>
> #+begin_verse
>
> a b
> c d
>
> e f
> g h
> #+end_verse
>
> ---- 8< ----
>
> With the fix Ihor committed today I have got
>
> ---- >8 ----
> text
> \begin{verse}
> \vspace*{1em}
> a b\\\empty
> c d\\\empty
> \vspace*{1em}
> e f\\\empty
> g h\\\empty
> \end{verse}
> ---- 8< ----
>
> My expectation was
> ---- >8 ----
> \begin{verse}
> a b\\\empty
> c d
>
> e f\\\empty
> g h
> \end{verse}
> ---- 8< ----
>
> I am surprised that \vspace is added instead of empty line between
> stanzas. Is there a reason to override LaTeX defaults? If such reason
> exists would not it better to add
>
>      \parskip=1em plus 0.5em minus 0.25em\relax
>
> before first verse line? Is hard coded rigid vertical space acceptable
> when high quality output is desired? \vspace before first line looks
> like a bug. Frankly speaking, nested calls of replace-regexp-in-string'
> makes the code hard to read.
>
> P.S. I have not found exact citation, but I noticed a mention: Lamport
> expected that verse environment would get negative feedback from poets.

First of all, I'm afraid that in my previous post I mixed up two
different issues: the verse block bug and my personal opinions about the
new added constant. I should have separated these topics, sorry.

I answer here what you comment about the verse blocks and later I will
open a different thread for the other issue.

Yes, you're right: that \vspace is the normal behavior of the block when
exporting to LaTeX, and it is certainly not quite correct. In LaTeX,
both the out-of-the-box verse environment and the one provided by the
'verse' package (which, typographically speaking, is the correct way to
represent verses in LaTeX) do not add a \vspace between stanzas. In the
LaTeX input the separation between stanzas is expressed by an (at least)
empty line. In fact, that space can be controlled by the verse package
with the stanzaskip \length. I remember that I commented on this anomaly
here on the list, a long time ago, but I can't find the original
message...

In my init I have redefined the verse block like this, so that there is
a white line between stanzas, not \vspace (I have also added some things
for the verse package, so that the numbering of verses is not broken).
So your example would produce a white line under \begin{verse}, not the
current \vspace, which would be irrelevant to LaTeX. But this is only a
hack:

(defun org-latex-verse-block (verse-block contents info)
"Transcode a VERSE-BLOCK element from Org to LaTeX.
CONTENTS is verse block contents.  INFO is a plist holding
contextual information."
(let*
(attr (concat
(if cent "[\\versewidth]" "")
(if lin (format "\n\\poemlines{%s}" lin) "")
(if latcode (format "\n%s" latcode) "")))
(vwidth (if versewidth (format "\\settowidth{\\versewidth}{%s}\n" versewidth) ""))
(linreset (if lin "\n\\poemlines{0}" "")))
(org-latex--wrap-label
verse-block
;; In a verse environment, add a line break to each newline
;; character and change each white space at beginning of a line
;; into a space of 1 em.  Also change each blank line with
;; a vertical space of 1 em.
(format "%s\\begin{verse}%s\n%s\\end{verse}%s"
vwidth
attr
(replace-regexp-in-string
"^[ \t]+" (lambda (m) (format "\\hspace*{%dem}" (length m)))
(replace-regexp-in-string
"\$$\\\\\\\\\n\$$+\$$[ \t]*\\\\\\\\\$$+" "\n"
(replace-regexp-in-string
"\$$[ \t]*\\\\\\\\\$$?[ \t]*\n" "\\\\\n"
(setq contents
(if lin
(replace-regexp-in-string "\$$\n\$$+\$$[ \t]*\n\$$+" "\\\\\\\\!\n\n"
contents)
contents)) nil t) nil t) nil t) linreset)
info)))

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Line breaks and brackets in LaTeX export (was: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX)
2022-10-16 15:04  Max Nikulin
2022-10-16 16:33    Verse block and separations (was: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX) Juan Manuel Macías
@ 2022-10-16 17:14    Juan Manuel Macías
2022-10-17  9:04      Ihor Radchenko
From: Juan Manuel Macías @ 2022-10-16 17:14 UTC (permalink / raw)
To: Max Nikulin; +Cc: emacs-orgmode

Max Nikulin writes:

> In my opinion Org has to apply a general solution merely because a table
> may be result of evaluation of an org-babel code block. I may be wrong,
> but I suspect that some users do not care what is the intermediate
> format when they need a PDF file.
>
>> And finally, I think that applying a general solution to this problem is
>> something that should be done(IMHO) in LaTeX and not in Org,
>
> Org has a bit more information concerning context of square brackets.
> They may be just text of a table cell or contents of an export snippet.
> In a LaTeX file they are just square brackets and there is no way to
> distinguish them.
>
> I believe, it is a design flaw in LaTeX that the \\ command does not
> have a counterpart with no optional * and [] arguments. For humans \\ is
> enough, but it is fragile when LaTeX is used as an intermediate export
> format. A part of the problem is that all 3rd party packages should be
> adapted for the robust \\ sibling. Likely it may be solved by additional
> pass in the exporter code but it will significantly increase its
> complexity. I have no idea what is appropriate place to discuss such
> issue with LaTeX developers.

The solution of adding \relax or \empty is a good one. The problem is
when it is applied massively and indiscriminately, bearing in mind that
it is to prevent a problem that is, frankly, very rare. \empty is
unlikely to have any side effects, but still, I don't like Org making
these decisions for me and adding LaTeX code that I didn't ask for, in
form or quantity. Isn't it possible to be more selective, like Pandoc
does (I think), and add what needs to be added only when the problem is
present? I don't know if the Pandoc thing has already been mentioned in
past discussions, because these days I haven't been paying attention to
the list.

In any case, I would prefer: a) a selective solution; and if a) is not
possible, at least b) a defcustom.

This is an example I came up with of how it could be fixed selectively
in LaTeX (big, big caveat: it's just a proof of concept and likely to
produce errors in other contexts. I think if a LaTeX package to fix this
isn't already out, it is either because the problem is very rare and the
solution for particular cases is known, or because there are drawbacks
inherent to LaTeX that I can't figure out right now):

\makeatletter

\def\my@protectlbrack{
\@ifnextchar{[}{\relax}{}
\@ifnextchar{*}{\relax}{}
}

\let\old@break\\
\def\\{%
\old@break\my@protectlbrack}

%% for tables

\let\old@tabularcr\@tabularcr
\def\@tabularcr{%
\old@tabularcr\my@protectlbrack}

% for use the old commands

\newcommand\oldbreak{\old@break}
\newcommand\oldbreakt{\old@tabularcr}

\makeatother

\begin{document}

lorem\\
[ipsum]

lorem\\
*ipsum

\begin{tabular}{cccc}
a & a & a & a \\
[a] & a & a & a \\
*a & a & a & a \\
\end{tabular}

\end{document}

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Verse block and separations (was: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX)
2022-10-16 16:33    Verse block and separations (was: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX) Juan Manuel Macías
@ 2022-10-17  8:54      Ihor Radchenko
2022-10-18  9:39        Verse block and separations Juan Manuel Macías
2022-10-17 14:48      Verse block and separations (was: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX) Max Nikulin
2022-10-19 11:08      Max Nikulin
To: Juan Manuel Macías; +Cc: Max Nikulin, emacs-orgmode

Juan Manuel Macías <maciaschain@posteo.net> writes:

> Yes, you're right: that \vspace is the normal behavior of the block when
> exporting to LaTeX, and it is certainly not quite correct. In LaTeX,
> both the out-of-the-box verse environment and the one provided by the
> 'verse' package (which, typographically speaking, is the correct way to
> represent verses in LaTeX) do not add a \vspace between stanzas. In the
> LaTeX input the separation between stanzas is expressed by an (at least)
> empty line. In fact, that space can be controlled by the verse package
> with the stanzaskip \length. I remember that I commented on this anomaly
> here on the list, a long time ago, but I can't find the original
> message...

I checked git log and the current approach for verse export dates back
to initial versions of ox-latex. I am pretty sure that it should be
improved.

> In my init I have redefined the verse block like this, so that there is
> a white line between stanzas, not \vspace (I have also added some things
> for the verse package, so that the numbering of verses is not broken).
> So your example would produce a white line under \begin{verse}, not the
> current \vspace, which would be irrelevant to LaTeX. But this is only a
> hack:

Could you provide a patch?

--
Org mode contributor,
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Line breaks and brackets in LaTeX export (was: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX)
2022-10-16 17:14    Line breaks and brackets in LaTeX export (was: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX) Juan Manuel Macías
@ 2022-10-17  9:04      Ihor Radchenko
2022-10-17 11:30        Line breaks and brackets in LaTeX export Juan Manuel Macías
To: Juan Manuel Macías, Daniel Fleischer; +Cc: Max Nikulin, emacs-orgmode

Juan Manuel Macías <maciaschain@posteo.net> writes:

> The solution of adding \relax or \empty is a good one. The problem is
> when it is applied massively and indiscriminately, bearing in mind that
> it is to prevent a problem that is, frankly, very rare. \empty is
> unlikely to have any side effects, but still, I don't like Org making
> these decisions for me and adding LaTeX code that I didn't ask for, in
> form or quantity. Isn't it possible to be more selective, like Pandoc
> does (I think), and add what needs to be added only when the problem is
> present? I don't know if the Pandoc thing has already been mentioned in
> past discussions, because these days I haven't been paying attention to
> the list.
>
> In any case, I would prefer: a) a selective solution; and if a) is not
> possible, at least b) a defcustom.

I tried to add the safe version of page break everywhere where there is
a risk of a page break followed by square brackets.

The key point: A valid Org file must be exported to a valid LaTeX file.

If you know a selective solution, patches are welcome. (I am also CCing
Daniel).

I see no issue with defcustom in general, but can you explain what
practical problems it is going to solve? I am not talking about
aesthetics of the exported LaTeX.

> This is an example I came up with of how it could be fixed selectively
> in LaTeX (big, big caveat: it's just a proof of concept and likely to
> produce errors in other contexts. I think if a LaTeX package to fix this
> isn't already out, it is either because the problem is very rare and the
> solution for particular cases is known, or because there are drawbacks
> inherent to LaTeX that I can't figure out right now):
>
> \makeatletter
>
> \def\my@protectlbrack{
>   \@ifnextchar{[}{\relax}{}
>   \@ifnextchar{*}{\relax}{}
> }
>
> \let\old@break\\
> \def\\{%
>   \old@break\my@protectlbrack}
>
> %% for tables
>
> \let\old@tabularcr\@tabularcr
> \def\@tabularcr{%
>   \old@tabularcr\my@protectlbrack}
>
> % for use the old commands
>
> \newcommand\oldbreak{\old@break}
> \newcommand\oldbreakt{\old@tabularcr}
>
> \makeatother
>
> \begin{document}
>
> lorem\\
> [ipsum]
>
> lorem\\
> *ipsum
>
> \begin{tabular}{cccc}
> a & a & a & a \\
> [a] & a & a & a \\
> *a & a & a & a \\
> \end{tabular}
>
>
> \end{document}

Won't it make it impossible to use
@@latex:\\@@
@@latex:[1em]@@

deliberately?

Note that we may also replace \\\empty with a single custom command. It
will look a bit cleaner. Not sure if it is any different from your
approach.

--
Org mode contributor,
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Line breaks and brackets in LaTeX export
2022-10-17  9:04      Ihor Radchenko
@ 2022-10-17 11:30        Juan Manuel Macías
2022-10-17 11:47          Ihor Radchenko
2022-10-17 15:01          Juan Manuel Macías
0 siblings, 2 replies; 53+ messages in thread
From: Juan Manuel Macías @ 2022-10-17 11:30 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: Daniel Fleischer, Max Nikulin, emacs-orgmode

> I see no issue with defcustom in general, but can you explain what
> practical problems it is going to solve? I am not talking about
> aesthetics of the exported LaTeX.

In my opinion it is not so much a question of aesthetics as of (let's
say) a certain conformity with what a LaTeX document should be. I think,
for example, in the case of a user who must submit his/her work in LaTeX
format to a publication, following the rules of that publication. It is
one thing for a LaTeX document to compile without error and quite
another for a LaTeX document to be or not to be formed according to good
practice. Would there be a problem sharing LaTeX documents with
unnecessary commands like \empty after all occurrences of \\?

Anyway. As for the compilation, it is highly unlikely that \empty will
cause any unexpected error. But LaTeX and its over 6000 packages is
unpredictable. It also seemed unlikely that \relax would cause any
problems, and catching up on the last discussion, it had to be replaced
by \empty because it returned an error just before \hline. \relax is one
of the recommended solutions from LaTeX, because it tells LaTeX that the
previous macro has finished expanding, but it is recommended keeping in
mind that the user will apply it only when needed, not everywhere. And
before \hline it doesn't make sense because there will never be an
a display equation "0pt]"

> I propose the following:
> 1. Merge my patch with \$0pt] safe page breaks > 2. Modify org-latex-template to replace unnecessary occurrences of > "\\[0pt]" in CONTENTS when org-latex-compact-latex (you may propose > other defcustom names) is non-nil. I believe, it is better to introduce a list of filter functions that may perform such optimizing. The only problem that I see with such approach is that if defined as defcustom then users changed its value will not see default filters added in new Org version. If an alist mapping hook name to enabled/disabled state is used then I am unsure how to determine order of hooks during mixing of standard and user defined functions. I raised a similar question during discussion of org-file-apps. I do not have a better proposal. ^ permalink raw reply [flat|nested] 53+ messages in thread * Re: Line breaks and brackets in LaTeX export 2022-10-19 3:57  Ihor Radchenko 2022-10-19 5:11  Max Nikulin @ 2022-10-29 2:36  Ihor Radchenko 2022-11-13 17:01  Max Nikulin 1 sibling, 1 reply; 53+ messages in thread From: Ihor Radchenko @ 2022-10-29 2:36 UTC (permalink / raw) To: Juan Manuel Macías Cc: Daniel Fleischer, Max Nikulin, emacs-orgmode, Ihor Radchenko, Vikas Rawal Ihor Radchenko <yantar92@posteo.net> writes: > Then [0pt] should it be. At least for now, before we have a cleaner > solution. > > See the attached patch. Applied onto main. https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=b93a61af9c93d21c56cf883630e52f36076e40bd I will look into filtering out unnecessary [0pt] instances later. The patch here is more urgent as it avoids the known case of breakage. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 53+ messages in thread * Re: Line breaks and brackets in LaTeX export 2022-10-22 15:55  Max Nikulin @ 2022-11-01 1:51  Ihor Radchenko 2022-11-01 16:07  Max Nikulin 0 siblings, 1 reply; 53+ messages in thread From: Ihor Radchenko @ 2022-11-01 1:51 UTC (permalink / raw) To: Max Nikulin; +Cc: emacs-orgmode Max Nikulin <manikulin@gmail.com> writes: > On 22/10/2022 12:15, Ihor Radchenko wrote: >> as long as the next line does not match >> "^[ \t]*\\[" > > Verse package defines \\! and \\>. > > The only precaution that search pattern should ignore \\\[0pt]$ that is
> a display equation "0pt]"

>> 2. Modify org-latex-template to replace unnecessary occurrences of
>>     "\$0pt]" in CONTENTS when org-latex-compact-latex (you may propose >> other defcustom names) is non-nil. > > I believe, it is better to introduce a list of filter functions that may > perform such optimizing. I don't think so. Users can already define filters. What we discuss here is a built-in filter. It should be separate from user customization and only expose on/off options. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 53+ messages in thread * Re: Line breaks and brackets in LaTeX export 2022-11-01 1:51  Ihor Radchenko @ 2022-11-01 16:07  Max Nikulin 2022-11-02 6:44  Ihor Radchenko 0 siblings, 1 reply; 53+ messages in thread From: Max Nikulin @ 2022-11-01 16:07 UTC (permalink / raw) To: emacs-orgmode On 01/11/2022 08:51, Ihor Radchenko wrote: > Max Nikulin writes: > >> On 22/10/2022 12:15, Ihor Radchenko wrote: >>> as long as the next line does not match >>> "^[ \t]*\\[" >> >> Verse package defines \\! and \\>. >> >> The only precaution that search pattern should ignore \\\[0pt]$ that is
>> a display equation "0pt]"
>

LaTeX verse package require "\\!" as stanza separator to get proper line
count, so square bracket on the next line is not the only character that
may change meaning of "\\". So "[*!>" (depending on context) should be
handled.

Search pattern should not match odd number of backslashes. It is a
common mistake to not care what character precedes the pattern.
Accordingly to Org syntax \\ line break may appear at the end of line
only, but a user may add for some reason @@latex:...@.

Another corner case when "\\[0pt]\n" should not be blindly replaced to
"\\\n" (I do not escape "\"). Imagine that you are going to describe
this optimizing filter

---- >8 ----
Optimizing filter will replace
#+begin_example
First\\[0pt]
Second
#+end_example
with
#+begin_example
First\\
Second
#+end_example
---- 8< -----

>>> 2. Modify org-latex-template to replace unnecessary occurrences of
>>>      "\\[0pt]" in CONTENTS when org-latex-compact-latex (you may propose
>>>      other defcustom names) is non-nil.
>>
>> I believe, it is better to introduce a list of filter functions that may
>> perform such optimizing.
>
> I don't think so. Users can already define filters. What we discuss here
> is a built-in filter. It should be separate from user customization and
> only expose on/off options.

If a separate defcustom is used for each built-in filter then it will be
inconvenient to review which filters are active from easy customization UI.

It might be better to use single defcustom of list type allowing to
disable particular built-in filters and to add user filters and to
define their order in respect to built-in ones. I am unsure concerning
convenient enough UI.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Line breaks and brackets in LaTeX export
2022-11-01 16:07                                Max Nikulin
@ 2022-11-02  6:44                                  Ihor Radchenko
To: Max Nikulin; +Cc: emacs-orgmode

Max Nikulin <manikulin@gmail.com> writes:

> Another corner case when "\\[0pt]\n" should not be blindly replaced to
> "\\\n" (I do not escape "\"). Imagine that you are going to describe
> this optimizing filter
>
> ---- >8 ----
> Optimizing filter will replace
> #+begin_example
> First\\[0pt]
> Second
> #+end_example
> with
> #+begin_example
> First\\
> Second
> #+end_example
> ---- 8< -----

This is a strong argument against auto-removing the \\[0pt].
Should we, instead of using exact "\\[0pt]" string for line breaks,
define a new LaTeX command and then clean it up? This will distinguish
between \\[0pt] added by users explicitly and the ones generated
automatically by Org.

--
Org mode contributor,
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Line breaks and brackets in LaTeX export
2022-11-02  6:44                                  Ihor Radchenko
2022-11-02 15:27                                      Max Nikulin
To: Max Nikulin; +Cc: emacs-orgmode

> Should we, instead of using exact "\\[0pt]" string for line breaks,
> define a new LaTeX command and then clean it up? This will distinguish
> between \\[0pt] added by users explicitly and the ones generated
> automatically by Org.

I guess it will not work for that fancy table package we discussed
above. Maybe the Max's idea with "\\[0pt]%some comment indicator"?
I am just worried that % does have side effects in LaTeX.

--
Org mode contributor,
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Line breaks and brackets in LaTeX export
@ 2022-11-02 15:27                                      Max Nikulin
From: Max Nikulin @ 2022-11-02 15:27 UTC (permalink / raw)
To: emacs-orgmode

On 02/11/2022 13:46, Ihor Radchenko wrote:
>
>> Should we, instead of using exact "\\[0pt]" string for line breaks,
>> define a new LaTeX command and then clean it up? This will distinguish
>> between \\[0pt] added by users explicitly and the ones generated
>> automatically by Org.
>
> I guess it will not work for that fancy table package we discussed
> above.

tabularray package will require \NewTableCommand

> Maybe the Max's idea with "\\[0pt]%some comment indicator"?
> I am just worried that % does have side effects in LaTeX.

"a% comment
b"

as "ab", dropping newline and starting spaces. Of course, advanced users
may redefine category of "%" from comment to something else (regular
character, command prefix like \, etc.) like it is done inside verbatim
environment.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Line breaks and brackets in LaTeX export
2022-11-02 15:27                                      Max Nikulin
2022-11-03 15:00                                          Juan Manuel Macías
To: Max Nikulin, Juan Manuel Macías; +Cc: emacs-orgmode

Max Nikulin <manikulin@gmail.com> writes:

> On 02/11/2022 13:46, Ihor Radchenko wrote:
>>
>>> Should we, instead of using exact "\\[0pt]" string for line breaks,
>>> define a new LaTeX command and then clean it up? This will distinguish
>>> between \\[0pt] added by users explicitly and the ones generated
>>> automatically by Org.
>>
>> I guess it will not work for that fancy table package we discussed
>> above.
>
> tabularray package will require \NewTableCommand
>
>> Maybe the Max's idea with "\\[0pt]%some comment indicator"?
>> I am just worried that % does have side effects in LaTeX.
>
>
> "a% comment
>    b"
>
> as "ab", dropping newline and starting spaces. Of course, advanced users
> may redefine category of "%" from comment to something else (regular
> character, command prefix like \, etc.) like it is done inside verbatim
> environment.

These arguments mean that auto-cleaning \\[0pt] is not always safe and
may be a subject of surrounding LaTeX context. Moreover, there is no
clear alternative to \\[0pt] that is guaranteed to work.
Thus, the whole idea with cleaning up the generated LaTeX cannot be
enabled by default, and I am not even sure if it is something we want to
implement in the core.

Juan, maybe you have some good alternative suggestions?

--
Org mode contributor,
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Line breaks and brackets in LaTeX export
@ 2022-11-03 15:00                                          Juan Manuel Macías
2022-11-03 15:33                                            Max Nikulin
From: Juan Manuel Macías @ 2022-11-03 15:00 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: Max Nikulin, emacs-orgmode

> These arguments mean that auto-cleaning \\[0pt] is not always safe and
> may be a subject of surrounding LaTeX context. Moreover, there is no
> clear alternative to \\[0pt] that is guaranteed to work.
> Thus, the whole idea with cleaning up the generated LaTeX cannot be
> enabled by default, and I am not even sure if it is something we want to
> implement in the core.
>
> Juan, maybe you have some good alternative suggestions?

I'm afraid not. I've been trying to follow the thread these past few
days and frankly I can't think of anything beyond all the options that

I agree that the safest solution (or the least compromised, depending on
how we look at it) is to add [0pt] in all cases. I believe this should
not bring any unexpected consequences. At least I've tried all the
tabular environments related packages that I know of and there don't
seem to be any problems. And in the case of the verse package and verse
numberings, the solution is ad hoc with the patch that I have planned (I
hope to find a gap these days to finish it and test it...): The idea of
the patch is to replace the current separation between stanzas that Org
adds (\vspace) with a white line, and (only) in that case it would be
necessary to ensure that the last verse of the stanza ends in \\!.

On the other hand, if 'factory-cleaning' unnecessary instances of [0pt]
is going to be problematic, I also agree not to add that action to the
exporter. I think it would be better to leave these things up to the
users, depending on their use cases. After all, it is not difficult to
do it using a specific export filter. Maybe add some tips and examples
of basic filters in the documentation, especially for users who don't
have a lot of experience writing filters?

BTW, I also hope that one day this LaTeX problem, which has been there
for so many years, will be solved (because it should be considered a
LaTeX bug, or a LaTeX design flaw, as Maxim commented in another
message).

Best regards,

Juan Manuel

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Line breaks and brackets in LaTeX export
2022-11-03 15:00                                          Juan Manuel Macías
@ 2022-11-03 15:33                                            Max Nikulin
2022-11-03 15:48                                              Juan Manuel Macías
0 siblings, 2 replies; 53+ messages in thread
From: Max Nikulin @ 2022-11-03 15:33 UTC (permalink / raw)
To: emacs-orgmode

>
>> These arguments mean that auto-cleaning \\[0pt] is not always safe and
>> may be a subject of surrounding LaTeX context.

I still believe that

something\\[0pt]%__ORG_EXPORT__

is quite safe to remove (depending on the following character) and
unlikely harmful if remained for some reason. Even tabularray
regexp-based parser ignores comments. To be perfect,
"\\[0pt]%__ORG_EXPORT__" emitted by user code should be escaped to e.g.
"\\[0pd]%__ORG_EXPORT____ORG_EXPORT__", but I have no idea if it can be
implemented in a reliable way.

An alternative is a new export framework.

On 03/11/2022 22:00, Juan Manuel Macías wrote:
>
> BTW, I also hope that one day this LaTeX problem, which has been there
> for so many years, will be solved (because it should be considered a
> LaTeX bug, or a LaTeX design flaw, as Maxim commented in another
> message).

I have not managed to convince even the tabularray developer. And I have
not tried to find a way to reach more LaTeX developers.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Line breaks and brackets in LaTeX export
2022-11-03 15:33                                            Max Nikulin
@ 2022-11-03 15:48                                              Juan Manuel Macías
2022-11-04  4:23                                              Ihor Radchenko
1 sibling, 0 replies; 53+ messages in thread
From: Juan Manuel Macías @ 2022-11-03 15:48 UTC (permalink / raw)
To: Max Nikulin; +Cc: emacs-orgmode

Max Nikulin writes:

> I have not managed to convince even the tabularray developer. And I
> have not tried to find a way to reach more LaTeX developers.

I think that people from the LaTeX team and the authors of the most
popular packages are often very active on tex.stackexchange.com

And the repo on GitHub for the LaTeX project: https://github.com/latex3

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Line breaks and brackets in LaTeX export
2022-11-03 15:33                                            Max Nikulin
2022-11-03 15:48                                              Juan Manuel Macías
2022-11-04  5:40                                                Max Nikulin
To: Max Nikulin; +Cc: emacs-orgmode

Max Nikulin <manikulin@gmail.com> writes:

>>
>>> These arguments mean that auto-cleaning \\[0pt] is not always safe and
>>> may be a subject of surrounding LaTeX context.
>
> I still believe that
>
>      something\\[0pt]%__ORG_EXPORT__
>
> is quite safe to remove (depending on the following character) and
> unlikely harmful if remained for some reason.

What about the side effect you mentioned in a previous email?

>
>		   "a% comment
>		      b"
>
>		   as "ab", dropping newline and starting spaces.

--
Org mode contributor,
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Line breaks and brackets in LaTeX export
@ 2022-11-04  5:40                                                Max Nikulin
From: Max Nikulin @ 2022-11-04  5:40 UTC (permalink / raw)
To: emacs-orgmode

On 04/11/2022 11:23, Ihor Radchenko wrote:
> Max Nikulin writes:
>>>
>>>> These arguments mean that auto-cleaning \\[0pt] is not always safe and
>>>> may be a subject of surrounding LaTeX context.
>>
>> I still believe that
>>
>>       something\\[0pt]%__ORG_EXPORT__
>>
>> is quite safe to remove (depending on the following character) and
>> unlikely harmful if remained for some reason.
>
> What about the side effect you mentioned in a previous email?
>
>>
>> 		   "a% comment
>> 		      b"
>>
>> 		   as "ab", dropping newline and starting spaces.

Unlike in Org, in TeX "a\\b" still contains a line break. Besides exotic
packages and user setups it should just work. After all, during normal
operation all "%__ORG_EXPORT__" should be stripped by a filter. They are
just marks where a decision is necessary whether to retain "[0pt]" or to
remove it.

On 03/11/2022 22:48, Juan Manuel Macías wrote:
> Max Nikulin writes:
>
>> I have not managed to convince even the tabularray developer. And I
>> have not tried to find a way to reach more LaTeX developers.
>
> I think that people from the LaTeX team and the authors of the most
> popular packages are often very active on tex.stackexchange.com

Stackexchange is for questions, not for discussions and bugs. At certain
point I was going to ask a question there, I even prepared a draft.
After that I decided to try a command expanding to nothing and found
\empty. I was not aware that tabularray uses a different approach to
parsing.

> And the repo on GitHub for the LaTeX project: https://github.com/latex3

https://github.com/latex3/latex3/issues may be an appropriate place,
thank you.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Line breaks and brackets in LaTeX export
2022-11-04  5:40                                                Max Nikulin
0 siblings, 0 replies; 53+ messages in thread
To: Max Nikulin, Daniel Fleischer; +Cc: emacs-orgmode

Max Nikulin <manikulin@gmail.com> writes:

>>> I still believe that
>>>
>>>       something\\[0pt]%__ORG_EXPORT__
>>>
>>> is quite safe to remove (depending on the following character) and
>>> unlikely harmful if remained for some reason.
>>
>> What about the side effect you mentioned in a previous email?
>>
>>>
>>> 		   "a% comment
>>> 		      b"
>>>
>>> 		   as "ab", dropping newline and starting spaces.
>
> Unlike in Org, in TeX "a\\b" still contains a line break. Besides exotic
> packages and user setups it should just work. After all, during normal
> operation all "%__ORG_EXPORT__" should be stripped by a filter. They are
> just marks where a decision is necessary whether to retain "[0pt]" or to
> remove it.

Thanks for the clarification.

So, we can

1. Change org-latex-line-break-safe to "\\\\[0pt]%__ORG_EXPORT__",
making it unique to distinguish Org-generated line breaks.
2. Establish a regexp criterion when it is safe to strip
"[0pt]%__ORG_EXPORT__" from the page break (I am still unsure what
will be the most reliable criteria here). If not safe, we just strip
the "%__ORG_EXPORT__" part.
3. Write a filter that replaces the above from final output.
Note: If we go __ORG_EXPORT__ way, it might be safe to leave the
filter on by default as long as we can find a really robust criteria
on when it is safe to drop [0pt] part.

--
Org mode contributor,
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Line breaks and brackets in LaTeX export
2022-10-29  2:36                      Ihor Radchenko
@ 2022-11-13 17:01                        Max Nikulin
0 siblings, 0 replies; 53+ messages in thread
From: Max Nikulin @ 2022-11-13 17:01 UTC (permalink / raw)
To: emacs-orgmode

On 29/10/2022 09:36, Ihor Radchenko wrote:
>
>> Then [0pt] should it be. At least for now, before we have a cleaner
>> solution.
>>
>> See the attached patch.
>
> Applied onto main.
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=b93a61af9c93d21c56cf883630e52f36076e40bd
>
> I will look into filtering out unnecessary [0pt] instances later.
> The patch here is more urgent as it avoids the known case of breakage.

Tabularray became aware of \empty:
https://github.com/lvjr/tabularray/issues/328

So in future this command may be used instead of [0pt].

^ permalink raw reply	[flat|nested] 53+ messages in thread

end of thread, other threads:[~2022-11-13 17:03 UTC | newest]

2022-10-15 21:35 [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX Juan Manuel Macías
2022-10-16 12:08    Juan Manuel Macías
2022-10-16 15:04  Max Nikulin
2022-10-16 16:33    Verse block and separations (was: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX) Juan Manuel Macías
2022-10-17  8:54      Ihor Radchenko
2022-10-18  9:39        Verse block and separations Juan Manuel Macías
2022-10-17 14:48      Verse block and separations (was: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX) Max Nikulin
2022-10-19 11:08      Max Nikulin
2022-10-19 11:24        Verse block and separations Juan Manuel Macías
2022-10-16 17:14    Line breaks and brackets in LaTeX export (was: [bug] org-latex-line-break-safe' breaks the export of verse blocks to LaTeX) Juan Manuel Macías
2022-10-17  9:04      Ihor Radchenko
2022-10-17 11:30        Line breaks and brackets in LaTeX export Juan Manuel Macías
2022-10-17 11:47          Ihor Radchenko
2022-10-17 12:27            Juan Manuel Macías
2022-10-17 15:01          Juan Manuel Macías
2022-10-17 16:46            Max Nikulin
2022-10-17 18:04              Juan Manuel Macías
2022-10-18 14:23                  Juan Manuel Macías
2022-10-19  5:11                      Max Nikulin
2022-10-19 11:16                        Juan Manuel Macías
2022-10-19 12:30                          Juan Manuel Macías
2022-10-19 17:07                            Max Nikulin
2022-10-20 16:55                              Juan Manuel Macías
2022-10-21 16:38                                  Max Nikulin
2022-10-21 19:32                                    Juan Manuel Macías
[not found]                          <ac290c60-3b54-5521-eb16-82e6611dc6e2@gmail.com>
2022-10-20 17:07                            Juan Manuel Macías
2022-10-29  2:36                      Ihor Radchenko
2022-11-13 17:01                        Max Nikulin
2022-10-18  4:39              Ihor Radchenko
2022-10-19 17:12                Max Nikulin
2022-10-20  5:07                  Ihor Radchenko
2022-10-20 17:15                    Max Nikulin
2022-10-21  3:41                      Ihor Radchenko
2022-10-21 16:32                        Max Nikulin
2022-10-22  5:15                          Ihor Radchenko
2022-10-22 12:26                            Juan Manuel Macías
2022-10-22 15:55                            Max Nikulin
2022-11-01 16:07                                Max Nikulin
2022-11-02  6:46                                    Ihor Radchenko
2022-11-02 15:27                                      Max Nikulin
2022-11-03  6:15                                        Ihor Radchenko
2022-11-03 15:00                                          Juan Manuel Macías
2022-11-03 15:33                                            Max Nikulin
2022-11-03 15:48                                              Juan Manuel Macías
2022-11-04  4:23                                              Ihor Radchenko
2022-11-04  5:40                                                Max Nikulin
2022-11-05  5:30                                                  Ihor Radchenko


Code repositories for project(s) associated with this public inbox

https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).