[PATCH] Wrap LaTeX snippets in $$ with markdown export

[PATCH] Wrap LaTeX snippets in $$ with markdown export

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 936 bytes --]


Hi All,

I anticipate that this change may be somewhat contentions because ox-md
explicitly follows only the original Markdown spec from 2003, however
I've thought this over and come to the conclusion that this change is
still in keeping with that, and beneficial.

Currently ox-md simply inherits the output from ox-html's handling of
LaTeX snippets. Needless to say, the original Markdown specification
does not mention LaTeX snippets. As such, by subtly tweaking the output
(either adding $$ or substituting out LaTeX-style \(\) / \[\] for $ /
$$) we are not deviating from the original specification any more than we
already are.

While I don't see any issue with this, I do see some benefits. Namely
that, of the many Markdown variants that now exist, many support LaTeX
snippets, but exclusively in the TeX $$ form. Hence, by applying this
change the overall utility of ox-md is increased.

Let me know what you think,

Timothy


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ox-md-Use-TeX-style-math-wrapping.patch --]
[-- Type: text/x-patch, Size: 2721 bytes --]

From b7f1b89a50752398672a642519534818d23c72a4 Mon Sep 17 00:00:00 2001
From: TEC <tec@tecosaur.com>
Date: Thu, 1 Apr 2021 00:25:41 +0800
Subject: [PATCH] ox-md: Use TeX-style $ math wrapping

* lisp/ox-md.el (org-md-latex-environment, org-md-latex-fragment):
These two new filters wrap LaTeX maths in $ / $$ TeX-style notation.
While ox-md endeavours to adhere to the original Markdown specification,
and not any particular variant, the original specification does not deal
with LaTeX fragments at all, and so this change does not reduce how
faithfully the original specification is followed.
There is a major upside to this though. Of the many Markdown variants
that have emerged, those that support LaTeX very often exclusively
support TeX style notation.  This change thus improves the utility of
the Markdown export for many use cases, and deviates no more from the
original specification that the current method.
---
 lisp/ox-md.el | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/lisp/ox-md.el b/lisp/ox-md.el
index b6b2c1728..c5c0e05a5 100644
--- a/lisp/ox-md.el
+++ b/lisp/ox-md.el
@@ -100,6 +100,8 @@ (org-export-define-derived-backend 'md 'html
 		     (italic . org-md-italic)
 		     (item . org-md-item)
 		     (keyword . org-md-keyword)
+                     (latex-environment . org-md-latex-environment)
+                     (latex-fragment . org-md-latex-fragment)
 		     (line-break . org-md-line-break)
 		     (link . org-md-link)
 		     (node-property . org-md-node-property)
@@ -460,6 +462,32 @@ (defun org-md-keyword (keyword contents info)
     (_ (org-export-with-backend 'html keyword contents info))))
 
 
+;;;; Latex Environment
+
+(defun org-md-latex-environment (latex-environment contents info)
+  "Transcode a LATEX-ENVIRONMENT object from Org to Markdown.
+CONTENTS is nil.  INFO is a plist holding contextual information."
+  (when (plist-get info :with-latex)
+    (concat "$$\n"
+            (org-html-latex-environment latex-environment contents info)
+            "$$\n")))
+
+
+;;;; Latex Fragment
+
+(defun org-md-latex-fragment (latex-fragment contents info)
+  "Transcode a LATEX-FRAGMENT object from Org to Markdown.
+CONTENTS is nil.  INFO is a plist holding contextual information."
+  (when (plist-get info :with-latex)
+    (let ((frag (org-html-latex-fragment contents info)))
+      (cond
+       ((string-match-p "^\\\\(" frag)
+        (concat "$" (substring frag 2 -2) "$"))
+       ((string-match-p "^\\\\\\[" frag)
+        (concat "$$" (substring frag 2 -2) "$$"))
+       (t (message "unrecognised fragment: %s" frag)
+          frag)))))
+
 ;;;; Line Break
 
 (defun org-md-line-break (_line-break _contents _info)
-- 
2.30.1

Re: [PATCH] Wrap LaTeX snippets in $$ with markdown export

[Berry, Charles via General discussions about Org-mode.]

> On Mar 31, 2021, at 9:41 AM, Timothy <tecosaur@gmail.com> wrote:
> 
> I anticipate that this change may be somewhat contentions because ox-md
> explicitly follows only the original Markdown spec from 2003, however
> I've thought this over and come to the conclusion that this change is
> still in keeping with that, and beneficial.


Will this handle LaTeX macros[1] gracefully and other things intended as raw LaTeX?

Best,
Chuck

[1] https://pandoc.org/MANUAL.html#latex-macros

Re: [PATCH] Wrap LaTeX snippets in $$ with markdown export

[Timothy]

> Will this handle LaTeX macros[1] gracefully and other things intended as raw LaTeX?
> [1] https://pandoc.org/MANUAL.html#latex-macros

This only affects LaTeX fragments, and then it's just changing \(\) /
\[\] for $ / $$. I don't think this has any effect on your concerns.

--
Timothy

Re: [PATCH] Wrap LaTeX snippets in $$ with markdown export

[Timothy]

Hello Nicolas,

In my plea for patch feedback email, Bastien said he'd like to leave
this patch to your discretion.

As such, it would be fantastic if you might be able to provide your
thoughts on this.

All the best,

Timothy.

Re: [PATCH] Wrap LaTeX snippets in $$ with markdown export

[Nicolas Goaziou]

Hello,

Timothy <tecosaur@gmail.com> writes:

> I anticipate that this change may be somewhat contentions because ox-md
> explicitly follows only the original Markdown spec from 2003, however
> I've thought this over and come to the conclusion that this change is
> still in keeping with that, and beneficial.
>
> Currently ox-md simply inherits the output from ox-html's handling of
> LaTeX snippets. Needless to say, the original Markdown specification
> does not mention LaTeX snippets. As such, by subtly tweaking the output
> (either adding $$ or substituting out LaTeX-style \(\) / \[\] for $ /
> $$) we are not deviating from the original specification any more than we
> already are.

This sounds reasonable.

> +(defun org-md-latex-environment (latex-environment contents info)
> +  "Transcode a LATEX-ENVIRONMENT object from Org to Markdown.
> +CONTENTS is nil.  INFO is a plist holding contextual information."
> +  (when (plist-get info :with-latex)
> +    (concat "$$\n"
> +            (org-html-latex-environment latex-environment contents info)
> +            "$$\n")))

Nitpick: I would use `format', also the final newline character is
useless, since it will be removed later during the export process.

> +;;;; Latex Fragment
> +
> +(defun org-md-latex-fragment (latex-fragment contents info)
> +  "Transcode a LATEX-FRAGMENT object from Org to Markdown.
> +CONTENTS is nil.  INFO is a plist holding contextual information."
> +  (when (plist-get info :with-latex)
> +    (let ((frag (org-html-latex-fragment contents info)))
> +      (cond
> +       ((string-match-p "^\\\\(" frag)
> +        (concat "$" (substring frag 2 -2) "$"))
> +       ((string-match-p "^\\\\\\[" frag)
> +        (concat "$$" (substring frag 2 -2) "$$"))
> +       (t (message "unrecognised fragment: %s" frag)
> +          frag)))))

Nitpick: I suggest to use `rx' macro. It really makes the code base
a better place.

You are missing some cases. The fragment could be $...$ or $$...$$
already, so you can return it as-is without sending the message.

Otherwise, it is a macro. We can assume it lives outside math mode. So
maybe the "Unrecognized fragment: %S" is in order in that situation. We
could also let HTML export back-end deal with it. I don't know what is
better.

WDYT?

Regards,
-- 
Nicolas Goaziou

Re: [PATCH] Wrap LaTeX snippets in $$ with markdown export

[Timothy]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> This sounds reasonable.

I'm glad to hear that!

>> +(defun org-md-latex-environment [...]
> Nitpick: I would use `format', also the final newline character is
> useless, since it will be removed later during the export process.

Actually, since writing this patch I'm not sure that $$-surrounding
\begin{}...\end{} environments is also a good idea. I'm inclined to
leave this out of the patch.

>> +;;;; Latex Fragment
>> +
>> +(defun org-md-latex-fragment (latex-fragment contents info)
>> +  "Transcode a LATEX-FRAGMENT object from Org to Markdown.
>> +CONTENTS is nil.  INFO is a plist holding contextual information."
>> +  (when (plist-get info :with-latex)
>> +    (let ((frag (org-html-latex-fragment contents info)))
>> +      (cond
>> +       ((string-match-p "^\\\\(" frag)
>> +        (concat "$" (substring frag 2 -2) "$"))
>> +       ((string-match-p "^\\\\\\[" frag)
>> +        (concat "$$" (substring frag 2 -2) "$$"))
>> +       (t (message "unrecognised fragment: %s" frag)
>> +          frag)))))
>
> Nitpick: I suggest to use `rx' macro. It really makes the code base
> a better place.

I do rather like the `rx' macro, however I'm not sure that
(rx bol "\\(") is really an improvement over "^\\\\(" in this particular
case.

> You are missing some cases. The fragment could be $...$ or $$...$$
> already, so you can return it as-is without sending the message.

Ah yep, after getting your thoughts on this message I'll send a revised
patch with fixes this (along with any other changes).

> Otherwise, it is a macro. We can assume it lives outside math mode. So
> maybe the "Unrecognized fragment: %S" is in order in that situation. We
> could also let HTML export back-end deal with it. I don't know what is
> better.

I'm not sure what to do here either. Perhaps nothing/HTML backend?

Finally, I think if this case (lone macro) is handled, there aren't any
possible "Unrecognized fragment"s that could be passed, and so that
condition would no longer be necessary?

--
Timothy.

Re: [PATCH] Wrap LaTeX snippets in $$ with markdown export

[Nicolas Goaziou]

Timothy <tecosaur@gmail.com> writes:

> Actually, since writing this patch I'm not sure that $$-surrounding
> \begin{}...\end{} environments is also a good idea. I'm inclined to
> leave this out of the patch.

Sounds good.

> I do rather like the `rx' macro, however I'm not sure that
> (rx bol "\\(") is really an improvement over "^\\\\(" in this particular
> case.

True, but I still prefer `rx'. I guess I wrote too many backslashes.
Anyway it's a nitpick. Do whatever you want.

>> Otherwise, it is a macro. We can assume it lives outside math mode. So
>> maybe the "Unrecognized fragment: %S" is in order in that situation. We
>> could also let HTML export back-end deal with it. I don't know what is
>> better.
>
> I'm not sure what to do here either. Perhaps nothing/HTML backend?

I suggest to re-send to HTML backend `org-export-data-with-backend', to
be consistent with the rest of un-handled objects.

> Finally, I think if this case (lone macro) is handled, there aren't any
> possible "Unrecognized fragment"s that could be passed, and so that
> condition would no longer be necessary?

Yup. If you handle $$ $ \( \[ and macros, you're set.

Regards,

Re: [PATCH] Wrap LaTeX snippets in $$ with markdown export

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 951 bytes --]


Hi Nicolas,

So, I've now fixed the handling of already-$-delaminated fragments, and
removed the $$ from around environments.

Sorry to complicate things, but, looking at what the HTML fallback
actually is, I've had some further thoughts. The HTML backend will
either leave the content as-is, run `org-latex-to-html-convert-command',
or run something from `org-preview-latex-process-alist' and include an
<img>.

Should one want to convert LaTeX images say to an SVG for HTML export, I
do not think one would want this behaviour to also apply to LaTeX being
exported to markdown.

As such I see two 'sensible' paths forward: (1) just include LaTeX
verbatim when :with-latex is non-nil, or (2) introduce
`org-markdown-with-latex' which can be set to t/'verbatim or 'html.

In the attached patch, I've taken the first approach. Let me know if
you'd like to take the second approach, or just inherit the HTML
behaviour for LaTeX after all.

--
Timothy


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ox-md-Use-TeX-style-math-wrapping.patch --]
[-- Type: text/x-patch, Size: 3018 bytes --]

From a02dbcb21dddb6d9239d13308a78a45738d9b7e6 Mon Sep 17 00:00:00 2001
From: Timothy <tecosaur@gmail.com>
Date: Thu, 1 Apr 2021 00:41:29 +0800
Subject: [PATCH] ox-md: Use TeX-style $ math wrapping

* lisp/ox-md.el (org-md-latex-environment, org-md-latex-fragment):
These two new filters wrap LaTeX maths in $ / $$ TeX-style notation.
While ox-md endeavours to adhere to the original Markdown specification,
and not any particular variant, the original specification does not deal
with LaTeX fragments at all, and so this change does not reduce how
faithfully the original specification is followed.
There is a major upside to this though. Of the many Markdown variants
that have emerged, those that support LaTeX very often exclusively
support TeX style notation.  This change thus improves the utility of
the Markdown export for many use cases, and deviates no more from the
original specification that the current method.
---
 lisp/ox-md.el | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/lisp/ox-md.el b/lisp/ox-md.el
index b6b2c1728..1ae0cf2d9 100644
--- a/lisp/ox-md.el
+++ b/lisp/ox-md.el
@@ -100,6 +100,8 @@ (org-export-define-derived-backend 'md 'html
 		     (italic . org-md-italic)
 		     (item . org-md-item)
 		     (keyword . org-md-keyword)
+                     (latex-environment . org-md-latex-environment)
+                     (latex-fragment . org-md-latex-fragment)
 		     (line-break . org-md-line-break)
 		     (link . org-md-link)
 		     (node-property . org-md-node-property)
@@ -460,6 +462,35 @@ (defun org-md-keyword (keyword contents info)
     (_ (org-export-with-backend 'html keyword contents info))))
 
 
+;;;; Latex Environment
+
+(defun org-md-latex-environment (latex-environment contents info)
+  "Transcode a LATEX-ENVIRONMENT object from Org to Markdown.
+CONTENTS is nil.  INFO is a plist holding contextual information."
+  (when (plist-get info :with-latex)
+    (let ((latex-frag (org-remove-indentation
+                       (org-element-property :value latex-environment)))
+          (label (org-html--reference latex-environment info t)))
+      (if (org-string-nw-p label)
+          (replace-regexp-in-string "\\`.*"
+                                    (format "\\&\n\\\\label{%s}" label)
+                                    latex-frag)
+        latex-frag))))
+
+;;;; Latex Fragment
+
+(defun org-md-latex-fragment (latex-fragment contents info)
+  "Transcode a LATEX-FRAGMENT object from Org to Markdown.
+CONTENTS is nil.  INFO is a plist holding contextual information."
+  (when (plist-get info :with-latex)
+    (let ((frag (org-element-property :value latex-fragment)))
+      (cond
+       ((string-match-p "^\\\\(" frag)
+        (concat "$" (substring frag 2 -2) "$"))
+       ((string-match-p "^\\\\\\[" frag)
+        (concat "$$" (substring frag 2 -2) "$$"))
+       (t frag))))) ; either already $-deliminated or a macro
+
 ;;;; Line Break
 
 (defun org-md-line-break (_line-break _contents _info)
-- 
2.31.1

Re: [PATCH] Wrap LaTeX snippets in $$ with markdown export

[Nicolas Goaziou]

Hello,

Timothy <tecosaur@gmail.com> writes:

> Should one want to convert LaTeX images say to an SVG for HTML export, I
> do not think one would want this behaviour to also apply to LaTeX being
> exported to markdown.

Why do you think so? This is consistent with everything else not
supported by vanilla Markdown.

> As such I see two 'sensible' paths forward: (1) just include LaTeX
> verbatim when :with-latex is non-nil, or (2) introduce
> `org-markdown-with-latex' which can be set to t/'verbatim or 'html.
>
> In the attached patch, I've taken the first approach. Let me know if
> you'd like to take the second approach, or just inherit the HTML
> behaviour for LaTeX after all.

Obeying to :with-latex property means "ox-md" is somehow actively
handling LaTeX fragments, which was not the initial intent, IIUC. The
first idea was to provide a sensible default for such objects, because
we're outside the specification anyway. I'd rather not overdo it.

WDYT?

Regards,
-- 
Nicolas Goaziou

Re: [PATCH] Wrap LaTeX snippets in $$ with markdown export

[Timothy]

Hello,

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

>> Should one want to convert LaTeX images say to an SVG for HTML export, I
>> do not think one would want this behaviour to also apply to LaTeX being
>> exported to markdown.
>
> Why do you think so? This is consistent with everything else not
> supported by vanilla Markdown.

To speak personally, at the moment I use mathjax for HTML. I'm quite
content with the markdown output from this patch without what's
discussed below.
However, if/when I can get the SVG baseline to work with dvisvgm I plan
on switching to that for HTML files. However, I'd consider this
behaviour undesirable for markdown.

The main reason why I'm hesitant about including <img>s in Markdown, is
because unlike <table> etc. the output now relies on external files.

>> As such I see two 'sensible' paths forward: (1) just include LaTeX
>> verbatim when :with-latex is non-nil, or (2) introduce
>> `org-markdown-with-latex' which can be set to t/'verbatim or 'html.
>>
> Obeying to :with-latex property means "ox-md" is somehow actively
> handling LaTeX fragments, which was not the initial intent, IIUC. The
> first idea was to provide a sensible default for such objects, because
> we're outside the specification anyway. I'd rather not overdo it.

I'm happy to strip this out of the patch (there's always advice for my
own config...), I just thought there may be people who like me are
interested in <img>s for LaTeX in HTML, but not in Markdown.

--
Timothy

Re: [PATCH] Wrap LaTeX snippets in $$ with markdown export

[Nicolas Goaziou]

Hello,

Timothy <tecosaur@gmail.com> writes:

> I just thought there may be people who like me are interested in
> <img>s for LaTeX in HTML, but not in Markdown.

Fair enough. Let's push your last patch, then.

Thank you.

Regards,
-- 
Nicolas Goaziou

Re: [PATCH] Wrap LaTeX snippets in $$ with markdown export

[Timothy]

Hi,

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

>> I just thought there may be people who like me are interested in
>> <img>s for LaTeX in HTML, but not in Markdown.
>
> Fair enough. Let's push your last patch, then.

Going off this, I've taken this as assent and just pushed my patch in
its current form as 981f25031.

--
Timothy

Re: [PATCH] Wrap LaTeX snippets in $$ with markdown export

[Nicolas Goaziou]

Hello,

Timothy <tecosaur@gmail.com> writes:

> Going off this, I've taken this as assent and just pushed my patch in
> its current form as 981f25031.

Thank you.

Regards,
-- 
Nicolas Goaziou