Re: [bug] Smart quotes: confusion of apostrophe with second level quotes

[Ihor Radchenko <yantar92@posteo.net>]

[-- Attachment #1: Type: text/plain, Size: 1088 bytes --]

Juan Manuel Macías <maciaschain@posteo.net> writes:

>   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
>   #+OPTIONS: ':t
>   #+language:es
>
>   "my friends' party and the students' papers"
>   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
>
> the above produces in LaTeX:
>
>   \guillemotleft{}my friends'' party and the students'' papers\guillemotright{}
> ...
> Perhaps a possible solution would be to allow the use of a specific,
> customizable character, other than an apostrophe, for second-level
> quotes. Or at least add some brief warning in the manual: in certain
> contexts it is safer to use a explicit Unicode character for the
> apostrophe.

I think that we can address examples like this simply by not replacing
unbalanced quotes. There is already some effort in the code towards such
treatment, but it is not complete.

Can you try the attached patch?


[-- Attachment #2: 0001-org-export-Do-not-treat-unpaired-and-as-smart-quotes.patch --]
[-- Type: text/x-patch, Size: 5726 bytes --]

From 4a034fbb0029ca7e635f629810a6179df4ca24d9 Mon Sep 17 00:00:00 2001
Message-ID: <4a034fbb0029ca7e635f629810a6179df4ca24d9.1711193777.git.yantar92@posteo.net>
From: Ihor Radchenko <yantar92@posteo.net>
Date: Sat, 23 Mar 2024 14:34:06 +0300
Subject: [PATCH] org-export: Do not treat unpaired ' and " as smart quotes
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* lisp/ox.el (org-export--smart-quote-status): When quotes are not
balanced, treat " literally and ' as apostrophes.
* testing/lisp/test-ox.el (test-org-export/activate-smart-quotes): Fix
test with unbalanced " and add new tests for unbalanced quotes.

Reported-by: Juan Manuel Macías <maciaschain@posteo.net>
Link: https://list.orgmode.org/orgmode/875xxfqdpt.fsf@posteo.net/
---
 lisp/ox.el              | 45 +++++++++++++++++++++++++++++++++++++++++
 testing/lisp/test-ox.el | 29 ++++++++++++++++++++++++--
 2 files changed, 72 insertions(+), 2 deletions(-)

diff --git a/lisp/ox.el b/lisp/ox.el
index 929b306dc..539d31d9d 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -5942,6 +5942,51 @@ (defun org-export--smart-quote-status (s info)
 	      (when current-status
 		(push (cons text (nreverse current-status)) full-status))))
 	  info nil org-element-recursive-objects)
+        ;; When quotes are not balanced, threat them as apostrophes.
+        (setq full-status (nreverse full-status))
+        (let (primary-openings secondary-openings)
+          (dolist (substatus full-status)
+            (let ((status (cdr substatus)))
+              (while status
+                (pcase (car status)
+                  (`apostrophe nil)
+                  (`primary-opening
+                   (push status primary-openings))
+                  (`secondary-opening
+                   (push status secondary-openings))
+                  (`secondary-closing
+                   (if secondary-openings
+                       ;; Remove matched opening.
+                       (pop secondary-openings)
+                     ;; No matching openings for a given closing.  Replace
+                     ;; it with apostrophe.
+                     (setcar status 'apostrophe)))
+                  (`primary-closing
+                   (when secondary-openings
+                     ;; Some secondary opening quotes are not closed
+                     ;; within "...".  Replace them all with apostrophes.
+                     (dolist (opening secondary-openings)
+                       (setcar opening 'apostrophe))
+                     (setq secondary-openings nil))
+                   (if primary-openings
+                       ;; Remove matched opening.
+                       (pop primary-openings)
+                     ;; No matching openings for a given closing.
+                     (error "This should no happen"))))
+                (setq status (cdr status)))))
+          (when primary-openings
+            ;; Trailing unclosed "
+            (unless (= 1 (length primary-openings))
+              (error "This should not happen"))
+            ;; Mark for not replacing.
+            (setcar (car primary-openings) nil)
+            ;; Mark all the secondary openings and closings after
+            ;; trailing unclosed " as apostrophes.
+            (let ((tail (car primary-openings)))
+              (while tail
+                (when (memq (car tail) '(secondary-opening secondary-closing))
+                  (setcar tail 'apostrophe))
+                (setq tail (cdr tail))))))
 	(puthash (cons parent (org-element-secondary-p s)) full-status cache)
 	(cdr (assq s full-status))))))
 
diff --git a/testing/lisp/test-ox.el b/testing/lisp/test-ox.el
index 01e082c9b..16e81c64b 100644
--- a/testing/lisp/test-ox.el
+++ b/testing/lisp/test-ox.el
@@ -4134,9 +4134,9 @@ (ert-deftest test-org-export/activate-smart-quotes ()
   ;; Opening quotes: at the beginning of a paragraph.
   (should
    (equal
-    '("&ldquo;begin")
+    '("&ldquo;begin&rdquo;")
     (let ((org-export-default-language "en"))
-      (org-test-with-parsed-data "\"begin"
+      (org-test-with-parsed-data "\"begin\""
 	(org-element-map tree 'plain-text
 	  (lambda (s) (org-export-activate-smart-quotes s :html info))
 	  info)))))
@@ -4267,6 +4267,31 @@ (ert-deftest test-org-export/activate-smart-quotes ()
 	    (org-test-with-parsed-data "*\"foo\"*"
 	      (org-element-map tree 'plain-text
 		(lambda (s) (org-export-activate-smart-quotes s :html info))
+		info nil nil t)))))
+  ;; Unmatched quotes.
+  (should
+   (equal '("\\guillemotleft{}my friends' party and the students' papers\\guillemotright{} \\guillemotleft{}``mothers''\\guillemotright{}")
+	  (let ((org-export-default-language "es"))
+	    (org-test-with-parsed-data
+                "\"my friends' party and the students' papers\" \"'mothers'\""
+	      (org-element-map tree 'plain-text
+		(lambda (s) (org-export-activate-smart-quotes s :latex info))
+		info nil nil t)))))
+  (should
+   (equal '("\"'mothers'")
+	  (let ((org-export-default-language "es"))
+	    (org-test-with-parsed-data
+                "\"'mothers'"
+	      (org-element-map tree 'plain-text
+		(lambda (s) (org-export-activate-smart-quotes s :latex info))
+		info nil nil t)))))
+  (should
+   (equal '("\\guillemotleft{}να 'ρθώ το βράδυ\\guillemotright{}")
+	  (let ((org-export-default-language "el"))
+	    (org-test-with-parsed-data
+                "\"να 'ρθώ το βράδυ\""
+	      (org-element-map tree 'plain-text
+		(lambda (s) (org-export-activate-smart-quotes s :latex info))
 		info nil nil t))))))
 
 
-- 
2.44.0


[-- Attachment #3: Type: text/plain, Size: 224 bytes --]


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>