bug#48149: 27.2; Wrong underline width when the line char has a width of 2

[Shingo Tanaka <shingo.fg8@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3079 bytes --]

Hi,

> Please note that using char-width cannot solve the problem of a
> character whose width depends on the font, because char-width is
> oblivious to fonts, it only knows about the character's codepoint.

I updated my patch proposal as attached to use window-text-pixel-size based
on Eli's advice.  Could you check it to see if it meets the expectation?  It
works good in my environment with some fonts of different char widths.

Here are some comments:

- New internal functions org-ascii--make-string and org-ascii--pixel-width
  are introduced just to improve code readability of this modification
  
- Line width is decided by org-ascii--make-string, which is a pixel width
  based make-string

- org-ascii--make-string uses org-ascii--pixel-width, which returns
  actual pixel width of characters and strings by using
  window-text-pixel-size with frame default font

- Line justification is also modified to be a pixel width basis

Since this is not a simple modification, I think we might need further
improvement, so any feedback is appreciated.  Especially, we could do better
for table alignment, as that is not very easy because the pixel width of line
character and that of space character is not always the same.

Anyway, I appreciate it if you can give it a try.  I am doing FSF signing
process in parallel just in case.

---
Shingo Tanaka


On Mon, 03 May 2021 01:23:02 +0900,
Eli Zaretskii wrote:
> 
> > From: Nicolas Goaziou <mail@nicolasgoaziou.fr>
> > Date: Sun, 02 May 2021 18:08:50 +0200
> > Cc: 48149@debbugs.gnu.org
> > 
> > > In case of 1), it correctly takes account of the case in which the character
> > > has a width of 2 in `org-ascii--build-title', by dividing the line width by
> > > `(char-width under-char)' (line 700-701), maybe because the character is user
> > > configurable and its width in unknown.  However, in case of 2) and
> > > 3), maybe because the characters is embedded in the code, it looks like only
> > > considering the character always has a width of 1.  But the reality is
> > > character ?─ or ?━ can have a width of 2 in the screen displayed with some
> > > fonts (ex. "Noto Sans Mono CJK JP"), and in that case the line width gets
> > > doubled of the expected width.
> > >
> > > Attached one is a potential patch.  The basic concepts are:
> > >
> > > a) Do the same in case of 2) and 3) as in case of 1)
> > >    (dividing the line width by `(char-width under-char)',
> > >     assuming `char-width-table' is correctly set)
> > >     
> > > b) Prefer the longer line width if the width is odd, even in case of 1)
> > >    (adding `(1- (char-width under-char))' to dividend,
> > >     just because it should be more beautiful ;-) )
> > 
> > Thank you. This looks good. I cannot apply it on "maint" branch,
> > however. Also, a proper commit message would be nice. Could you send an
> > updated patch?
> 
> Please note that using char-width cannot solve the problem of a
> character whose width depends on the font, because char-width is
> oblivious to fonts, it only knows about the character's codepoint.

[-- Attachment #2: ox-ascii.el.patch --]
[-- Type: application/octet-stream, Size: 7108 bytes --]

--- ox-ascii.el.org	2021-03-26 09:28:44.000000000 +0900
+++ ox-ascii.el	2021-05-09 17:23:40.421912931 +0900
@@ -440,8 +440,11 @@
 ;; `org-ascii--describe-links' creates notes about links for
 ;; insertion at the end of a section.  It uses
 ;; `org-ascii--unique-links' to get the list of links to describe.
-;; Eventually, `org-ascii--translate' translates a string according
+;; `org-ascii--translate' translates a string according
 ;; to language and charset specification.
+;; `org-ascii--make-string' provides pixel width based `make-string' for
+;; better text alignment, by using `org-ascii--pixel-width' which returns
+;; actual pixel width of characters and strings.
 
 
 (defun org-ascii--fill-string (s text-width info &optional justify)
@@ -481,9 +484,19 @@
     (let ((fill-column text-width)
 	  ;; Disable `adaptive-fill-mode' so it doesn't prevent
 	  ;; filling lines matching `adaptive-fill-regexp'.
-	  (adaptive-fill-mode nil))
+	  (adaptive-fill-mode nil)
+          linestr pwidth)
       (while (< (point) (point-max))
-	(justify-current-line how)
+        (setq linestr (buffer-substring (point) (line-end-position))
+              pwidth (max (- (* (org-ascii--pixel-width ?\s) text-width)
+                             (org-ascii--pixel-width linestr))
+                          0))
+        (insert (org-ascii--make-string (pcase how
+                                          (`left 0)
+                                          (`center (/ pwidth 2))
+                                          (`right pwidth))
+                                        ?\s 0 (- text-width
+                                                 (string-width linestr))))
 	(forward-line)))
     (buffer-string)))
 
@@ -697,9 +710,8 @@
 			      (plist-get info :ascii-underline))))))
 	 (and under-char
 	      (concat "\n"
-		      (make-string (/ (string-width first-part)
-				      (char-width under-char))
-				   under-char))))))))
+		      (org-ascii--make-string
+                       (org-ascii--pixel-width first-part) under-char))))))))
 
 (defun org-ascii--has-caption-p (element _info)
   "Non-nil when ELEMENT has a caption affiliated keyword.
@@ -750,9 +762,9 @@
    (unless scope
      (let ((title (org-ascii--translate "Table of Contents" info)))
        (concat title "\n"
-	       (make-string
-		(string-width title)
-		(if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_))
+               (org-ascii--make-string
+                (org-ascii--pixel-width title)
+                (if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_))
 	       "\n\n")))
    (let ((text-width
 	  (if keyword (org-ascii--current-text-width keyword info)
@@ -779,8 +791,9 @@
   (let ((title (org-ascii--translate "List of Listings" info)))
     (concat
      title "\n"
-     (make-string (string-width title)
-		  (if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_))
+     (org-ascii--make-string
+      (org-ascii--pixel-width title)
+      (if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_))
      "\n\n"
      (let ((text-width
 	    (if keyword (org-ascii--current-text-width keyword info)
@@ -819,8 +832,9 @@
   (let ((title (org-ascii--translate "List of Tables" info)))
     (concat
      title "\n"
-     (make-string (string-width title)
-		  (if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_))
+     (org-ascii--make-string
+      (org-ascii--pixel-width title)
+      (if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_))
      "\n\n"
      (let ((text-width
 	    (if keyword (org-ascii--current-text-width keyword info)
@@ -1033,7 +1047,7 @@
 	     ;; Format TITLE.  It may be filled if it is too wide,
 	     ;; that is wider than the two thirds of the total width.
 	     (title-len (min (apply #'max
-				    (mapcar #'length
+				    (mapcar #'string-width
 					    (org-split-string
 					     (concat title "\n" subtitle) "\n")))
 			     (/ (* 2 text-width) 3)))
@@ -1041,12 +1055,19 @@
 	     (formatted-subtitle (when (org-string-nw-p subtitle)
 				   (org-ascii--fill-string subtitle title-len info)))
 	     (line
-	      (make-string
-	       (min (+ (max title-len
-			    (string-width (or author ""))
-			    (string-width (or email "")))
-		       2)
-		    text-width) (if utf8p ?━ ?_))))
+              (org-ascii--make-string
+               (apply #'max
+                      (mapcar #'org-ascii--pixel-width
+	                      (org-split-string
+		               (concat formatted-title "\n"
+                                       formatted-subtitle "\n"
+                                       (if author (concat author "\n") "")
+                                       (if email (concat email "\n") ""))
+                               "\n")))
+               (if utf8p ?━ ?_)
+               2 text-width)))
+        (message "%s" line)
+        (message "%s" formatted-title)
 	(org-ascii--justify-lines
 	 (concat line "\n"
 		 (unless utf8p "\n")
@@ -1082,9 +1103,9 @@
 	    (let ((title (org-ascii--translate "Footnotes" info)))
 	      (concat
 	       title "\n"
-	       (make-string
-		(string-width title)
-		(if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_))))
+               (org-ascii--make-string
+                (org-ascii--pixel-width title)
+                (if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_))))
 	    "\n\n"
 	    (let ((text-width (- (plist-get info :ascii-text-width)
 				 global-margin)))
@@ -1150,6 +1171,29 @@
   (let ((charset (intern (format ":%s" (plist-get info :ascii-charset)))))
     (org-export-translate s charset info)))
 
+(defun org-ascii--make-string (pwidth init &optional adjust limit)
+  "Return a newly created string of width PWIDTH, with INIT in each element.
+PWIDTH is in pixel and the actual width is a minimum longer than
+PWIDTH, unless either ADJUST or LIMIT is specified.  ADJUST is
+the number of characters added or removed from the string if the
+value is positive or negative integer.  LIMIT is the maximum
+allowed number of characters of the string."
+  (let* ((init-pwidth (org-ascii--pixel-width init))
+         (width (/ (+ pwidth (1- init-pwidth)) init-pwidth)))
+    (if (integerp adjust) (setq width (+ width adjust)))
+    (if (integerp limit) (setq width (min width (max limit 0))))
+    (make-string width init)))
+
+(defun org-ascii--pixel-width (obj)
+  "Return OBJ pixel width in display.
+OBJ is supposed to be a character or a string.  Returned pixel
+width is based on frame default font."
+  (with-temp-buffer
+    (insert obj)
+    (switch-to-buffer (current-buffer))
+    (car (window-text-pixel-size
+          nil 1 (1+ (if (sequencep obj) (length obj) 1))))))
+
 
 \f
 ;;; Transcode Functions
@@ -1960,7 +2004,9 @@
 					'identity info t)
 				      cell))
 			 lcorner)
-		       (make-string (+ 2 width) (string-to-char horiz))
+                       (org-ascii--make-string
+                        (* (+ 2 width) (org-ascii--pixel-width ?\s))
+                        (string-to-char horiz))
 		       (cond
 			((not (memq 'right borders)) nil)
 			((eq (car (last (org-element-contents table-row))) cell)