emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* bug#48149: 27.2; Wrong underline width when the line char has a width of 2
       [not found] <87fsz62b1w.wl-shingo.fg8@gmail.com>
@ 2021-05-02  7:17 ` Eli Zaretskii
       [not found] ` <837dkh8uzi.fsf__22492.4155323365$1619940077$gmane$org@gnu.org>
  2021-05-02 16:08 ` Nicolas Goaziou
  2 siblings, 0 replies; 6+ messages in thread
From: Eli Zaretskii @ 2021-05-02  7:17 UTC (permalink / raw)
  To: Shingo Tanaka; +Cc: 48149

> Date: Sun, 02 May 2021 10:12:43 +0900
> From: Shingo Tanaka <shingo.fg8@gmail.com>
> 
> When exporting org-mode document to plain text (either ascii/unicode/utf-8)
> with `org-export-dispatch', Emacs inserts lines under headlines, inline
> tasks, table rows and titles of the document, TOC, list of listings, list of
> tables and footnotes.  The problem is it inserts too long (double width) line
> when the line character has a width of 2.
> 
> Those lines are made of 3 types of characters below (in ox-ascii.el):
> 1) org-ascii-underline
> 2) (if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_)
> 3) (if utf8p ?━ ?_)
> 
> In case of 1), it correctly takes account of the case in which the character
> has a width of 2 in `org-ascii--build-title', by dividing the line width by
> `(char-width under-char)' (line 700-701), maybe because the character is user
> configurable and its width in unknown.  However, in case of 2) and
> 3), maybe because the characters is embedded in the code, it looks like only
> considering the character always has a width of 1.  But the reality is
> character ?─ or ?━ can have a width of 2 in the screen displayed with some
> fonts (ex. "Noto Sans Mono CJK JP"), and in that case the line width gets
> doubled of the expected width.
> 
> Attached one is a potential patch.  The basic concepts are:
> 
> a) Do the same in case of 2) and 3) as in case of 1)
>    (dividing the line width by `(char-width under-char)',
>     assuming `char-width-table' is correctly set)
>     
> b) Prefer the longer line width if the width is odd, even in case of 1)
>    (adding `(1- (char-width under-char))' to dividend,
>     just because it should be more beautiful ;-) )

You reported a similar bug already, and I replied there that TRT in
these cases is to use window-text-pixel-size, which will automatically
account for the actual width on display of any characters and any
fonts specified for displaying them.  char-width is an approximation,
and is accurate only on TTY frames.




^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#48149: 27.2; Wrong underline width when the line char has a width of 2
       [not found] ` <837dkh8uzi.fsf__22492.4155323365$1619940077$gmane$org@gnu.org>
@ 2021-05-02  8:36   ` Rudolf Schlatte
  2021-05-02  9:16     ` Eli Zaretskii
  0 siblings, 1 reply; 6+ messages in thread
From: Rudolf Schlatte @ 2021-05-02  8:36 UTC (permalink / raw)
  To: 48149

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Sun, 02 May 2021 10:12:43 +0900
>> From: Shingo Tanaka <shingo.fg8@gmail.com>
>> 
>> When exporting org-mode document to plain text (either ascii/unicode/utf-8)
>> with `org-export-dispatch', Emacs inserts lines under headlines, inline
>> tasks, table rows and titles of the document, TOC, list of listings, list of
>> tables and footnotes.  The problem is it inserts too long (double width) line
>> when the line character has a width of 2.
>> 
>> Those lines are made of 3 types of characters below (in ox-ascii.el):
>> 1) org-ascii-underline
>> 2) (if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_)
>> 3) (if utf8p ?━ ?_)
>> 
>> In case of 1), it correctly takes account of the case in which the character
>> has a width of 2 in `org-ascii--build-title', by dividing the line width by
>> `(char-width under-char)' (line 700-701), maybe because the character is user
>> configurable and its width in unknown.  However, in case of 2) and
>> 3), maybe because the characters is embedded in the code, it looks like only
>> considering the character always has a width of 1.  But the reality is
>> character ?─ or ?━ can have a width of 2 in the screen displayed with some
>> fonts (ex. "Noto Sans Mono CJK JP"), and in that case the line width gets
>> doubled of the expected width.
>> 
>> Attached one is a potential patch.  The basic concepts are:
>> 
>> a) Do the same in case of 2) and 3) as in case of 1)
>>    (dividing the line width by `(char-width under-char)',
>>     assuming `char-width-table' is correctly set)
>>     
>> b) Prefer the longer line width if the width is odd, even in case of 1)
>>    (adding `(1- (char-width under-char))' to dividend,
>>     just because it should be more beautiful ;-) )
>
> You reported a similar bug already, and I replied there that TRT in
> these cases is to use window-text-pixel-size, which will automatically
> account for the actual width on display of any characters and any
> fonts specified for displaying them.  char-width is an approximation,
> and is accurate only on TTY frames.

Isn't the primary result of org-export a plain (UTF-8) text file,
instead of an emacs buffer to be displayed in a GUI or TTY frame?

If so, maybe the criterion for correctness should be that "cat
filename.txt" looks as expected in a terminal, even if opening that file
in Emacs shows lines of different lengths due to variable-pitch faces
etc.





^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#48149: 27.2; Wrong underline width when the line char has a width of 2
  2021-05-02  8:36   ` Rudolf Schlatte
@ 2021-05-02  9:16     ` Eli Zaretskii
  0 siblings, 0 replies; 6+ messages in thread
From: Eli Zaretskii @ 2021-05-02  9:16 UTC (permalink / raw)
  To: Rudolf Schlatte; +Cc: 48149

> From: Rudolf Schlatte <rudi@constantly.at>
> Date: Sun, 02 May 2021 10:36:14 +0200
> 
> > You reported a similar bug already, and I replied there that TRT in
> > these cases is to use window-text-pixel-size, which will automatically
> > account for the actual width on display of any characters and any
> > fonts specified for displaying them.  char-width is an approximation,
> > and is accurate only on TTY frames.
> 
> Isn't the primary result of org-export a plain (UTF-8) text file,
> instead of an emacs buffer to be displayed in a GUI or TTY frame?
> 
> If so, maybe the criterion for correctness should be that "cat
> filename.txt" looks as expected in a terminal, even if opening that file
> in Emacs shows lines of different lengths due to variable-pitch faces
> etc.

If the result is supposed to be displayed only on text-mode terminals,
then indeed string-width is the way to go (assuming that the terminal
in question will use fonts that will not break the alignment).
However, if the result is supposed to be displayed by a GUI program
such as Emacs, then string-width will not produce accurate results.

Maybe this is not important in this kind of export, in which case I
apologize for the noise.




^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#48149: 27.2; Wrong underline width when the line char has a width of 2
       [not found] <87fsz62b1w.wl-shingo.fg8@gmail.com>
  2021-05-02  7:17 ` bug#48149: 27.2; Wrong underline width when the line char has a width of 2 Eli Zaretskii
       [not found] ` <837dkh8uzi.fsf__22492.4155323365$1619940077$gmane$org@gnu.org>
@ 2021-05-02 16:08 ` Nicolas Goaziou
  2021-05-02 16:23   ` Eli Zaretskii
  2 siblings, 1 reply; 6+ messages in thread
From: Nicolas Goaziou @ 2021-05-02 16:08 UTC (permalink / raw)
  To: Shingo Tanaka; +Cc: 48149

Hello,

Shingo Tanaka <shingo.fg8@gmail.com> writes:

> When exporting org-mode document to plain text (either ascii/unicode/utf-8)
> with `org-export-dispatch', Emacs inserts lines under headlines, inline
> tasks, table rows and titles of the document, TOC, list of listings, list of
> tables and footnotes.  The problem is it inserts too long (double width) line
> when the line character has a width of 2.
>
> Those lines are made of 3 types of characters below (in ox-ascii.el):
> 1) org-ascii-underline
> 2) (if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_)
> 3) (if utf8p ?━ ?_)
>
> In case of 1), it correctly takes account of the case in which the character
> has a width of 2 in `org-ascii--build-title', by dividing the line width by
> `(char-width under-char)' (line 700-701), maybe because the character is user
> configurable and its width in unknown.  However, in case of 2) and
> 3), maybe because the characters is embedded in the code, it looks like only
> considering the character always has a width of 1.  But the reality is
> character ?─ or ?━ can have a width of 2 in the screen displayed with some
> fonts (ex. "Noto Sans Mono CJK JP"), and in that case the line width gets
> doubled of the expected width.
>
> Attached one is a potential patch.  The basic concepts are:
>
> a) Do the same in case of 2) and 3) as in case of 1)
>    (dividing the line width by `(char-width under-char)',
>     assuming `char-width-table' is correctly set)
>     
> b) Prefer the longer line width if the width is odd, even in case of 1)
>    (adding `(1- (char-width under-char))' to dividend,
>     just because it should be more beautiful ;-) )

Thank you. This looks good. I cannot apply it on "maint" branch,
however. Also, a proper commit message would be nice. Could you send an
updated patch?

Moreover, have you signed FSF papers already? This is above limit for
tiny changes.

Regards,
-- 
Nicolas Goaziou




^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#48149: 27.2; Wrong underline width when the line char has a width of 2
  2021-05-02 16:08 ` Nicolas Goaziou
@ 2021-05-02 16:23   ` Eli Zaretskii
  2021-05-09 13:57     ` Shingo Tanaka
  0 siblings, 1 reply; 6+ messages in thread
From: Eli Zaretskii @ 2021-05-02 16:23 UTC (permalink / raw)
  To: Nicolas Goaziou; +Cc: shingo.fg8, 48149

> From: Nicolas Goaziou <mail@nicolasgoaziou.fr>
> Date: Sun, 02 May 2021 18:08:50 +0200
> Cc: 48149@debbugs.gnu.org
> 
> > In case of 1), it correctly takes account of the case in which the character
> > has a width of 2 in `org-ascii--build-title', by dividing the line width by
> > `(char-width under-char)' (line 700-701), maybe because the character is user
> > configurable and its width in unknown.  However, in case of 2) and
> > 3), maybe because the characters is embedded in the code, it looks like only
> > considering the character always has a width of 1.  But the reality is
> > character ?─ or ?━ can have a width of 2 in the screen displayed with some
> > fonts (ex. "Noto Sans Mono CJK JP"), and in that case the line width gets
> > doubled of the expected width.
> >
> > Attached one is a potential patch.  The basic concepts are:
> >
> > a) Do the same in case of 2) and 3) as in case of 1)
> >    (dividing the line width by `(char-width under-char)',
> >     assuming `char-width-table' is correctly set)
> >     
> > b) Prefer the longer line width if the width is odd, even in case of 1)
> >    (adding `(1- (char-width under-char))' to dividend,
> >     just because it should be more beautiful ;-) )
> 
> Thank you. This looks good. I cannot apply it on "maint" branch,
> however. Also, a proper commit message would be nice. Could you send an
> updated patch?

Please note that using char-width cannot solve the problem of a
character whose width depends on the font, because char-width is
oblivious to fonts, it only knows about the character's codepoint.




^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#48149: 27.2; Wrong underline width when the line char has a width of 2
  2021-05-02 16:23   ` Eli Zaretskii
@ 2021-05-09 13:57     ` Shingo Tanaka
  0 siblings, 0 replies; 6+ messages in thread
From: Shingo Tanaka @ 2021-05-09 13:57 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: shingo.fg8, mail, 48149

[-- Attachment #1: Type: text/plain, Size: 3079 bytes --]

Hi,

> Please note that using char-width cannot solve the problem of a
> character whose width depends on the font, because char-width is
> oblivious to fonts, it only knows about the character's codepoint.

I updated my patch proposal as attached to use window-text-pixel-size based
on Eli's advice.  Could you check it to see if it meets the expectation?  It
works good in my environment with some fonts of different char widths.

Here are some comments:

- New internal functions org-ascii--make-string and org-ascii--pixel-width
  are introduced just to improve code readability of this modification
  
- Line width is decided by org-ascii--make-string, which is a pixel width
  based make-string

- org-ascii--make-string uses org-ascii--pixel-width, which returns
  actual pixel width of characters and strings by using
  window-text-pixel-size with frame default font

- Line justification is also modified to be a pixel width basis

Since this is not a simple modification, I think we might need further
improvement, so any feedback is appreciated.  Especially, we could do better
for table alignment, as that is not very easy because the pixel width of line
character and that of space character is not always the same.

Anyway, I appreciate it if you can give it a try.  I am doing FSF signing
process in parallel just in case.

---
Shingo Tanaka


On Mon, 03 May 2021 01:23:02 +0900,
Eli Zaretskii wrote:
> 
> > From: Nicolas Goaziou <mail@nicolasgoaziou.fr>
> > Date: Sun, 02 May 2021 18:08:50 +0200
> > Cc: 48149@debbugs.gnu.org
> > 
> > > In case of 1), it correctly takes account of the case in which the character
> > > has a width of 2 in `org-ascii--build-title', by dividing the line width by
> > > `(char-width under-char)' (line 700-701), maybe because the character is user
> > > configurable and its width in unknown.  However, in case of 2) and
> > > 3), maybe because the characters is embedded in the code, it looks like only
> > > considering the character always has a width of 1.  But the reality is
> > > character ?─ or ?━ can have a width of 2 in the screen displayed with some
> > > fonts (ex. "Noto Sans Mono CJK JP"), and in that case the line width gets
> > > doubled of the expected width.
> > >
> > > Attached one is a potential patch.  The basic concepts are:
> > >
> > > a) Do the same in case of 2) and 3) as in case of 1)
> > >    (dividing the line width by `(char-width under-char)',
> > >     assuming `char-width-table' is correctly set)
> > >     
> > > b) Prefer the longer line width if the width is odd, even in case of 1)
> > >    (adding `(1- (char-width under-char))' to dividend,
> > >     just because it should be more beautiful ;-) )
> > 
> > Thank you. This looks good. I cannot apply it on "maint" branch,
> > however. Also, a proper commit message would be nice. Could you send an
> > updated patch?
> 
> Please note that using char-width cannot solve the problem of a
> character whose width depends on the font, because char-width is
> oblivious to fonts, it only knows about the character's codepoint.

[-- Attachment #2: ox-ascii.el.patch --]
[-- Type: application/octet-stream, Size: 7108 bytes --]

--- ox-ascii.el.org	2021-03-26 09:28:44.000000000 +0900
+++ ox-ascii.el	2021-05-09 17:23:40.421912931 +0900
@@ -440,8 +440,11 @@
 ;; `org-ascii--describe-links' creates notes about links for
 ;; insertion at the end of a section.  It uses
 ;; `org-ascii--unique-links' to get the list of links to describe.
-;; Eventually, `org-ascii--translate' translates a string according
+;; `org-ascii--translate' translates a string according
 ;; to language and charset specification.
+;; `org-ascii--make-string' provides pixel width based `make-string' for
+;; better text alignment, by using `org-ascii--pixel-width' which returns
+;; actual pixel width of characters and strings.
 
 
 (defun org-ascii--fill-string (s text-width info &optional justify)
@@ -481,9 +484,19 @@
     (let ((fill-column text-width)
 	  ;; Disable `adaptive-fill-mode' so it doesn't prevent
 	  ;; filling lines matching `adaptive-fill-regexp'.
-	  (adaptive-fill-mode nil))
+	  (adaptive-fill-mode nil)
+          linestr pwidth)
       (while (< (point) (point-max))
-	(justify-current-line how)
+        (setq linestr (buffer-substring (point) (line-end-position))
+              pwidth (max (- (* (org-ascii--pixel-width ?\s) text-width)
+                             (org-ascii--pixel-width linestr))
+                          0))
+        (insert (org-ascii--make-string (pcase how
+                                          (`left 0)
+                                          (`center (/ pwidth 2))
+                                          (`right pwidth))
+                                        ?\s 0 (- text-width
+                                                 (string-width linestr))))
 	(forward-line)))
     (buffer-string)))
 
@@ -697,9 +710,8 @@
 			      (plist-get info :ascii-underline))))))
 	 (and under-char
 	      (concat "\n"
-		      (make-string (/ (string-width first-part)
-				      (char-width under-char))
-				   under-char))))))))
+		      (org-ascii--make-string
+                       (org-ascii--pixel-width first-part) under-char))))))))
 
 (defun org-ascii--has-caption-p (element _info)
   "Non-nil when ELEMENT has a caption affiliated keyword.
@@ -750,9 +762,9 @@
    (unless scope
      (let ((title (org-ascii--translate "Table of Contents" info)))
        (concat title "\n"
-	       (make-string
-		(string-width title)
-		(if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_))
+               (org-ascii--make-string
+                (org-ascii--pixel-width title)
+                (if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_))
 	       "\n\n")))
    (let ((text-width
 	  (if keyword (org-ascii--current-text-width keyword info)
@@ -779,8 +791,9 @@
   (let ((title (org-ascii--translate "List of Listings" info)))
     (concat
      title "\n"
-     (make-string (string-width title)
-		  (if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_))
+     (org-ascii--make-string
+      (org-ascii--pixel-width title)
+      (if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_))
      "\n\n"
      (let ((text-width
 	    (if keyword (org-ascii--current-text-width keyword info)
@@ -819,8 +832,9 @@
   (let ((title (org-ascii--translate "List of Tables" info)))
     (concat
      title "\n"
-     (make-string (string-width title)
-		  (if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_))
+     (org-ascii--make-string
+      (org-ascii--pixel-width title)
+      (if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_))
      "\n\n"
      (let ((text-width
 	    (if keyword (org-ascii--current-text-width keyword info)
@@ -1033,7 +1047,7 @@
 	     ;; Format TITLE.  It may be filled if it is too wide,
 	     ;; that is wider than the two thirds of the total width.
 	     (title-len (min (apply #'max
-				    (mapcar #'length
+				    (mapcar #'string-width
 					    (org-split-string
 					     (concat title "\n" subtitle) "\n")))
 			     (/ (* 2 text-width) 3)))
@@ -1041,12 +1055,19 @@
 	     (formatted-subtitle (when (org-string-nw-p subtitle)
 				   (org-ascii--fill-string subtitle title-len info)))
 	     (line
-	      (make-string
-	       (min (+ (max title-len
-			    (string-width (or author ""))
-			    (string-width (or email "")))
-		       2)
-		    text-width) (if utf8p ?━ ?_))))
+              (org-ascii--make-string
+               (apply #'max
+                      (mapcar #'org-ascii--pixel-width
+	                      (org-split-string
+		               (concat formatted-title "\n"
+                                       formatted-subtitle "\n"
+                                       (if author (concat author "\n") "")
+                                       (if email (concat email "\n") ""))
+                               "\n")))
+               (if utf8p ?━ ?_)
+               2 text-width)))
+        (message "%s" line)
+        (message "%s" formatted-title)
 	(org-ascii--justify-lines
 	 (concat line "\n"
 		 (unless utf8p "\n")
@@ -1082,9 +1103,9 @@
 	    (let ((title (org-ascii--translate "Footnotes" info)))
 	      (concat
 	       title "\n"
-	       (make-string
-		(string-width title)
-		(if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_))))
+               (org-ascii--make-string
+                (org-ascii--pixel-width title)
+                (if (eq (plist-get info :ascii-charset) 'utf-8) ?─ ?_))))
 	    "\n\n"
 	    (let ((text-width (- (plist-get info :ascii-text-width)
 				 global-margin)))
@@ -1150,6 +1171,29 @@
   (let ((charset (intern (format ":%s" (plist-get info :ascii-charset)))))
     (org-export-translate s charset info)))
 
+(defun org-ascii--make-string (pwidth init &optional adjust limit)
+  "Return a newly created string of width PWIDTH, with INIT in each element.
+PWIDTH is in pixel and the actual width is a minimum longer than
+PWIDTH, unless either ADJUST or LIMIT is specified.  ADJUST is
+the number of characters added or removed from the string if the
+value is positive or negative integer.  LIMIT is the maximum
+allowed number of characters of the string."
+  (let* ((init-pwidth (org-ascii--pixel-width init))
+         (width (/ (+ pwidth (1- init-pwidth)) init-pwidth)))
+    (if (integerp adjust) (setq width (+ width adjust)))
+    (if (integerp limit) (setq width (min width (max limit 0))))
+    (make-string width init)))
+
+(defun org-ascii--pixel-width (obj)
+  "Return OBJ pixel width in display.
+OBJ is supposed to be a character or a string.  Returned pixel
+width is based on frame default font."
+  (with-temp-buffer
+    (insert obj)
+    (switch-to-buffer (current-buffer))
+    (car (window-text-pixel-size
+          nil 1 (1+ (if (sequencep obj) (length obj) 1))))))
+
 
 \f
 ;;; Transcode Functions
@@ -1960,7 +2004,9 @@
 					'identity info t)
 				      cell))
 			 lcorner)
-		       (make-string (+ 2 width) (string-to-char horiz))
+                       (org-ascii--make-string
+                        (* (+ 2 width) (org-ascii--pixel-width ?\s))
+                        (string-to-char horiz))
 		       (cond
 			((not (memq 'right borders)) nil)
 			((eq (car (last (org-element-contents table-row))) cell)

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-05-09 13:59 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <87fsz62b1w.wl-shingo.fg8@gmail.com>
2021-05-02  7:17 ` bug#48149: 27.2; Wrong underline width when the line char has a width of 2 Eli Zaretskii
     [not found] ` <837dkh8uzi.fsf__22492.4155323365$1619940077$gmane$org@gnu.org>
2021-05-02  8:36   ` Rudolf Schlatte
2021-05-02  9:16     ` Eli Zaretskii
2021-05-02 16:08 ` Nicolas Goaziou
2021-05-02 16:23   ` Eli Zaretskii
2021-05-09 13:57     ` Shingo Tanaka

Code repositories for project(s) associated with this inbox:

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).