From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id WF/yNyjaEWFtvwAAgWs5BA (envelope-from ) for ; Tue, 10 Aug 2021 03:45:12 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id GKqlMyjaEWFPYQAA1q6Kng (envelope-from ) for ; Tue, 10 Aug 2021 01:45:12 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 37DDB8924 for ; Tue, 10 Aug 2021 03:45:12 +0200 (CEST) Received: from localhost ([::1]:56124 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mDGpJ-0006N4-UW for larch@yhetil.org; Mon, 09 Aug 2021 21:45:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59898) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mDGoc-0006LL-G9 for emacs-orgmode@gnu.org; Mon, 09 Aug 2021 21:44:26 -0400 Received: from mail-pj1-x1036.google.com ([2607:f8b0:4864:20::1036]:33613) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mDGoa-0005eE-TO for emacs-orgmode@gnu.org; Mon, 09 Aug 2021 21:44:26 -0400 Received: by mail-pj1-x1036.google.com with SMTP id 28-20020a17090a031cb0290178dcd8a4d1so565284pje.0 for ; Mon, 09 Aug 2021 18:44:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=references:user-agent:from:to:subject:date:message-id:in-reply-to :mime-version; bh=6k7Hbohq0XUd9aeHlSNNh7u28fNUDMCrmqdpb4E/7Vk=; b=cEk4XgImJmprLBmi8a7wGF6J2+uKtxrlIATzfOOnrkACoHqkOU/RgdzyUx7sFePEgX gK2Du/tLJuz8gOh4X81BmRvEO5P7zTOQD/iZPXQ3ufdbkC+QNmA+Zhkti4ZBuBfXdFyO Didbhk8blQwb3mC0kw8OrGzSLr/pKaddf5atyNBN46IURyO7jzz1hdNnzYl2NV0/djdP 10GUdWVfG/gRFKXHys6lT9xgO0odxgXMKJvMKhJDV9hgsxmSWiz47UMeZ3hby+iOVSu6 1JEL2wuC2B1dV9RuHGkq+tZNSt2RBEiTuKzqD8qw7vxd1qI1voZ0gq1/FJMtO3gzFkdt ynKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:references:user-agent:from:to:subject:date :message-id:in-reply-to:mime-version; bh=6k7Hbohq0XUd9aeHlSNNh7u28fNUDMCrmqdpb4E/7Vk=; b=LKp62i6nHB6jlSydPVfAqGw4p/LHO1tKtCG6Pw3xBJKuHsEQe2q1AOZYCTzPMPa7Xe yxZeb0r1lqXK/mEsZUn1+zHc0NsEPYyzeSXrCgfq+hVpWqDf/GOnTd7f+JKeAHWkO4eF z918fR7X82OjmQx8ZxpI4/s6DhFFThoGwsWc0oZcSCPDvhurHqJdE2iTdDtxMoWCTzTM eilXiI0hAFeU/lX7jsPQQ6f/lHqYdwdwxzpEvdLn5ZHL1eJnUDJDeIwIVKD7GDrVIdOW hCKD1iJ7Pzn+3To0PaN34OQy5deSOA3YKTuKyHfKOaG370HEnzYLHMXzcMYnnwxI2lT1 ZNuQ== X-Gm-Message-State: AOAM532OXojDbUQbGkP3TkMixq2HAr/sE7+wP4IoAxilTF12i/KtppD1 oCW+LBi0R7D1lIxNRcmVzcui4VFxdgw= X-Google-Smtp-Source: ABdhPJxuWlkIAEAA8PsjsaMINGKzJsPWxhiI8ODuCa+zc8MFrjyM5A2q07qPmr3Q1eMLrBD+EARF5g== X-Received: by 2002:a17:90a:74c8:: with SMTP id p8mr16638890pjl.214.1628559862656; Mon, 09 Aug 2021 18:44:22 -0700 (PDT) Received: from tim-desktop (106-69-140-77.dyn.iinet.net.au. [106.69.140.77]) by smtp.gmail.com with ESMTPSA id c133sm4447160pfb.174.2021.08.09.18.44.21 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Aug 2021 18:44:22 -0700 (PDT) References: User-agent: mu4e 1.6.2; emacs 28.0.50 From: Tim Cross To: emacs-orgmode@gnu.org Subject: Re: [BUG] Fill-paragraph issue with hidden emphasis markers and composed characters Date: Tue, 10 Aug 2021 11:34:23 +1000 Message-ID: <87pmumgjl4.fsf@gmail.com> In-reply-to: X-Woof-Bug: confirmed MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::1036; envelope-from=theophilusx@gmail.com; helo=mail-pj1-x1036.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1628559912; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=6k7Hbohq0XUd9aeHlSNNh7u28fNUDMCrmqdpb4E/7Vk=; b=Y8ukzUGOc+a0sY+Y+eEK6b3d3KmGWiDsOAyfvVF+7mdkHM8XvNXOEC4B0LCE4pGhBaTuF8 x9N0tvRbnQbvixaYbhb4tsNwcN23C0GqNn5EW5JYiSJHXJMjUC8bJS3op3/iwxO0wC5bUt dq6Phs0eciVfVp98sp/mo4VNE1GkjGMUoy9Z1dGRc+GilHpYywiK/TwvB7ZEMyA4OvYqBw F7EH/DBlxETovCODcIkE61PZSqOJxcQ+ZpecOONRQu2hxneGWFC6iTeuoaNMkwVOByapJq AcQJ6haKEuV0zknoiyMzaQFdvanhRczHaBN4n5gMEcWBrxD4G7MaW0omCigrhg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1628559912; a=rsa-sha256; cv=none; b=WKu4SGcT9aLd3pRoMJE3i4csIXja/wzyQUau3bQ/Ek5tchEN1ccnnlhd6pNNLFFjQL3saj h9bD0R/dd24nBFWMEGSsgWv7MRNhu68U46vtdw4SYffvPIRTdkCcvCDDYecxrcAw1nNtMB AcCKMZQ/9YyumxUWcZdIMClAcWYxcntHLIkTwMVaI3ej19dzRwBPYEtpr28DZ0xMCK9upx nwHay4Ys126DseOswAKVxLMNbo7GF0bttEdaQzIGOaXG/l0U+llfh54afuvR675G602CCU EnaqUMTc7fPfMZ5U8ekgBtDdiNZvEZ+7pyw1Ag0VmIn8tn5abHHdHs7AqbZVRA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=cEk4XgIm; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Spam-Score: -3.11 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=cEk4XgIm; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Queue-Id: 37DDB8924 X-Spam-Score: -3.11 X-Migadu-Scanner: scn1.migadu.com X-TUID: oD2K2EyflQVH I was able to reproduce this using Emacs 28.0.50 and emacs -Q. It is important to ensure both of the below paragraphs are completely 'unwrapped' i.e. all on one line and then wrap them with M-q. The result seems counter-intuitive to me - the second paragraph with an extra character is wrapped correctly. The first paragraph has the first line broken too early. This is likely going to be a very difficult issue to fix. I'm not sure how fill column mode is aware of add on features like prettify symbols and hidden emphasis markers. Obviously it is aware at some level because it does wrap the second paragraph correctly. My wild guess would be that the second one works because the additional character breaks up the 'special' handling and allows fill column to correctly calculate the 'virtual' line length. Lucas Viana writes: > Dear list, > > I'm not sure if this is a general issue with fill-paragraph or with the way org hides emphasis markers, but I'm facing a problem with bare emacs + > org. > > Steps to reproduce: > 1. Start emacs -Q (no configuration) > 2. Set some variables: > > (setq org-pretty-entities t > org-hide-emphasis-markers t > fill-column 80) > > 3. Create an empty org-mode buffer with the following paragraph: > "" > This is a /italic sentence for a buggy/ \alpha\beta\gamma\delta\nu\epsilon > paragraph in terms of filling, see, it's not filled yet and still the line above > ends early. > "" > > So far so good, and with the previous config the emphasis markers are now hidden and the greek letters appear as unicode characters. > > 4. Hit M-q (either fill-paragraph or org-fill-paragraph, both have the same issue) > The first line wrongly wraps after "buggy" and leaves a big empty space after that, as if the greek letters were not composed. > > 5. In the same buffer, compare the behaviour with this paragraph (very similar, but note the ! character after "buggy"): > "" > This is a /italic sentence for a buggy/ ! \alpha\beta\gamma\delta\nu\epsilon > paragraph in terms of filling, see, it's not filled yet and still the line above > ends early. > "" > > When I fill this slightly different paragraph with M-q, it wraps the text in the right way and the first line is completely filled. > > So the issue happens when a hidden emphasis marker is immediately followed (maybe after a space) by a composed character like \alpha. I am also > facing the same issue when the characters are composed by prettify-symbols-mode instead of org-pretty-entities, and also with other types of > hidden emphasis markers. If org-hide-emphasis-markers is disabled then the issue doesn't happen. > > Thanks in advance! > > Emacs : GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.29, cairo version 1.17.4) > of 2021-07-09 > Package: Org mode version 9.5 (9.5-b83ae59 @ /home/lucas/.emacs.d/.local/straight/build-28.0.50/org/)