From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id YEpzFHCW12Yx7wAA62LTzQ:P1 (envelope-from ) for ; Tue, 03 Sep 2024 23:06:24 +0000 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id YEpzFHCW12Yx7wAA62LTzQ (envelope-from ) for ; Wed, 04 Sep 2024 01:06:24 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=kA53WmIr; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1725404784; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=HFzVO4LydSlQPy71F/+Fnirn7rMPwJjo13LBwaixgok=; b=q53pS/o/litHVafWyFohQ5wjVbw00kduiu8jmgKQF/zztnZljU7zFcXZ/Jc/8/bUOM1AC5 iPp/GkKGySIzCb7PvlmzvG6o7kpY+VGpIYcPioysp8KcAp5h59pDjXvQ73+0j8jhT+B8Eu OvTxVyyYrH3n65ip07wyVzH3+5UCPZaFjiVTSHYG9Hx0Zj1zkxKtTrASW0tbr2qzcOzdT9 4ffQxI+HSBDcqGJDESvpyMW3KXGa+fkJxbzjJzjIP7TA/tNGyINYrOdd8DjEcMkzjtutYk 6NQsoxjWDF9GyOMJjQDIitRoyzCGInq04ldtyldYm8YJwLBe0aispddktb+Ifw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=kA53WmIr; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=key1; d=yhetil.org; t=1725404784; a=rsa-sha256; cv=none; b=uJShTmv1j+FuA//jCFSZdZjlcGqcV+ahMoNWLl6OH/f0KFSyKngEvWIcs2vkt9k15lKrlU UpHL7pALQRfK1Qz/vnGNSgvUwb5e+pZEBIe06TXu3QJCpjik0cXx3+EO7AdNk5U1pP1Kpe K2N3KpyAaz6E9J+Q+yd8V1Xu9ya6AlatEe2r5ltheu0sYMgjBBuZ+Mt3vBaGpEKBNToQS6 YmHxv0237emfHG0nYvqUspaauxMqfsltMjp7A2ji8IHySCRWQpPSS6YZ2vnQaabhiUlG69 epCFqqRMaq6I3etuC9yxPct9f6WeYe0YVtAt81cLEfzw1eKR8VpM+E3m8diL/Q== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C8FC225628 for ; Wed, 4 Sep 2024 01:06:23 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slcab-00065e-TF; Tue, 03 Sep 2024 19:05:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slcaa-00065H-1S for emacs-orgmode@gnu.org; Tue, 03 Sep 2024 19:05:32 -0400 Received: from mail-lf1-x129.google.com ([2a00:1450:4864:20::129]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slcaX-0003Wo-JP for emacs-orgmode@gnu.org; Tue, 03 Sep 2024 19:05:31 -0400 Received: by mail-lf1-x129.google.com with SMTP id 2adb3069b0e04-5343617fdddso10202795e87.0 for ; Tue, 03 Sep 2024 16:05:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725404727; x=1726009527; darn=gnu.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=HFzVO4LydSlQPy71F/+Fnirn7rMPwJjo13LBwaixgok=; b=kA53WmIrgoSmodnss3xCd0AXNvzU2HV4e1qUBmxk3mO7wzkV0FzakAEQvRSRLcMrtF haahhCiaIGWZ7SxCrCG3NR0fYlArCKBzPaZZMQ7UWdfHgchjsAEt0mnSlOqY3muWUMRe x0oBuaGrS/OdP8oIjagZtarCQkilyNiRCOGIGpytyaFy1B6gKBcgNq3wydUrgdCEiaBR kW7cPTcZZlgOSQRwyk26Ic/QqWHyouQKOoYYPqAK0z6vs0b/gVGVj/bXQg85n9JyDafy e/ExGlECFNj7UBiPmhWtLlj1MtxzFWa9k8lhnBrRmlSSY2t+BhgfnAqS8A+oIo0wzS4Q PcbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725404727; x=1726009527; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=HFzVO4LydSlQPy71F/+Fnirn7rMPwJjo13LBwaixgok=; b=j85gq3j+nmko9jZgoom0GZNxYtvOKotgk9O6iqxINS7cM94s9oyZqz88oS/Gew35u7 SHXFQssyRR59tDFWu+AIuIP+zQvNJdnZRFmVoFS13pkVAVwTJ3Ghr8ZLVoYMe5NjvvYk 86xy/OixsX9RdXrmcMUhDzcyzKkzhwGx4q1eqpfseEbVxZUL1FxxJl+Vj7OOzq7SJ7Pz uikh/7Z3O1AObvJw9ZsE2S4cpr805BMtHEgynnDRJoeoX72OZS5G9hv7yWeCf0mjRLdv 4wDRcCdawrhfc++gNdKcNvsuPHzm/6GrVwzaKXIphjaaFd1a/iIF2ZtDV3EWCfnfedEc RUGg== X-Forwarded-Encrypted: i=1; AJvYcCUZhWnfbr27LWRPriwM8FuHGiMtulC8aB05jJMZbdMN4vv3oI/nn4lZ64yYfyLi9ozKe+NbcHQsGA9myWHg@gnu.org X-Gm-Message-State: AOJu0YyTJl57bP5cnxfuzDeRhNOgu8xuu7dMOc5x5yqsxzeRcQ2s4Oty Qyifxi48VlwfjUjfViOYvxnwviwEzqITD50nZE0IQ0zBUTOYbp0iskwIHhIFASpdgihR0e42W58 I644M52nB+IggIcEjiAlmWnck9xMLfcNiSoQ= X-Google-Smtp-Source: AGHT+IGCnkw/oE+8rRCJB7DddLsdwB/XyQDNMnxYxQRM0Vn6btmje1LWmOXubhGVWZOGLENwooG0jMg2KIE1HMqhJao= X-Received: by 2002:a05:6512:32c8:b0:52c:9ae0:beed with SMTP id 2adb3069b0e04-53546bfbf9dmr12440369e87.52.1725404726164; Tue, 03 Sep 2024 16:05:26 -0700 (PDT) MIME-Version: 1.0 References: <16ED0C3D-9C4E-49C0-B908-3E759BC31992@gmail.com> <87frvitp30.fsf@gmail.com> <87cyo8udza.fsf@localhost> <87le2rplrq.fsf@gmail.com> <87cylkn72b.fsf@localhost> <871q20ag88.fsf@gmail.com> In-Reply-To: <871q20ag88.fsf@gmail.com> From: Jun Inoue Date: Wed, 4 Sep 2024 08:04:55 +0900 Message-ID: Subject: Re: [Pre-PATCH] Overhaul of the LaTeX preview system To: Yaroslav Drachov Cc: Ihor Radchenko , Karthik Chikmagalur , orgmode@tec.tecosaur.net, emacs-orgmode@gnu.org Content-Type: multipart/alternative; boundary="000000000000cc045606213f17ec" Received-SPF: pass client-ip=2a00:1450:4864:20::129; envelope-from=jun.lambda@gmail.com; helo=mail-lf1-x129.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Spam-Score: -7.30 X-Migadu-Queue-Id: C8FC225628 X-Migadu-Scanner: mx10.migadu.com X-Migadu-Spam-Score: -7.30 X-TUID: Ed2FyKMElIxY --000000000000cc045606213f17ec Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Yaroslav, I had a similar issue. The problem there was that the relative path ../../../../var/folders/3m/9xzpnkks773bfdwzqlvr421r0000gn/T/org-tex-BYOo4y-= 000000001.svg was being expanded in the wrong directory. Can you see if the problem reproduces with the attached patch? (You could also just find the string "{?svgfile}" in (alist-get 'dvisvgm org-latex-preview-process-alist) and replace it with "{?svgpath}". Here's a copy of the email I sent to this list regarding the issue. It seems to have fallen through the cracks, so it would be great if someone could have a look, independently of Yaroslav's issue if it turns out to be different. Recently, the previews stopped working after a system update. * Symptom On macOS with dvisvgm 3.2+, org-latex-preview can't typeset LaTeX fragments at all in a lot of files. It gives an error like the following: error in process filter: Opening input file: No such file or directory, /var/var/folders/8g/k689fc1j3gg2ny7xrzqchjsm0000gn/T/org-tex-ZqKlQw-0000000= 01.svg [2 times] ** Steps to Reproduce Put dvisvgm 3.2+ on your path and create a .org file anywhere outside temporary-file-directory that is shallower than (i.e. has fewer slashes than) temporary-file-directory. Try to preview any LaTeX fragment in that file. This should theoretically reproduce on non-macOS, but macOS has an especially deep temporary-file-directory that looks like /var/folders/8g/k689fc1j3gg2ny7xrzqchjsm0000g, making it easier to reproduce. If temporary-file-directory is /tmp on your system, I think you have to make the .org file in the root directory or change temporary-file-directory to a deeper directory (haven't tried). * Cause This is because org-latex-preview: 1. Runs dvisvgm in the same directory as the .org file, and 2. Tells dvisvgm to report the output SVG file names as relative paths using ?svgfile, but 3. Then expands those file names under temporary-file-directory in org-latex-preview--dvisvgm-filter. So dvisvgm relativizes in one path, then org-latex-preview expands in another. For instance, if I try to typeset fragments in /Users/jun/org/test/test.org= , then dvisvgm reports: output written to: ../../../../var/folders/8g/k689fc1j3gg2ny7xrzqchjsm0000gn/T/org-tex-CWRtgi-= 000000001.svg where /var/folders/8g... is the absolute path of the output file. This is expanded under temporary-file-directory, which is /var/folders/8g/k689fc1j3gg2ny7xrzqchjsm0000gn/T, leading org-latex-preview to busy-wait for files to arrive in /var/var/folders/8g/k689fc1j3gg2ny7xrzqchjsm0000gn/T. I think this is hard to reproduce on most non-macOS systems where the temporary-file-directory is shallow. ?svgfile contains as many ../ as there are slashes in the directory hosting the .org file, and expanding that relative to (say) /tmp will effectively remove all those ../, making it expand to the correct absolute path by accident. * Fix Use ?svgpath instead of ?svgfile. Here's a patch that applies to commit 9584a76a843e2e8122799e5653bb9120fe568f64 of [[ https://git.tecosaur.net/tec/org-mode.git]]. modified lisp/org-latex-preview.el @@ -200,7 +200,7 @@ Place-holders only used by `:image-converter': (list (concat "dvisvgm --page=3D1- --optimize --clipjoin --relative --no-fonts" (if (>=3D org-latex-preview--dvisvgm3-minor-version 2) - " -v3 --message=3D'processing page {?pageno}= : output written to {?svgfile}'" "") + " -v3 --message=3D'processing page {?pageno}= : output written to {?svgpath}'" "") " --bbox=3Dpreview -o %B-%%9p.svg %f")))))) (defcustom org-latex-preview-compiler-command-map @@ -2792,7 +2792,7 @@ EXTENDED-INFO, and displayed in the buffer." (when (save-excursion (re-search-forward "output written to \\(.*.svg\\)$" end t)) (setq fragment-info (nth (1- page) fragments)) - (plist-put fragment-info :path (expand-file-name (match-string 1) temporary-file-directory)) + (plist-put fragment-info :path (match-string 1)) (when (save-excursion (re-search-forward "^ page is empty" end t)) (unless (plist-get fragment-info :error) ** Environment GNU Emacs 29.1 (build 1, aarch64-apple-darwin21.6.0, Carbon Version 165 AppKit 2113.6) of 2023-08-09 (Brew tap railwaycat emacs-mac) dvisvgm 3.2.2 On Wed, Sep 4, 2024 at 12:51=E2=80=AFAM Karthik Chikmagalur < karthikchikmagalur@gmail.com> wrote: > >> Sorry for delay. > >> > >> 1. pdflatex > >> 2. processing page 1: output written to > ../../../../var/folders/3m/9xzpnkks773bfdwzqlvr421r0000gn/T/org-tex-BYOo4= y-000000001.svg > >> 3. nil > > > > Karthik, any thoughts? > > > > It will be useful to know if that svg exists at that path. I've had > reports from other Mac users complaining about dvisvgm reporting the wron= g > path. > > Karthik > > --=20 Jun Inoue --000000000000cc045606213f17ec Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Yaroslav, I had a similar issue.=C2=A0 The problem th= ere was that the relative path ../../../../var/folders/3m/9xzpnkks773bfdwzq= lvr421r0000gn/T/org-tex-BYOo4y-000000001.svg was being expanded in the wron= g directory.=C2=A0 Can you see if the problem reproduces with the attached = patch?=C2=A0 (You could also just find the string "{?svgfile}" in= (alist-get 'dvisvgm org-latex-preview-process-alist) and replace it wi= th "{?svgpath}".


Here's a copy of the email I sent to this list regard= ing the issue.=C2=A0 It seems to have fallen through the cracks, so it woul= d be great if someone could have a look, independently of Yaroslav's is= sue if it turns out to be different.

Recently,= the previews stopped working after a system update.

* Symptom
On macOS with dvisvgm 3.2+,= org-latex-preview can't typeset LaTeX fragments at all in a lot of fil= es.=C2=A0 It gives an error like the following:

error in process fil= ter: Opening input file: No such file or directory, /var/var/folders/8g/k68= 9fc1j3gg2ny7xrzqchjsm0000gn/T/org-tex-ZqKlQw-000000001.svg [2 times]
<= div>
** Steps to Reproduce
Put dvisvgm 3.2+ on your path and create a .org file anywhere outside=20 temporary-file-directory that is shallower than (i.e. has fewer slashes=20 than)=C2=A0temporary-file-directory.=C2=A0 Try to preview any LaTeX fragmen= t in that file.

This should theoretically reproduce on non-macOS, but macOS has an=20 especially deep temporary-file-directory that looks like=20 /var/folders/8g/k689fc1j3gg2ny7xrzqchjsm0000g, making it easier to=20 reproduce.=C2=A0 If temporary-file-directory is /tmp on your system, I thin= k=20 you have to make the .org file in the root directory or change=20 temporary-file-directory to a deeper directory (haven't tried).

* Cause
This is because org-latex-preview:
1. R= uns dvisvgm in the same directory as the .o= rg file, and
2. Tells dvisvgm to report = the output SVG file names as relative paths using ?svgfile, but
3. Then = expands those file names under temporary-file-directory in org-latex-previe= w--dvisvgm-filter.

So dvisvgm relativizes in one path, then org-latex-preview e= xpands in another.

For instance, if I try to typeset fragments in /U= sers/jun/org/test/test.org, then dvisvgm reports:

output writ= ten to: ../../../../var/folders/8g/k689fc1j3gg2ny7xrzqchjsm0000gn/T/org-tex= -CWRtgi-000000001.svg

where /var/folders/8g... is the absolute path of the output file.=C2=A0 This is= =20 expanded under temporary-file-directory, which is /var/folders/8g/k689fc1j3= gg2ny7xrzqchjsm0000gn/T, leading org-latex-preview to busy-wait for files t= o arrive in /var/var/folders/8g/k689fc1j3gg2ny7xrzqchjsm0000gn/T.

* Fix
Use ?svgpath instead of ?svgfile.=C2=A0 Here's a = patch that applies to commit 9584a76a843e2e8122799e5653bb9120fe568f64 of [[= = https://git.tecosaur.net/tec/org-mode.git]].

modified =C2=A0 lis= p/org-latex-preview.el
@@ -200,7 +200,7 @@ Place-holders only used by `:= image-converter':
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 (list
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0(concat "dvisvgm --pa= ge=3D1- --optimize --clipjoin --relative --no-fonts"
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0(if (>=3D org-latex-preview--dvisvgm3-minor-version 2)
- = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0" -v3 --message=3D'processing page = {?pageno}: output written to {?svgfile}'" "")
+ =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0" -v3 --message=3D'processing page {?pa= geno}: output written to {?svgpath}'" "")
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0" --bbox=3Dpreview -o %B-%%9p.svg %f"))))))
=C2= =A0
=C2=A0(defcustom org-latex-preview-compiler-command-map
@@ -2792,= 7 +2792,7 @@ EXTENDED-INFO, and displayed in the buffer."
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0(when (save-excursion
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(re-search-forward "output wr= itten to \\(.*.svg\\)$" end t))
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0(setq fragment-info (nth (1- page) fragments))
- =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0(plist-put fragment-info :path (expand-file-name (match-st= ring 1) temporary-file-directory))
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(= plist-put fragment-info :path (match-string 1))
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0(when (save-excursion
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(re-search-forward "^ =C2=A0page= is empty" end t))
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= (unless (plist-get fragment-info :error)

** Environment
GNU Emacs 29.1 (build 1, aarch64-apple-darwin21.6.0, Carbon Version 165=20 AppKit 2113.6) of 2023-08-09 (Brew tap railwaycat emacs-mac)

dvisvgm 3.2.2

On Wed, Sep 4, 2024= at 12:51=E2=80=AFAM Karthik Chikmagalur <karthikchikmagalur@gmail.com> wrote:
>> Sorry for delay.
>>
>> 1. pdflatex
>> 2. processing page 1: output written to ../../../../var/folders/3m= /9xzpnkks773bfdwzqlvr421r0000gn/T/org-tex-BYOo4y-000000001.svg
>> 3. nil
>
> Karthik, any thoughts?
>

It will be useful to know if that svg exists at that path.=C2=A0 I've h= ad reports from other Mac users complaining about dvisvgm reporting the wro= ng path.

Karthik



--
Jun Inoue --000000000000cc045606213f17ec--