From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id mNtLMUVHkWL9SgAAbAwnHQ (envelope-from ) for ; Fri, 27 May 2022 23:48:53 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id UEBYMEVHkWL2UQEAG6o9tA (envelope-from ) for ; Fri, 27 May 2022 23:48:53 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 5FAE13AB5D for ; Fri, 27 May 2022 23:48:53 +0200 (CEST) Received: from localhost ([::1]:53338 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nuhpE-0007zv-4W for larch@yhetil.org; Fri, 27 May 2022 17:48:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53488) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nuhnV-0006KY-Qo for emacs-orgmode@gnu.org; Fri, 27 May 2022 17:47:08 -0400 Received: from mail-yb1-xb33.google.com ([2607:f8b0:4864:20::b33]:43643) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nuhnU-0004Oc-BH for emacs-orgmode@gnu.org; Fri, 27 May 2022 17:47:05 -0400 Received: by mail-yb1-xb33.google.com with SMTP id s14so9933447ybc.10 for ; Fri, 27 May 2022 14:47:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:from:date:message-id:subject:to; bh=PMAKffuEKkJ/B/QOlKwzjLfkkaoWJOGZ3jLSeAqSllU=; b=gKikq1iWjkNBXokerMfiGcJmKTUNuV8lJdXfuaKQRDaXfa7rCJTrPv9fzZlkeY68ah Idk4EjF8C4opRzCAVie2lS60DP9W6+QwaoFmueRmCYne/PsidTn4NGvTrNH7cLbLijFo U3yG+yzk8E423ws/tPWXMZ70dzkmMKGgHyI1iJcOkTn9Vy7O6s/5/VcPpgScfqrW4S1Y bNTqGabhlGUrvuEJANjEi3avgF9gAmez1RXtI5kBGyLKNMnErSVH1wOm96B6Z7+3fZ4H 800RpEo3Vjc3iPD/2shlUP9D2Fj2BpfPk3yBnLKm9Ds7btN1w7yg7LSvVKPecO6DoZ2w Dx8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=PMAKffuEKkJ/B/QOlKwzjLfkkaoWJOGZ3jLSeAqSllU=; b=GsrNOIfGFr/Vt6Ux7N+dSH6SgQqexHV9BxYzLRJZ9ZPiZ/Eclx7GLS0BAKIEz4aktm 9ClDAY9abR8M04Vz7oi3EqFH8FkHyt92dWerUbcfQhdTtfDDn9fwFdhBcMUzOxKKO3kU E4BAN66+FPITr+tfz8zJwRSAmatQTrqBsJpDAc5vhWkVbh7RLHK81bHCdR/qk3ZveX8y 8INk/nPkAaNEvJcm2WCzracwwd+eXMZOzF2x2hVwh67qEHRwbcIu+ZouTt7/pDjn/aO8 OHPK2zFRqdAmSV+lS2M6zAwEBJhMorS0W1yw4bITGqBzLB6+cdhZXD271V0MsVVNR1OC CyTA== X-Gm-Message-State: AOAM5319FuCWVjif2xXVz7LBUtR3RwxqzL0q7JVwhPMHVG0YqfuMG1Ig XdcbriyvbJnlBnZhW/LfK6jeutC5h5cgYkOP9jxx+0ky X-Google-Smtp-Source: ABdhPJzUP4b3eF4tH8ZUsyM7Doltf+XLf1b7lKB5pqIUSIoExb2ZNFW77VDZfDaaqfGA5k690q233HIDH3M5RXQlqhw= X-Received: by 2002:a25:9742:0:b0:64e:2c40:b33e with SMTP id h2-20020a259742000000b0064e2c40b33emr42582920ybo.455.1653688022873; Fri, 27 May 2022 14:47:02 -0700 (PDT) MIME-Version: 1.0 From: Tom Gillespie Date: Fri, 27 May 2022 14:46:52 -0700 Message-ID: Subject: org-persist-gc and tramp To: emacs-orgmode , Ihor Radchenko Content-Type: text/plain; charset="UTF-8" Received-SPF: pass client-ip=2607:f8b0:4864:20::b33; envelope-from=tgbugs@gmail.com; helo=mail-yb1-xb33.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1653688133; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=PMAKffuEKkJ/B/QOlKwzjLfkkaoWJOGZ3jLSeAqSllU=; b=QCdE2CrgnSR4fGlxyRMXHw1LIi+2wrAblhZNUhKhg+Kq7mWjEdosi7jhUMGB/fw4LKo5yL wBieMhKB/y9IwwYJgJBGgD44tuLAxJZ/fDjphn1H+CjQQggeqbfOa3jYcSXlAUgxXpj1Yo VnxiSo9afEpq0avtGXKixwOT3F/KcMsYS5XZfVkqw4GucV5qVoOE9bUhWQ0SwQa5R1hA7I N//K3qLJSWLvMjZQXLOCt6ZTVj6HjqJwx3dWbH5ov0XNgCKLLQMdoKZk9xyk4z8Oxxda0B MTQL4mUDwoyS+7FADOEKHzAAXAk4/+q2+zOoZUNvZJ4GO6FqHKlA6nVlqu/otg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1653688133; a=rsa-sha256; cv=none; b=D9VneAHyYMDvyUmaHgIEqQtlghqWpM0AtkHVeXGP9YsY8OKpA5Mgi/3aRIQ9rEiomo2cz6 7GB+jDk/GVRkmgLg6i5tOsPs7pxaeaViNexkYWWyweCgIBJh01Rh8VcUGEpSNG5FRZ2O+H 4yMsbY/CMqhVupVXEZTGCABXmDOoY07LFH4HMf72gjq+kX+XLHJfNWG9sc5eSZQyavwKCq UDSaDsgXvtN+YE2YtECcqL9H+nHa/5P+XtLth3KJu53o7/e+66+w8pt+LmByUETiYF1H/L ge/EbdUV/nWLLt77W2h45vHgW4YUdd+Divti7hM+7w9Si42YaHnwPOQvse+lDw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=gKikq1iW; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.05 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=gKikq1iW; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 5FAE13AB5D X-Spam-Score: -4.05 X-Migadu-Scanner: scn1.migadu.com X-TUID: 8hXm/O4ADqJD While debugging an unrelated issue I noticed that tramp was initiating a new connection every time save-buffers-kill-emacs ran. I eventually tracked it down to org-persist-gc running on kill-emacs-hook. It turns out that org-persist--index held a record for a remote file, and the transparency of tramp means that emacs will happily open any number of remote connections to check whether such files exist on exit. This could produce unexpected and quite bad behavior for users that edit many remote org files. It also leaks information that a user has closed emacs over the network even if they have not intentionally made any remote connections. I see a couple potential fixes. - skip gc for remote files - never persist remote files - +only gc remote files when their cache is expired+ I think the last option might be the most reasonable compromise? Er, nope, I was looking at an old/variant cache location that still had :expiry values in the plist, but it looks like the most recent version does not, so we are left with option 1 or option 2. Given that the network latency is likely to dominate accessing such files the time to reparse if we don't persist (option 2) seems like it will be small? I think for option 1 to work safely there would need to be a way to periodically gc remote files, maybe when another file on that remote was accessed so an existing tramp connection would be used? Thoughts? Tom