From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 0PUNE6L3vWGRLQAAgWs5BA (envelope-from ) for ; Sat, 18 Dec 2021 16:00:50 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id MPKmDqL3vWEzRAAAbx9fmQ (envelope-from ) for ; Sat, 18 Dec 2021 15:00:50 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id AE483A84C for ; Sat, 18 Dec 2021 16:00:49 +0100 (CET) Received: from localhost ([::1]:48114 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1myaZu-0008Vl-Lp for larch@yhetil.org; Sat, 18 Dec 2021 09:20:50 -0500 Received: from eggs.gnu.org ([209.51.188.92]:36886) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1myaYx-0008EZ-B6 for emacs-orgmode@gnu.org; Sat, 18 Dec 2021 09:19:51 -0500 Received: from [2607:f8b0:4864:20::52a] (port=47046 helo=mail-pg1-x52a.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1myaYv-00088l-GS for emacs-orgmode@gnu.org; Sat, 18 Dec 2021 09:19:51 -0500 Received: by mail-pg1-x52a.google.com with SMTP id r138so4939275pgr.13 for ; Sat, 18 Dec 2021 06:19:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=SPQe/xFq5jc7Q2Epf4LYcd4s/jLgmrb+1bDQlzePxlc=; b=Thy7H6QCroQ/ZI84WVTqiCk6fcDIMv8LkZqtQw98W7sD1CfvTXcrGnmRf0GVpsaJfj VZtfKVIOR44if9FdMvuV4cgkgpvfAKB4zuV1SzazSgPLXzGeBkYJ+Rg7jCHCIe8plz5S ZssXTOhoceXHBUrx5wf93hmVnZnOwlngywPv+8gaGZb9RF80Wi8bCZrYLiGwN7zwhHtW AbWoZbrgP7KvRrrRhecRiFbHm86X0FtRsQ7k5N1Nlm0zW/TMCD36xlMXKUCGg/50JiDf 5lc3/4sHDorBoaVjtsHx3uDUn/stX2uUKtSoPjbbSLC63Nr4TSideT21VXXYfQIU/zt9 XH+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=SPQe/xFq5jc7Q2Epf4LYcd4s/jLgmrb+1bDQlzePxlc=; b=1KO3Z4naRW2DPnzmUBOvm5esCGr4s8YQZiW17gTfdD4KKVdi/EnutowSjje65XasDz S+LWDLArlRdsdxaeLCopoFwkOhDfn8s29vF+OxdH7JSkQzOy+mbZf20JAyuWq1hPF39L +WdbWNwRQQvsfuUFzH/959hkD5F97CaxrH9aK0SX0kBXjL4KEd75Y4rRk3VAOYQEJXNt 5SM3o1RoGaicUnd2VbKqtHeT6kJWFwxK02qVJI397zLIDIc8DePbKbCj4jQu5NnXgF6b A17fI4hYJYihtMfQQpNnDpNL/Uc2EpN0egEc3g9pRfliPKAJzqHh2wje4MJiUL0fTNfp +4zg== X-Gm-Message-State: AOAM531z0io1nSTDEb2/9dfVcLux3RPLfCmF47KEVd5aMmLMQzV/6tvw Xb1nEhMBXLP5UjctxahoimgFBflMR4wGRWpk X-Google-Smtp-Source: ABdhPJwznsw/zMuSfP5MaBIqisDqPEue4ikR803Fqi0bN1i848M35c8GK42CwZbgCqBaYYhE77PIiQ== X-Received: by 2002:a63:6a83:: with SMTP id f125mr7134166pgc.340.1639837188252; Sat, 18 Dec 2021 06:19:48 -0800 (PST) Received: from localhost ([50.7.251.66]) by smtp.gmail.com with ESMTPSA id i9sm5359681pjs.4.2021.12.18.06.19.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 18 Dec 2021 06:19:47 -0800 (PST) From: Ihor Radchenko To: Lele Gaifax Subject: Re: org-persist cache for remote files In-Reply-To: <87ilvmqujm.fsf@metapensiero.it> References: <87o87kb64s.fsf@yandex.com> <87tuhcfc3n.fsf@localhost> <87czo0nqw4.fsf@yandex.com> <87o87kchi2.fsf@localhost> <8735owawla.fsf@yandex.com> <87lf2oc9sv.fsf@localhost> <87ee8fncmw.fsf@yandex.com> <87fssvdfxj.fsf@localhost> <87wnk3c58c.fsf@metapensiero.it> <87bl1eilt6.fsf@localhost> <87ilvmqujm.fsf@metapensiero.it> Date: Sat, 18 Dec 2021 22:21:06 +0800 Message-ID: <87y24i2dv1.fsf@localhost> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Host-Lookup-Failed: Reverse DNS lookup failed for 2607:f8b0:4864:20::52a (failed) Received-SPF: pass client-ip=2607:f8b0:4864:20::52a; envelope-from=yantar92@gmail.com; helo=mail-pg1-x52a.google.com X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: emacs-orgmode@gnu.org Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1639839649; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=SPQe/xFq5jc7Q2Epf4LYcd4s/jLgmrb+1bDQlzePxlc=; b=XvRB/5oar6B/lrEoP8Qp8d5l72EHUi60fqvtMUhsZNnF2KZKRqW1uHP85QNRRomDPC9eDd 3ckTzXMi0A88imB3rzrl1h2xTu+tQO952+/Zp4rYyWNVSDyu9qpafLNSlOTxxzDdGRTLh9 tNuv7O2y1zt+gRnAcxyeVIIlpBzxF8sGdUScC4LMJLR5L4lw5CQZy5AW4OT9QazZ5cQjDS JO4+OLvMbDbmQg9pR3MA1MmoRnEYyks6AWSh5INeGvguPQ8qv+H0yXmgHxYjTA4NyPTBpo SF9znf/zXFzKgyisbIx0nFL4GtowwOQmYSVcgbaxcs8gIOUgF2k75KuGEePk3Q== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1639839649; a=rsa-sha256; cv=none; b=ZCFTkvj2Q6Myevg7vNLe3ZXTaY45LM/mNeAa7CzMqFauqbsohRgGOB+UWuF2EpertkuHPW gkxTV91KbQxKU3JWKks0CrsYNdfX/+A3aUxgm/p7enA5lYVBMTwOq+ghuPihau79tG7qn0 TOelV19IVZnKfxTsaNXfduJ4qiJMS910CwZjfe7ZesWcm0+JgjyjZw6QxHt93/TjoUUKTB QW0VekTY1yvEiD+o/O/jaR9n5k3ufzcaLsZ16kj+aWOkMbZWPj+95ETvOJP+uRSOt8za/d 6KonYN6/0yTop1HP7WNmuH2NhS2VHVu4fgsKqOrAPD0LJFWZxYw0HK7VcGyEuA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Thy7H6QC; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -8.71 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Thy7H6QC; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: AE483A84C X-Spam-Score: -8.71 X-Migadu-Scanner: scn0.migadu.com X-TUID: m5f77bWIFuHG --=-=-= Content-Type: text/plain Lele Gaifax writes: >> Maybe we should better make this a user option? > > Yes, that's what I had in mind: when (say) org-persist-do-not-auto-gc-remotes > is enabled, the cleanup procedure would ignore remote Org documents and a new > explicit (interactive) org-persist-gc-remotes would take care of them. > >> Or we can go a different route and just not garbage collect remote files >> until cache gets too large. > > Yes, a threshold would work too. Can you test the attached 2 patches? They do not seem to break local files, but I do not have remote file caches to test on. Best, Ihor --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-org-persist.el-Make-org-persist-directory-a-custom-v.patch >From 592608d3c48cb1e63f9b4427576f0698ca190e9a Mon Sep 17 00:00:00 2001 Message-Id: <592608d3c48cb1e63f9b4427576f0698ca190e9a.1639837163.git.yantar92@gmail.com> From: Ihor Radchenko Date: Sat, 18 Dec 2021 22:13:24 +0800 Subject: [PATCH] org-persist.el: Make org-persist-directory a custom variable * lisp/org-persist.el (org-persist): Create org-persist customisation group. (org-persist-directory): Convert into defcustom. --- lisp/org-persist.el | 34 +++++++++++++++++++++------------- 1 file changed, 21 insertions(+), 13 deletions(-) diff --git a/lisp/org-persist.el b/lisp/org-persist.el index f06d4fc61..90f9a3bc9 100644 --- a/lisp/org-persist.el +++ b/lisp/org-persist.el @@ -35,19 +35,27 @@ (declare-function org-back-to-heading "org" (&optional invisible-ok)) (declare-function org-next-visible-heading "org" (arg)) (declare-function org-at-heading-p "org" (&optional invisible-not-ok)) -(defvar org-persist-directory (expand-file-name - (org-file-name-concat - (let ((cache-dir (when (fboundp 'xdg-cache-home) - (xdg-cache-home)))) - (if (or (seq-empty-p cache-dir) - (not (file-exists-p cache-dir)) - (file-exists-p (org-file-name-concat - user-emacs-directory - "org-persist"))) - user-emacs-directory - cache-dir)) - "org-persist/")) - "Directory where the data is stored.") + +(defgroup org-persist nil + "Persistent cache for Org mode." + :tag "Org persist" + :group 'org) + +(defcustom org-persist-directory (expand-file-name + (org-file-name-concat + (let ((cache-dir (when (fboundp 'xdg-cache-home) + (xdg-cache-home)))) + (if (or (seq-empty-p cache-dir) + (not (file-exists-p cache-dir)) + (file-exists-p (org-file-name-concat + user-emacs-directory + "org-persist"))) + user-emacs-directory + cache-dir)) + "org-persist/")) + "Directory where the data is stored." + :group 'org-persist + :type 'directory) (defvar org-persist-index-file "index" "File name used to store the data index.") -- 2.32.0 --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-org-persist.el-Customise-persistence-of-data-for-rem.patch >From de89eca758ca1972d75c8830aee224376a1e364e Mon Sep 17 00:00:00 2001 Message-Id: From: Ihor Radchenko Date: Sat, 18 Dec 2021 22:14:24 +0800 Subject: [PATCH] org-persist.el: Customise persistence of data for remote files * lisp/org-persist.el (org-persist-remote-files): New defcustom defining strategy for persisting data associated with remote files. Separate strategy is necessary to avoid unexpected password promtps while garbage collecting the cache: remote files may require server login. (org-persist-gc): Decide whether to store remote file data according to `org-persist-remote-files'. Reported in https://list.orgmode.org/87bl1eilt6.fsf@localhost/T/#u --- lisp/org-persist.el | 41 ++++++++++++++++++++++++++++++++++------- 1 file changed, 34 insertions(+), 7 deletions(-) diff --git a/lisp/org-persist.el b/lisp/org-persist.el index 90f9a3bc9..613290afc 100644 --- a/lisp/org-persist.el +++ b/lisp/org-persist.el @@ -57,6 +57,22 @@ (defcustom org-persist-directory (expand-file-name :group 'org-persist :type 'directory) +(defcustom org-persist-remote-files 100 + "Whether to keep persistent data for remote files. + +When this variable is nil, never save persitent data associated with +remote files. When `t', always keep the data. When +`check-existance', contact remote server containing the file and only +keep the data when the file exists on the server. + +Note that the last option `check-existance' may cause Emacs to show +password prompts to log in." + :group 'org-persist + :type '(choice (const :tag "Never" nil) + (const :tag "Always" t) + (number :tag "Keep note more than X files") + (const :tag "Check if exist on remote" 'check-existence))) + (defvar org-persist-index-file "index" "File name used to store the data index.") @@ -287,15 +303,26 @@ (defun org-persist-read-all-buffer () (defun org-persist-gc () "Remove stored data for not existing files or unregistered variables." - (let (new-index) + (let (new-index (remote-files-num 0)) (dolist (index org-persist--index) - (let ((file (plist-get index :path)) - (persist-file (when (plist-get index :persist-file) - (org-file-name-concat - org-persist-directory - (plist-get index :persist-file))))) + (let* ((file (plist-get index :path)) + (file-remote (when file (file-remote-p file))) + (persist-file (when (plist-get index :persist-file) + (org-file-name-concat + org-persist-directory + (plist-get index :persist-file))))) (when (and file persist-file) - (if (file-exists-p file) + (when (and file-remote persist-file) + (cl-incf remote-files-num)) + (if (if (not file-remote) + (file-exists-p file) + (pcase org-persist-remote-files + ('t t) + ('check-existence + (file-exists-p file)) + ((pred #'numberp) + (<= org-persist-remote-files remote-files-num)) + (_ nil))) (push index new-index) (when (file-exists-p persist-file) (delete-file persist-file) -- 2.32.0 --=-=-=--