After today's pull of org-mode I get the warning that org-persist cannot read its index. This is an entirely new warning (as of today) and I assume is the result of the recent commits in org-persist. Please, can this be corrected. The warning is irritating, especially so since I do not use org-persist and have no wish to do so. Best wishes, Colin Baxter.
Colin Baxter 😺 <m43cap@yandex.com> writes: > After today's pull of org-mode I get the warning that org-persist cannot > read its index. This is an entirely new warning (as of today) and I > assume is the result of the recent commits in org-persist. > Please, can this be corrected. The warning is irritating, especially so > since I do not use org-persist and have no wish to do so. Fixed. Note that org-persist is enabled by default together with org-element-cache. You can indeed disable it, but, when enabled, org-persist should improve loading times and overall performance. Best, Ihor
>>>>> Ihor Radchenko <yantar92@gmail.com> writes:
> Colin Baxter 😺 <m43cap@yandex.com> writes:
>> After today's pull of org-mode I get the warning that org-persist
>> cannot read its index. This is an entirely new warning (as of
>> today) and I assume is the result of the recent commits in
>> org-persist.
>> Please, can this be corrected. The warning is irritating,
>> especially so since I do not use org-persist and have no wish to
>> do so.
> Fixed.
Thank you. I cloned org-mode afresh and got the same warning during
'make' - I presume that too is now fixed.
> Note that org-persist is enabled by default together with
> org-element-cache. You can indeed disable it,
How can I do this?
Best wishes,
Colin Baxter 😺 <m43cap@yandex.com> writes: > Thank you. I cloned org-mode afresh and got the same warning during > 'make' - I presume that too is now fixed. The warning should never appear now. I removed it completely. > > Note that org-persist is enabled by default together with > > org-element-cache. You can indeed disable it, > > How can I do this? If you want to disable cache persistence, just set `org-element-cache-persistent' to nil. It is documented in ORG-NEWS. If you find ORG-NEWS not clear, please let me know. I will try to improve the news record. If you want to disable cache completely, you can set `org-element-use-cache' to nil. However, I am hoping to make cache enabled all the time in future. If you have specific reasons to avoid org-element-cache, may you share them? Best, Ihor
>>>>> Ihor Radchenko <yantar92@gmail.com> writes:
r> Colin Baxter 😺 <m43cap@yandex.com> writes:
>> Thank you. I cloned org-mode afresh and got the same warning
>> during 'make' - I presume that too is now fixed.
r> The warning should never appear now. I removed it completely.
>> > Note that org-persist is enabled by default together with >
>> org-element-cache. You can indeed disable it,
>>
>> How can I do this?
r> If you want to disable cache persistence, just set
r> `org-element-cache-persistent' to nil. It is documented in
r> ORG-NEWS. If you find ORG-NEWS not clear, please let me know. I
r> will try to improve the news record.
Thank you.
r> If you want to disable cache completely, you can set
r> `org-element-use-cache' to nil. However, I am hoping to make
r> cache enabled all the time in future. If you have specific
r> reasons to avoid org-element-cache, may you share them?
I may not have understood things fully, but it seems that the
org-persist directory is set only by the
org-persist-path. Unfortunately, the resulting org-persist directory in
~/.cache is no good for me since I daily clear out my ~/.cache using
bleachbit. I can't find a way of changing org-persist-path and grep
finds the variable only in ORG-NEWS.
Best wishes,
Colin.
Colin Baxter 😺 <m43cap@yandex.com> writes:
> I may not have understood things fully, but it seems that the
> org-persist directory is set only by the
> org-persist-path. Unfortunately, the resulting org-persist directory in
> ~/.cache is no good for me since I daily clear out my ~/.cache using
> bleachbit. I can't find a way of changing org-persist-path and grep
> finds the variable only in ORG-NEWS.
Oops. It was my bad (fixed now). Should be `org-persist-directory'. You
can set it to other location in your init.el.
Also, deleting the cache would not make any harm, except possible slower
Org startup. org-persist should handle such case routinely. If you are
using bleachbit for privacy reasons, you may want to keep the default
cache directory: i.e. headline titles are stored in cache (unless you
are using org-crypt).
Best,
Ihot
>>>>> Ihor Radchenko <yantar92@gmail.com> writes:
> Colin Baxter 😺 <m43cap@yandex.com> writes:
>> I may not have understood things fully, but it seems that the
>> org-persist directory is set only by the
>> org-persist-path. Unfortunately, the resulting org-persist
>> directory in ~/.cache is no good for me since I daily clear out
>> my ~/.cache using bleachbit. I can't find a way of changing
>> org-persist-path and grep finds the variable only in ORG-NEWS.
> Oops. It was my bad (fixed now). Should be
> `org-persist-directory'. You can set it to other location in your
> init.el.
Ok, thanks - that works - I used ~/.emacs.d/org-persist/. It might be
worth mentioning that the user should not create the
org-persist-directory himself. I did this originally (mkdir
~/.emacs.d/org-persist <RET>, touch ~/.emacs.d/org-persist/index <RET>)
and got various parsing error messages. Everything worked fine after I
deleted my own handiwork and left the directory creation to org-persist.
I had originally set org-element-use-cache to nil because I remember
reading somewhere that the nil setting would help prevent emacs from
hanging in org-mode.
I'll certainly begin using org-persist and see what happens.
Thanks for your help.
Best wishes,
Colin.
Colin Baxter 😺 <m43cap@yandex.com> writes: > Ok, thanks - that works - I used ~/.emacs.d/org-persist/. It might be > worth mentioning that the user should not create the > org-persist-directory himself. I did this originally (mkdir > ~/.emacs.d/org-persist <RET>, touch ~/.emacs.d/org-persist/index <RET>) > and got various parsing error messages. Everything worked fine after I > deleted my own handiwork and left the directory creation to org-persist. Thanks for reporting! I did not consider corrupted index file. Fixed now. > I had originally set org-element-use-cache to nil because I remember > reading somewhere that the nil setting would help prevent emacs from > hanging in org-mode. The cache code has been refactored. I tried my best to fix all the bugs causing the hangs and I am not seeing hangs anymore. That's why I took a risk to enable cache by default on main. > I'll certainly begin using org-persist and see what happens. Thanks! The new cache makes agenda/sparse tree searches at least several times faster (I even got >10x on some agendas). But we need more testing. Best, Ihor
On 10/19/21, Ihor Radchenko <yantar92@gmail.com> wrote:
> enabled all the time in future. If you have specific reasons to avoid
> org-element-cache, may you share them?
fwiw, long ago, i disabled org element cache due to buffer corruption.
perhaps that has been fixed and tested.
also i presume this is all on main not bugfix [i haven't upgraded
maint yet; first thing i will do is try the archiving improvement].
Samuel Wales <samologist@gmail.com> writes: > On 10/19/21, Ihor Radchenko <yantar92@gmail.com> wrote: >> enabled all the time in future. If you have specific reasons to avoid >> org-element-cache, may you share them? > > fwiw, long ago, i disabled org element cache due to buffer corruption. > perhaps that has been fixed and tested. I hope I fixed it. At least on my side. Need other users to test. Currently, cache even self-verifies itself to double check corruption. > also i presume this is all on main not bugfix [i haven't upgraded > maint yet; first thing i will do is try the archiving improvement]. It is indeed not on bugfix. Just on main. Best, Ihor
Hi,
Ihor Radchenko <yantar92@gmail.com> writes:
>> I had originally set org-element-use-cache to nil because I remember
>> reading somewhere that the nil setting would help prevent emacs from
>> hanging in org-mode.
>
> The cache code has been refactored. I tried my best to fix all the bugs
> causing the hangs and I am not seeing hangs anymore. That's why I took a
> risk to enable cache by default on main.
today I spent the required time to investigate on one issue I had with my
Doom-based profile, that uses the "master" version of Orgmode, and found it
related to the org-persist functionality.
Basically since some time ago, quitting Emacs I was sometime prompted for a
password, some other time it took a considerable time to close the instance.
Digging down, I found that it was due to the persistent cache, that contained
several entries about remote (Tramp) documents: upon closing Emacs, the
org-persist-gc function is triggered, and the index is traversed to remove
dangling entries, performing a file-exist-p on each filename. For remote
files, that obviously mean prompting for the remote password (when the
ssh-agent does not carry it) or spending a considerable time in the network
round trip.
I wonder if there is some mechanism I could use to either prevent caching of
non-local documents or to avoid the check on existence in the org-persist-gc.
Thanks for any hint,
bye, lele.
--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it | -- Fortunato Depero, 1929.
Lele Gaifax <lele@metapensiero.it> writes: > Digging down, I found that it was due to the persistent cache, that contained > several entries about remote (Tramp) documents: upon closing Emacs, the > org-persist-gc function is triggered, and the index is traversed to remove > dangling entries, performing a file-exist-p on each filename. For remote > files, that obviously mean prompting for the remote password (when the > ssh-agent does not carry it) or spending a considerable time in the network > round trip. Thanks for reporting! I am not using tramp regularly and I did not think about this use case. > I wonder if there is some mechanism I could use to either prevent caching of > non-local documents or to avoid the check on existence in the org-persist-gc. I can easily make a change that always garbage-collect non-local documents without trying to connect to remote server. This can be one solution of your problem. However, I am not sure if it is the best solution. May I know more about your use cases? Do you frequently open large remote Org files? If you do, trashing their cache may not be optimal. Maybe we should better make this a user option? Or we can go a different route and just not garbage collect remote files until cache gets too large. WDYT? Best, Ihor
Ihor Radchenko <yantar92@gmail.com> writes: > Lele Gaifax <lele@metapensiero.it> writes: > >> I wonder if there is some mechanism I could use to either prevent caching >> of non-local documents or to avoid the check on existence in the >> org-persist-gc. > > I can easily make a change that always garbage-collect non-local > documents without trying to connect to remote server. This can be one > solution of your problem. However, I am not sure if it is the best > solution. > > May I know more about your use cases? Do you frequently open large > remote Org files? If you do, trashing their cache may not be optimal. I do not open remote Org files very often, but surely happens now and then, typically from my laptop to see/edit something I keep on my desktop PC. > Maybe we should better make this a user option? Yes, that's what I had in mind: when (say) org-persist-do-not-auto-gc-remotes is enabled, the cleanup procedure would ignore remote Org documents and a new explicit (interactive) org-persist-gc-remotes would take care of them. > Or we can go a different route and just not garbage collect remote files > until cache gets too large. Yes, a threshold would work too. Thanks a lot, ciao, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. lele@metapensiero.it | -- Fortunato Depero, 1929.
[-- Attachment #1: Type: text/plain, Size: 615 bytes --] Lele Gaifax <lele@metapensiero.it> writes: >> Maybe we should better make this a user option? > > Yes, that's what I had in mind: when (say) org-persist-do-not-auto-gc-remotes > is enabled, the cleanup procedure would ignore remote Org documents and a new > explicit (interactive) org-persist-gc-remotes would take care of them. > >> Or we can go a different route and just not garbage collect remote files >> until cache gets too large. > > Yes, a threshold would work too. Can you test the attached 2 patches? They do not seem to break local files, but I do not have remote file caches to test on. Best, Ihor [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-org-persist.el-Make-org-persist-directory-a-custom-v.patch --] [-- Type: text/x-diff, Size: 2638 bytes --] From 592608d3c48cb1e63f9b4427576f0698ca190e9a Mon Sep 17 00:00:00 2001 Message-Id: <592608d3c48cb1e63f9b4427576f0698ca190e9a.1639837163.git.yantar92@gmail.com> From: Ihor Radchenko <yantar92@gmail.com> Date: Sat, 18 Dec 2021 22:13:24 +0800 Subject: [PATCH] org-persist.el: Make org-persist-directory a custom variable * lisp/org-persist.el (org-persist): Create org-persist customisation group. (org-persist-directory): Convert into defcustom. --- lisp/org-persist.el | 34 +++++++++++++++++++++------------- 1 file changed, 21 insertions(+), 13 deletions(-) diff --git a/lisp/org-persist.el b/lisp/org-persist.el index f06d4fc61..90f9a3bc9 100644 --- a/lisp/org-persist.el +++ b/lisp/org-persist.el @@ -35,19 +35,27 @@ (declare-function org-back-to-heading "org" (&optional invisible-ok)) (declare-function org-next-visible-heading "org" (arg)) (declare-function org-at-heading-p "org" (&optional invisible-not-ok)) -(defvar org-persist-directory (expand-file-name - (org-file-name-concat - (let ((cache-dir (when (fboundp 'xdg-cache-home) - (xdg-cache-home)))) - (if (or (seq-empty-p cache-dir) - (not (file-exists-p cache-dir)) - (file-exists-p (org-file-name-concat - user-emacs-directory - "org-persist"))) - user-emacs-directory - cache-dir)) - "org-persist/")) - "Directory where the data is stored.") + +(defgroup org-persist nil + "Persistent cache for Org mode." + :tag "Org persist" + :group 'org) + +(defcustom org-persist-directory (expand-file-name + (org-file-name-concat + (let ((cache-dir (when (fboundp 'xdg-cache-home) + (xdg-cache-home)))) + (if (or (seq-empty-p cache-dir) + (not (file-exists-p cache-dir)) + (file-exists-p (org-file-name-concat + user-emacs-directory + "org-persist"))) + user-emacs-directory + cache-dir)) + "org-persist/")) + "Directory where the data is stored." + :group 'org-persist + :type 'directory) (defvar org-persist-index-file "index" "File name used to store the data index.") -- 2.32.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #3: 0001-org-persist.el-Customise-persistence-of-data-for-rem.patch --] [-- Type: text/x-diff, Size: 3480 bytes --] From de89eca758ca1972d75c8830aee224376a1e364e Mon Sep 17 00:00:00 2001 Message-Id: <de89eca758ca1972d75c8830aee224376a1e364e.1639837255.git.yantar92@gmail.com> From: Ihor Radchenko <yantar92@gmail.com> Date: Sat, 18 Dec 2021 22:14:24 +0800 Subject: [PATCH] org-persist.el: Customise persistence of data for remote files * lisp/org-persist.el (org-persist-remote-files): New defcustom defining strategy for persisting data associated with remote files. Separate strategy is necessary to avoid unexpected password promtps while garbage collecting the cache: remote files may require server login. (org-persist-gc): Decide whether to store remote file data according to `org-persist-remote-files'. Reported in https://list.orgmode.org/87bl1eilt6.fsf@localhost/T/#u --- lisp/org-persist.el | 41 ++++++++++++++++++++++++++++++++++------- 1 file changed, 34 insertions(+), 7 deletions(-) diff --git a/lisp/org-persist.el b/lisp/org-persist.el index 90f9a3bc9..613290afc 100644 --- a/lisp/org-persist.el +++ b/lisp/org-persist.el @@ -57,6 +57,22 @@ (defcustom org-persist-directory (expand-file-name :group 'org-persist :type 'directory) +(defcustom org-persist-remote-files 100 + "Whether to keep persistent data for remote files. + +When this variable is nil, never save persitent data associated with +remote files. When `t', always keep the data. When +`check-existance', contact remote server containing the file and only +keep the data when the file exists on the server. + +Note that the last option `check-existance' may cause Emacs to show +password prompts to log in." + :group 'org-persist + :type '(choice (const :tag "Never" nil) + (const :tag "Always" t) + (number :tag "Keep note more than X files") + (const :tag "Check if exist on remote" 'check-existence))) + (defvar org-persist-index-file "index" "File name used to store the data index.") @@ -287,15 +303,26 @@ (defun org-persist-read-all-buffer () (defun org-persist-gc () "Remove stored data for not existing files or unregistered variables." - (let (new-index) + (let (new-index (remote-files-num 0)) (dolist (index org-persist--index) - (let ((file (plist-get index :path)) - (persist-file (when (plist-get index :persist-file) - (org-file-name-concat - org-persist-directory - (plist-get index :persist-file))))) + (let* ((file (plist-get index :path)) + (file-remote (when file (file-remote-p file))) + (persist-file (when (plist-get index :persist-file) + (org-file-name-concat + org-persist-directory + (plist-get index :persist-file))))) (when (and file persist-file) - (if (file-exists-p file) + (when (and file-remote persist-file) + (cl-incf remote-files-num)) + (if (if (not file-remote) + (file-exists-p file) + (pcase org-persist-remote-files + ('t t) + ('check-existence + (file-exists-p file)) + ((pred #'numberp) + (<= org-persist-remote-files remote-files-num)) + (_ nil))) (push index new-index) (when (file-exists-p persist-file) (delete-file persist-file) -- 2.32.0
Ihor Radchenko <yantar92@gmail.com> writes: > Lele Gaifax <lele@metapensiero.it> writes: > >>> Maybe we should better make this a user option? >> >> Yes, that's what I had in mind: when (say) org-persist-do-not-auto-gc-remotes >> is enabled, the cleanup procedure would ignore remote Org documents and a new >> explicit (interactive) org-persist-gc-remotes would take care of them. >> >>> Or we can go a different route and just not garbage collect remote files >>> until cache gets too large. >> >> Yes, a threshold would work too. > > Can you test the attached 2 patches? They do not seem to break local > files, but I do not have remote file caches to test on. Yes, I'll be able to do some test in a couple of hours. Thank you! In the meanwhile, here below some notes: > ... > Subject: [PATCH] org-persist.el: Customise persistence of data for remote > files > > * lisp/org-persist.el (org-persist-remote-files): New defcustom > defining strategy for persisting data associated with remote files. > Separate strategy is necessary to avoid unexpected password promtps > while garbage collecting the cache: remote files may require server > login. > ... s/promtps/prompts > ... > +(defcustom org-persist-remote-files 100 > + "Whether to keep persistent data for remote files. > + > +When this variable is nil, never save persitent data associated with > +remote files. When `t', always keep the data. When > +`check-existance', contact remote server containing the file and only > +keep the data when the file exists on the server. s/check-existance/check-existence Also, shouldn't the doc mention the meaning of the numberp, expecially since that's the default? > +Note that the last option `check-existance' may cause Emacs to show > +password prompts to log in." Thanks again, bye, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. lele@metapensiero.it | -- Fortunato Depero, 1929.
[-- Attachment #1: Type: text/plain, Size: 132 bytes --] Lele Gaifax <lele@metapensiero.it> writes: > In the meanwhile, here below some notes: Thanks! See the updated patch. Best, Ihor [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-org-persist.el-Customise-persistence-of-data-for-rem.patch --] [-- Type: text/x-diff, Size: 3556 bytes --] From 92878fab54c26800562409f3c686f065b7523a61 Mon Sep 17 00:00:00 2001 Message-Id: <92878fab54c26800562409f3c686f065b7523a61.1639838789.git.yantar92@gmail.com> From: Ihor Radchenko <yantar92@gmail.com> Date: Sat, 18 Dec 2021 22:14:24 +0800 Subject: [PATCH] org-persist.el: Customise persistence of data for remote files * lisp/org-persist.el (org-persist-remote-files): New defcustom defining strategy for persisting data associated with remote files. Separate strategy is necessary to avoid unexpected password promtps while garbage collecting the cache: remote files may require server login. (org-persist-gc): Decide whether to store remote file data according to `org-persist-remote-files'. Reported in https://list.orgmode.org/87bl1eilt6.fsf@localhost/T/#u --- lisp/org-persist.el | 42 +++++++++++++++++++++++++++++++++++------- 1 file changed, 35 insertions(+), 7 deletions(-) diff --git a/lisp/org-persist.el b/lisp/org-persist.el index 90f9a3bc9..4c0673c72 100644 --- a/lisp/org-persist.el +++ b/lisp/org-persist.el @@ -57,6 +57,23 @@ (defcustom org-persist-directory (expand-file-name :group 'org-persist :type 'directory) +(defcustom org-persist-remote-files 100 + "Whether to keep persistent data for remote files. + +When this variable is nil, never save persitent data associated with +remote files. When `t', always keep the data. When +`check-existence', contact remote server containing the file and only +keep the data when the file exists on the server. When a number, keep +up to that number persistent values for remote files. + +Note that the last option `check-existence' may cause Emacs to show +password prompts to log in." + :group 'org-persist + :type '(choice (const :tag "Never" nil) + (const :tag "Always" t) + (number :tag "Keep note more than X files") + (const :tag "Check if exist on remote" 'check-existence))) + (defvar org-persist-index-file "index" "File name used to store the data index.") @@ -287,15 +304,26 @@ (defun org-persist-read-all-buffer () (defun org-persist-gc () "Remove stored data for not existing files or unregistered variables." - (let (new-index) + (let (new-index (remote-files-num 0)) (dolist (index org-persist--index) - (let ((file (plist-get index :path)) - (persist-file (when (plist-get index :persist-file) - (org-file-name-concat - org-persist-directory - (plist-get index :persist-file))))) + (let* ((file (plist-get index :path)) + (file-remote (when file (file-remote-p file))) + (persist-file (when (plist-get index :persist-file) + (org-file-name-concat + org-persist-directory + (plist-get index :persist-file))))) (when (and file persist-file) - (if (file-exists-p file) + (when (and file-remote persist-file) + (cl-incf remote-files-num)) + (if (if (not file-remote) + (file-exists-p file) + (pcase org-persist-remote-files + ('t t) + ('check-existence + (file-exists-p file)) + ((pred #'numberp) + (<= org-persist-remote-files remote-files-num)) + (_ nil))) (push index new-index) (when (file-exists-p persist-file) (delete-file persist-file) -- 2.32.0
Ihor Radchenko <yantar92@gmail.com> writes: > Lele Gaifax <lele@metapensiero.it> writes: > >> In the meanwhile, here below some notes: > > Thanks! See the updated patch. Great, you're so fast! :-) I couldn't test yet, but I saw a couple of other glitches, sorry for not noticing first. > ... > +(defcustom org-persist-remote-files 100 > + "Whether to keep persistent data for remote files. > + > +When this variable is nil, never save persitent data associated with > +remote files. When `t', always keep the data. When > +`check-existence', contact remote server containing the file and only > +keep the data when the file exists on the server. When a number, keep > +up to that number persistent values for remote files. > ... s/persitent/persistent > +Note that the last option `check-existence' may cause Emacs to show > +password prompts to log in." I'd remove "last" above, since `check-existence' is not the last (described) option anymore. > + :group 'org-persist > + :type '(choice (const :tag "Never" nil) > + (const :tag "Always" t) > + (number :tag "Keep note more than X files") Now that you explained the meaning, maybe better to be explicit here, mentioning "X remote files"? Thanks&bye, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. lele@metapensiero.it | -- Fortunato Depero, 1929.
Lele Gaifax <lele@metapensiero.it> writes:
>> + :group 'org-persist
>> + :type '(choice (const :tag "Never" nil)
>> + (const :tag "Always" t)
>> + (number :tag "Keep note more than X files")
>
> Now that you explained the meaning, maybe better to be explicit here,
> mentioning "X remote files"?
Sigh, re-reading, I see one more glitch there: s/Keep note/Keep not/
I'm sorry, but real work swallowed all my day, and could not try this out yet.
Hopefully I will be able to do some test tomorrow morning.
bye, lele.
--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it | -- Fortunato Depero, 1929.
[-- Attachment #1: Type: text/plain, Size: 214 bytes --] Lele Gaifax <lele@metapensiero.it> writes: > I couldn't test yet, but I saw a couple of other glitches, sorry for not > noticing first. Thanks for the proof reading. I updated the patch accordingly. Best, Ihor [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-org-persist.el-Customise-persistence-of-data-for-rem.patch --] [-- Type: text/x-diff, Size: 3579 bytes --] From 557e7cec6dd22b09468b82650feed2020f97d781 Mon Sep 17 00:00:00 2001 Message-Id: <557e7cec6dd22b09468b82650feed2020f97d781.1639907677.git.yantar92@gmail.com> From: Ihor Radchenko <yantar92@gmail.com> Date: Sat, 18 Dec 2021 22:14:24 +0800 Subject: [PATCH] org-persist.el: Customise persistence of data for remote files * lisp/org-persist.el (org-persist-remote-files): New defcustom defining strategy for persisting data associated with remote files. Separate strategy is necessary to avoid unexpected password prompts while garbage collecting the cache: remote files may require server login. (org-persist-gc): Decide whether to store remote file data according to `org-persist-remote-files'. Reported in https://list.orgmode.org/87bl1eilt6.fsf@localhost/T/#u --- lisp/org-persist.el | 42 +++++++++++++++++++++++++++++++++++------- 1 file changed, 35 insertions(+), 7 deletions(-) diff --git a/lisp/org-persist.el b/lisp/org-persist.el index 90f9a3bc9..8e8ecc196 100644 --- a/lisp/org-persist.el +++ b/lisp/org-persist.el @@ -57,6 +57,23 @@ (defcustom org-persist-directory (expand-file-name :group 'org-persist :type 'directory) +(defcustom org-persist-remote-files 100 + "Whether to keep persistent data for remote files. + +When this variable is nil, never save persistent data associated with +remote files. When `t', always keep the data. When a number, keep up +to that number persistent values for remote files. When +`check-existence', contact remote server containing the file and only +keep the data when the file exists on the server. + +Note that setting value to `check-existence' will require logging in +to remote servers. Emacs may show multiple password prompts." + :group 'org-persist + :type '(choice (const :tag "Never" nil) + (const :tag "Always" t) + (number :tag "Up to X files") + (const :tag "Check if exist on remote" 'check-existence))) + (defvar org-persist-index-file "index" "File name used to store the data index.") @@ -287,15 +304,26 @@ (defun org-persist-read-all-buffer () (defun org-persist-gc () "Remove stored data for not existing files or unregistered variables." - (let (new-index) + (let (new-index (remote-files-num 0)) (dolist (index org-persist--index) - (let ((file (plist-get index :path)) - (persist-file (when (plist-get index :persist-file) - (org-file-name-concat - org-persist-directory - (plist-get index :persist-file))))) + (let* ((file (plist-get index :path)) + (file-remote (when file (file-remote-p file))) + (persist-file (when (plist-get index :persist-file) + (org-file-name-concat + org-persist-directory + (plist-get index :persist-file))))) (when (and file persist-file) - (if (file-exists-p file) + (when (and file-remote persist-file) + (cl-incf remote-files-num)) + (if (if (not file-remote) + (file-exists-p file) + (pcase org-persist-remote-files + ('t t) + ('check-existence + (file-exists-p file)) + ((pred #'numberp) + (<= org-persist-remote-files remote-files-num)) + (_ nil))) (push index new-index) (when (file-exists-p persist-file) (delete-file persist-file) -- 2.32.0
Hi Ihor, I had some trouble testing your patches, and the outcome is still unclear to me: apparently (again, cannot be sure this is not caused by "improper" application of the patches) the new version of org-persist, with default value for o-p-remote-files (100), removes any remote file from the persistent index, without even prompting for a password. I will try again (and harder) as time permits. ciao, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. lele@metapensiero.it | -- Fortunato Depero, 1929.
Lele Gaifax <lele@metapensiero.it> writes:
> Hi Ihor,
>
> I had some trouble testing your patches, and the outcome is still unclear to
> me: apparently (again, cannot be sure this is not caused by "improper"
> application of the patches) the new version of org-persist, with default value
> for o-p-remote-files (100), removes any remote file from the persistent index,
> without even prompting for a password.
Oops. Try:
- (<= org-persist-remote-files remote-files-num))
+ (>= org-persist-remote-files remote-files-num))
Best,
Ihor