emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
From: Ihor Radchenko <yantar92@gmail.com>
To: Eric S Fraga <e.fraga@ucl.ac.uk>
Cc: Emacs Org mode mailing list <emacs-orgmode@gnu.org>
Subject: Re: when does :cache not cache?
Date: Sun, 12 Dec 2021 15:45:50 +0800	[thread overview]
Message-ID: <875yruxo41.fsf@localhost> (raw)
In-Reply-To: <87mtn1o5mn.fsf@ucl.ac.uk>

[-- Attachment #1: Type: text/plain, Size: 1113 bytes --]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> in the paper I am writing, I have a number of gnuplot src blocks, some
> of which process a significant amount of data so take some time to
> generate the actual plots.  The data are static so caching the results
> make sense.  However, even though I have ":cache yes" on each of these
> named src blocks, and I have (for good measure), the property
> "header-args:gnuplot" set to ":cache yes" as well, the plots are being
> regenerated each time I export to PDF via LaTeX.

I was able to reproduce. The reason why caching does not work is related
to the way :var assignments work in ob-gnuplot. We dump the table data
into temporary files and refer to those files in generated gnuplot
script body. The temporary files names change on every execution and the
gnuplot script hash is never going to be the same.

I am attaching tentative patch to fix the issue.
The patch introduces a new functionality to ob-core.el allowing more
stable temporary file names. I am not sure if my implementation is the
best way to solve the problem, so comments are welcome.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-gnuplot.el-Make-cache-argument-work-with-var-assi.patch --]
[-- Type: text/x-diff, Size: 5773 bytes --]

From ac11b4d08edd577b29a398296364b4340096a6ae Mon Sep 17 00:00:00 2001
Message-Id: <ac11b4d08edd577b29a398296364b4340096a6ae.1639294626.git.yantar92@gmail.com>
From: Ihor Radchenko <yantar92@gmail.com>
Date: Sun, 12 Dec 2021 15:31:35 +0800
Subject: [PATCH] ob-gnuplot.el: Make :cache argument work with :var

* lisp/ob-core.el (org-babel-temporary-stable-directory): New variable
holding a temporary directory name that does not change between Emacs
(org-babel-remove-temporary-stable-directory): New function removing
`org-babel-temporary-stable-directory' on Emacs shutdown.
(org-babel-temp-stable-file): Generate stable temporary file name for
object storage.  The file name is constant with for equal objects.
(org-babel-execute-src-block): Explicitly identify that if the result
is cached.
* lisp/ob-gnuplot.el (org-babel-gnuplot-process-vars): Make use of
`org-babel-stable-file' to make expanded body stable with respect to
:var assignments.

Fixes https://orgmode.org/list/87mtn1o5mn.fsf@ucl.ac.uk
 lisp/ob-core.el    | 52 +++++++++++++++++++++++++++++++++++++++++++++-
 lisp/ob-gnuplot.el |  5 ++++-
 2 files changed, 55 insertions(+), 2 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 7a9467b0e..d572423b7 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -735,7 +735,8 @@ (defun org-babel-execute-src-block (&optional arg info params)
 	    (skip-chars-forward " \t")
 	    (let ((result (org-babel-read-result)))
-	      (message (replace-regexp-in-string "%" "%%" (format "%S" result)))
+	      (message (format "Cached: %s"
+                               (replace-regexp-in-string "%" "%%" (format "%S" result))))
 	 ((org-babel-confirm-evaluate info)
 	  (let* ((lang (nth 0 info))
@@ -3112,6 +3113,22 @@   (defvar org-babel-temporary-directory
 Used by `org-babel-temp-file'.  This directory will be removed on
 Emacs shutdown."))
+(defvar org-babel-temporary-stable-directory)
+(unless (or noninteractive (boundp 'org-babel-temporary-stable-directory))
+  (defvar org-babel-temporary-stable-directory
+    (or (and (boundp 'org-babel-temporary-stable-directory)
+	     (file-exists-p org-babel-temporary-stable-directory)
+	     org-babel-temporary-stable-directory)
+        (condition-case nil
+            (make-directory
+	     (expand-file-name
+              "babel-stable"
+              (temporary-file-directory)))
+          (t nil)))
+    "Directory to hold temporary files created to execute code blocks.
+Used by `org-babel-temp-file'.  This directory will be removed on
+Emacs shutdown."))
 (defcustom org-babel-remote-temporary-directory "/tmp/"
   "Directory to hold temporary files on remote hosts."
   :group 'org-babel
@@ -3155,6 +3172,30 @@ (defun org-babel-temp-file (prefix &optional suffix)
       (make-temp-file prefix nil suffix))))
+(defun org-babel-temp-stable-file (data prefix &optional suffix)
+  "Create a temporary file in the `org-babel-remove-temporary-stable-directory'.
+The file name is stable with respect to DATA.  The file name is
+constructed like the following: PREFIXDATAhashSUFFIX."
+  (if (file-remote-p default-directory)
+      (let* ((prefix
+              (concat (file-remote-p default-directory)
+                      (expand-file-name
+		       prefix org-babel-temporary-stable-directory)))
+             (path (concat prefix (format "%s" (sxhash data)) (or suffix ""))))
+        (with-temp-file path)
+        path)
+    (let* ((temporary-file-directory
+	    (or (and (boundp 'org-babel-temporary-stable-directory)
+		     (file-exists-p org-babel-temporary-stable-directory)
+		     org-babel-temporary-stable-directory)
+	        temporary-file-directory))
+           (path (concat
+                  (expand-file-name
+		   prefix org-babel-temporary-stable-directory)
+                  (format "%s" (sxhash data)) (or suffix ""))))
+      (with-temp-file path)
+      path)))
 (defun org-babel-remove-temporary-directory ()
   "Remove `org-babel-temporary-directory' on Emacs shutdown."
   (when (and (boundp 'org-babel-temporary-directory)
@@ -3178,7 +3219,16 @@ (defun org-babel-remove-temporary-directory ()
 		  "[directory not defined]"))))))
+(defun org-babel-remove-temporary-stable-directory ()
+  "Remove `org-babel-temporary-stable-directory' and on Emacs shutdown."
+  (when (and (boundp 'org-babel-temporary-stable-directory)
+	     (file-exists-p org-babel-temporary-stable-directory))
+    (let ((org-babel-temporary-directory
+           org-babel-temporary-stable-directory))
+      (org-babel-remove-temporary-directory))))
 (add-hook 'kill-emacs-hook #'org-babel-remove-temporary-directory)
+(add-hook 'kill-emacs-hook #'org-babel-remove-temporary-stable-directory)
 (defun org-babel-one-header-arg-safe-p (pair safe-list)
   "Determine if the PAIR is a safe babel header arg according to SAFE-LIST.
diff --git a/lisp/ob-gnuplot.el b/lisp/ob-gnuplot.el
index 8c4a5957b..f831b4996 100644
--- a/lisp/ob-gnuplot.el
+++ b/lisp/ob-gnuplot.el
@@ -94,7 +94,10 @@ (defun org-babel-gnuplot-process-vars (params)
 	       (let* ((first  (car val))
 		      (tablep (or (listp first) (symbolp first))))
 		 (if tablep val (mapcar 'list val)))
-	       (org-babel-temp-file "gnuplot-") params)
+               ;; Make temporary file name stable with respect to data.
+               ;; If we do not do it, :cache argument becomes useless.
+               (org-babel-temp-stable-file params "gnuplot-")
+               params)
 	    (if (and (stringp val)
 		     (file-remote-p val)  ;; check if val is a remote file
 		     (file-exists-p val)) ;; call to file-exists-p is slow, maybe remove it

  reply	other threads:[~2021-12-12  7:45 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-22  7:55 when does :cache not cache? Eric S Fraga
2021-12-12  7:45 ` Ihor Radchenko [this message]
2021-12-13  8:13   ` Eric S Fraga
2022-06-09 13:35     ` Ihor Radchenko
2022-06-09 13:41       ` Fraga, Eric
2022-07-04 13:06   ` Ihor Radchenko
2022-07-08 14:59     ` Fraga, Eric
2022-07-08 15:08       ` Ihor Radchenko
2022-07-08 15:19       ` Fraga, Eric
  -- strict thread matches above, loose matches on Subject: below --
2021-10-23  9:13 Emmanuel Charpentier
2021-10-23 10:09 ` Eric S Fraga
2021-10-23 15:15   ` Eric S Fraga
2021-10-24 11:49 Emmanuel Charpentier
2021-10-25 10:58 ` Eric S Fraga

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information: https://www.orgmode.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=875yruxo41.fsf@localhost \
    --to=yantar92@gmail.com \
    --cc=e.fraga@ucl.ac.uk \
    --cc=emacs-orgmode@gnu.org \


* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).