From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id cK3yEoiotWF4OQEAgWs5BA (envelope-from ) for ; Sun, 12 Dec 2021 08:45:12 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id uECyDoiotWH7QwAAB5/wlQ (envelope-from ) for ; Sun, 12 Dec 2021 07:45:12 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 9CE9D351E2 for ; Sun, 12 Dec 2021 08:45:11 +0100 (CET) Received: from localhost ([::1]:36994 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mwJXf-00012g-S9 for larch@yhetil.org; Sun, 12 Dec 2021 02:45:07 -0500 Received: from eggs.gnu.org ([209.51.188.92]:40532) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwJX6-00012Y-RM for emacs-orgmode@gnu.org; Sun, 12 Dec 2021 02:44:32 -0500 Received: from [2a00:1450:4864:20::52b] (port=33446 helo=mail-ed1-x52b.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mwJX4-0005kz-VJ for emacs-orgmode@gnu.org; Sun, 12 Dec 2021 02:44:32 -0500 Received: by mail-ed1-x52b.google.com with SMTP id t5so42248008edd.0 for ; Sat, 11 Dec 2021 23:44:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=TbyZuEkpLAlrju1U0FjLaskFT+sT5zQ01952/iwUDHc=; b=cRnPUkG5j28POWZO5U8dzPMOefweXmQMXjrq7jceTogkBDeOtvPZ7XpGk09VEacl+L ZF1f7tmANGH0yZzbS5aBXXAj57PSHgy4jP6qj/mTMp9h7RUkHPRQ1mfldnDzuX6JgGGB odAoIOMmDk4a0V1XQeXa4fQjlsg8ewaQWsalZagOx19Af5jOD4EFZ8vdKyg6pi5gXVeC 8QcPq3qEY2pqf+cNlEombb9maxHc1WHhxP5ASY2WttPhmGaW31j3Cpn2CE7TyRimGirY oj63zeBI6nhrRzsrv/SzRdFUZFBNhvKr1iUu2E/bRpTGjIUZ6JAr4zmbjdIeEt3QsQHS Qmjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=TbyZuEkpLAlrju1U0FjLaskFT+sT5zQ01952/iwUDHc=; b=onDb9O7AXVSDT9E/nkJviGgx5FnaOwYwAMNtStI4zjPDA3vxZ0tWb4VvsnyjSKAEKS 6zrUsb/9e3OZJCI6QjawbrtE816O3lt7hC6JacOwqQt9+4vgqoU6euwFbWyyK5cYp5fi EvBnG2fGJSQw04KJgy53s98xnqQXBSwYmOlIQ4ZDlpfB7zCSnmZOStt2gDMraGaQqkIJ bI4KwlJNw2GMcWnFnaHsxYZnSXVdLGP7ll85C6C9/U2Q+px3Hw+WK66hsNed0TcAuyKQ hfy6tOE+M0fm0rjuBfxiqiYvo7JzmAS1YHrsbqrkp+jfP6UbAwDJXh2iTNuEt6cCbU47 dAmg== X-Gm-Message-State: AOAM533fWuwALkcUCRKhjSfggUr+mg5CjkWJt6+XgQ03msPXryON3fII +uUsI5e7eCt9lnTYb5HPm7c= X-Google-Smtp-Source: ABdhPJzgfOh69fvT170+pBzp3bM3JNE6SB4T6DOQ6OT/XCAUUOluVqtb7gSWUDk7bD+c4AbcLs8BKg== X-Received: by 2002:a05:6402:3549:: with SMTP id f9mr55530483edd.23.1639295069476; Sat, 11 Dec 2021 23:44:29 -0800 (PST) Received: from localhost ([91.210.107.150]) by smtp.gmail.com with ESMTPSA id z25sm4062241ejd.80.2021.12.11.23.44.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 11 Dec 2021 23:44:29 -0800 (PST) From: Ihor Radchenko To: Eric S Fraga Subject: Re: when does :cache not cache? In-Reply-To: <87mtn1o5mn.fsf@ucl.ac.uk> References: <87mtn1o5mn.fsf@ucl.ac.uk> X-Woof-Patch: yes Date: Sun, 12 Dec 2021 15:45:50 +0800 Message-ID: <875yruxo41.fsf@localhost> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::52b (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::52b; envelope-from=yantar92@gmail.com; helo=mail-ed1-x52b.google.com X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Emacs Org mode mailing list Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1639295111; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=TbyZuEkpLAlrju1U0FjLaskFT+sT5zQ01952/iwUDHc=; b=J6VKY1byDbdY+cm0a0soYQtwIuedBcxetLLHVobHGemF5PsYcLV2mjunkweXXrKi0Wf+li 00wa1QUcvwJ0SjWhH61wWtoNAtpmgcHn+/yQftK2WICnU4ISQwCIBmwbe7eBPa4sYaJJxg D7P726zEs8wMLck8D8S/Hv7Ybgw160pwgiGSBwXqZPCb18CMo4kxBY1f0M3WYWB8hboBsS LL4ucdt1EhVGEVKB9SgjOQ79ay778E7L90zsAmWjchANjNfC/Lo37X9OD4yZzuK9IdrA1G Y3SKuM1Kny4cwehukpX0NHFp3At0VX+NX3D8Ylwbj+gzOAJni7SnHPOeMORXnQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1639295111; a=rsa-sha256; cv=none; b=p5fM2z0Vb+PSwVFbjSVuoo9d9GhEvFVVLGHSH+Qy6TqsBrecO2tF2qXjMqDDsUQ6o8ZH3V 87up7dNYC9abmM8wyHA84RO7WyoZgyt2sJsVMSeMHY+8mRncnFvh74grUnNUM1CojXrZ2d bAOWWdYdzJ72VDQ5of7sA9Xpk0cvToaGBDggO/jVNaVjUx5cJea/GbCmGWOpdMiyIiXjxT eZVQ/JgIv2z7yjQK6es+gMrpeF/NX4TFtIGblBjcDeuhkOZq6GPItnCBFQqYYqDn9JatiF dP6gyPiuelHSYr27v66XJ9BVJZnCRLBc6uIeKE7pXgYhSUoknXa1Eq2/G4WKsQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=cRnPUkG5; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -2.66 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=cRnPUkG5; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 9CE9D351E2 X-Spam-Score: -2.66 X-Migadu-Scanner: scn1.migadu.com X-TUID: E9biuuESFTqi --=-=-= Content-Type: text/plain Eric S Fraga writes: > in the paper I am writing, I have a number of gnuplot src blocks, some > of which process a significant amount of data so take some time to > generate the actual plots. The data are static so caching the results > make sense. However, even though I have ":cache yes" on each of these > named src blocks, and I have (for good measure), the property > "header-args:gnuplot" set to ":cache yes" as well, the plots are being > regenerated each time I export to PDF via LaTeX. I was able to reproduce. The reason why caching does not work is related to the way :var assignments work in ob-gnuplot. We dump the table data into temporary files and refer to those files in generated gnuplot script body. The temporary files names change on every execution and the gnuplot script hash is never going to be the same. I am attaching tentative patch to fix the issue. The patch introduces a new functionality to ob-core.el allowing more stable temporary file names. I am not sure if my implementation is the best way to solve the problem, so comments are welcome. Best, Ihor --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-ob-gnuplot.el-Make-cache-argument-work-with-var-assi.patch >From ac11b4d08edd577b29a398296364b4340096a6ae Mon Sep 17 00:00:00 2001 Message-Id: From: Ihor Radchenko Date: Sun, 12 Dec 2021 15:31:35 +0800 Subject: [PATCH] ob-gnuplot.el: Make :cache argument work with :var assignments * lisp/ob-core.el (org-babel-temporary-stable-directory): New variable holding a temporary directory name that does not change between Emacs sessions. (org-babel-remove-temporary-stable-directory): New function removing `org-babel-temporary-stable-directory' on Emacs shutdown. (org-babel-temp-stable-file): Generate stable temporary file name for object storage. The file name is constant with for equal objects. (org-babel-execute-src-block): Explicitly identify that if the result is cached. * lisp/ob-gnuplot.el (org-babel-gnuplot-process-vars): Make use of `org-babel-stable-file' to make expanded body stable with respect to :var assignments. Fixes https://orgmode.org/list/87mtn1o5mn.fsf@ucl.ac.uk --- lisp/ob-core.el | 52 +++++++++++++++++++++++++++++++++++++++++++++- lisp/ob-gnuplot.el | 5 ++++- 2 files changed, 55 insertions(+), 2 deletions(-) diff --git a/lisp/ob-core.el b/lisp/ob-core.el index 7a9467b0e..d572423b7 100644 --- a/lisp/ob-core.el +++ b/lisp/ob-core.el @@ -735,7 +735,8 @@ (defun org-babel-execute-src-block (&optional arg info params) (forward-line) (skip-chars-forward " \t") (let ((result (org-babel-read-result))) - (message (replace-regexp-in-string "%" "%%" (format "%S" result))) + (message (format "Cached: %s" + (replace-regexp-in-string "%" "%%" (format "%S" result)))) result))) ((org-babel-confirm-evaluate info) (let* ((lang (nth 0 info)) @@ -3112,6 +3113,22 @@ (defvar org-babel-temporary-directory Used by `org-babel-temp-file'. This directory will be removed on Emacs shutdown.")) +(defvar org-babel-temporary-stable-directory) +(unless (or noninteractive (boundp 'org-babel-temporary-stable-directory)) + (defvar org-babel-temporary-stable-directory + (or (and (boundp 'org-babel-temporary-stable-directory) + (file-exists-p org-babel-temporary-stable-directory) + org-babel-temporary-stable-directory) + (condition-case nil + (make-directory + (expand-file-name + "babel-stable" + (temporary-file-directory))) + (t nil))) + "Directory to hold temporary files created to execute code blocks. +Used by `org-babel-temp-file'. This directory will be removed on +Emacs shutdown.")) + (defcustom org-babel-remote-temporary-directory "/tmp/" "Directory to hold temporary files on remote hosts." :group 'org-babel @@ -3155,6 +3172,30 @@ (defun org-babel-temp-file (prefix &optional suffix) temporary-file-directory))) (make-temp-file prefix nil suffix)))) +(defun org-babel-temp-stable-file (data prefix &optional suffix) + "Create a temporary file in the `org-babel-remove-temporary-stable-directory'. +The file name is stable with respect to DATA. The file name is +constructed like the following: PREFIXDATAhashSUFFIX." + (if (file-remote-p default-directory) + (let* ((prefix + (concat (file-remote-p default-directory) + (expand-file-name + prefix org-babel-temporary-stable-directory))) + (path (concat prefix (format "%s" (sxhash data)) (or suffix "")))) + (with-temp-file path) + path) + (let* ((temporary-file-directory + (or (and (boundp 'org-babel-temporary-stable-directory) + (file-exists-p org-babel-temporary-stable-directory) + org-babel-temporary-stable-directory) + temporary-file-directory)) + (path (concat + (expand-file-name + prefix org-babel-temporary-stable-directory) + (format "%s" (sxhash data)) (or suffix "")))) + (with-temp-file path) + path))) + (defun org-babel-remove-temporary-directory () "Remove `org-babel-temporary-directory' on Emacs shutdown." (when (and (boundp 'org-babel-temporary-directory) @@ -3178,7 +3219,16 @@ (defun org-babel-remove-temporary-directory () org-babel-temporary-directory "[directory not defined]")))))) +(defun org-babel-remove-temporary-stable-directory () + "Remove `org-babel-temporary-stable-directory' and on Emacs shutdown." + (when (and (boundp 'org-babel-temporary-stable-directory) + (file-exists-p org-babel-temporary-stable-directory)) + (let ((org-babel-temporary-directory + org-babel-temporary-stable-directory)) + (org-babel-remove-temporary-directory)))) + (add-hook 'kill-emacs-hook #'org-babel-remove-temporary-directory) +(add-hook 'kill-emacs-hook #'org-babel-remove-temporary-stable-directory) (defun org-babel-one-header-arg-safe-p (pair safe-list) "Determine if the PAIR is a safe babel header arg according to SAFE-LIST. diff --git a/lisp/ob-gnuplot.el b/lisp/ob-gnuplot.el index 8c4a5957b..f831b4996 100644 --- a/lisp/ob-gnuplot.el +++ b/lisp/ob-gnuplot.el @@ -94,7 +94,10 @@ (defun org-babel-gnuplot-process-vars (params) (let* ((first (car val)) (tablep (or (listp first) (symbolp first)))) (if tablep val (mapcar 'list val))) - (org-babel-temp-file "gnuplot-") params) + ;; Make temporary file name stable with respect to data. + ;; If we do not do it, :cache argument becomes useless. + (org-babel-temp-stable-file params "gnuplot-") + params) (if (and (stringp val) (file-remote-p val) ;; check if val is a remote file (file-exists-p val)) ;; call to file-exists-p is slow, maybe remove it -- 2.32.0 --=-=-=--