From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Eric Schulte" Subject: Re: [PATCH] sha1 hash of latex fragments to avoid regeneration Date: Tue, 17 Nov 2009 10:02:51 -0700 Message-ID: References: <6978E5D0-BBB7-4B45-8270-11FDE93F6726@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1NARSM-00030h-B2 for emacs-orgmode@gnu.org; Tue, 17 Nov 2009 12:03:06 -0500 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1NARSG-0002yd-HZ for emacs-orgmode@gnu.org; Tue, 17 Nov 2009 12:03:05 -0500 Received: from [199.232.76.173] (port=37565 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1NARSG-0002yX-D0 for emacs-orgmode@gnu.org; Tue, 17 Nov 2009 12:03:00 -0500 Received: from mail-pz0-f181.google.com ([209.85.222.181]:46005) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1NARSF-0005KZ-Kv for emacs-orgmode@gnu.org; Tue, 17 Nov 2009 12:03:00 -0500 Received: by pzk11 with SMTP id 11so112680pzk.14 for ; Tue, 17 Nov 2009 09:02:57 -0800 (PST) In-Reply-To: <6978E5D0-BBB7-4B45-8270-11FDE93F6726@gmail.com> (Carsten Dominik's message of "Tue, 17 Nov 2009 16:36:48 +0100") List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Carsten Dominik Cc: Org Mode Hi Carsten, I just pulled, reloaded, and re-ran my simple tests and the patch appears to have been applied successfully. Thanks -- Eric Carsten Dominik writes: > Hi Eric, > > I had a problem while pushing, please verify that the patch got in > correctly. Thanks! > > - Carsten > > On Nov 17, 2009, at 4:24 PM, Eric Schulte wrote: > >> Carsten Dominik writes: >> >>> Wow, this is fantastic! >>> >>> Do you think it is ready to be included (because you say first >>> pass...) >>> >> >> Yes, >> >> I said first pass because I had only done minimal testing. However >> all >> indications are that it works, and there are no further changes I >> would >> like to make, so if it looks good to you I would say "yes", please >> apply >> it. >> >> Thanks -- Eric >> >>> >>> - Carsten >>> >>> On Nov 17, 2009, at 3:42 AM, Eric Schulte wrote: >>> >>>> "Eric Schulte" writes: >>>> >>>>> Hi Carsten, >>>>> >>>>> Thanks for the feedback, I have comments inline below >>>>> >>>>> Carsten Dominik writes: >>>> >>>> [...] >>>> >>>>>> Now, I am sure that you are already planning to do the same >>>>>> for ditaa images etc? >>>>> >>>>> of course :) >>>> >>>> A first pass at a patch implementing caching of ditaa and dot images >>>> generated by org-exp-blocks is attached. It seems to work in all >>>> initial tests, and it will only remove files which match the >>>> following >>>> pattern >>>> >>>> "beginning-of-file-name_\\([[:alnum:]]+\\)\\.extension" >>>> >>>> such that the length of the part matched by \\([[:alnum:]]+\\) is 40 >>>> characters. >>>> >>>> Best -- Eric >>>> >>>> From bd90d519a7d3e16732f4f377ccce51bdb97cef90 Mon Sep 17 00:00:00 >>>> 2001 >>>> From: Eric Schulte >>>> Date: Mon, 16 Nov 2009 19:33:11 -0700 >>>> Subject: [PATCH] use sha1 hash keys to cache ditaa and dot images >>>> when exporting through org-exp-blocks >>>> >>>> --- >>>> lisp/ChangeLog | 6 ++++ >>>> lisp/org-exp-blocks.el | 60 +++++++++++++++++++++++++++++++++++++ >>>> +--------- >>>> 2 files changed, 54 insertions(+), 12 deletions(-) >>>> >>>> diff --git a/lisp/ChangeLog b/lisp/ChangeLog >>>> index 5f83aaa..c2d44fa 100755 >>>> --- a/lisp/ChangeLog >>>> +++ b/lisp/ChangeLog >>>> @@ -1,3 +1,9 @@ >>>> +2009-11-17 Eric Schulte >>>> + >>>> + * org-exp-blocks.el (org-export-blocks-format-ditaa): Use sha1 >>>> + hash keys to cache and re-use images generated by the >>>> + org-exp-blocks interface to ditaa and dot. >>>> + >>>> 2009-11-16 Carsten Dominik >>>> >>>> * org-html.el (org-export-html-home/up-format): Add an ID to the >>>> diff --git a/lisp/org-exp-blocks.el b/lisp/org-exp-blocks.el >>>> index 71e1608..72fe6c4 100644 >>>> --- a/lisp/org-exp-blocks.el >>>> +++ b/lisp/org-exp-blocks.el >>>> @@ -217,9 +217,15 @@ Specify the path at which the image should be >>>> saved as the first >>>> element of headers, any additional elements of headers will be >>>> passed to the ditaa utility as command line arguments." >>>> (message "ditaa-formatting...") >>>> - (let ((out-file (if headers (car headers))) >>>> - (args (if (cdr headers) (mapconcat 'identity (cdr headers) " "))) >>>> - (data-file (make-temp-file "org-ditaa"))) >>>> + (let* ((args (if (cdr headers) (mapconcat 'identity (cdr headers) >>>> " "))) >>>> + (data-file (make-temp-file "org-ditaa")) >>>> + (hash (sha1 (prin1-to-string (list body args)))) >>>> + (raw-out-file (if headers (car headers))) >>>> + (out-file-parts (if (string-match >>>> "\\(.+\\)\\.\\([^\\.]+\\) >>>> $" raw-out-file) >>>> + (cons (match-string 1 raw-out-file) >>>> + (match-string 2 raw-out-file)) >>>> + (cons raw-out-file "png"))) >>>> + (out-file (concat (car out-file-parts) "_" hash "." (cdr >>>> out-file-parts)))) >>>> (unless (file-exists-p org-ditaa-jar-path) >>>> (error (format "Could not find ditaa.jar at %s" org-ditaa-jar- >>>> path))) >>>> (setq body (if (string-match "^\\([^:\\|:[^ ]\\)" body) >>>> @@ -229,9 +235,21 @@ passed to the ditaa utility as command line >>>> arguments." >>>> "\n"))) >>>> (cond >>>> ((or htmlp latexp docbookp) >>>> - (with-temp-file data-file (insert body)) >>>> - (message (concat "java -jar " org-ditaa-jar-path " " args " " >>>> data-file " " out-file)) >>>> - (shell-command (concat "java -jar " org-ditaa-jar-path " " >>>> args " " data-file " " out-file)) >>>> + (unless (file-exists-p out-file) >>>> + (mapc ;; remove old hashed versions of this file >>>> + (lambda (file) >>>> + (when (and (string-match (concat (regexp-quote (car out- >>>> file-parts)) >>>> + "_\\([[:alnum:]]+\\)\ >>>> \." >>>> + (regexp-quote (cdr out- >>>> file-parts))) >>>> + file) >>>> + (= (length (match-string 1 out-file)) 40)) >>>> + (delete-file (expand-file-name file >>>> + (file-name-directory >>>> out-file))))) >>>> + (directory-files (or (file-name-directory out-file) >>>> + default-directory))) >>>> + (with-temp-file data-file (insert body)) >>>> + (message (concat "java -jar " org-ditaa-jar-path " " args " >>>> " data-file " " out-file)) >>>> + (shell-command (concat "java -jar " org-ditaa-jar-path " " >>>> args " " data-file " " out-file))) >>>> (format "\n[[file:%s]]\n" out-file)) >>>> (t (concat >>>> "\n#+BEGIN_EXAMPLE\n" >>>> @@ -259,14 +277,32 @@ digraph data_relationships { >>>> } >>>> #+end_dot" >>>> (message "dot-formatting...") >>>> - (let ((out-file (if headers (car headers))) >>>> - (args (if (cdr headers) (mapconcat 'identity (cdr headers) " "))) >>>> - (data-file (make-temp-file "org-ditaa"))) >>>> + (let* ((args (if (cdr headers) (mapconcat 'identity (cdr headers) >>>> " "))) >>>> + (data-file (make-temp-file "org-ditaa")) >>>> + (hash (sha1 (prin1-to-string (list body args)))) >>>> + (raw-out-file (if headers (car headers))) >>>> + (out-file-parts (if (string-match >>>> "\\(.+\\)\\.\\([^\\.]+\\) >>>> $" raw-out-file) >>>> + (cons (match-string 1 raw-out-file) >>>> + (match-string 2 raw-out-file)) >>>> + (cons raw-out-file "png"))) >>>> + (out-file (concat (car out-file-parts) "_" hash "." (cdr >>>> out-file-parts)))) >>>> (cond >>>> ((or htmlp latexp docbookp) >>>> - (with-temp-file data-file (insert body)) >>>> - (message (concat "dot " data-file " " args " -o " out-file)) >>>> - (shell-command (concat "dot " data-file " " args " -o " out- >>>> file)) >>>> + (unless (file-exists-p out-file) >>>> + (mapc ;; remove old hashed versions of this file >>>> + (lambda (file) >>>> + (when (and (string-match (concat (regexp-quote (car out- >>>> file-parts)) >>>> + "_\\([[:alnum:]]+\\)\ >>>> \." >>>> + (regexp-quote (cdr out- >>>> file-parts))) >>>> + file) >>>> + (= (length (match-string 1 out-file)) 40)) >>>> + (delete-file (expand-file-name file >>>> + (file-name-directory >>>> out-file))))) >>>> + (directory-files (or (file-name-directory out-file) >>>> + default-directory))) >>>> + (with-temp-file data-file (insert body)) >>>> + (message (concat "dot " data-file " " args " -o " out- >>>> file)) >>>> + (shell-command (concat "dot " data-file " " args " -o " >>>> out- >>>> file))) >>>> (format "\n[[file:%s]]\n" out-file)) >>>> (t (concat >>>> "\n#+BEGIN_EXAMPLE\n" >>>> -- >>>> 1.6.4.73.gc144 >>>> >>> >>> - Carsten > > - Carsten