From mboxrd@z Thu Jan 1 00:00:00 1970 From: Carsten Dominik Subject: Re: [PATCH] sha1 hash of latex fragments to avoid regeneration Date: Tue, 17 Nov 2009 16:36:48 +0100 Message-ID: <6978E5D0-BBB7-4B45-8270-11FDE93F6726@gmail.com> References: Mime-Version: 1.0 (Apple Message framework v936) Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Return-path: Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1NAQQ9-0001oB-E3 for emacs-orgmode@gnu.org; Tue, 17 Nov 2009 10:56:45 -0500 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1NAQQ4-0001kr-Rz for emacs-orgmode@gnu.org; Tue, 17 Nov 2009 10:56:44 -0500 Received: from [199.232.76.173] (port=45697 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1NAQQ4-0001ko-Eu for emacs-orgmode@gnu.org; Tue, 17 Nov 2009 10:56:40 -0500 Received: from mail-bw0-f215.google.com ([209.85.218.215]:51147) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1NAQ74-0008Qk-96 for emacs-orgmode@gnu.org; Tue, 17 Nov 2009 10:37:02 -0500 Received: by bwz7 with SMTP id 7so109997bwz.26 for ; Tue, 17 Nov 2009 07:37:01 -0800 (PST) In-Reply-To: List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Eric Schulte Cc: Org Mode Hi Eric, I had a problem while pushing, please verify that the patch got in correctly. Thanks! - Carsten On Nov 17, 2009, at 4:24 PM, Eric Schulte wrote: > Carsten Dominik writes: > >> Wow, this is fantastic! >> >> Do you think it is ready to be included (because you say first >> pass...) >> > > Yes, > > I said first pass because I had only done minimal testing. However > all > indications are that it works, and there are no further changes I > would > like to make, so if it looks good to you I would say "yes", please > apply > it. > > Thanks -- Eric > >> >> - Carsten >> >> On Nov 17, 2009, at 3:42 AM, Eric Schulte wrote: >> >>> "Eric Schulte" writes: >>> >>>> Hi Carsten, >>>> >>>> Thanks for the feedback, I have comments inline below >>>> >>>> Carsten Dominik writes: >>> >>> [...] >>> >>>>> Now, I am sure that you are already planning to do the same >>>>> for ditaa images etc? >>>> >>>> of course :) >>> >>> A first pass at a patch implementing caching of ditaa and dot images >>> generated by org-exp-blocks is attached. It seems to work in all >>> initial tests, and it will only remove files which match the >>> following >>> pattern >>> >>> "beginning-of-file-name_\\([[:alnum:]]+\\)\\.extension" >>> >>> such that the length of the part matched by \\([[:alnum:]]+\\) is 40 >>> characters. >>> >>> Best -- Eric >>> >>> From bd90d519a7d3e16732f4f377ccce51bdb97cef90 Mon Sep 17 00:00:00 >>> 2001 >>> From: Eric Schulte >>> Date: Mon, 16 Nov 2009 19:33:11 -0700 >>> Subject: [PATCH] use sha1 hash keys to cache ditaa and dot images >>> when exporting through org-exp-blocks >>> >>> --- >>> lisp/ChangeLog | 6 ++++ >>> lisp/org-exp-blocks.el | 60 +++++++++++++++++++++++++++++++++++++ >>> +--------- >>> 2 files changed, 54 insertions(+), 12 deletions(-) >>> >>> diff --git a/lisp/ChangeLog b/lisp/ChangeLog >>> index 5f83aaa..c2d44fa 100755 >>> --- a/lisp/ChangeLog >>> +++ b/lisp/ChangeLog >>> @@ -1,3 +1,9 @@ >>> +2009-11-17 Eric Schulte >>> + >>> + * org-exp-blocks.el (org-export-blocks-format-ditaa): Use sha1 >>> + hash keys to cache and re-use images generated by the >>> + org-exp-blocks interface to ditaa and dot. >>> + >>> 2009-11-16 Carsten Dominik >>> >>> * org-html.el (org-export-html-home/up-format): Add an ID to the >>> diff --git a/lisp/org-exp-blocks.el b/lisp/org-exp-blocks.el >>> index 71e1608..72fe6c4 100644 >>> --- a/lisp/org-exp-blocks.el >>> +++ b/lisp/org-exp-blocks.el >>> @@ -217,9 +217,15 @@ Specify the path at which the image should be >>> saved as the first >>> element of headers, any additional elements of headers will be >>> passed to the ditaa utility as command line arguments." >>> (message "ditaa-formatting...") >>> - (let ((out-file (if headers (car headers))) >>> - (args (if (cdr headers) (mapconcat 'identity (cdr headers) " "))) >>> - (data-file (make-temp-file "org-ditaa"))) >>> + (let* ((args (if (cdr headers) (mapconcat 'identity (cdr headers) >>> " "))) >>> + (data-file (make-temp-file "org-ditaa")) >>> + (hash (sha1 (prin1-to-string (list body args)))) >>> + (raw-out-file (if headers (car headers))) >>> + (out-file-parts (if (string-match >>> "\\(.+\\)\\.\\([^\\.]+\\) >>> $" raw-out-file) >>> + (cons (match-string 1 raw-out-file) >>> + (match-string 2 raw-out-file)) >>> + (cons raw-out-file "png"))) >>> + (out-file (concat (car out-file-parts) "_" hash "." (cdr >>> out-file-parts)))) >>> (unless (file-exists-p org-ditaa-jar-path) >>> (error (format "Could not find ditaa.jar at %s" org-ditaa-jar- >>> path))) >>> (setq body (if (string-match "^\\([^:\\|:[^ ]\\)" body) >>> @@ -229,9 +235,21 @@ passed to the ditaa utility as command line >>> arguments." >>> "\n"))) >>> (cond >>> ((or htmlp latexp docbookp) >>> - (with-temp-file data-file (insert body)) >>> - (message (concat "java -jar " org-ditaa-jar-path " " args " " >>> data-file " " out-file)) >>> - (shell-command (concat "java -jar " org-ditaa-jar-path " " >>> args " " data-file " " out-file)) >>> + (unless (file-exists-p out-file) >>> + (mapc ;; remove old hashed versions of this file >>> + (lambda (file) >>> + (when (and (string-match (concat (regexp-quote (car out- >>> file-parts)) >>> + "_\\([[:alnum:]]+\\)\ >>> \." >>> + (regexp-quote (cdr out- >>> file-parts))) >>> + file) >>> + (= (length (match-string 1 out-file)) 40)) >>> + (delete-file (expand-file-name file >>> + (file-name-directory >>> out-file))))) >>> + (directory-files (or (file-name-directory out-file) >>> + default-directory))) >>> + (with-temp-file data-file (insert body)) >>> + (message (concat "java -jar " org-ditaa-jar-path " " args " >>> " data-file " " out-file)) >>> + (shell-command (concat "java -jar " org-ditaa-jar-path " " >>> args " " data-file " " out-file))) >>> (format "\n[[file:%s]]\n" out-file)) >>> (t (concat >>> "\n#+BEGIN_EXAMPLE\n" >>> @@ -259,14 +277,32 @@ digraph data_relationships { >>> } >>> #+end_dot" >>> (message "dot-formatting...") >>> - (let ((out-file (if headers (car headers))) >>> - (args (if (cdr headers) (mapconcat 'identity (cdr headers) " "))) >>> - (data-file (make-temp-file "org-ditaa"))) >>> + (let* ((args (if (cdr headers) (mapconcat 'identity (cdr headers) >>> " "))) >>> + (data-file (make-temp-file "org-ditaa")) >>> + (hash (sha1 (prin1-to-string (list body args)))) >>> + (raw-out-file (if headers (car headers))) >>> + (out-file-parts (if (string-match >>> "\\(.+\\)\\.\\([^\\.]+\\) >>> $" raw-out-file) >>> + (cons (match-string 1 raw-out-file) >>> + (match-string 2 raw-out-file)) >>> + (cons raw-out-file "png"))) >>> + (out-file (concat (car out-file-parts) "_" hash "." (cdr >>> out-file-parts)))) >>> (cond >>> ((or htmlp latexp docbookp) >>> - (with-temp-file data-file (insert body)) >>> - (message (concat "dot " data-file " " args " -o " out-file)) >>> - (shell-command (concat "dot " data-file " " args " -o " out- >>> file)) >>> + (unless (file-exists-p out-file) >>> + (mapc ;; remove old hashed versions of this file >>> + (lambda (file) >>> + (when (and (string-match (concat (regexp-quote (car out- >>> file-parts)) >>> + "_\\([[:alnum:]]+\\)\ >>> \." >>> + (regexp-quote (cdr out- >>> file-parts))) >>> + file) >>> + (= (length (match-string 1 out-file)) 40)) >>> + (delete-file (expand-file-name file >>> + (file-name-directory >>> out-file))))) >>> + (directory-files (or (file-name-directory out-file) >>> + default-directory))) >>> + (with-temp-file data-file (insert body)) >>> + (message (concat "dot " data-file " " args " -o " out- >>> file)) >>> + (shell-command (concat "dot " data-file " " args " -o " >>> out- >>> file))) >>> (format "\n[[file:%s]]\n" out-file)) >>> (t (concat >>> "\n#+BEGIN_EXAMPLE\n" >>> -- >>> 1.6.4.73.gc144 >>> >> >> - Carsten - Carsten