Re: [PATCH] sha1 hash of latex fragments to avoid regeneration

[Carsten Dominik <carsten.dominik@gmail.com>]

Hi Eric,

I had a problem while pushing, please verify that the patch got in  
correctly.  Thanks!

- Carsten

On Nov 17, 2009, at 4:24 PM, Eric Schulte wrote:

> Carsten Dominik <carsten.dominik@gmail.com> writes:
>
>> Wow, this is fantastic!
>>
>> Do you think it is ready to be included (because you say first  
>> pass...)
>>
>
> Yes,
>
> I said first pass because I had only done minimal testing.  However  
> all
> indications are that it works, and there are no further changes I  
> would
> like to make, so if it looks good to you I would say "yes", please  
> apply
> it.
>
> Thanks -- Eric
>
>>
>> - Carsten
>>
>> On Nov 17, 2009, at 3:42 AM, Eric Schulte wrote:
>>
>>> "Eric Schulte" <schulte.eric@gmail.com> writes:
>>>
>>>> Hi Carsten,
>>>>
>>>> Thanks for the feedback, I have comments inline below
>>>>
>>>> Carsten Dominik <carsten.dominik@gmail.com> writes:
>>>
>>> [...]
>>>
>>>>> Now, I am sure that you are already planning to do the same
>>>>> for ditaa images etc?
>>>>
>>>> of course :)
>>>
>>> A first pass at a patch implementing caching of ditaa and dot images
>>> generated by org-exp-blocks is attached.  It seems to work in all
>>> initial tests, and it will only remove files which match the  
>>> following
>>> pattern
>>>
>>> "beginning-of-file-name_\\([[:alnum:]]+\\)\\.extension"
>>>
>>> such that the length of the part matched by \\([[:alnum:]]+\\) is 40
>>> characters.
>>>
>>> Best -- Eric
>>>
>>> From bd90d519a7d3e16732f4f377ccce51bdb97cef90 Mon Sep 17 00:00:00  
>>> 2001
>>> From: Eric Schulte <schulte.eric@gmail.com>
>>> Date: Mon, 16 Nov 2009 19:33:11 -0700
>>> Subject: [PATCH] use sha1 hash keys to cache ditaa and dot images
>>> when exporting through org-exp-blocks
>>>
>>> ---
>>> lisp/ChangeLog         |    6 ++++
>>> lisp/org-exp-blocks.el |   60 +++++++++++++++++++++++++++++++++++++
>>> +---------
>>> 2 files changed, 54 insertions(+), 12 deletions(-)
>>>
>>> diff --git a/lisp/ChangeLog b/lisp/ChangeLog
>>> index 5f83aaa..c2d44fa 100755
>>> --- a/lisp/ChangeLog
>>> +++ b/lisp/ChangeLog
>>> @@ -1,3 +1,9 @@
>>> +2009-11-17  Eric Schulte  <schulte.eric@gmail.com>
>>> +
>>> +	* org-exp-blocks.el (org-export-blocks-format-ditaa): Use sha1
>>> +	hash keys to cache and re-use images generated by the
>>> +	org-exp-blocks interface to ditaa and dot.
>>> +
>>> 2009-11-16  Carsten Dominik  <carsten.dominik@gmail.com>
>>>
>>> 	* org-html.el (org-export-html-home/up-format): Add an ID to the
>>> diff --git a/lisp/org-exp-blocks.el b/lisp/org-exp-blocks.el
>>> index 71e1608..72fe6c4 100644
>>> --- a/lisp/org-exp-blocks.el
>>> +++ b/lisp/org-exp-blocks.el
>>> @@ -217,9 +217,15 @@ Specify the path at which the image should be
>>> saved as the first
>>> element of headers, any additional elements of headers will be
>>> passed to the ditaa utility as command line arguments."
>>>  (message "ditaa-formatting...")
>>> -  (let ((out-file (if headers (car headers)))
>>> -	(args (if (cdr headers) (mapconcat 'identity (cdr headers) " ")))
>>> -	(data-file (make-temp-file "org-ditaa")))
>>> +  (let* ((args (if (cdr headers) (mapconcat 'identity (cdr headers)
>>> " ")))
>>> +         (data-file (make-temp-file "org-ditaa"))
>>> +         (hash (sha1 (prin1-to-string (list body args))))
>>> +         (raw-out-file (if headers (car headers)))
>>> +         (out-file-parts (if (string-match
>>> "\\(.+\\)\\.\\([^\\.]+\\)
>>> $" raw-out-file)
>>> +                             (cons (match-string 1 raw-out-file)
>>> +                                   (match-string 2 raw-out-file))
>>> +                           (cons raw-out-file "png")))
>>> +         (out-file (concat (car out-file-parts) "_" hash "." (cdr
>>> out-file-parts))))
>>>    (unless (file-exists-p org-ditaa-jar-path)
>>>      (error (format "Could not find ditaa.jar at %s" org-ditaa-jar-
>>> path)))
>>>    (setq body (if (string-match "^\\([^:\\|:[^ ]\\)" body)
>>> @@ -229,9 +235,21 @@ passed to the ditaa utility as command line
>>> arguments."
>>> 			    "\n")))
>>>    (cond
>>>     ((or htmlp latexp docbookp)
>>> -      (with-temp-file data-file (insert body))
>>> -      (message (concat "java -jar " org-ditaa-jar-path " " args " "
>>> data-file " " out-file))
>>> -      (shell-command (concat "java -jar " org-ditaa-jar-path " "
>>> args " " data-file " " out-file))
>>> +      (unless (file-exists-p out-file)
>>> +        (mapc ;; remove old hashed versions of this file
>>> +         (lambda (file)
>>> +           (when (and (string-match (concat (regexp-quote (car out-
>>> file-parts))
>>> +                                            "_\\([[:alnum:]]+\\)\ 
>>> \."
>>> +                                            (regexp-quote (cdr out-
>>> file-parts)))
>>> +                                    file)
>>> +                      (= (length (match-string 1 out-file)) 40))
>>> +             (delete-file (expand-file-name file
>>> +                                            (file-name-directory
>>> out-file)))))
>>> +         (directory-files (or (file-name-directory out-file)
>>> +                              default-directory)))
>>> +        (with-temp-file data-file (insert body))
>>> +        (message (concat "java -jar " org-ditaa-jar-path " " args "
>>> " data-file " " out-file))
>>> +        (shell-command (concat "java -jar " org-ditaa-jar-path " "
>>> args " " data-file " " out-file)))
>>>      (format "\n[[file:%s]]\n" out-file))
>>>     (t (concat
>>> 	 "\n#+BEGIN_EXAMPLE\n"
>>> @@ -259,14 +277,32 @@ digraph data_relationships {
>>> }
>>> #+end_dot"
>>>  (message "dot-formatting...")
>>> -  (let ((out-file (if headers (car headers)))
>>> -	(args (if (cdr headers) (mapconcat 'identity (cdr headers) " ")))
>>> -	(data-file (make-temp-file "org-ditaa")))
>>> +  (let* ((args (if (cdr headers) (mapconcat 'identity (cdr headers)
>>> " ")))
>>> +         (data-file (make-temp-file "org-ditaa"))
>>> +         (hash (sha1 (prin1-to-string (list body args))))
>>> +         (raw-out-file (if headers (car headers)))
>>> +         (out-file-parts (if (string-match
>>> "\\(.+\\)\\.\\([^\\.]+\\)
>>> $" raw-out-file)
>>> +                             (cons (match-string 1 raw-out-file)
>>> +                                   (match-string 2 raw-out-file))
>>> +                           (cons raw-out-file "png")))
>>> +         (out-file (concat (car out-file-parts) "_" hash "." (cdr
>>> out-file-parts))))
>>>    (cond
>>>     ((or htmlp latexp docbookp)
>>> -      (with-temp-file data-file (insert body))
>>> -      (message (concat "dot " data-file " " args " -o " out-file))
>>> -      (shell-command (concat "dot " data-file " " args " -o " out-
>>> file))
>>> +      (unless (file-exists-p out-file)
>>> +        (mapc ;; remove old hashed versions of this file
>>> +         (lambda (file)
>>> +           (when (and (string-match (concat (regexp-quote (car out-
>>> file-parts))
>>> +                                            "_\\([[:alnum:]]+\\)\ 
>>> \."
>>> +                                            (regexp-quote (cdr out-
>>> file-parts)))
>>> +                                    file)
>>> +                      (= (length (match-string 1 out-file)) 40))
>>> +             (delete-file (expand-file-name file
>>> +                                            (file-name-directory
>>> out-file)))))
>>> +         (directory-files (or (file-name-directory out-file)
>>> +                              default-directory)))
>>> +        (with-temp-file data-file (insert body))
>>> +        (message (concat "dot " data-file " " args " -o " out- 
>>> file))
>>> +        (shell-command (concat "dot " data-file " " args " -o "
>>> out-
>>> file)))
>>>      (format "\n[[file:%s]]\n" out-file))
>>>     (t (concat
>>> 	 "\n#+BEGIN_EXAMPLE\n"
>>> --
>>> 1.6.4.73.gc144
>>>
>>
>> - Carsten

- Carsten