From mboxrd@z Thu Jan 1 00:00:00 1970 From: Francesco Pizzolante Subject: Re: [PATCH] sha1 hash of latex fragments to avoid regeneration Date: Wed, 02 Dec 2009 11:35:05 +0100 Message-ID: <87ws15zmsm.fsf@missioncriticalit.com> References: <6978E5D0-BBB7-4B45-8270-11FDE93F6726@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: In-Reply-To: (Eric Schulte's message of "Tue, 17 Nov 2009 10:02:51 -0700") List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org-mXXj517/zsQ@public.gmane.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org-mXXj517/zsQ@public.gmane.org To: mailing-list-org-mode Hi, > I just pulled, reloaded, and re-ran my simple tests and the patch > appears to have been applied successfully. This idea of caching images is really great and works well. Nevertheless, I have a remark about the way it is implemented. When I write the following code: --8<---------------cut here---------------start------------->8--- #+BEGIN_ditaa procedure.png --8<---------------cut here---------------end--------------->8--- I expect that: - the system generates a `procedure.png' file in order for me to check it in the repository; - that the `procedure.png' file is referenced everywhere (i.e. in the exported HTML code or LaTeX code). The current way it works is: 1. The `procedure_XXX.png' is committed to the repository; 2. I alter my ditaa code, and export the new document; 3. The `procedure_XXX.png' file has been removed from my working copy and a new file `procedure_YYY.png' has been added in my working copy. Thus, in order to commit the new files as they are, I need to: 1. revert the `procedure_XXX.png' file, in order to delete it afterward; 2. add the `procedure_YYY.png' file to the repository. A nice solution would have been that all ditaa exports are done under the filename `procedure.png', and that empty files such as `procedure.png.ZZZ.cache' are created to "store" the fact that they have already been generated. That way, we even can ignore all the `*.cache' files from our working copy, when checking for updates. What do you think? Francesco > > Thanks -- Eric > > Carsten Dominik writes: > >> Hi Eric, >> >> I had a problem while pushing, please verify that the patch got in >> correctly. Thanks! >> >> - Carsten >> >> On Nov 17, 2009, at 4:24 PM, Eric Schulte wrote: >> >>> Carsten Dominik writes: >>> >>>> Wow, this is fantastic! >>>> >>>> Do you think it is ready to be included (because you say first >>>> pass...) >>>> >>> >>> Yes, >>> >>> I said first pass because I had only done minimal testing. However >>> all >>> indications are that it works, and there are no further changes I >>> would >>> like to make, so if it looks good to you I would say "yes", please >>> apply >>> it. >>> >>> Thanks -- Eric >>> >>>> >>>> - Carsten >>>> >>>> On Nov 17, 2009, at 3:42 AM, Eric Schulte wrote: >>>> >>>>> "Eric Schulte" writes: >>>>> >>>>>> Hi Carsten, >>>>>> >>>>>> Thanks for the feedback, I have comments inline below >>>>>> >>>>>> Carsten Dominik writes: >>>>> >>>>> [...] >>>>> >>>>>>> Now, I am sure that you are already planning to do the same >>>>>>> for ditaa images etc? >>>>>> >>>>>> of course :) >>>>> >>>>> A first pass at a patch implementing caching of ditaa and dot images >>>>> generated by org-exp-blocks is attached. It seems to work in all >>>>> initial tests, and it will only remove files which match the >>>>> following >>>>> pattern >>>>> >>>>> "beginning-of-file-name_\\([[:alnum:]]+\\)\\.extension" >>>>> >>>>> such that the length of the part matched by \\([[:alnum:]]+\\) is 40 >>>>> characters. >>>>> >>>>> Best -- Eric >>>>> >>>>> From bd90d519a7d3e16732f4f377ccce51bdb97cef90 Mon Sep 17 00:00:00 >>>>> 2001 >>>>> From: Eric Schulte >>>>> Date: Mon, 16 Nov 2009 19:33:11 -0700 >>>>> Subject: [PATCH] use sha1 hash keys to cache ditaa and dot images >>>>> when exporting through org-exp-blocks >>>>> >>>>> --- >>>>> lisp/ChangeLog | 6 ++++ >>>>> lisp/org-exp-blocks.el | 60 +++++++++++++++++++++++++++++++++++++ >>>>> +--------- >>>>> 2 files changed, 54 insertions(+), 12 deletions(-) >>>>> >>>>> diff --git a/lisp/ChangeLog b/lisp/ChangeLog >>>>> index 5f83aaa..c2d44fa 100755 >>>>> --- a/lisp/ChangeLog >>>>> +++ b/lisp/ChangeLog >>>>> @@ -1,3 +1,9 @@ >>>>> +2009-11-17 Eric Schulte >>>>> + >>>>> + * org-exp-blocks.el (org-export-blocks-format-ditaa): Use sha1 >>>>> + hash keys to cache and re-use images generated by the >>>>> + org-exp-blocks interface to ditaa and dot. >>>>> + >>>>> 2009-11-16 Carsten Dominik >>>>> >>>>> * org-html.el (org-export-html-home/up-format): Add an ID to the >>>>> diff --git a/lisp/org-exp-blocks.el b/lisp/org-exp-blocks.el >>>>> index 71e1608..72fe6c4 100644 >>>>> --- a/lisp/org-exp-blocks.el >>>>> +++ b/lisp/org-exp-blocks.el >>>>> @@ -217,9 +217,15 @@ Specify the path at which the image should be >>>>> saved as the first >>>>> element of headers, any additional elements of headers will be >>>>> passed to the ditaa utility as command line arguments." >>>>> (message "ditaa-formatting...") >>>>> - (let ((out-file (if headers (car headers))) >>>>> - (args (if (cdr headers) (mapconcat 'identity (cdr headers) " "))) >>>>> - (data-file (make-temp-file "org-ditaa"))) >>>>> + (let* ((args (if (cdr headers) (mapconcat 'identity (cdr headers) >>>>> " "))) >>>>> + (data-file (make-temp-file "org-ditaa")) >>>>> + (hash (sha1 (prin1-to-string (list body args)))) >>>>> + (raw-out-file (if headers (car headers))) >>>>> + (out-file-parts (if (string-match >>>>> "\\(.+\\)\\.\\([^\\.]+\\) >>>>> $" raw-out-file) >>>>> + (cons (match-string 1 raw-out-file) >>>>> + (match-string 2 raw-out-file)) >>>>> + (cons raw-out-file "png"))) >>>>> + (out-file (concat (car out-file-parts) "_" hash "." (cdr >>>>> out-file-parts)))) >>>>> (unless (file-exists-p org-ditaa-jar-path) >>>>> (error (format "Could not find ditaa.jar at %s" org-ditaa-jar- >>>>> path))) >>>>> (setq body (if (string-match "^\\([^:\\|:[^ ]\\)" body) >>>>> @@ -229,9 +235,21 @@ passed to the ditaa utility as command line >>>>> arguments." >>>>> "\n"))) >>>>> (cond >>>>> ((or htmlp latexp docbookp) >>>>> - (with-temp-file data-file (insert body)) >>>>> - (message (concat "java -jar " org-ditaa-jar-path " " args " " >>>>> data-file " " out-file)) >>>>> - (shell-command (concat "java -jar " org-ditaa-jar-path " " >>>>> args " " data-file " " out-file)) >>>>> + (unless (file-exists-p out-file) >>>>> + (mapc ;; remove old hashed versions of this file >>>>> + (lambda (file) >>>>> + (when (and (string-match (concat (regexp-quote (car out- >>>>> file-parts)) >>>>> + "_\\([[:alnum:]]+\\)\ >>>>> \." >>>>> + (regexp-quote (cdr out- >>>>> file-parts))) >>>>> + file) >>>>> + (= (length (match-string 1 out-file)) 40)) >>>>> + (delete-file (expand-file-name file >>>>> + (file-name-directory >>>>> out-file))))) >>>>> + (directory-files (or (file-name-directory out-file) >>>>> + default-directory))) >>>>> + (with-temp-file data-file (insert body)) >>>>> + (message (concat "java -jar " org-ditaa-jar-path " " args " >>>>> " data-file " " out-file)) >>>>> + (shell-command (concat "java -jar " org-ditaa-jar-path " " >>>>> args " " data-file " " out-file))) >>>>> (format "\n[[file:%s]]\n" out-file)) >>>>> (t (concat >>>>> "\n#+BEGIN_EXAMPLE\n" >>>>> @@ -259,14 +277,32 @@ digraph data_relationships { >>>>> } >>>>> #+end_dot" >>>>> (message "dot-formatting...") >>>>> - (let ((out-file (if headers (car headers))) >>>>> - (args (if (cdr headers) (mapconcat 'identity (cdr headers) " "))) >>>>> - (data-file (make-temp-file "org-ditaa"))) >>>>> + (let* ((args (if (cdr headers) (mapconcat 'identity (cdr headers) >>>>> " "))) >>>>> + (data-file (make-temp-file "org-ditaa")) >>>>> + (hash (sha1 (prin1-to-string (list body args)))) >>>>> + (raw-out-file (if headers (car headers))) >>>>> + (out-file-parts (if (string-match >>>>> "\\(.+\\)\\.\\([^\\.]+\\) >>>>> $" raw-out-file) >>>>> + (cons (match-string 1 raw-out-file) >>>>> + (match-string 2 raw-out-file)) >>>>> + (cons raw-out-file "png"))) >>>>> + (out-file (concat (car out-file-parts) "_" hash "." (cdr >>>>> out-file-parts)))) >>>>> (cond >>>>> ((or htmlp latexp docbookp) >>>>> - (with-temp-file data-file (insert body)) >>>>> - (message (concat "dot " data-file " " args " -o " out-file)) >>>>> - (shell-command (concat "dot " data-file " " args " -o " out- >>>>> file)) >>>>> + (unless (file-exists-p out-file) >>>>> + (mapc ;; remove old hashed versions of this file >>>>> + (lambda (file) >>>>> + (when (and (string-match (concat (regexp-quote (car out- >>>>> file-parts)) >>>>> + "_\\([[:alnum:]]+\\)\ >>>>> \." >>>>> + (regexp-quote (cdr out- >>>>> file-parts))) >>>>> + file) >>>>> + (= (length (match-string 1 out-file)) 40)) >>>>> + (delete-file (expand-file-name file >>>>> + (file-name-directory >>>>> out-file))))) >>>>> + (directory-files (or (file-name-directory out-file) >>>>> + default-directory))) >>>>> + (with-temp-file data-file (insert body)) >>>>> + (message (concat "dot " data-file " " args " -o " out- >>>>> file)) >>>>> + (shell-command (concat "dot " data-file " " args " -o " >>>>> out- >>>>> file))) >>>>> (format "\n[[file:%s]]\n" out-file)) >>>>> (t (concat >>>>> "\n#+BEGIN_EXAMPLE\n" >>>>> -- >>>>> 1.6.4.73.gc144 >>>>> >>>> >>>> - Carsten >> >> - Carsten > > > _______________________________________________ > Emacs-orgmode mailing list > Remember: use `Reply All' to send replies to the list. > Emacs-orgmode-mXXj517/zsQ@public.gmane.org > http://lists.gnu.org/mailman/listinfo/emacs-orgmode -- ________________________________________________ Francesco Pizzolante - fpz-djc/iPCCuDYQheJpep6IedvLeJWuRmrY@public.gmane.org M i s s i o n C r i t i c a l I T Phone ... +32 2-757.10.15 Fax ..... +32 2-759.27.60 _______________________________________________ Emacs-orgmode mailing list Please use `Reply All' to send replies to the list. Emacs-orgmode-mXXj517/zsQ@public.gmane.org http://lists.gnu.org/mailman/listinfo/emacs-orgmode