From mboxrd@z Thu Jan 1 00:00:00 1970 From: Carsten Dominik Subject: Re: [PATCH] sha1 hash of latex fragments to avoid regeneration Date: Mon, 16 Nov 2009 07:57:22 +0100 Message-ID: References: Mime-Version: 1.0 (Apple Message framework v936) Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Return-path: Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1N9wRA-0002LD-Vl for emacs-orgmode@gnu.org; Mon, 16 Nov 2009 02:55:49 -0500 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1N9wR6-0002Jg-4L for emacs-orgmode@gnu.org; Mon, 16 Nov 2009 02:55:48 -0500 Received: from [199.232.76.173] (port=60171 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1N9wR5-0002Jd-VM for emacs-orgmode@gnu.org; Mon, 16 Nov 2009 02:55:44 -0500 Received: from ey-out-1920.google.com ([74.125.78.150]:39031) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1N9wR5-0003KU-3v for emacs-orgmode@gnu.org; Mon, 16 Nov 2009 02:55:43 -0500 Received: by ey-out-1920.google.com with SMTP id 3so1719539eyh.34 for ; Sun, 15 Nov 2009 23:55:41 -0800 (PST) In-Reply-To: List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Eric Schulte Cc: Org Mode Hi Eric, this is fantastic, thank you for implementing it. I have wanted some speedup for this for a long time. I think your implementation still suffers from one issue: The produced image also depends on the variables org-format-latex- options, org-format-latex-header, org-export-latex-package-alist, and on the `forbuffer' flag (because images made for display in the buffer and fo HTML export generaly need different resolution). One way to deal with this would be to make a list containing the values of these four variables and using prin1-to-string to convert this list into a string, and then to prepend this string to TXT when creating the hash. Now, I am sure that you are already planning to do the same for ditaa images etc? That would be a treat, because ditaa can be terribly slow for complex figures, and this would speed up the cycle when writing document by quite a bit. There is one further issue: Cleaning up images that are no longer used. With the LaTeX fragments it is not a big problem, because there live in a special directory. This would be a bigger concern for ditaa images etc which tend to live in the same directory as the source. Maybe that could be solved by 1. Making sure that each image still have a name like "blue", so that the name now would be "blus_loooooonghashvalue.png" or so. 2. Maybe creating a command that will look for orphaned images and remove them, by looking for the hash in the name and checking access times. I am not sure if this is needed, and not sure what would be the best way to implement it. After looking at these things, I would be *very* happy to accept this patch. - Carsten On Nov 16, 2009, at 1:07 AM, Eric Schulte wrote: > Hi, > > The attached patch changes the latex fragment image generation so that > it saves images into files named by the sha1 hash of the latex source > code. By checking for the existence of image files before image > generation the regeneration of identical images is avoided. > > In practice I find that this greatly speeds up export to html and the > `org-preview-latex-fragment' command. > > Cheers -- Eric > > From 13e1c48fa6cac43b0c87ca0fbc8e349f7a9fa864 Mon Sep 17 00:00:00 2001 > From: Eric Schulte > Date: Sun, 15 Nov 2009 17:00:09 -0700 > Subject: [PATCH] latex fragment images cached using sha1 hash keys > > Latex fragment images are now saved in files named by the sha1 hash > of the latex text used to create the image. By checking if files > exist before images generation the regeneration of identical latex > images is avoided. > --- > lisp/ChangeLog | 6 ++++++ > lisp/org.el | 18 +++++++----------- > 2 files changed, 13 insertions(+), 11 deletions(-) > > diff --git a/lisp/ChangeLog b/lisp/ChangeLog > index 339f248..f18755c 100755 > --- a/lisp/ChangeLog > +++ b/lisp/ChangeLog > @@ -1,3 +1,9 @@ > +2009-11-16 Eric Schulte > + > + * org.el (org-format-latex): Latex images are now saved to files > + named by the sha1 hash of the latex source text avoiding > + regeneration of identical images. > + > 2009-11-15 Carsten Dominik > > * org-wl.el (org-wl-store-link): Handle the case that > diff --git a/lisp/org.el b/lisp/org.el > index bf6573b..46348fc 100644 > --- a/lisp/org.el > +++ b/lisp/org.el > @@ -14550,15 +14550,9 @@ Some of the options can be changed using > the variable > (opt org-format-latex-options) > (matchers (plist-get opt :matchers)) > (re-list org-latex-regexps) > - (cnt 0) txt link beg end re e checkdir > + (cnt 0) txt hash link beg end re e checkdir > executables-checked > m n block linkfile movefile ov) > - ;; Check if there are old images files with this prefix, and > remove them > - (when (file-directory-p todir) > - (mapc 'delete-file > - (directory-files > - todir 'full > - (concat (regexp-quote prefixnodir) "_[0-9]+\\.png$")))) > ;; Check the different regular expressions > (while (setq e (pop re-list)) > (setq m (car e) re (nth 1 e) n (nth 2 e) > @@ -14576,9 +14570,10 @@ Some of the options can be changed using > the variable > (setq txt (match-string n) > beg (match-beginning n) end (match-end n) > cnt (1+ cnt) > - linkfile (format "%s_%04d.png" prefix cnt) > - movefile (format "%s_%04d.png" absprefix cnt) > link (concat block "[[file:" linkfile "]]" block)) > + (setq hash (sha1 txt) > + linkfile (format "%s_%s.png" prefix hash) > + movefile (format "%s_%s.png" absprefix hash)) > (if msg (message msg cnt)) > (goto-char beg) > (unless checkdir ; make sure the directory exists > @@ -14592,8 +14587,9 @@ Some of the options can be changed using the > variable > "dvipng" "needed to convert LaTeX fragments to images") > (setq executables-checked t)) > > - (org-create-formula-image > - txt movefile opt forbuffer) > + (unless (file-exists-p movefile) > + (org-create-formula-image > + txt movefile opt forbuffer)) > (if overlays > (progn > (mapc (lambda (o) > -- > 1.6.4.73.gc144 > > _______________________________________________ > Emacs-orgmode mailing list > Remember: use `Reply All' to send replies to the list. > Emacs-orgmode@gnu.org > http://lists.gnu.org/mailman/listinfo/emacs-orgmode - Carsten