Matt Lundin writes: > This patch does two things: > > 1. It prevents org-publish from perpetually republishing files > containing includes. Currently, if an included file changes, it is > published, as is the file that includes it. However, until changes are > made in the file that includes it (which may be never) its modification > time remain older than that of the included file, so it will be > republished every time org-publish is called, even if neither file has > changed. > > - Note: This patch fixes this behavior by updating the modification > time of the file that contains the includes. If this is deemed too > destructive/meddlesome, we could consider alternate behaviors, such > as setting the cached time with a simple float-time. Let me know > what would be best. > > 2. It optimizes checking for included files. Currently, > org-publish-cache-file-needs-publishing visits every file in a project > to check for includes. On my underpowered box, this takes a long time > (over a minute) on a project with 1000+ files, thus defeating the > purpose of the cache. This patch causes org-publish to store information > about included files in the cache itself: > > a. If a file has changed it updates information about the files it > includes and stores it in the cache. > > b. If a file has not changed, it checks the cache for included files > and sees if any of those files have been updated (thus preventing the > need to check every file in the project for includes) > > Best, > Matt > > From 94a8061bb30e1992213fb8e71ee949d336d37435 Mon Sep 17 00:00:00 2001 > From: Matt Lundin > Date: Tue, 12 Aug 2014 07:51:44 -0500 > Subject: [PATCH] Fix and optimize publish cache check > > * lisp/ox-publish.el: (org-publish-cache-file-needs-publishing) Fix > org-publish-cache-file-needs-publishing to change timestamp of files > containing includes. Speed up check for includes by storing > information about included files in cache itself. > > This patch ensures that org-publish-cache-file-needs-publishing does > not keep publishing a file containing includes by updating the > modification time of that file. It also speeds up publishing by > caching information about included files, thus keeping > org-publish-cache-file-needs-publishing from having to visit every > file just to check includes (which can take a long time on a project > containing hundreds or thousands of files). > --- > lisp/ox-publish.el | 60 +++++++++++++++++++++++++++++++----------------------- > 1 file changed, 35 insertions(+), 25 deletions(-) > > diff --git a/lisp/ox-publish.el b/lisp/ox-publish.el > index df40572..228411f 100644 > --- a/lisp/ox-publish.el > +++ b/lisp/ox-publish.el > @@ -1168,32 +1168,42 @@ the file including them will be republished as well." > (pstamp (org-publish-cache-get key)) > (org-inhibit-startup t) > (visiting (find-buffer-visiting filename)) > - included-files-ctime buf) > - (when (equal (file-name-extension filename) "org") > - (setq buf (find-file (expand-file-name filename))) > + (ctime (org-publish-cache-ctime-of-src filename)) > + (needsp (or (null pstamp) (< pstamp ctime))) > + includes buf) > + ;; if the file needs publishing, refresh the included-files cache property > + (when (and needsp > + (equal (file-name-extension filename) "org")) > + (setq buf (find-file-noselect (expand-file-name filename))) > (with-current-buffer buf > - (goto-char (point-min)) > - (while (re-search-forward "^[ \t]*#\\+INCLUDE:" nil t) > - (let* ((element (org-element-at-point)) > - (included-file > - (and (eq (org-element-type element) 'keyword) > - (let ((value (org-element-property :value element))) > - (and value > - (string-match "^\\(\".+?\"\\|\\S-+\\)" value) > - (org-remove-double-quotes > - (match-string 1 value))))))) > - (when included-file > - (add-to-list 'included-files-ctime > - (org-publish-cache-ctime-of-src > - (expand-file-name included-file)) > - t))))) > - (unless visiting (kill-buffer buf))) > - (if (null pstamp) t > - (let ((ctime (org-publish-cache-ctime-of-src filename))) > - (or (< pstamp ctime) > - (when included-files-ctime > - (not (null (delq nil (mapcar (lambda (ct) (< ctime ct)) > - included-files-ctime)))))))))) > + (save-excursion > + (goto-char (point-min)) > + (while (re-search-forward "^[ \t]*#\\+INCLUDE:" nil t) > + (let* ((element (org-element-at-point)) > + (included-file > + (and (eq (org-element-type element) 'keyword) > + (let ((value (org-element-property :value element))) > + (and value > + (string-match "^\\(\".+?\"\\|\\S-+\\)" value) > + (org-remove-double-quotes > + (match-string 1 value))))))) > + (when included-file (add-to-list 'includes (expand-file-name included-file))))))) > + (unless visiting (kill-buffer buf)) > + (when includes (org-publish-cache-set-file-property filename :includes includes))) > + ;; return t if needsp or if included files have changed > + (or needsp > + (when (delq nil > + (mapcar (lambda (file) > + (let ((ct (org-publish-cache-ctime-of-src file))) > + (and (file-exists-p file) > + (< ctime ct)))) > + (org-publish-cache-get-file-property filename :includes))) > + ;; update the timestamp of the published file, but only if buffer is not modified > + (setq buf (find-file-noselect (expand-file-name filename))) > + (with-current-buffer buf > + (when (not (buffer-modified-p)) (set-buffer-modified-p t) (save-buffer))) > + (unless visiting (kill-buffer buf)) > + t)))) > > (defun org-publish-cache-set-file-property > (filename property value &optional project-name)