From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id jYvJNoGL8l+pOAAA0tVLHw (envelope-from ) for ; Mon, 04 Jan 2021 03:29:05 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id aTA8MoGL8l96FAAAbx9fmQ (envelope-from ) for ; Mon, 04 Jan 2021 03:29:05 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 40CD69403D5 for ; Mon, 4 Jan 2021 03:29:05 +0000 (UTC) Received: from localhost ([::1]:41230 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kwGYK-0001gR-10 for larch@yhetil.org; Sun, 03 Jan 2021 22:29:04 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:60348) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kwGXo-0001gJ-SV for emacs-orgmode@gnu.org; Sun, 03 Jan 2021 22:28:32 -0500 Received: from out1.migadu.com ([2001:41d0:2:863f::]:43786) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kwGXh-0006YX-4m for emacs-orgmode@gnu.org; Sun, 03 Jan 2021 22:28:32 -0500 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kyleam.com; s=key1; t=1609730899; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=AGgjgoApGz0ywSTTIo40bRP0O8geKGQxQIqzz+WqqDc=; b=X8MTkFCcFN/96ciRxMTrhOQgRnD6D3Q56vsYQ/k4NLhUFqzWBw4A3wgskbrHGDLzMvIFg7 iwtJlkYR3o8k6/pCI9sq0bmhFQi2/jsXwNbhAHeKC38O7Xh2pZB4f4Xf2y08m1YpnH5+K3 f9pBaY1c0BIudn35qwA+gWL4jfo9Fpm0xZ1KyFbEqHSVZizhnknPk1Xfo5Z5MoECw6kUHX CR0HmJ7c8mt27HoM6BShc8t9ZKrvnEAZJc8VBICniLRzRY8tglYR9DRH4kQBGeI+AqDoYj YQ2/i5I1TAhcGZJR5R/a2YXmktUsuzz8WB5FTbe8i6acEV4ZpW9nrI4Pflh2Iw== From: Kyle Meyer To: Emily Bourke Subject: Re: [PATCH] ox-publish.el: Speed up org-publish-cache-file-needs-publishing In-Reply-To: Message-ID: <87y2h9bctq.fsf@kyleam.com> MIME-Version: 1.0 Content-Type: text/plain X-Migadu-Auth-User: kyle@kyleam.com Date: Mon, 04 Jan 2021 03:28:19 GMT Received-SPF: pass client-ip=2001:41d0:2:863f::; envelope-from=kyle@kyleam.com; helo=out1.migadu.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "emacs-orgmode@gnu.org" Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -2.53 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=kyleam.com header.s=key1 header.b=X8MTkFCc; dmarc=none; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Queue-Id: 40CD69403D5 X-Spam-Score: -2.53 X-Migadu-Scanner: scn1.migadu.com X-TUID: GkAdEZPVjJ+p Thank you for the patch. Emily Bourke writes: > I found publishing when there were no changes to be slower than > expected. Profiling showed me that > `org-publish-cache-file-needs-publishing' was invoking the > `after-find-file' hooks, which I don't think is necessary. > > I've changed it to avoid doing that, by using `with-temp-buffer' and > `insert-file-contents', and noticed a significant increase in speed. > > Is there any reason I'm missing for using `find-file-noselect' in this > case? Nothing jumps out to me. For large files that are already visited, I suppose find-file-noselect returning an existing buffer can be faster, so relevant factors would include how many Org files a project has, how large they are, and how many of those are visited in the current session. My guess is that using with-temp-buffer and insert-file-contents would be a net gain, though that gain would be narrowed some if the temporary buffer was put into org-mode rather than kept in fundamental-mode (more below). > Subject: [PATCH] ox-publish.el: Speed up > org-publish-cache-file-needs-publishing > > * lisp/ox-publish.el (org-publish-cache-file-needs-publishing): Use > `with-temp-buffer' with `insert-file-contents' instead of > `find-file-noselect'. This avoids running the `after-find-file' hook, > which can make it significantly faster. This reads to me like after-find-file is the hook itself. Perhaps something like this would be clearer: "... avoids calling after-find-file and running find-file-hook, ...". > diff --git a/lisp/ox-publish.el b/lisp/ox-publish.el > index 7bb2fed6e..e967286cf 100644 > --- a/lisp/ox-publish.el > +++ b/lisp/ox-publish.el > @@ -1290,29 +1290,26 @@ the file including them will be republished as well." > (org-inhibit-startup t) > included-files-ctime) > (when (equal (file-name-extension filename) "org") > - (let ((visiting (find-buffer-visiting filename)) > - (buf (find-file-noselect filename)) > - (case-fold-search t)) > - (unwind-protect > - (with-current-buffer buf > - (goto-char (point-min)) > - (while (re-search-forward "^[ \t]*#\\+INCLUDE:" nil t) > - (let ((element (org-element-at-point))) > - (when (eq 'keyword (org-element-type element)) > - (let* ((value (org-element-property :value element)) > - (filename > - (and (string-match "\\`\\(\".+?\"\\|\\S-+\\)" value) > - (let ((m (org-strip-quotes > - (match-string 1 value)))) > - ;; Ignore search suffix. > - (if (string-match "::.*?\\'" m) > - (substring m 0 (match-beginning 0)) > - m))))) > - (when filename > - (push (org-publish-cache-ctime-of-src > - (expand-file-name filename)) > - included-files-ctime))))))) > - (unless visiting (kill-buffer buf))))) > + (let ((case-fold-search t)) > + (with-temp-buffer > + (insert-file-contents filename) > + (goto-char (point-min)) The goto-char call can be dropped now because insert-file-contents inserts after point. Unlike the previous code, this doesn't activate org-mode in the buffer. That gives a speedup. And I don't spot any code downstream that depends on the major mode being org-mode, so it's probably safe, though perhaps there's a subtle change in behavior here (e.g., related to syntax table). If org-mode isn't called, the org-inhibit-startup binding above could be dropped. > + (while (re-search-forward "^[ \t]*#\\+INCLUDE:" nil t) > + (let ((element (org-element-at-point))) > + (when (eq 'keyword (org-element-type element)) > + (let* ((value (org-element-property :value element)) > + (filename > + (and (string-match "\\`\\(\".+?\"\\|\\S-+\\)" value) > + (let ((m (org-strip-quotes > + (match-string 1 value)))) > + ;; Ignore search suffix. > + (if (string-match "::.*?\\'" m) > + (substring m 0 (match-beginning 0)) > + m))))) > + (when filename > + (push (org-publish-cache-ctime-of-src > + (expand-file-name filename)) This introduces a regression. With the previous code, the find-file-noselect call led to default-directory being set to the Org file's directory, and then this expand-file call on the included file was relative to that. With the new code, default-directory isn't changed, so it points to a non-existing or incorrect file unless the current default-directory and the Org file's happen to match. > + included-files-ctime)))))))))