emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* [ox-publish, patch] More flexible sitemaps
@ 2016-05-19 15:39 Rasmus
  2016-05-22 22:58 ` Nicolas Goaziou
  0 siblings, 1 reply; 8+ messages in thread
From: Rasmus @ 2016-05-19 15:39 UTC (permalink / raw)
  To: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 690 bytes --]

Hi,

I've long wanted to use ox to auto-generate something that looks like a
blog index.

This patch makes ox sitemaps a bit more flexible.  For instance, it would
allow me to use something like this for ‘:sitemap-file-entry-format’,

    :sitemap-file-entry-format "* [[file:%l][%t]]
    #+include: \"%f::lead\"

    [[file:%l][Read more]]"

Which would come out as;

    * [[file:link][Title]]
    #+Include: "file.org::lead"

    [[File:link][Read more]]

For the tests I did, it matches the "old" sitemap for list and tree.

WDYT?

I would particularly like feedback on simplification for the ordering of
the tree’ed filenames.

Rasmus

-- 
This space is left intentionally blank

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ox-publish-More-flexible-sitemaps.patch --]
[-- Type: text/x-diff, Size: 18582 bytes --]

From e6b35524ba0959b6ca4057555325ec7d755248da Mon Sep 17 00:00:00 2001
From: Rasmus <rasmus@gmx.us>
Date: Sun, 27 Mar 2016 17:33:06 +0200
Subject: [PATCH 1/2] ox-publish: More flexible sitemaps

* lisp/ox-publish.el (org-publish-sitemap-file-entry-format): Support
  more formatters.
(org-publish-sitemap-dir-entry-format): New defcustom.
(org-publish-org-sitemap): Use new variables and functions.
(org-publish-org-sitemap-as-list): New function.
(org-publish--tree-assoc): New function.
(org-pubish--order-files-by-dir-tree): New function.
(org-publish-find-title): New function.
(org-publish-find-subtitle): New function.
(org-publish-org-sitemap-as-tree): New function.
(org-publish--find-property): Find arbirary property.
(org-publish-project-alist): Document changes.
* doc/org.texi (Sitemap): Update documentation.
---
 doc/org.texi       |  20 ++--
 lisp/ox-publish.el | 319 ++++++++++++++++++++++++++++++++++++++---------------
 2 files changed, 241 insertions(+), 98 deletions(-)

diff --git a/doc/org.texi b/doc/org.texi
index 025baaa..b3517c0 100644
--- a/doc/org.texi
+++ b/doc/org.texi
@@ -14570,8 +14570,9 @@ becomes @file{sitemap.html}).
 
 @item @code{:sitemap-function}
 @tab Plug-in function to use for generation of the sitemap.
-Defaults to @code{org-publish-org-sitemap}, which generates a plain list
-of links to all files in the project.
+Defaults to @code{org-publish-org-sitemap}, which generates a plain list of
+links to all files in the project.  See further details in
+@code{org-publish-project-alist}.
 
 @item @code{:sitemap-sort-folders}
 @tab Where folders should appear in the sitemap.  Set this to @code{first}
@@ -14590,12 +14591,9 @@ a file is retrieved with @code{org-publish-find-date}.
 @tab Should sorting be case-sensitive?  Default @code{nil}.
 
 @item @code{:sitemap-file-entry-format}
-@tab With this option one can tell how a sitemap's entry is formatted in the
-sitemap.  This is a format string with some escape sequences: @code{%t} stands
-for the title of the file, @code{%a} stands for the author of the file and
-@code{%d} stands for the date of the file.  The date is retrieved with the
-@code{org-publish-find-date} function and formatted with
-@code{org-publish-sitemap-date-format}.  Default @code{%t}.
+@item @code{:sitemap-dir-entry-format}
+@tab With this option one can tell how the entries of the sitemap is
+formatted.  See @code{org-publish-sitemap-file-entry-format} for details.
 
 @item @code{:sitemap-date-format}
 @tab Format string for the @code{format-time-string} function that tells how
@@ -14607,6 +14605,12 @@ a sitemap entry's date is to be formatted.  This property bypasses
 Useful to have cool URIs (see @uref{http://www.w3.org/Provider/Style/URI}).
 Defaults to @code{nil}.
 
+@item @code{:sitemap-preamble}
+@item @code{:sitemap-postamble}
+@tab Preamble and postamble for sitemap.  Useful for inserting
+    @code{#+OPTIONS}, footers etc.  See @code{org-publish-sitemap-preamble}
+    for details.
+
 @end multitable
 
 @node Generating an index
diff --git a/lisp/ox-publish.el b/lisp/ox-publish.el
index 8ccba99..b791e9a 100644
--- a/lisp/ox-publish.el
+++ b/lisp/ox-publish.el
@@ -41,6 +41,8 @@
 (require 'cl-lib)
 (require 'format-spec)
 (require 'ox)
+(autoload 'message-flatten-list "message")
+(autoload 'dired-tree-lessp "dired-aux")
 
 
 \f
@@ -217,10 +219,15 @@ a site-map of files or summary page for a given project.
 
   `:sitemap-style'
 
-    Can be `list' (site-map is just an itemized list of the
-    titles of the files involved) or `tree' (the directory
-    structure of the source files is reflected in the site-map).
-    Defaults to `tree'.
+    By default `list' (site-map is a list of files) or
+    `tree' (the directory structure of the source files is
+    reflected in the site-map).  Defaults to `tree'.  Files are
+    formatted according to `:sitemap-file-entry-format',
+    directories according to `:sitemap-dir-entry-format'.  To add
+    new styles STYLE define a new function
+    `org-publish-org-sitemap-as-STYLE' that takes a list of files
+    and project-plist as arguments (assuming `:sitemap-function'
+    is `org-publish-org-sitemap').
 
   `:sitemap-sans-extension'
 
@@ -228,6 +235,20 @@ a site-map of files or summary page for a given project.
     cool URIs (see http://www.w3.org/Provider/Style/URI).
     Defaults to nil.
 
+  `:sitemap-file-entry-format'
+  `:sitemap-dir-entry-format'
+
+    Format of filenames and directories included in the sitemap.
+    See `org-publish-sitemap-file-entry-format' for details.
+
+  `:sitemap-preamble'
+  `:sitemap-postamble'
+
+    Preamble and postamble for sitemap.  Useful for inserting
+    #+OPTIONS: keywords, footers etc.  See
+    `org-publish-sitemap-preamble' for details.
+
+
 If you create a site-map file, adjust the sorting like this:
 
   `:sitemap-sort-folders'
@@ -322,15 +343,64 @@ See `format-time-string' for allowed formatters."
   :group 'org-export-publish
   :type 'string)
 
-(defcustom org-publish-sitemap-file-entry-format "%t"
+(defcustom org-publish-sitemap-file-entry-format "%i [[file:%l][%t]]"
   "Format string for site-map file entry.
-You could use brackets to delimit on what part the link will be.
+
+This format string can contain these elements:
 
 %t is the title.
+%s is the subtitle.
 %a is the author.
-%d is the date formatted using `org-publish-sitemap-date-format'."
+%l is the link.
+%h is a leveled headline relative to the base directory.
+%i is an indented item relative to the base directory.
+%d is the date formatted using `org-publish-sitemap-date-format'.
+%f is the directory or filename relative to the base directory.
+%F is the plain directory or filename.
+
+See also `org-publish-sitemap-dir-entry-format'."
   :group 'org-export-publish
-  :type 'string)
+  :type 'string
+  :version "25.1"
+  :package-version '(Org . "9.0"))
+
+(defcustom org-publish-sitemap-dir-entry-format "%i %f"
+  "Format string for site-map file entry.
+See also `org-publish-sitemap-file-entry-format'."
+  :group 'org-export-publish
+  :type 'string
+  :version "25.1"
+  :package-version '(Org . "9.0"))
+
+(defcustom org-publish-sitemap-preamble nil
+  "Sitemap preamble.
+
+Can be either a string, a list of strings, or a function that
+takes a project-plist as an argument and return a string."
+  :group 'org-export-publish
+  :type '(choice
+	  (const :tag "None" nil)
+	  (string :tag "String")
+	  (repeat :tag "List of strings"
+		  (string :tag "String"))
+	  (function :tag "Function"))
+  :version "25.1"
+  :package-version '(Org . "9.0"))
+
+(defcustom org-publish-sitemap-postamble nil
+  "Sitemap postamble.
+
+Can be either a string, a list of strings, or a function that
+takes a project-plist as an argument and return a string."
+  :group 'org-export-publish
+  :type '(choice
+	  (const :tag "None" nil)
+	  (string :tag "String")
+	  (repeat :tag "List of strings"
+		  (string :tag "String"))
+	  (function :tag "Function"))
+  :version "25.1"
+  :package-version '(Org . "9.0"))
 
 
 \f
@@ -399,6 +469,7 @@ This splices all the components into the list."
 (defvar org-publish-sitemap-requested)
 (defvar org-publish-sitemap-date-format)
 (defvar org-publish-sitemap-file-entry-format)
+(defvar org-publish-sitemap-dir-entry-format)
 (defun org-publish-compare-directory-files (a b)
   "Predicate for `sort', that sorts folders and files for sitemap."
   (let ((retval t))
@@ -690,7 +761,16 @@ If `:auto-sitemap' is set, publish the sitemap too.  If
 		   org-publish-sitemap-date-format))
 	      (org-publish-sitemap-file-entry-format
 	       (or (plist-get project-plist :sitemap-file-entry-format)
-		   org-publish-sitemap-file-entry-format)))
+		   org-publish-sitemap-file-entry-format))
+	      (org-publish-sitemap-dir-entry-format
+	       (or (plist-get project-plist :sitemap-dir-entry-format)
+		   org-publish-sitemap-dir-entry-format))
+	      (org-publish-sitemap-preamble
+	       (or (plist-get project-plist :sitemap-preamble)
+		   org-publish-sitemap-preamble))
+	      (org-publish-sitemap-postamble
+	       (or (plist-get project-plist :sitemap-postamble)
+		   org-publish-sitemap-postamble)))
 	  (funcall sitemap-function project sitemap-filename)))
       ;; Publish all files from PROJECT excepted "theindex.org".  Its
       ;; publishing will be deferred until "theindex.inc" is
@@ -715,112 +795,171 @@ If `:auto-sitemap' is set, publish the sitemap too.  If
 (defun org-publish-org-sitemap (project &optional sitemap-filename)
   "Create a sitemap of pages in set defined by PROJECT.
 Optionally set the filename of the sitemap with SITEMAP-FILENAME.
+
 Default for SITEMAP-FILENAME is `sitemap.org'."
   (let* ((project-plist (cdr project))
 	 (dir (file-name-as-directory
 	       (plist-get project-plist :base-directory)))
-	 (localdir (file-name-directory dir))
-	 (indent-str (make-string 2 ?\ ))
-	 (exclude-regexp (plist-get project-plist :exclude))
-	 (files (nreverse
-		 (org-publish-get-base-files project exclude-regexp)))
 	 (sitemap-filename (concat dir (or sitemap-filename "sitemap.org")))
+	 (files (nreverse
+		 ;; Sitemap shouldn't list itself.
+		 (cl-delete-if (lambda (f)
+				 (equal (file-truename f)
+					(file-truename sitemap-filename)))
+			       (org-publish-get-base-files
+				project
+				(plist-get project-plist :exclude)))))
 	 (sitemap-title (or (plist-get project-plist :sitemap-title)
-			  (concat "Sitemap for project " (car project))))
-	 (sitemap-style (or (plist-get project-plist :sitemap-style)
-			    'tree))
-	 (sitemap-sans-extension
-	  (plist-get project-plist :sitemap-sans-extension))
+			    (concat "Sitemap for project " (car project))))
 	 (visiting (find-buffer-visiting sitemap-filename))
-	 file sitemap-buffer)
-    (with-current-buffer
-	(let ((org-inhibit-startup t))
-	  (setq sitemap-buffer
-		(or visiting (find-file sitemap-filename))))
+	 (sitemap-buffer (or visiting (find-file sitemap-filename)))
+	 (insert-pre-or-postamble (function (lambda (pre-or-postamble)
+					      (when pre-or-postamble
+						(cond ((stringp pre-or-postamble) pre-or-postamble)
+						      ((listp pre-or-postamble)
+						       (mapconcat 'identity preamble "\n"))
+						      ((functionp pre-or-postamble)
+						       (funcall pre-or-postamble project-plist))
+						      (t (error (concat "unknown `:sitemap-preamble' or "
+									"`:sitemap-postamble' format")))))))))
+    (with-current-buffer (let ((org-inhibit-startup t)) sitemap-buffer)
       (erase-buffer)
       (insert (concat "#+TITLE: " sitemap-title "\n\n"))
-      (while (setq file (pop files))
-	(let ((link (file-relative-name file dir))
-	      (oldlocal localdir))
-	  (when sitemap-sans-extension
-	    (setq link (file-name-sans-extension link)))
-	  ;; sitemap shouldn't list itself
-	  (unless (equal (file-truename sitemap-filename)
-			 (file-truename file))
-	    (if (eq sitemap-style 'list)
-		(message "Generating list-style sitemap for %s" sitemap-title)
-	      (message "Generating tree-style sitemap for %s" sitemap-title)
-	      (setq localdir (concat (file-name-as-directory dir)
-				     (file-name-directory link)))
-	      (unless (string= localdir oldlocal)
-		(if (string= localdir dir)
-		    (setq indent-str (make-string 2 ?\ ))
-		  (let ((subdirs
-			 (split-string
-			  (directory-file-name
-			   (file-name-directory
-			    (file-relative-name localdir dir))) "/"))
-			(subdir "")
-			(old-subdirs (split-string
-				      (file-relative-name oldlocal dir) "/")))
-		    (setq indent-str (make-string 2 ?\ ))
-		    (while (string= (car old-subdirs) (car subdirs))
-		      (setq indent-str (concat indent-str (make-string 2 ?\ )))
-		      (pop old-subdirs)
-		      (pop subdirs))
-		    (dolist (d subdirs)
-		      (setq subdir (concat subdir d "/"))
-		      (insert (concat indent-str " + " d "\n"))
-		      (setq indent-str (make-string
-					(+ (length indent-str) 2) ?\ )))))))
-	    ;; This is common to 'flat and 'tree
-	    (let ((entry
-		   (org-publish-format-file-entry
-		    org-publish-sitemap-file-entry-format file project-plist))
-		  (regexp "\\(.*\\)\\[\\([^][]+\\)\\]\\(.*\\)"))
-	      (cond ((string-match-p regexp entry)
-		     (string-match regexp entry)
-		     (insert (concat indent-str " + " (match-string 1 entry)
-				     "[[file:" link "]["
-				     (match-string 2 entry)
-				     "]]" (match-string 3 entry) "\n")))
-		    (t
-		     (insert (concat indent-str " + [[file:" link "]["
-				     entry
-				     "]]\n"))))))))
+      ;; Insert sitemap-preamble.
+      (funcall insert-pre-or-postamble
+	       (plist-get project-plist :sitemap-preamble))
+      ;; Call function to build sitemap based on files and the project-plist.
+      (insert (funcall (intern
+			(concat "org-publish-org-sitemap-as-"
+				(symbol-name (or (plist-get project-plist :sitemap-style) 'tree))))
+		       files project-plist))
+      ;; Insert sitemap-postamble.
+      (funcall insert-pre-or-postamble
+	       (plist-get project-plist :sitemap-postamble))
       (save-buffer))
     (or visiting (kill-buffer sitemap-buffer))))
 
-(defun org-publish-format-file-entry (fmt file project-plist)
-  (format-spec
-   fmt
-   `((?t . ,(org-publish-find-title file t))
-     (?d . ,(format-time-string org-publish-sitemap-date-format
-				(org-publish-find-date file)))
-     (?a . ,(or (plist-get project-plist :author) user-full-name)))))
+(defun org-publish-org-sitemap-as-list (files project-plist)
+  "Insert FILES as simple list separated by newlines.
+PROJECT-PLIST holds the project information."
+  (mapconcat
+   (lambda (file) (org-publish-format-file-entry
+	      org-publish-sitemap-file-entry-format
+	      file project-plist))
+   files "\n"))
+
+(defun org-publish--dir-parent (dir)
+  "Return directory parent of DIR"
+  (let ((dir (file-name-directory dir)))
+    (substring dir 0 (string-match-p "[^/]+/?\\'" dir))))
+
+(defun org-publish--tree-assoc (key tree)
+  "Traverse TREE to find list for which the car is `equal' to KEY."
+  (and (consp tree)
+       (cl-destructuring-bind (tree-car . tree-cdr) tree
+	 (if (equal tree-car key) tree
+	   (or (org-publish--tree-assoc key tree-car)
+	       (org-publish--tree-assoc key tree-cdr))))))
+
+(defun org-pubish--order-files-by-dir-tree (files)
+  "Order FILES according to the file tree."
+  (let* ((dirs (sort
+		(delq nil (delete-dups (mapcar 'file-name-directory files)))
+		'dired-tree-lessp))
+         (file-list (list (pop dirs))))
+    (dolist (dir dirs)
+      (or (nconc (org-publish--tree-assoc
+		  (org-publish--dir-parent dir)
+		  file-list)
+		 (list (list dir)))
+	  (nconc file-list dir)))
+    (dolist (file files)
+      (nconc (org-publish--tree-assoc
+	      (file-name-directory file) file-list)
+             (list file)))
+    (message-flatten-list file-list)))
+
+(defun org-publish-org-sitemap-as-tree (files project-plist)
+  "Insert FILES as a tree.
+PROJECT-PLIST holds the project information."
+  (mapconcat (lambda (elm)
+	       (org-publish-format-file-entry
+		(cond
+		 ((directory-name-p elm) org-publish-sitemap-dir-entry-format)
+		 (t org-publish-sitemap-file-entry-format))
+		elm project-plist))
+	     (org-pubish--order-files-by-dir-tree files)
+	     "\n"))
 
-(defun org-publish-find-title (file &optional reset)
-  "Find the title of FILE in project."
+(defun org-publish-format-file-entry (fmt file project-plist)
+  "Format FILE according to the format-string FMT.
+PROJECT-PLIST is a plist holding project options.
+See also `org-publish-sitemap-file-entry-format'.
+"
+  (let ((basedir (file-truename (plist-get project-plist :base-directory))))
+    (when (and (file-exists-p file)
+	       (not (equal file basedir)))
+      (let* ((filename (file-relative-name file basedir))
+	     (dirname (file-name-directory filename))
+	     (depth (if (or (eq 'list (plist-get project-plist :sitemap-style))
+			    (not dirname))
+			1
+		      (+ (if (not (directory-name-p filename)) 1 0)
+			 (length (split-string (file-name-directory filename) "/" t)))))
+	     (link (funcall (if (plist-get project-plist :sitemap-sans-extension)
+				'file-name-sans-extension
+			      'identity)
+			    filename)))
+	(format-spec
+	 fmt
+	 `((?t . ,(and (not (directory-name-p file)) (org-publish-find-title file t)))
+	   (?s . ,(and (not (directory-name-p file)) (org-publish-find-subtitle file t)))
+	   (?f . ,filename)
+	   (?F . ,(directory-file-name
+		    (if (directory-name-p filename)
+			(file-relative-name
+			 dirname (org-publish--dir-parent dirname))
+		      (file-relative-name filename dirname))))
+	   (?l . ,link)
+	   (?h . ,(concat (make-string depth ?*)))
+	   (?i . ,(concat (make-string (* 2 depth) ? ) "-"))
+	   (?d . ,(and (not (directory-name-p file))
+		       (format-time-string
+			(or (plist-get project-plist :sitemap-date-format)
+			    org-publish-sitemap-date-format)
+			(org-publish-find-date file))))
+	   (?a . ,(or (plist-get project-plist :author) user-full-name))))))))
+
+(defun org-publish--find-property (file property &optional reset)
+  "Find the PROPERTY of FILE in project"
   (or
-   (and (not reset) (org-publish-cache-get-file-property file :title nil t))
+   (and (not reset) (org-publish-cache-get-file-property file property nil t))
    (let* ((org-inhibit-startup t)
 	  (visiting (find-buffer-visiting file))
 	  (buffer (or visiting (find-file-noselect file))))
      (with-current-buffer buffer
-       (let ((title
-	      (let ((property
+       (let ((value
+	      (let ((found-property
 		     (plist-get
 		      ;; protect local variables in open buffers
 		      (if visiting
 			  (org-export-with-buffer-copy (org-export-get-environment))
 			(org-export-get-environment))
-		      :title)))
-		(if property
-		    (org-no-properties (org-element-interpret-data property))
+		      property)))
+		(if found-property
+		    (org-no-properties (org-element-interpret-data found-property))
 		  (file-name-nondirectory (file-name-sans-extension file))))))
 	 (unless visiting (kill-buffer buffer))
-	 (org-publish-cache-set-file-property file :title title)
-	 title)))))
+	 (org-publish-cache-set-file-property file property value)
+	 value)))))
+
+(defun org-publish-find-title (file &optional reset)
+  "Find the title of FILE in project."
+  (org-publish--find-property file :title reset))
+
+(defun org-publish-find-subtitle (file &optional reset)
+  "Find the title of FILE in project."
+  (org-publish--find-property file :subtitle reset))
 
 (defun org-publish-find-date (file)
   "Find the date of FILE in project.
-- 
2.8.2


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [ox-publish, patch] More flexible sitemaps
  2016-05-19 15:39 [ox-publish, patch] More flexible sitemaps Rasmus
@ 2016-05-22 22:58 ` Nicolas Goaziou
  2016-05-27 16:41   ` Rasmus
  0 siblings, 1 reply; 8+ messages in thread
From: Nicolas Goaziou @ 2016-05-22 22:58 UTC (permalink / raw)
  To: Rasmus; +Cc: emacs-orgmode

Hello,

Rasmus <rasmus@gmx.us> writes:

> I've long wanted to use ox to auto-generate something that looks like a
> blog index.
>
> This patch makes ox sitemaps a bit more flexible.  For instance, it would
> allow me to use something like this for ‘:sitemap-file-entry-format’,
>
>     :sitemap-file-entry-format "* [[file:%l][%t]]
>     #+include: \"%f::lead\"
>
>     [[file:%l][Read more]]"
>
> Which would come out as;
>
>     * [[file:link][Title]]
>     #+Include: "file.org::lead"
>
>     [[File:link][Read more]]
>
> For the tests I did, it matches the "old" sitemap for list and tree.
>
> WDYT?

It sounds interesting. Usual nitpicking follows.

> * lisp/ox-publish.el (org-publish-sitemap-file-entry-format): Support
>   more formatters.
> (org-publish-sitemap-dir-entry-format): New defcustom.
> (org-publish-org-sitemap): Use new variables and functions.
> (org-publish-org-sitemap-as-list): New function.
> (org-publish--tree-assoc): New function.
> (org-pubish--order-files-by-dir-tree): New function.
> (org-publish-find-title): New function.

This is not a new function.

> (org-publish-find-subtitle): New function.
> (org-publish-org-sitemap-as-tree): New function.
> (org-publish--find-property): Find arbirary property.
> (org-publish-project-alist): Document changes.
> * doc/org.texi (Sitemap): Update documentation.

All in all, I think this deserves to be split into 3 patches: one for
the preamble-postamble feature, another one for implementing
`org-publish--find-property' and associated refactoring, and the latter
for the sitemap itself.

> +(autoload 'message-flatten-list "message")
> +(autoload 'dired-tree-lessp "dired-aux")

I hope we can avoid these. In particular, why are you using
`dired-tree-lessp' instead of `org-publish-compare-directory-files'?

> @@ -399,6 +469,7 @@ This splices all the components into the list."
>  (defvar org-publish-sitemap-requested)
>  (defvar org-publish-sitemap-date-format)
>  (defvar org-publish-sitemap-file-entry-format)
> +(defvar org-publish-sitemap-dir-entry-format)

The above is not necessary.

> +	 (files (nreverse
> +		 ;; Sitemap shouldn't list itself.
> +		 (cl-delete-if (lambda (f)
> +				 (equal (file-truename f)
> +					(file-truename sitemap-filename)))

See `file-equal-p'.

> +			       (org-publish-get-base-files
> +				project
> +				(plist-get project-plist :exclude)))))
>  	 (sitemap-title (or (plist-get project-plist :sitemap-title)
> -			  (concat "Sitemap for project " (car project))))
> -	 (sitemap-style (or (plist-get project-plist :sitemap-style)
> -			    'tree))
> -	 (sitemap-sans-extension
> -	  (plist-get project-plist :sitemap-sans-extension))
> +			    (concat "Sitemap for project " (car project))))
>  	 (visiting (find-buffer-visiting sitemap-filename))
> -	 file sitemap-buffer)
> -    (with-current-buffer
> -	(let ((org-inhibit-startup t))
> -	  (setq sitemap-buffer
> -		(or visiting (find-file sitemap-filename))))
> +	 (sitemap-buffer (or visiting (find-file sitemap-filename)))
> +	 (insert-pre-or-postamble (function (lambda (pre-or-postamble)

No need to wrap `function' around `lambda'.  Also, it doesn't "insert"
anything, does it? IOW, isn't the name a bit confusing ?

> +					      (when pre-or-postamble

You can include the `when' in the cond:

  (cond ((not pre-or-postamble) nil)
        ((stringp pre-or-postamble) ...)
        ...)
> +						(cond ((stringp pre-or-postamble) pre-or-postamble)
> +						      ((listp pre-or-postamble)
> +						       (mapconcat 'identity preamble "\n"))
> +						      ((functionp pre-or-postamble)
> +						       (funcall pre-or-postamble project-plist))
> +						      (t (error (concat "unknown `:sitemap-preamble' or "
> +									"`:sitemap-postamble' format")))))))))
> +    (with-current-buffer (let ((org-inhibit-startup t)) sitemap-buffer)

You can drop the `let' part, which is a no-op here.

> +      ;; Insert sitemap-preamble.
> +      (funcall insert-pre-or-postamble
> +	       (plist-get project-plist :sitemap-preamble))
> +      ;; Call function to build sitemap based on files and the project-plist.
> +      (insert (funcall (intern
> +			(concat "org-publish-org-sitemap-as-"
> +				(symbol-name (or (plist-get project-plist :sitemap-style) 'tree))))
> +		       files project-plist))

(intern (format "org-publish-org-sitemap-as-%s" (or (plist-get ...) 'tree))

You may want to check that it does exist before and raise an error
otherwise.

> +      ;; Insert sitemap-postamble.
> +      (funcall insert-pre-or-postamble
> +	       (plist-get project-plist :sitemap-postamble))
>        (save-buffer))

Not directly related, but `save-buffer' here is suspicious. It may be
better to use `with-temp-file' instead.

> +(defun org-publish-org-sitemap-as-list (files project-plist)
> +  "Insert FILES as simple list separated by newlines.
> +PROJECT-PLIST holds the project information."
> +  (mapconcat
> +   (lambda (file) (org-publish-format-file-entry
> +	      org-publish-sitemap-file-entry-format
> +	      file project-plist))
> +   files "\n"))

Mind indentation

  (lambda (file)
    (org-publish-format-file-entry ...))

> +(defun org-publish--dir-parent (dir)
> +  "Return directory parent of DIR"
> +  (let ((dir (file-name-directory dir)))
> +    (substring dir 0 (string-match-p "[^/]+/?\\'" dir))))

  (file-name-directory (directory-file-name dir))

> +(defun org-publish--tree-assoc (key tree)
> +  "Traverse TREE to find list for which the car is `equal' to KEY."
> +  (and (consp tree)
> +       (cl-destructuring-bind (tree-car . tree-cdr) tree
> +	 (if (equal tree-car key) tree
> +	   (or (org-publish--tree-assoc key tree-car)
> +	       (org-publish--tree-assoc key tree-cdr))))))
> +
> +(defun org-pubish--order-files-by-dir-tree (files)
> +  "Order FILES according to the file tree."
> +  (let* ((dirs (sort
> +		(delq nil (delete-dups (mapcar 'file-name-directory files)))
> +		'dired-tree-lessp))
> +         (file-list (list (pop dirs))))
> +    (dolist (dir dirs)
> +      (or (nconc (org-publish--tree-assoc
> +		  (org-publish--dir-parent dir)
> +		  file-list)
> +		 (list (list dir)))
> +	  (nconc file-list dir)))
> +    (dolist (file files)
> +      (nconc (org-publish--tree-assoc
> +	      (file-name-directory file) file-list)
> +             (list file)))
> +    (message-flatten-list file-list)))

I don't understand why you need the 2 functions above. You are working
with plain lists, not nested ones. Besides, once the file name are
standardized, isn't tree order equivalent to lexicographic one?

> +  (let ((basedir (file-truename (plist-get project-plist :base-directory))))
> +    (when (and (file-exists-p file)
> +	       (not (equal file basedir)))

`file-equal-p'

> +      (let* ((filename (file-relative-name file basedir))
> +	     (dirname (file-name-directory filename))
> +	     (depth (if (or (eq 'list (plist-get project-plist :sitemap-style))
> +			    (not dirname))
> +			1
> +		      (+ (if (not (directory-name-p filename)) 1 0)
> +			 (length (split-string (file-name-directory filename) "/" t)))))
> +	     (link (funcall (if (plist-get project-plist :sitemap-sans-extension)
> +				'file-name-sans-extension
> +			      'identity)

#'file-name-sans-extension and #'identity

> +	(format-spec
> +	 fmt
> +	 `((?t . ,(and (not (directory-name-p file)) (org-publish-find-title file t)))
> +	   (?s . ,(and (not (directory-name-p file)) (org-publish-find-subtitle file t)))
> +	   (?f . ,filename)
> +	   (?F . ,(directory-file-name
> +		    (if (directory-name-p filename)
> +			(file-relative-name
> +			 dirname (org-publish--dir-parent dirname))
> +		      (file-relative-name filename dirname))))
> +	   (?l . ,link)
> +	   (?h . ,(concat (make-string depth ?*)))
> +	   (?i . ,(concat (make-string (* 2 depth) ? ) "-"))

  (make-string (* 2 depth) ?\s)
  
> +	   (?d . ,(and (not (directory-name-p file))
> +		       (format-time-string
> +			(or (plist-get project-plist :sitemap-date-format)
> +			    org-publish-sitemap-date-format)
> +			(org-publish-find-date file))))
> +	   (?a . ,(or (plist-get project-plist :author) user-full-name))))))))
> +

> +(defun org-publish-find-subtitle (file &optional reset)
> +  "Find the title of FILE in project."
> +  (org-publish--find-property file :subtitle reset))

I don't think this would work. :subtitle is not defined in default
export properties, it is back-end specific. `org-export-get-environment'
without any argument, doesn't catch these. You need to somehow provide
it the back-end.

Regards,

-- 
Nicolas Goaziou

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ox-publish, patch] More flexible sitemaps
  2016-05-22 22:58 ` Nicolas Goaziou
@ 2016-05-27 16:41   ` Rasmus
  2016-06-01 15:34     ` Nicolas Goaziou
  0 siblings, 1 reply; 8+ messages in thread
From: Rasmus @ 2016-05-27 16:41 UTC (permalink / raw)
  To: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 3010 bytes --]

Hi,

Thanks for the comments.

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

>> (org-publish-find-subtitle): New function.
>> (org-publish-org-sitemap-as-tree): New function.
>> (org-publish--find-property): Find arbirary property.
>> (org-publish-project-alist): Document changes.
>> * doc/org.texi (Sitemap): Update documentation.
>
> All in all, I think this deserves to be split into 3 patches: one for
> the preamble-postamble feature, another one for implementing
> `org-publish--find-property' and associated refactoring, and the latter
> for the sitemap itself.

This was by far the hardest part...

>> +(autoload 'message-flatten-list "message")
>> +(autoload 'dired-tree-lessp "dired-aux")
>
> I hope we can avoid these. In particular, why are you using
> `dired-tree-lessp' instead of `org-publish-compare-directory-files'?


AFAIK, org-publish-compare-directory-files can’t be used on its own.  It
requires variables to be bound around it,
e.g. org-publish-sitemap-ignore-case.  See org-publish-get-base-files.
(I don’t know why the project plist isn’t being passed around.)

Anyway, I’m not using neither of these files anymore.

Aside: the way ox-publish order files is not super, IMO, as it takes
directory into account even if one use a flat list and chronological
ordering.  That’s a problem for another day, though.

>> +(defun org-publish--tree-assoc (key tree)
>> +(defun org-pubish--order-files-by-dir-tree (files)
>
> I don't understand why you need the 2 functions above. You are working
> with plain lists, not nested ones. Besides, once the file name are
> standardized, isn't tree order equivalent to lexicographic one?

Removed.

>> +(defun org-publish-find-subtitle (file &optional reset)
>> +  "Find the title of FILE in project."
>> +  (org-publish--find-property file :subtitle reset))
>
> I don't think this would work. :subtitle is not defined in default
> export properties, it is back-end specific. `org-export-get-environment'
> without any argument, doesn't catch these. You need to somehow provide
> it the back-end.

You are right.  The "problem" is that the sitemap is done in a before the
backend is known.  So we can’t do (org-export-get-environment BACKEND).

OTOH, only allowing ox.el keywords is too limiting IMO.  E.g. a blog
sitemap wouldn’t be able to include keywords, though a desirable format
for each entry is

    TITLE
    DATE KEYWORDS
    #+INCLUDE: "file.org::lead"

But KEYWORDS is specific to ox-html...

A couple of workarounds:

1. Guess the backend(s) from the name of :publishing-function.  This is
   not very robust...
2. Find the keyword by search in the file, thus sidestepping
   org-export-get-environment.  It is is not without problems.  E.g. if my
   file contains this, I’d probably only key1.
      #+Keywords: key1
      #+Keywords: key2
3. Go through backend for org-export-get-environment until something is
   Found.  I do this now.

Rasmus

-- 
This is the kind of tedious nonsense up with which I will not put

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ox-publish-Tiny-refactor.patch --]
[-- Type: text/x-diff, Size: 1880 bytes --]

From 4e18d9141c509b03cc5579517547f6e7e73088f3 Mon Sep 17 00:00:00 2001
From: Rasmus <rasmus@gmx.us>
Date: Fri, 27 May 2016 16:46:14 +0200
Subject: [PATCH 1/6] ox-publish: Tiny refactor

* lisp/ox-publish.el: (org-publish-compare-directory-files):
(org-publish-get-base-files-1): Change test.
* lisp/ox-publish.el (org-publish-initialize-cache): Tiny fix.
---
 lisp/ox-publish.el | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lisp/ox-publish.el b/lisp/ox-publish.el
index 8ccba99..d60562b 100644
--- a/lisp/ox-publish.el
+++ b/lisp/ox-publish.el
@@ -467,7 +467,7 @@ matching the regexp SKIP-DIR when recursing through BASE-DIR."
 		      (and skip-file (string-match skip-file fnd))
 		      (not (file-exists-p (file-truename f)))
 		      (not (string-match match fnd)))
-	    (cl-pushnew f org-publish-temp-files)))))))
+	    (cl-pushnew f org-publish-temp-files :test #'file-equal-p)))))))
 
 (defun org-publish-get-base-files (project &optional exclude-regexp)
   "Return a list of all files in PROJECT.
@@ -511,7 +511,7 @@ matching filenames."
     (setq org-publish-temp-files nil)
     (when org-publish-sitemap-requested
       (cl-pushnew (expand-file-name (concat base-dir sitemap-filename))
-		  org-publish-temp-files))
+		  org-publish-temp-files :test #'file-equal-p))
     (org-publish-get-base-files-1 base-dir recurse match
 				  ;; FIXME distinguish exclude regexp
 				  ;; for skip-file and skip-dir?
@@ -1172,7 +1172,7 @@ If FREE-CACHE, empty the cache."
 
       (if cexists (load-file cache-file)
 	(setq org-publish-cache
-	      (make-hash-table :test 'equal :weakness nil :size 100))
+	      (make-hash-table :test #'equal :weakness nil :size 100))
 	(org-publish-cache-set ":project:" project-name)
 	(org-publish-cache-set ":cache-file:" cache-file))
       (unless cexists (org-publish-write-cache-file nil))))
-- 
2.8.3


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-ox-publish-Get-property-from-file.patch --]
[-- Type: text/x-diff, Size: 4152 bytes --]

From 410dc552de2e846f7eed9f98275acffba45f5be0 Mon Sep 17 00:00:00 2001
From: Rasmus <rasmus@gmx.us>
Date: Fri, 27 May 2016 15:46:31 +0200
Subject: [PATCH 2/6] ox-publish: Get property from file

* lisp/ox-publish.el (org-publish-find-property): New function.
(org-publish-find-title): Use new function.
(org-publish-find-date): Use new function.
---
 lisp/ox-publish.el | 62 +++++++++++++++++++++++++++---------------------------
 1 file changed, 31 insertions(+), 31 deletions(-)

diff --git a/lisp/ox-publish.el b/lisp/ox-publish.el
index d60562b..480c8de 100644
--- a/lisp/ox-publish.el
+++ b/lisp/ox-publish.el
@@ -799,28 +799,38 @@ Default for SITEMAP-FILENAME is `sitemap.org'."
 				(org-publish-find-date file)))
      (?a . ,(or (plist-get project-plist :author) user-full-name)))))
 
+
+(defun org-publish-find-property (file property &optional reset)
+  "Find the PROPERTY of FILE in project.
+PROPERTY can be a string or a symbol-property."
+  (unless (or (file-directory-p file) (directory-name-p file))
+    (let ((prop (cond ((stringp property)
+                       (intern (downcase (format ":%s" property))))
+                      (t property))))
+      (and (not reset) (org-publish-cache-get-file-property file prop nil t))
+      (let* ((org-inhibit-startup t)
+             (visiting (find-buffer-visiting file))
+             (buffer (or visiting (find-file-noselect file)))
+             (case-fold-search t))
+        (with-current-buffer buffer
+          ;; protect local variables in open buffers
+          (org-export-with-buffer-copy
+           (let* ((backends (append (list nil)
+                                    (mapcar 'org-export-backend-name
+                                            org-export-registered-backends)))
+                  (value (cl-loop for backend in backends
+                                  when (org-no-properties
+                                        (org-element-interpret-data
+                                         (plist-get (org-export-get-environment backend)
+                                                    prop)))
+                                  return it)))
+             (unless visiting (kill-buffer buffer))
+             (org-publish-cache-set-file-property file prop value)
+             value)))))))
+
 (defun org-publish-find-title (file &optional reset)
   "Find the title of FILE in project."
-  (or
-   (and (not reset) (org-publish-cache-get-file-property file :title nil t))
-   (let* ((org-inhibit-startup t)
-	  (visiting (find-buffer-visiting file))
-	  (buffer (or visiting (find-file-noselect file))))
-     (with-current-buffer buffer
-       (let ((title
-	      (let ((property
-		     (plist-get
-		      ;; protect local variables in open buffers
-		      (if visiting
-			  (org-export-with-buffer-copy (org-export-get-environment))
-			(org-export-get-environment))
-		      :title)))
-		(if property
-		    (org-no-properties (org-element-interpret-data property))
-		  (file-name-nondirectory (file-name-sans-extension file))))))
-	 (unless visiting (kill-buffer buffer))
-	 (org-publish-cache-set-file-property file :title title)
-	 title)))))
+  (org-publish-find-property file :title reset))
 
 (defun org-publish-find-date (file)
   "Find the date of FILE in project.
@@ -829,17 +839,7 @@ If FILE is an Org file and provides a DATE keyword use it.  In
 any other case use the file system's modification time.  Return
 time in `current-time' format."
   (if (file-directory-p file) (nth 5 (file-attributes file))
-    (let* ((org-inhibit-startup t)
-	   (visiting (find-buffer-visiting file))
-	   (file-buf (or visiting (find-file-noselect file nil)))
-	   (date (plist-get
-		  (with-current-buffer file-buf
-		    (if visiting
-			(org-export-with-buffer-copy
-			 (org-export-get-environment))
-		      (org-export-get-environment)))
-		  :date)))
-      (unless visiting (kill-buffer file-buf))
+    (let* ((date (org-publish-find-property file :date t)))
       ;; DATE is a secondary string.  If it contains a timestamp,
       ;; convert it to internal format.  Otherwise, use FILE
       ;; modification time.
-- 
2.8.3


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0003-ox-publish-Optionally-add-dirs-to-project-files.patch --]
[-- Type: text/x-diff, Size: 4023 bytes --]

From c16e6faf21890f8cb572133d40dc2c3545821a23 Mon Sep 17 00:00:00 2001
From: Rasmus <rasmus@gmx.us>
Date: Fri, 27 May 2016 16:50:28 +0200
Subject: [PATCH 3/6] ox-publish: Optionally add dirs to project files

* lisp/ox-publish.el (org-publish-sitemap-file-entry-format):
(org-publish-compare-directory-files): New optional argument that allows
inclusion of directories in `org-publish-temp-files'.
---
 lisp/ox-publish.el | 30 ++++++++++++++++++++++--------
 1 file changed, 22 insertions(+), 8 deletions(-)

diff --git a/lisp/ox-publish.el b/lisp/ox-publish.el
index 480c8de..8a72349 100644
--- a/lisp/ox-publish.el
+++ b/lisp/ox-publish.el
@@ -439,20 +439,21 @@ This splices all the components into the list."
     retval))
 
 (defun org-publish-get-base-files-1
-    (base-dir &optional recurse match skip-file skip-dir)
+    (base-dir &optional recurse match skip-file skip-dir include-dirs)
   "Set `org-publish-temp-files' with files from BASE-DIR directory.
 If RECURSE is non-nil, check BASE-DIR recursively.  If MATCH is
 non-nil, restrict this list to the files matching the regexp
 MATCH.  If SKIP-FILE is non-nil, skip file matching the regexp
 SKIP-FILE.  If SKIP-DIR is non-nil, don't check directories
-matching the regexp SKIP-DIR when recursing through BASE-DIR."
+matching the regexp SKIP-DIR when recursing through BASE-DIR.
+If INCLUDE-DIRS is non-nil include directories"
   (let ((all-files (if (not recurse) (directory-files base-dir t match)
 		     ;; If RECURSE is non-nil, we want all files
 		     ;; matching MATCH and sub-directories.
 		     (cl-remove-if-not
 		      (lambda (file)
 			(or (file-directory-p file)
-			    (and match (string-match match file))))
+			    (and match (string-match-p match file))))
 		      (directory-files base-dir t)))))
     (dolist (f (if (not org-publish-sitemap-requested) all-files
 		 (sort all-files #'org-publish-compare-directory-files)))
@@ -461,15 +462,18 @@ matching the regexp SKIP-DIR when recursing through BASE-DIR."
 	(if (and fd-p recurse
 		 (not (string-match "^\\.+$" fnd))
 		 (if skip-dir (not (string-match skip-dir fnd)) t))
-	    (org-publish-get-base-files-1
-	     f recurse match skip-file skip-dir)
+	    (progn
+	      (and include-dirs
+		   (cl-pushnew f org-publish-temp-files :test #'file-equal-p))
+	      (org-publish-get-base-files-1
+	       f recurse match skip-file skip-dir include-dirs))
 	  (unless (or fd-p		; This is a directory.
 		      (and skip-file (string-match skip-file fnd))
 		      (not (file-exists-p (file-truename f)))
 		      (not (string-match match fnd)))
 	    (cl-pushnew f org-publish-temp-files :test #'file-equal-p)))))))
 
-(defun org-publish-get-base-files (project &optional exclude-regexp)
+(defun org-publish-get-base-files (project &optional exclude-regexp include-dirs)
   "Return a list of all files in PROJECT.
 If EXCLUDE-REGEXP is set, this will be used to filter out
 matching filenames."
@@ -515,10 +519,20 @@ matching filenames."
     (org-publish-get-base-files-1 base-dir recurse match
 				  ;; FIXME distinguish exclude regexp
 				  ;; for skip-file and skip-dir?
-				  exclude-regexp exclude-regexp)
+				  exclude-regexp exclude-regexp include-dirs)
     (dolist (f include-list org-publish-temp-files)
       (cl-pushnew (expand-file-name (concat base-dir f))
-		  org-publish-temp-files))))
+		  org-publish-temp-files :test #'file-equal-p))
+    ;; Remove directories from `org-publish-temp-files' that do not
+    ;; hold project files.
+    (when include-dirs
+      (let* ((dirs (cl-remove-if-not 'file-directory-p org-publish-temp-files))
+	     (file-dirs (mapcar 'file-name-directory
+				(cl-remove-if 'file-directory-p org-publish-temp-files))))
+	(setq org-publish-temp-files
+	      (cl-set-difference org-publish-temp-files
+				 (cl-set-difference dirs file-dirs :test #'file-equal-p)
+				 :test #'file-equal-p))))))
 
 (defun org-publish-get-project-from-filename (filename &optional up)
   "Return the project that FILENAME belongs to."
-- 
2.8.3


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #5: 0004-ox-publish-More-flexible-sitemap-formatting.patch --]
[-- Type: text/x-diff, Size: 58452 bytes --]

From c8ab9a3bf0f4f4fdf086ea3c085a327cc582cd94 Mon Sep 17 00:00:00 2001
From: Rasmus <rasmus@gmx.us>
Date: Fri, 27 May 2016 16:55:37 +0200
Subject: [PATCH 4/6] ox-publish: More flexible sitemap formatting

* lisp/ox-publish.el (org-publish-project-alist): Document changes.
(org-publish-sitemap-sort-ignore-case): New default value.
(org-publish-sitemap-file-entry-format): New defcustom.
(org-publish-org-sitemap): Refactored to use new functions.
(org-publish-org-sitemap-as-list): New function.
(org-publish-org-sitemap-as-tree): New function.
(org-publish-format-file-entry): Refactored to be more flexible.
---
 lisp/ox-publish.el | 988 +++++++++++++++++++++++++++++------------------------
 1 file changed, 545 insertions(+), 443 deletions(-)

diff --git a/lisp/ox-publish.el b/lisp/ox-publish.el
index 8a72349..d94ca04 100644
--- a/lisp/ox-publish.el
+++ b/lisp/ox-publish.el
@@ -217,10 +217,15 @@ a site-map of files or summary page for a given project.
 
   `:sitemap-style'
 
-    Can be `list' (site-map is just an itemized list of the
-    titles of the files involved) or `tree' (the directory
-    structure of the source files is reflected in the site-map).
-    Defaults to `tree'.
+    By default `list' (site-map is a list of files) or
+    `tree' (the directory structure of the source files is
+    reflected in the site-map).  Defaults to `tree'.  Files are
+    formatted according to `:sitemap-file-entry-format',
+    directories according to `:sitemap-dir-entry-format'.  To add
+    new styles STYLE define a new function
+    `org-publish-org-sitemap-as-STYLE' that takes a list of files
+    and project-plist as arguments (assuming `:sitemap-function'
+    is `org-publish-org-sitemap').
 
   `:sitemap-sans-extension'
 
@@ -228,6 +233,12 @@ a site-map of files or summary page for a given project.
     cool URIs (see http://www.w3.org/Provider/Style/URI).
     Defaults to nil.
 
+  `:sitemap-file-entry-format'
+  `:sitemap-dir-entry-format'
+
+    Format of filenames and directories included in the sitemap.
+    See `org-publish-sitemap-file-entry-format' for details.
+
 If you create a site-map file, adjust the sorting like this:
 
   `:sitemap-sort-folders'
@@ -322,15 +333,82 @@ See `format-time-string' for allowed formatters."
   :group 'org-export-publish
   :type 'string)
 
-(defcustom org-publish-sitemap-file-entry-format "%t"
+(defcustom org-publish-sitemap-file-entry-format "%- [[file:%link][%TITLE]]"
   "Format string for site-map file entry.
-You could use brackets to delimit on what part the link will be.
 
-%t is the title.
-%a is the author.
-%d is the date formatted using `org-publish-sitemap-date-format'."
+Most keywords can be accessed using a the name of the keyword
+prefixed with \"%\", e.g. \"%TITLE\" or \"%KEYWORDS\".  In
+addition, the following keywords are defined.
+
+  %fullpath
+    The full path of the file.
+
+  %relpath
+    The relative path of the file.
+
+  %filename
+  %f
+    The filename.
+
+  %link
+  %l
+    The link.
+
+  %*
+    Leveled headline relative to the base directory.
+
+  %*
+    Leveled item relative to the base directory.
+
+  %author
+    The author of the document, the project, or `user-full-name'.
+
+  %date
+     Date formatted using `org-publish-sitemap-date-format'.
+
+See also `org-publish-sitemap-dir-entry-format'."
   :group 'org-export-publish
-  :type 'string)
+  :type 'string
+  :version "25.2"
+  :package-version '(Org . "9.0"))
+
+(defcustom org-publish-sitemap-dir-entry-format "%- %f"
+  "Format string for site-map file entry.
+
+The following keywords are defined.
+
+  %fullpath
+    The full path of the file.
+
+  %relpath
+    The relative path of the file.
+
+  %filename
+  %f
+    The filename.
+
+  %link
+  %l
+    The link.
+
+  %*
+    Leveled headline relative to the base directory.
+
+  %*
+    Leveled item relative to the base directory.
+
+  %author
+    The author of the document, the project, or `user-full-name'.
+
+  %date
+     Date formatted using `org-publish-sitemap-date-format'.
+
+See also `org-publish-sitemap-file-entry-format'."
+  :group 'org-export-publish
+  :type 'string
+  :version "25.2"
+  :package-version '(Org . "9.0"))
+
 
 
 \f
@@ -340,7 +418,7 @@ You could use brackets to delimit on what part the link will be.
 (defun org-publish-timestamp-filename (filename &optional pub-dir pub-func)
   "Return path to timestamp file for filename FILENAME."
   (setq filename (concat filename "::" (or pub-dir "") "::"
-			 (format "%s" (or pub-func ""))))
+                         (format "%s" (or pub-func ""))))
   (concat "X" (if (fboundp 'sha1) (sha1 filename) (md5 filename))))
 
 (defun org-publish-needed-p
@@ -353,11 +431,11 @@ Right now we cannot do this, because we do not know under what
 file name the file will be stored - the publishing function can
 still decide about that independently."
   (let ((rtn (if (not org-publish-use-timestamps-flag) t
-	       (org-publish-cache-file-needs-publishing
-		filename pub-dir pub-func base-dir))))
+               (org-publish-cache-file-needs-publishing
+                filename pub-dir pub-func base-dir))))
     (if rtn (message "Publishing file %s using `%s'" filename pub-func)
       (when org-publish-list-skipped-files
-	(message   "Skipping unmodified file %s" filename)))
+        (message   "Skipping unmodified file %s" filename)))
     rtn))
 
 (defun org-publish-update-timestamp
@@ -365,7 +443,7 @@ still decide about that independently."
   "Update publishing timestamp for file FILENAME.
 If there is no timestamp, create one."
   (let ((key (org-publish-timestamp-filename filename pub-dir pub-func))
-	(stamp (org-publish-cache-ctime-of-src filename)))
+        (stamp (org-publish-cache-ctime-of-src filename)))
     (org-publish-cache-set key stamp)))
 
 (defun org-publish-remove-all-timestamps ()
@@ -386,11 +464,11 @@ This splices all the components into the list."
   (let ((rest projects-alist) rtn p components)
     (while (setq p (pop rest))
       (if (setq components (plist-get (cdr p) :components))
-	  (setq rest (append
-		      (mapcar (lambda (x) (assoc x org-publish-project-alist))
-			      components)
-		      rest))
-	(push p rtn)))
+          (setq rest (append
+                      (mapcar (lambda (x) (assoc x org-publish-project-alist))
+                              components)
+                      rest))
+        (push p rtn)))
     (nreverse (delete-dups (delq nil rtn)))))
 
 (defvar org-publish-sitemap-sort-files)
@@ -405,35 +483,35 @@ This splices all the components into the list."
     (when (or org-publish-sitemap-sort-files org-publish-sitemap-sort-folders)
       ;; First we sort files:
       (when org-publish-sitemap-sort-files
-	(pcase org-publish-sitemap-sort-files
-	  (`alphabetically
-	   (let* ((adir (file-directory-p a))
-		  (aorg (and (string-match "\\.org$" a) (not adir)))
-		  (bdir (file-directory-p b))
-		  (borg (and (string-match "\\.org$" b) (not bdir)))
-		  (A (if aorg (concat (file-name-directory a)
-				      (org-publish-find-title a)) a))
-		  (B (if borg (concat (file-name-directory b)
-				      (org-publish-find-title b)) b)))
-	     (setq retval (if org-publish-sitemap-ignore-case
-			      (not (string-lessp (upcase B) (upcase A)))
-			    (not (string-lessp B A))))))
-	  ((or `anti-chronologically `chronologically)
-	   (let* ((adate (org-publish-find-date a))
-		  (bdate (org-publish-find-date b))
-		  (A (+ (lsh (car adate) 16) (cadr adate)))
-		  (B (+ (lsh (car bdate) 16) (cadr bdate))))
-	     (setq retval
-		   (if (eq org-publish-sitemap-sort-files 'chronologically)
-		       (<= A B)
-		     (>= A B)))))))
+        (pcase org-publish-sitemap-sort-files
+          (`alphabetically
+           (let* ((adir (file-directory-p a))
+                  (aorg (and (string-match "\\.org$" a) (not adir)))
+                  (bdir (file-directory-p b))
+                  (borg (and (string-match "\\.org$" b) (not bdir)))
+                  (A (if aorg (concat (file-name-directory a)
+                                      (org-publish-find-title a)) a))
+                  (B (if borg (concat (file-name-directory b)
+                                      (org-publish-find-title b)) b)))
+             (setq retval (if org-publish-sitemap-ignore-case
+                              (not (string-lessp (upcase B) (upcase A)))
+                            (not (string-lessp B A))))))
+          ((or `anti-chronologically `chronologically)
+           (let* ((adate (org-publish-find-date a))
+                  (bdate (org-publish-find-date b))
+                  (A (+ (lsh (car adate) 16) (cadr adate)))
+                  (B (+ (lsh (car bdate) 16) (cadr bdate))))
+             (setq retval
+                   (if (eq org-publish-sitemap-sort-files 'chronologically)
+                       (<= A B)
+                     (>= A B)))))))
       ;; Directory-wise wins:
       (when org-publish-sitemap-sort-folders
         ;; a is directory, b not:
         (cond
          ((and (file-directory-p a) (not (file-directory-p b)))
           (setq retval (equal org-publish-sitemap-sort-folders 'first)))
-	 ;; a is not a directory, but b is:
+         ;; a is not a directory, but b is:
          ((and (not (file-directory-p a)) (file-directory-p b))
           (setq retval (equal org-publish-sitemap-sort-folders 'last))))))
     retval))
@@ -448,65 +526,65 @@ SKIP-FILE.  If SKIP-DIR is non-nil, don't check directories
 matching the regexp SKIP-DIR when recursing through BASE-DIR.
 If INCLUDE-DIRS is non-nil include directories"
   (let ((all-files (if (not recurse) (directory-files base-dir t match)
-		     ;; If RECURSE is non-nil, we want all files
-		     ;; matching MATCH and sub-directories.
-		     (cl-remove-if-not
-		      (lambda (file)
-			(or (file-directory-p file)
-			    (and match (string-match-p match file))))
-		      (directory-files base-dir t)))))
+                     ;; If RECURSE is non-nil, we want all files
+                     ;; matching MATCH and sub-directories.
+                     (cl-remove-if-not
+                      (lambda (file)
+                        (or (file-directory-p file)
+                            (and match (string-match-p match file))))
+                      (directory-files base-dir t)))))
     (dolist (f (if (not org-publish-sitemap-requested) all-files
-		 (sort all-files #'org-publish-compare-directory-files)))
+                 (sort all-files #'org-publish-compare-directory-files)))
       (let ((fd-p (file-directory-p f))
-	    (fnd (file-name-nondirectory f)))
-	(if (and fd-p recurse
-		 (not (string-match "^\\.+$" fnd))
-		 (if skip-dir (not (string-match skip-dir fnd)) t))
-	    (progn
-	      (and include-dirs
-		   (cl-pushnew f org-publish-temp-files :test #'file-equal-p))
-	      (org-publish-get-base-files-1
-	       f recurse match skip-file skip-dir include-dirs))
-	  (unless (or fd-p		; This is a directory.
-		      (and skip-file (string-match skip-file fnd))
-		      (not (file-exists-p (file-truename f)))
-		      (not (string-match match fnd)))
-	    (cl-pushnew f org-publish-temp-files :test #'file-equal-p)))))))
+            (fnd (file-name-nondirectory f)))
+        (if (and fd-p recurse
+                 (not (string-match "^\\.+$" fnd))
+                 (if skip-dir (not (string-match skip-dir fnd)) t))
+            (progn
+              (and include-dirs
+                   (cl-pushnew f org-publish-temp-files :test #'file-equal-p))
+              (org-publish-get-base-files-1
+               f recurse match skip-file skip-dir include-dirs))
+          (unless (or fd-p		; This is a directory.
+                      (and skip-file (string-match skip-file fnd))
+                      (not (file-exists-p (file-truename f)))
+                      (not (string-match match fnd)))
+            (cl-pushnew f org-publish-temp-files :test #'file-equal-p)))))))
 
 (defun org-publish-get-base-files (project &optional exclude-regexp include-dirs)
   "Return a list of all files in PROJECT.
 If EXCLUDE-REGEXP is set, this will be used to filter out
 matching filenames."
   (let* ((project-plist (cdr project))
-	 (base-dir (file-name-as-directory
-		    (plist-get project-plist :base-directory)))
-	 (include-list (plist-get project-plist :include))
-	 (recurse (plist-get project-plist :recursive))
-	 (extension (or (plist-get project-plist :base-extension) "org"))
-	 ;; sitemap-... variables are dynamically scoped for
-	 ;; org-publish-compare-directory-files:
-	 (org-publish-sitemap-requested
-	  (plist-get project-plist :auto-sitemap))
-	 (sitemap-filename
-	  (or (plist-get project-plist :sitemap-filename) "sitemap.org"))
-	 (org-publish-sitemap-sort-folders
-	  (if (plist-member project-plist :sitemap-sort-folders)
-	      (plist-get project-plist :sitemap-sort-folders)
-	    org-publish-sitemap-sort-folders))
-	 (org-publish-sitemap-sort-files
-	  (cond ((plist-member project-plist :sitemap-sort-files)
-		 (plist-get project-plist :sitemap-sort-files))
-		;; For backward compatibility:
-		((plist-member project-plist :sitemap-alphabetically)
-		 (if (plist-get project-plist :sitemap-alphabetically)
-		     'alphabetically nil))
-		(t org-publish-sitemap-sort-files)))
-	 (org-publish-sitemap-ignore-case
-	  (if (plist-member project-plist :sitemap-ignore-case)
-	      (plist-get project-plist :sitemap-ignore-case)
-	    org-publish-sitemap-sort-ignore-case))
-	 (match (if (eq extension 'any) "^[^\\.]"
-		  (concat "^[^\\.].*\\.\\(" extension "\\)$"))))
+         (base-dir (file-name-as-directory
+                    (plist-get project-plist :base-directory)))
+         (include-list (plist-get project-plist :include))
+         (recurse (plist-get project-plist :recursive))
+         (extension (or (plist-get project-plist :base-extension) "org"))
+         ;; sitemap-... variables are dynamically scoped for
+         ;; org-publish-compare-directory-files:
+         (org-publish-sitemap-requested
+          (plist-get project-plist :auto-sitemap))
+         (sitemap-filename
+          (or (plist-get project-plist :sitemap-filename) "sitemap.org"))
+         (org-publish-sitemap-sort-folders
+          (if (plist-member project-plist :sitemap-sort-folders)
+              (plist-get project-plist :sitemap-sort-folders)
+            org-publish-sitemap-sort-folders))
+         (org-publish-sitemap-sort-files
+          (cond ((plist-member project-plist :sitemap-sort-files)
+                 (plist-get project-plist :sitemap-sort-files))
+                ;; For backward compatibility:
+                ((plist-member project-plist :sitemap-alphabetically)
+                 (if (plist-get project-plist :sitemap-alphabetically)
+                     'alphabetically nil))
+                (t org-publish-sitemap-sort-files)))
+         (org-publish-sitemap-ignore-case
+          (if (plist-member project-plist :sitemap-ignore-case)
+              (plist-get project-plist :sitemap-ignore-case)
+            org-publish-sitemap-sort-ignore-case))
+         (match (if (eq extension 'any) "^[^\\.]"
+                  (concat "^[^\\.].*\\.\\(" extension "\\)$"))))
     ;; Make sure `org-publish-sitemap-sort-folders' has an accepted
     ;; value.
     (unless (memq org-publish-sitemap-sort-folders '(first last))
@@ -515,14 +593,14 @@ matching filenames."
     (setq org-publish-temp-files nil)
     (when org-publish-sitemap-requested
       (cl-pushnew (expand-file-name (concat base-dir sitemap-filename))
-		  org-publish-temp-files :test #'file-equal-p))
+                  org-publish-temp-files :test #'file-equal-p))
     (org-publish-get-base-files-1 base-dir recurse match
-				  ;; FIXME distinguish exclude regexp
-				  ;; for skip-file and skip-dir?
-				  exclude-regexp exclude-regexp include-dirs)
+                                  ;; FIXME distinguish exclude regexp
+                                  ;; for skip-file and skip-dir?
+                                  exclude-regexp exclude-regexp include-dirs)
     (dolist (f include-list org-publish-temp-files)
       (cl-pushnew (expand-file-name (concat base-dir f))
-		  org-publish-temp-files :test #'file-equal-p))
+                  org-publish-temp-files :test #'file-equal-p))
     ;; Remove directories from `org-publish-temp-files' that do not
     ;; hold project files.
     (when include-dirs
@@ -537,31 +615,31 @@ matching filenames."
 (defun org-publish-get-project-from-filename (filename &optional up)
   "Return the project that FILENAME belongs to."
   (let* ((filename (expand-file-name filename))
-	 project-name)
+         project-name)
 
     (catch 'p-found
       (dolist (prj org-publish-project-alist)
-	(unless (plist-get (cdr prj) :components)
-	  ;; [[info:org:Selecting%20files]] shows how this is supposed to work:
-	  (let* ((r (plist-get (cdr prj) :recursive))
-		 (b (expand-file-name (file-name-as-directory
-				       (plist-get (cdr prj) :base-directory))))
-		 (x (or (plist-get (cdr prj) :base-extension) "org"))
-		 (e (plist-get (cdr prj) :exclude))
-		 (i (plist-get (cdr prj) :include))
-		 (xm (concat "^" b (if r ".+" "[^/]+") "\\.\\(" x "\\)$")))
-	    (when
-		(or (and i
-			 (member filename
-				 (dolist (file i) (expand-file-name file b))))
-		    (and (not (and e (string-match e filename)))
-			 (string-match xm filename)))
-	      (setq project-name (car prj))
-	      (throw 'p-found project-name))))))
+        (unless (plist-get (cdr prj) :components)
+          ;; [[info:org:Selecting%20files]] shows how this is supposed to work:
+          (let* ((r (plist-get (cdr prj) :recursive))
+                 (b (expand-file-name (file-name-as-directory
+                                       (plist-get (cdr prj) :base-directory))))
+                 (x (or (plist-get (cdr prj) :base-extension) "org"))
+                 (e (plist-get (cdr prj) :exclude))
+                 (i (plist-get (cdr prj) :include))
+                 (xm (concat "^" b (if r ".+" "[^/]+") "\\.\\(" x "\\)$")))
+            (when
+                (or (and i
+                         (member filename
+                                 (dolist (file i) (expand-file-name file b))))
+                    (and (not (and e (string-match e filename)))
+                         (string-match xm filename)))
+              (setq project-name (car prj))
+              (throw 'p-found project-name))))))
     (when up
       (dolist (prj org-publish-project-alist)
-	(if (member project-name (plist-get (cdr prj) :components))
-	    (setq project-name (car prj)))))
+        (if (member project-name (plist-get (cdr prj) :components))
+            (setq project-name (car prj)))))
     (assoc project-name org-publish-project-alist)))
 
 
@@ -585,26 +663,26 @@ Return output file name."
   (unless (or (not pub-dir) (file-exists-p pub-dir)) (make-directory pub-dir t))
   ;; Check if a buffer visiting FILENAME is already open.
   (let* ((org-inhibit-startup t)
-	 (visiting (find-buffer-visiting filename))
-	 (work-buffer (or visiting (find-file-noselect filename))))
+         (visiting (find-buffer-visiting filename))
+         (work-buffer (or visiting (find-file-noselect filename))))
     (unwind-protect
-	(with-current-buffer work-buffer
-	  (let ((output (org-export-output-file-name extension nil pub-dir)))
-	    (org-export-to-file backend output
-	      nil nil nil (plist-get plist :body-only)
-	      ;; Add `org-publish--store-crossrefs' and
-	      ;; `org-publish-collect-index' to final output filters.
-	      ;; The latter isn't dependent on `:makeindex', since we
-	      ;; want to keep it up-to-date in cache anyway.
-	      (org-combine-plists
-	       plist
-	       `(:crossrefs
-		 ,(org-publish-cache-get-file-property
-		   (expand-file-name filename) :crossrefs nil t)
-		 :filter-final-output
-		 (org-publish--store-crossrefs
-		  org-publish-collect-index
-		  ,@(plist-get plist :filter-final-output)))))))
+        (with-current-buffer work-buffer
+          (let ((output (org-export-output-file-name extension nil pub-dir)))
+            (org-export-to-file backend output
+              nil nil nil (plist-get plist :body-only)
+              ;; Add `org-publish--store-crossrefs' and
+              ;; `org-publish-collect-index' to final output filters.
+              ;; The latter isn't dependent on `:makeindex', since we
+              ;; want to keep it up-to-date in cache anyway.
+              (org-combine-plists
+               plist
+               `(:crossrefs
+                 ,(org-publish-cache-get-file-property
+                   (expand-file-name filename) :crossrefs nil t)
+                 :filter-final-output
+                 (org-publish--store-crossrefs
+                  org-publish-collect-index
+                  ,@(plist-get plist :filter-final-output)))))))
       ;; Remove opened buffer in the process.
       (unless visiting (kill-buffer work-buffer)))))
 
@@ -620,8 +698,8 @@ Return output file name."
     (make-directory pub-dir t))
   (let ((output (expand-file-name (file-name-nondirectory filename) pub-dir)))
     (or (equal (expand-file-name (file-name-directory filename))
-	       (file-name-as-directory (expand-file-name pub-dir)))
-	(copy-file filename output t))
+               (file-name-as-directory (expand-file-name pub-dir)))
+        (copy-file filename output t))
     ;; Return file name.
     output))
 
@@ -637,46 +715,46 @@ This is needed, since this function is used to publish single
 files, when entire projects are published (see
 `org-publish-projects')."
   (let* ((project
-	  (or project
-	      (or (org-publish-get-project-from-filename filename)
-		  (error "File %s not part of any known project"
-			 (abbreviate-file-name filename)))))
-	 (project-plist (cdr project))
-	 (ftname (expand-file-name filename))
-	 (publishing-function
-	  (let ((fun (plist-get project-plist :publishing-function)))
-	    (cond ((null fun) (error "No publishing function chosen"))
-		  ((listp fun) fun)
-		  (t (list fun)))))
-	 (base-dir
-	  (file-name-as-directory
-	   (expand-file-name
-	    (or (plist-get project-plist :base-directory)
-		(error "Project %s does not have :base-directory defined"
-		       (car project))))))
-	 (pub-dir
-	  (file-name-as-directory
-	   (file-truename
-	    (or (eval (plist-get project-plist :publishing-directory))
-		(error "Project %s does not have :publishing-directory defined"
-		       (car project))))))
-	 tmp-pub-dir)
+          (or project
+              (or (org-publish-get-project-from-filename filename)
+                  (error "File %s not part of any known project"
+                         (abbreviate-file-name filename)))))
+         (project-plist (cdr project))
+         (ftname (expand-file-name filename))
+         (publishing-function
+          (let ((fun (plist-get project-plist :publishing-function)))
+            (cond ((null fun) (error "No publishing function chosen"))
+                  ((listp fun) fun)
+                  (t (list fun)))))
+         (base-dir
+          (file-name-as-directory
+           (expand-file-name
+            (or (plist-get project-plist :base-directory)
+                (error "Project %s does not have :base-directory defined"
+                       (car project))))))
+         (pub-dir
+          (file-name-as-directory
+           (file-truename
+            (or (eval (plist-get project-plist :publishing-directory))
+                (error "Project %s does not have :publishing-directory defined"
+                       (car project))))))
+         tmp-pub-dir)
 
     (unless no-cache (org-publish-initialize-cache (car project)))
 
     (setq tmp-pub-dir
-	  (file-name-directory
-	   (concat pub-dir
-		   (and (string-match (regexp-quote base-dir) ftname)
-			(substring ftname (match-end 0))))))
+          (file-name-directory
+           (concat pub-dir
+                   (and (string-match (regexp-quote base-dir) ftname)
+                        (substring ftname (match-end 0))))))
     ;; Allow chain of publishing functions.
     (dolist (f publishing-function)
       (when (org-publish-needed-p filename pub-dir f tmp-pub-dir base-dir)
-	(let ((output (funcall f project-plist filename tmp-pub-dir)))
-	  (org-publish-update-timestamp filename pub-dir f base-dir)
-	  (run-hook-with-args 'org-publish-after-publishing-hook
-			      filename
-			      output))))
+        (let ((output (funcall f project-plist filename tmp-pub-dir)))
+          (org-publish-update-timestamp filename pub-dir f base-dir)
+          (run-hook-with-args 'org-publish-after-publishing-hook
+                              filename
+                              output))))
     ;; Make sure to write cache to file after successfully publishing
     ;; a file, so as to minimize impact of a publishing failure.
     (org-publish-write-cache-file)))
@@ -688,42 +766,45 @@ If `:auto-sitemap' is set, publish the sitemap too.  If
   (dolist (project (org-publish-expand-projects projects))
     (let ((project-plist (cdr project)))
       (let ((f (plist-get project-plist :preparation-function)))
-	(cond ((consp f) (mapc #'funcall f))
-	      ((functionp f) (funcall f))))
+        (cond ((consp f) (mapc #'funcall f))
+              ((functionp f) (funcall f))))
       ;; Each project uses its own cache file.
       (org-publish-initialize-cache (car project))
       (when  (plist-get project-plist :auto-sitemap)
-	(let ((sitemap-filename
-	       (or (plist-get project-plist :sitemap-filename)
-		   "sitemap.org"))
-	      (sitemap-function
-	       (or (plist-get project-plist :sitemap-function)
-		   #'org-publish-org-sitemap))
-	      (org-publish-sitemap-date-format
-	       (or (plist-get project-plist :sitemap-date-format)
-		   org-publish-sitemap-date-format))
-	      (org-publish-sitemap-file-entry-format
-	       (or (plist-get project-plist :sitemap-file-entry-format)
-		   org-publish-sitemap-file-entry-format)))
-	  (funcall sitemap-function project sitemap-filename)))
+        (let ((sitemap-filename
+               (or (plist-get project-plist :sitemap-filename)
+                   "sitemap.org"))
+              (sitemap-function
+               (or (plist-get project-plist :sitemap-function)
+                   #'org-publish-org-sitemap))
+              (org-publish-sitemap-date-format
+               (or (plist-get project-plist :sitemap-date-format)
+                   org-publish-sitemap-date-format))
+              (org-publish-sitemap-file-entry-format
+               (or (plist-get project-plist :sitemap-file-entry-format)
+                   org-publish-sitemap-file-entry-format))
+              (org-publish-sitemap-dir-entry-format
+               (or (plist-get project-plist :sitemap-dir-entry-format)
+                   org-publish-sitemap-dir-entry-format))
+          (funcall sitemap-function project sitemap-filename)))
       ;; Publish all files from PROJECT excepted "theindex.org".  Its
       ;; publishing will be deferred until "theindex.inc" is
       ;; populated.
       (let ((theindex
-	     (expand-file-name "theindex.org"
-			       (plist-get project-plist :base-directory)))
-	    (exclude-regexp (plist-get project-plist :exclude)))
-	(dolist (file (org-publish-get-base-files project exclude-regexp))
-	  (unless (equal file theindex) (org-publish-file file project t)))
-	;; Populate "theindex.inc", if needed, and publish
-	;; "theindex.org".
-	(when (plist-get project-plist :makeindex)
-	  (org-publish-index-generate-theindex
-	   project (plist-get project-plist :base-directory))
-	  (org-publish-file theindex project t)))
+             (expand-file-name "theindex.org"
+                               (plist-get project-plist :base-directory)))
+            (exclude-regexp (plist-get project-plist :exclude)))
+        (dolist (file (org-publish-get-base-files project exclude-regexp))
+          (unless (equal file theindex) (org-publish-file file project t)))
+        ;; Populate "theindex.inc", if needed, and publish
+        ;; "theindex.org".
+        (when (plist-get project-plist :makeindex)
+          (org-publish-index-generate-theindex
+           project (plist-get project-plist :base-directory))
+          (org-publish-file theindex project t)))
       (let ((f (plist-get project-plist :completion-function)))
-	(cond ((consp f) (mapc #'funcall f))
-	      ((functionp f) (funcall f))))
+        (cond ((consp f) (mapc #'funcall f))
+              ((functionp f) (funcall f))))
       (org-publish-write-cache-file))))
 
 (defun org-publish-org-sitemap (project &optional sitemap-filename)
@@ -731,88 +812,109 @@ If `:auto-sitemap' is set, publish the sitemap too.  If
 Optionally set the filename of the sitemap with SITEMAP-FILENAME.
 Default for SITEMAP-FILENAME is `sitemap.org'."
   (let* ((project-plist (cdr project))
-	 (dir (file-name-as-directory
-	       (plist-get project-plist :base-directory)))
-	 (localdir (file-name-directory dir))
-	 (indent-str (make-string 2 ?\ ))
-	 (exclude-regexp (plist-get project-plist :exclude))
-	 (files (nreverse
-		 (org-publish-get-base-files project exclude-regexp)))
-	 (sitemap-filename (concat dir (or sitemap-filename "sitemap.org")))
-	 (sitemap-title (or (plist-get project-plist :sitemap-title)
-			  (concat "Sitemap for project " (car project))))
-	 (sitemap-style (or (plist-get project-plist :sitemap-style)
-			    'tree))
-	 (sitemap-sans-extension
-	  (plist-get project-plist :sitemap-sans-extension))
-	 (visiting (find-buffer-visiting sitemap-filename))
-	 file sitemap-buffer)
-    (with-current-buffer
-	(let ((org-inhibit-startup t))
-	  (setq sitemap-buffer
-		(or visiting (find-file sitemap-filename))))
-      (erase-buffer)
-      (insert (concat "#+TITLE: " sitemap-title "\n\n"))
-      (while (setq file (pop files))
-	(let ((link (file-relative-name file dir))
-	      (oldlocal localdir))
-	  (when sitemap-sans-extension
-	    (setq link (file-name-sans-extension link)))
-	  ;; sitemap shouldn't list itself
-	  (unless (equal (file-truename sitemap-filename)
-			 (file-truename file))
-	    (if (eq sitemap-style 'list)
-		(message "Generating list-style sitemap for %s" sitemap-title)
-	      (message "Generating tree-style sitemap for %s" sitemap-title)
-	      (setq localdir (concat (file-name-as-directory dir)
-				     (file-name-directory link)))
-	      (unless (string= localdir oldlocal)
-		(if (string= localdir dir)
-		    (setq indent-str (make-string 2 ?\ ))
-		  (let ((subdirs
-			 (split-string
-			  (directory-file-name
-			   (file-name-directory
-			    (file-relative-name localdir dir))) "/"))
-			(subdir "")
-			(old-subdirs (split-string
-				      (file-relative-name oldlocal dir) "/")))
-		    (setq indent-str (make-string 2 ?\ ))
-		    (while (string= (car old-subdirs) (car subdirs))
-		      (setq indent-str (concat indent-str (make-string 2 ?\ )))
-		      (pop old-subdirs)
-		      (pop subdirs))
-		    (dolist (d subdirs)
-		      (setq subdir (concat subdir d "/"))
-		      (insert (concat indent-str " + " d "\n"))
-		      (setq indent-str (make-string
-					(+ (length indent-str) 2) ?\ )))))))
-	    ;; This is common to 'flat and 'tree
-	    (let ((entry
-		   (org-publish-format-file-entry
-		    org-publish-sitemap-file-entry-format file project-plist))
-		  (regexp "\\(.*\\)\\[\\([^][]+\\)\\]\\(.*\\)"))
-	      (cond ((string-match-p regexp entry)
-		     (string-match regexp entry)
-		     (insert (concat indent-str " + " (match-string 1 entry)
-				     "[[file:" link "]["
-				     (match-string 2 entry)
-				     "]]" (match-string 3 entry) "\n")))
-		    (t
-		     (insert (concat indent-str " + [[file:" link "]["
-				     entry
-				     "]]\n"))))))))
-      (save-buffer))
-    (or visiting (kill-buffer sitemap-buffer))))
+         (dir (file-name-as-directory
+               (plist-get project-plist :base-directory)))
+         (sitemap-filename (concat dir (or sitemap-filename "sitemap.org")))
+         (files (nreverse
+                 ;; Sitemap shouldn't list itself.
+                 (cl-remove sitemap-filename
+                            (org-publish-get-base-files
+                             project
+                             (plist-get project-plist :exclude) t)
+                            :test #'equal)))
+         (sitemap-title (or (plist-get project-plist :sitemap-title)
+                            (concat "Sitemap for project " (car project)))))
+    (with-temp-file sitemap-filename
+      (insert
+       (mapconcat
+        'identity
+        (list
+         (format "#+TITLE: %s" sitemap-title)
+         ;; Call function to build sitemap based on files and the project-plist.
+         (let* ((style (or (plist-get project-plist :sitemap-style) 'tree))
+                (fun (intern (format "org-publish-org-sitemap-as-%s" style))))
+           (unless (functionp fun)
+             (error "Unknown function %s for :sitemap-style %s."
+                    fun style))
+           (funcall fun files project-plist))) "\n")))))
+
+(defun org-publish-org-sitemap-as-list (files project-plist)
+  "Insert FILES as simple list separated by newlines.
+PROJECT-PLIST holds the project information."
+  (mapconcat
+   (lambda (file)
+     (org-publish-format-file-entry
+      org-publish-sitemap-file-entry-format file project-plist))
+   (cl-delete-if 'file-directory-p files) "\n"))
+
+(defun org-publish-org-sitemap-as-tree (files project-plist)
+  "Insert FILES as a tree.
+PROJECT-PLIST holds the project information."
+  (mapconcat (lambda (elm)
+               (org-publish-format-file-entry
+                (cond
+                 ((file-directory-p elm) org-publish-sitemap-dir-entry-format)
+                 (t org-publish-sitemap-file-entry-format))
+                elm project-plist))
+             (cl-remove (plist-get project-plist :base-directory)
+                        files :test #'file-equal-p)
+             "\n"))
 
 (defun org-publish-format-file-entry (fmt file project-plist)
-  (format-spec
-   fmt
-   `((?t . ,(org-publish-find-title file t))
-     (?d . ,(format-time-string org-publish-sitemap-date-format
-				(org-publish-find-date file)))
-     (?a . ,(or (plist-get project-plist :author) user-full-name)))))
-
+  "Format FILE according to the format-string FMT.
+PROJECT-PLIST is a plist holding project options.
+See also `org-publish-sitemap-file-entry-format'.
+"
+  (let* ((basedir (file-truename (plist-get project-plist :base-directory)))
+         (filename (file-relative-name file basedir))
+         (dirname (file-name-directory filename))
+         (dirp (file-directory-p file))
+         (depth (if (or (eq 'list (plist-get project-plist :sitemap-style))
+                        (not dirname))
+                    1
+                  (+ (if (not (directory-name-p filename)) 1 0)
+                     (length (split-string (file-name-directory filename) "/" t)))))
+         (link (funcall (if (plist-get project-plist :sitemap-sans-extension)
+                            #'file-name-sans-extension
+                          #'identity)
+                        filename))
+         (relfilepath (if (file-equal-p file basedir) ""
+                        (directory-file-name
+                         (if dirp
+                             (file-relative-name
+                              filename (file-name-directory (directory-file-name filename)))
+                           (file-relative-name filename dirname)))))
+         (date (unless dirp
+                 (format-time-string
+                  (or (plist-get project-plist :sitemap-date-format)
+                      org-publish-sitemap-date-format)
+                  (org-publish-find-date file))))
+         (info (list (list "fullpath" file)
+                     (list "relpath" filename)
+                     (list "filename" relfilepath)
+                     (list "f" relfilepath)
+                     (list "l" link)
+                     (list "link" link)
+                     (list "*" (concat (make-string depth ?*)))
+                     (list "-" (concat (make-string (* 2 depth) ?\s) "-"))
+                     (list "date" (format-time-string
+                                   (or (plist-get project-plist :sitemap-date-format)
+                                       org-publish-sitemap-date-format)
+                                   (org-publish-find-date file)))
+                     (list "author" (or (unless dirp (org-publish-find-property file :author t))
+                                        (plist-get project-plist :author)
+                                        user-full-name))
+                     ;; TODO: Insert legacy values here...
+                     )))
+    (with-temp-buffer
+      (save-excursion (insert fmt))
+      (while (re-search-forward "%\\([a-zA-Z-*]+\\>?\\)" nil t)
+        (save-excursion
+          (replace-match (or (cadr (assoc-string (match-string 1) info t))
+                             (save-match-data (org-publish-find-property file (match-string 1) t))
+                             (error "Could not expand \"%s\" while formatting sitemap entry for \"%s\""
+                                    (match-string 1) file)))))
+      (buffer-string))))
 
 (defun org-publish-find-property (file property &optional reset)
   "Find the PROPERTY of FILE in project.
@@ -858,12 +960,12 @@ time in `current-time' format."
       ;; convert it to internal format.  Otherwise, use FILE
       ;; modification time.
       (cond ((let ((ts (and (consp date) (assq 'timestamp date))))
-	       (and ts
-		    (let ((value (org-element-interpret-data ts)))
-		      (and (org-string-nw-p value)
-			   (org-time-string-to-time value))))))
-	    ((file-exists-p file) (nth 5 (file-attributes file)))
-	    (t (error "No such file: \"%s\"" file))))))
+               (and ts
+                    (let ((value (org-element-interpret-data ts)))
+                      (and (org-string-nw-p value)
+                           (org-time-string-to-time value))))))
+            ((file-exists-p file) (nth 5 (file-attributes file)))
+            (t (error "No such file: \"%s\"" file))))))
 
 
 \f
@@ -885,27 +987,27 @@ files in PROJECT.  With a non-nil optional argument ASYNC,
 publishing will be done asynchronously, in another process."
   (interactive
    (list (assoc (completing-read "Publish project: "
-				 org-publish-project-alist nil t)
-		org-publish-project-alist)
-	 current-prefix-arg))
+                                 org-publish-project-alist nil t)
+                org-publish-project-alist)
+         current-prefix-arg))
   (let ((project (if (not (stringp project)) project
-		   ;; If this function is called in batch mode,
-		   ;; PROJECT is still a string here.
-		   (assoc project org-publish-project-alist))))
+                   ;; If this function is called in batch mode,
+                   ;; PROJECT is still a string here.
+                   (assoc project org-publish-project-alist))))
     (cond
      ((not project))
      (async
       (org-export-async-start (lambda (_) nil)
-	`(let ((org-publish-use-timestamps-flag
-		,(and (not force) org-publish-use-timestamps-flag)))
-	   ;; Expand components right now as external process may not
-	   ;; be aware of complete `org-publish-project-alist'.
-	   (org-publish-projects
-	    ',(org-publish-expand-projects (list project))))))
+        `(let ((org-publish-use-timestamps-flag
+                ,(and (not force) org-publish-use-timestamps-flag)))
+           ;; Expand components right now as external process may not
+           ;; be aware of complete `org-publish-project-alist'.
+           (org-publish-projects
+            ',(org-publish-expand-projects (list project))))))
      (t (save-window-excursion
-	  (let ((org-publish-use-timestamps-flag
-		 (and (not force) org-publish-use-timestamps-flag)))
-	    (org-publish-projects (list project))))))))
+          (let ((org-publish-use-timestamps-flag
+                 (and (not force) org-publish-use-timestamps-flag)))
+            (org-publish-projects (list project))))))))
 
 ;;;###autoload
 (defun org-publish-all (&optional force async)
@@ -917,16 +1019,16 @@ in another process."
   (interactive "P")
   (if async
       (org-export-async-start (lambda (_) nil)
-	`(progn
-	   (when ',force (org-publish-remove-all-timestamps))
-	   (let ((org-publish-use-timestamps-flag
-		  (if ',force nil ,org-publish-use-timestamps-flag)))
-	     (org-publish-projects ',org-publish-project-alist))))
+        `(progn
+           (when ',force (org-publish-remove-all-timestamps))
+           (let ((org-publish-use-timestamps-flag
+                  (if ',force nil ,org-publish-use-timestamps-flag)))
+             (org-publish-projects ',org-publish-project-alist))))
     (when force (org-publish-remove-all-timestamps))
     (save-window-excursion
       (let ((org-publish-use-timestamps-flag
-	     (if force nil org-publish-use-timestamps-flag)))
-	(org-publish-projects org-publish-project-alist)))))
+             (if force nil org-publish-use-timestamps-flag)))
+        (org-publish-projects org-publish-project-alist)))))
 
 
 ;;;###autoload
@@ -938,14 +1040,14 @@ asynchronously, in another process."
   (interactive "P")
   (let ((file (buffer-file-name (buffer-base-buffer))))
     (if async
-	(org-export-async-start (lambda (_) nil)
-	  `(let ((org-publish-use-timestamps-flag
-		  (if ',force nil ,org-publish-use-timestamps-flag)))
-	     (org-publish-file ,file)))
+        (org-export-async-start (lambda (_) nil)
+          `(let ((org-publish-use-timestamps-flag
+                  (if ',force nil ,org-publish-use-timestamps-flag)))
+             (org-publish-file ,file)))
       (save-window-excursion
-	(let ((org-publish-use-timestamps-flag
-	       (if force nil org-publish-use-timestamps-flag)))
-	  (org-publish-file file))))))
+        (let ((org-publish-use-timestamps-flag
+               (if force nil org-publish-use-timestamps-flag)))
+          (org-publish-file file))))))
 
 ;;;###autoload
 (defun org-publish-current-project (&optional force async)
@@ -955,10 +1057,10 @@ the project."
   (interactive "P")
   (save-window-excursion
     (let ((project (org-publish-get-project-from-filename
-		    (buffer-file-name (buffer-base-buffer)) 'up)))
+                    (buffer-file-name (buffer-base-buffer)) 'up)))
       (if project (org-publish project force async)
-	(error "File %s is not part of any known project"
-	       (buffer-file-name (buffer-base-buffer)))))))
+        (error "File %s is not part of any known project"
+               (buffer-file-name (buffer-base-buffer)))))))
 
 
 \f
@@ -984,23 +1086,23 @@ its CDR is a string."
      file :index
      (delete-dups
       (org-element-map (plist-get info :parse-tree) 'keyword
-	(lambda (k)
-	  (when (equal (org-element-property :key k) "INDEX")
-	    (let ((parent (org-export-get-parent-headline k)))
-	      (list (org-element-property :value k)
-		    file
-		    (cond
-		     ((not parent) nil)
-		     ((let ((id (org-element-property :ID parent)))
-			(and id (cons 'id id))))
-		     ((let ((id (org-element-property :CUSTOM_ID parent)))
-			(and id (cons 'custom-id id))))
-		     (t (cons 'name
-			      ;; Remove statistics cookie.
-			      (replace-regexp-in-string
-			       "\\[[0-9]+%\\]\\|\\[[0-9]+/[0-9]+\\]" ""
-			       (org-element-property :raw-value parent)))))))))
-	info))))
+        (lambda (k)
+          (when (equal (org-element-property :key k) "INDEX")
+            (let ((parent (org-export-get-parent-headline k)))
+              (list (org-element-property :value k)
+                    file
+                    (cond
+                     ((not parent) nil)
+                     ((let ((id (org-element-property :ID parent)))
+                        (and id (cons 'id id))))
+                     ((let ((id (org-element-property :CUSTOM_ID parent)))
+                        (and id (cons 'custom-id id))))
+                     (t (cons 'name
+                              ;; Remove statistics cookie.
+                              (replace-regexp-in-string
+                               "\\[[0-9]+%\\]\\|\\[[0-9]+/[0-9]+\\]" ""
+                               (org-element-property :raw-value parent)))))))))
+        info))))
   ;; Return output unchanged.
   output)
 
@@ -1009,68 +1111,68 @@ its CDR is a string."
 PROJECT is the project the index relates to.  DIRECTORY is the
 publishing directory."
   (let ((all-files (org-publish-get-base-files
-		    project (plist-get (cdr project) :exclude)))
-	full-index)
+                    project (plist-get (cdr project) :exclude)))
+        full-index)
     ;; Compile full index and sort it alphabetically.
     (dolist (file all-files
-		  (setq full-index
-			(sort (nreverse full-index)
-			      (lambda (a b) (string< (downcase (car a))
-						(downcase (car b)))))))
+                  (setq full-index
+                        (sort (nreverse full-index)
+                              (lambda (a b) (string< (downcase (car a))
+                                                (downcase (car b)))))))
       (let ((index (org-publish-cache-get-file-property file :index)))
-	(dolist (term index)
-	  (unless (member term full-index) (push term full-index)))))
+        (dolist (term index)
+          (unless (member term full-index) (push term full-index)))))
     ;; Write "theindex.inc" in DIRECTORY.
     (with-temp-file (expand-file-name "theindex.inc" directory)
       (let ((current-letter nil) (last-entry nil))
-	(dolist (idx full-index)
-	  (let* ((entry (org-split-string (car idx) "!"))
-		 (letter (upcase (substring (car entry) 0 1)))
-		 ;; Transform file into a path relative to publishing
-		 ;; directory.
-		 (file (file-relative-name
-			(nth 1 idx)
-			(plist-get (cdr project) :base-directory))))
-	    ;; Check if another letter has to be inserted.
-	    (unless (string= letter current-letter)
-	      (insert (format "* %s\n" letter)))
-	    ;; Compute the first difference between last entry and
-	    ;; current one: it tells the level at which new items
-	    ;; should be added.
-	    (let* ((rank
-		    (if (equal entry last-entry) (1- (length entry))
-		      (cl-loop for n from 0 to (length entry)
-			       unless (equal (nth n entry) (nth n last-entry))
-			       return n)))
-		   (len (length (nthcdr rank entry))))
-	      ;; For each term after the first difference, create
-	      ;; a new sub-list with the term as body.  Moreover,
-	      ;; linkify the last term.
-	      (dotimes (n len)
-		(insert
-		 (concat
-		  (make-string (* (+ rank n) 2) ? ) "  - "
-		  (if (not (= (1- len) n)) (nth (+ rank n) entry)
-		    ;; Last term: Link it to TARGET, if possible.
-		    (let ((target (nth 2 idx)))
-		      (format
-		       "[[%s][%s]]"
-		       ;; Destination.
-		       (pcase (car target)
-			 (`nil (format "file:%s" file))
-			 (`id (format "id:%s" (cdr target)))
-			 (`custom-id (format "file:%s::#%s" file (cdr target)))
-			 (_ (format "file:%s::*%s" file (cdr target))))
-		       ;; Description.
-		       (car (last entry)))))
-		  "\n"))))
-	    (setq current-letter letter last-entry entry))))
+        (dolist (idx full-index)
+          (let* ((entry (org-split-string (car idx) "!"))
+                 (letter (upcase (substring (car entry) 0 1)))
+                 ;; Transform file into a path relative to publishing
+                 ;; directory.
+                 (file (file-relative-name
+                        (nth 1 idx)
+                        (plist-get (cdr project) :base-directory))))
+            ;; Check if another letter has to be inserted.
+            (unless (string= letter current-letter)
+              (insert (format "* %s\n" letter)))
+            ;; Compute the first difference between last entry and
+            ;; current one: it tells the level at which new items
+            ;; should be added.
+            (let* ((rank
+                    (if (equal entry last-entry) (1- (length entry))
+                      (cl-loop for n from 0 to (length entry)
+                               unless (equal (nth n entry) (nth n last-entry))
+                               return n)))
+                   (len (length (nthcdr rank entry))))
+              ;; For each term after the first difference, create
+              ;; a new sub-list with the term as body.  Moreover,
+              ;; linkify the last term.
+              (dotimes (n len)
+                (insert
+                 (concat
+                  (make-string (* (+ rank n) 2) ? ) "  - "
+                  (if (not (= (1- len) n)) (nth (+ rank n) entry)
+                    ;; Last term: Link it to TARGET, if possible.
+                    (let ((target (nth 2 idx)))
+                      (format
+                       "[[%s][%s]]"
+                       ;; Destination.
+                       (pcase (car target)
+                         (`nil (format "file:%s" file))
+                         (`id (format "id:%s" (cdr target)))
+                         (`custom-id (format "file:%s::#%s" file (cdr target)))
+                         (_ (format "file:%s::*%s" file (cdr target))))
+                       ;; Description.
+                       (car (last entry)))))
+                  "\n"))))
+            (setq current-letter letter last-entry entry))))
       ;; Create "theindex.org", if it doesn't exist yet, and provide
       ;; a default index file.
       (let ((index.org (expand-file-name "theindex.org" directory)))
-	(unless (file-exists-p index.org)
-	  (with-temp-file index.org
-	    (insert "#+TITLE: Index\n\n#+INCLUDE: \"theindex.inc\"\n\n")))))))
+        (unless (file-exists-p index.org)
+          (with-temp-file index.org
+            (insert "#+TITLE: Index\n\n#+INCLUDE: \"theindex.inc\"\n\n")))))))
 
 
 \f
@@ -1095,7 +1197,7 @@ This function is meant to be used as a final output filter.  See
    ;; `:internal-references', with references as strings removed.  See
    ;; `org-export-get-reference' for details.
    (cl-remove-if (lambda (pair) (stringp (car pair)))
-		 (plist-get info :internal-references)))
+                 (plist-get info :internal-references)))
   ;; Return output unchanged.
   output)
 
@@ -1115,27 +1217,27 @@ It only makes sense to use this if export back-end builds
 references with `org-export-get-reference'."
   (if (not org-publish-cache)
       (progn
-	(message "Reference %S in file %S cannot be resolved without publishing"
-		 search
-		 file)
-	"MissingReference")
+        (message "Reference %S in file %S cannot be resolved without publishing"
+                 search
+                 file)
+        "MissingReference")
     (let* ((filename (expand-file-name file))
-	   (crossrefs
-	    (org-publish-cache-get-file-property filename :crossrefs nil t))
-	   (cells (org-export-string-to-search-cell search)))
+           (crossrefs
+            (org-publish-cache-get-file-property filename :crossrefs nil t))
+           (cells (org-export-string-to-search-cell search)))
       (or
        ;; Look for reference associated to search cells triggered by
        ;; LINK.  It can match when targeted file has been published
        ;; already.
        (let ((known (cdr (cl-some (lambda (c) (assoc c crossrefs)) cells))))
-	 (and known (org-export-format-reference known)))
+         (and known (org-export-format-reference known)))
        ;; Search cell is unknown so far.  Generate a new internal
        ;; reference that will be used when the targeted file will be
        ;; published.
        (let ((new (org-export-new-reference crossrefs)))
-	 (dolist (cell cells) (push (cons cell new) crossrefs))
-	 (org-publish-cache-set-file-property filename :crossrefs crossrefs)
-	 (org-export-format-reference new))))))
+         (dolist (cell cells) (push (cons cell new) crossrefs))
+         (org-publish-cache-set-file-property filename :crossrefs crossrefs)
+         (org-export-format-reference new))))))
 
 
 \f
@@ -1152,13 +1254,13 @@ If FREE-CACHE, empty the cache."
       (error "Cannot find cache-file name in `org-publish-write-cache-file'"))
     (with-temp-file cache-file
       (let (print-level print-length)
-	(insert "(setq org-publish-cache \
+        (insert "(setq org-publish-cache \
 \(make-hash-table :test 'equal :weakness nil :size 100))\n")
-	(maphash (lambda (k v)
-		   (insert
-		    (format "(puthash %S %s%S org-publish-cache)\n"
-			    k (if (or (listp v) (symbolp v)) "'" "") v)))
-		 org-publish-cache)))
+        (maphash (lambda (k v)
+                   (insert
+                    (format "(puthash %S %s%S org-publish-cache)\n"
+                            k (if (or (listp v) (symbolp v)) "'" "") v)))
+                 org-publish-cache)))
     (when free-cache (org-publish-reset-cache))))
 
 (defun org-publish-initialize-cache (project-name)
@@ -1172,23 +1274,23 @@ If FREE-CACHE, empty the cache."
     (make-directory org-publish-timestamp-directory t))
   (unless (file-directory-p org-publish-timestamp-directory)
     (error "Org publish timestamp: %s is not a directory"
-	   org-publish-timestamp-directory))
+           org-publish-timestamp-directory))
 
   (unless (and org-publish-cache
-	       (string= (org-publish-cache-get ":project:") project-name))
+               (string= (org-publish-cache-get ":project:") project-name))
     (let* ((cache-file
-	    (concat
-	     (expand-file-name org-publish-timestamp-directory)
-	     project-name ".cache"))
-	   (cexists (file-exists-p cache-file)))
+            (concat
+             (expand-file-name org-publish-timestamp-directory)
+             project-name ".cache"))
+           (cexists (file-exists-p cache-file)))
 
       (when org-publish-cache (org-publish-reset-cache))
 
       (if cexists (load-file cache-file)
-	(setq org-publish-cache
-	      (make-hash-table :test #'equal :weakness nil :size 100))
-	(org-publish-cache-set ":project:" project-name)
-	(org-publish-cache-set ":cache-file:" cache-file))
+        (setq org-publish-cache
+              (make-hash-table :test #'equal :weakness nil :size 100))
+        (org-publish-cache-set ":project:" project-name)
+        (org-publish-cache-set ":cache-file:" cache-file))
       (unless cexists (org-publish-write-cache-file nil))))
   org-publish-cache)
 
@@ -1209,39 +1311,39 @@ the file including them will be republished as well."
     (error
      "`org-publish-cache-file-needs-publishing' called, but no cache present"))
   (let* ((case-fold-search t)
-	 (key (org-publish-timestamp-filename filename pub-dir pub-func))
-	 (pstamp (org-publish-cache-get key))
-	 (org-inhibit-startup t)
-	 (visiting (find-buffer-visiting filename))
-	 included-files-ctime buf)
+         (key (org-publish-timestamp-filename filename pub-dir pub-func))
+         (pstamp (org-publish-cache-get key))
+         (org-inhibit-startup t)
+         (visiting (find-buffer-visiting filename))
+         included-files-ctime buf)
     (when (equal (file-name-extension filename) "org")
       (setq buf (find-file (expand-file-name filename)))
       (with-current-buffer buf
-	(goto-char (point-min))
-	(while (re-search-forward "^[ \t]*#\\+INCLUDE:" nil t)
-	  (let* ((element (org-element-at-point))
-		 (included-file
+        (goto-char (point-min))
+        (while (re-search-forward "^[ \t]*#\\+INCLUDE:" nil t)
+          (let* ((element (org-element-at-point))
+                 (included-file
                   (and (eq (org-element-type element) 'keyword)
                        (let ((value (org-element-property :value element)))
                          (and value
                               (string-match
-			       "\\`\\(\".+?\"\\|\\S-+\\)\\(?:\\s-+\\|$\\)"
-			       value)
+                               "\\`\\(\".+?\"\\|\\S-+\\)\\(?:\\s-+\\|$\\)"
+                               value)
                               (let ((m (match-string 1 value)))
                                 (org-remove-double-quotes
-				 ;; Ignore search suffix.
+                                 ;; Ignore search suffix.
                                  (if (string-match "\\(::\\(.*?\\)\\)\"?\\'" m)
                                      (substring m 0 (match-beginning 0))
                                    m))))))))
-	    (when included-file
-	      (push (org-publish-cache-ctime-of-src
-		     (expand-file-name included-file))
-		    included-files-ctime)))))
+            (when included-file
+              (push (org-publish-cache-ctime-of-src
+                     (expand-file-name included-file))
+                    included-files-ctime)))))
       (unless visiting (kill-buffer buf)))
     (or (null pstamp)
-	(let ((ctime (org-publish-cache-ctime-of-src filename)))
-	  (or (< pstamp ctime)
-	      (cl-some (lambda (ct) (< ctime ct)) included-files-ctime))))))
+        (let ((ctime (org-publish-cache-ctime-of-src filename)))
+          (or (< pstamp ctime)
+              (cl-some (lambda (ct) (< ctime ct)) included-files-ctime))))))
 
 (defun org-publish-cache-set-file-property
   (filename property value &optional project-name)
@@ -1265,12 +1367,12 @@ be created, unless NO-CREATE is not nil."
   (if project-name (org-publish-initialize-cache project-name))
   (let ((pl (org-publish-cache-get filename)) retval)
     (if pl
-	(if (plist-member pl property)
-	    (setq retval (plist-get pl property))
-	  (setq retval default))
+        (if (plist-member pl property)
+            (setq retval (plist-get pl property))
+          (setq retval default))
       ;; no pl yet:
       (unless no-create
-	(org-publish-cache-set filename (list property default)))
+        (org-publish-cache-set filename (list property default)))
       (setq retval default))
     retval))
 
@@ -1292,11 +1394,11 @@ Returns value on success, else nil."
 (defun org-publish-cache-ctime-of-src (file)
   "Get the ctime of FILE as an integer."
   (let ((attr (file-attributes
-	       (expand-file-name (or (file-symlink-p file) file)
-				 (file-name-directory file)))))
+               (expand-file-name (or (file-symlink-p file) file)
+                                 (file-name-directory file)))))
     (if (not attr) (error "No such file: \"%s\"" file)
       (+ (lsh (car (nth 5 attr)) 16)
-	 (cadr (nth 5 attr))))))
+         (cadr (nth 5 attr))))))
 
 
 (provide 'ox-publish)
-- 
2.8.3


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #6: 0005-ox-publish-Sitemap-preamble-and-postamble.patch --]
[-- Type: text/x-diff, Size: 7669 bytes --]

From 1858dc5401e1d8520aa712983ee8372ff8657e5b Mon Sep 17 00:00:00 2001
From: Rasmus <rasmus@gmx.us>
Date: Fri, 27 May 2016 17:26:03 +0200
Subject: [PATCH 5/6] ox-publish: Sitemap preamble and postamble

* lisp/ox-publish.el (org-publish-sitemap-preamble):
(org-publish-sitemap-postamble): New defcustom.
(org-publish-projects):
(org-publish-org-sitemap): Use new defcustom
---
 lisp/ox-publish.el | 135 ++++++++++++++++++++++++++++++++++++++---------------
 1 file changed, 97 insertions(+), 38 deletions(-)

diff --git a/lisp/ox-publish.el b/lisp/ox-publish.el
index d94ca04..0fde0cc 100644
--- a/lisp/ox-publish.el
+++ b/lisp/ox-publish.el
@@ -239,6 +239,14 @@ a site-map of files or summary page for a given project.
     Format of filenames and directories included in the sitemap.
     See `org-publish-sitemap-file-entry-format' for details.
 
+  `:sitemap-preamble'
+  `:sitemap-postamble'
+
+    Preamble and postamble for sitemap.  Useful for inserting
+    #+OPTIONS: keywords, footers etc.  See
+    `org-publish-sitemap-preamble' for details.
+
+
 If you create a site-map file, adjust the sorting like this:
 
   `:sitemap-sort-folders'
@@ -409,6 +417,35 @@ See also `org-publish-sitemap-file-entry-format'."
   :version "25.2"
   :package-version '(Org . "9.0"))
 
+(defcustom org-publish-sitemap-preamble nil
+  "Sitemap preamble.
+
+Can be either a string, a list of strings, or a function that
+takes a project-plist as an argument and return a string."
+  :group 'org-export-publish
+  :type '(choice
+	  (const :tag "None" nil)
+	  (string :tag "String")
+	  (repeat :tag "List of strings"
+		  (string :tag "String"))
+	  (function :tag "Function"))
+  :version "25.2"
+  :package-version '(Org . "9.0"))
+
+(defcustom org-publish-sitemap-postamble nil
+  "Sitemap postamble.
+
+Can be either a string, a list of strings, or a function that
+takes a project-plist as an argument and return a string."
+  :group 'org-export-publish
+  :type '(choice
+	  (const :tag "None" nil)
+	  (string :tag "String")
+	  (repeat :tag "List of strings"
+		  (string :tag "String"))
+	  (function :tag "Function"))
+  :version "25.2"
+  :package-version '(Org . "9.0"))
 
 
 \f
@@ -771,22 +808,28 @@ If `:auto-sitemap' is set, publish the sitemap too.  If
       ;; Each project uses its own cache file.
       (org-publish-initialize-cache (car project))
       (when  (plist-get project-plist :auto-sitemap)
-        (let ((sitemap-filename
-               (or (plist-get project-plist :sitemap-filename)
-                   "sitemap.org"))
-              (sitemap-function
-               (or (plist-get project-plist :sitemap-function)
-                   #'org-publish-org-sitemap))
-              (org-publish-sitemap-date-format
-               (or (plist-get project-plist :sitemap-date-format)
-                   org-publish-sitemap-date-format))
-              (org-publish-sitemap-file-entry-format
-               (or (plist-get project-plist :sitemap-file-entry-format)
-                   org-publish-sitemap-file-entry-format))
-              (org-publish-sitemap-dir-entry-format
-               (or (plist-get project-plist :sitemap-dir-entry-format)
-                   org-publish-sitemap-dir-entry-format))
-          (funcall sitemap-function project sitemap-filename)))
+	(let ((sitemap-filename
+	       (or (plist-get project-plist :sitemap-filename)
+		   "sitemap.org"))
+	      (sitemap-function
+	       (or (plist-get project-plist :sitemap-function)
+		   #'org-publish-org-sitemap))
+	      (org-publish-sitemap-date-format
+	       (or (plist-get project-plist :sitemap-date-format)
+		   org-publish-sitemap-date-format))
+	      (org-publish-sitemap-file-entry-format
+	       (or (plist-get project-plist :sitemap-file-entry-format)
+		   org-publish-sitemap-file-entry-format))
+	      (org-publish-sitemap-dir-entry-format
+	       (or (plist-get project-plist :sitemap-dir-entry-format)
+		   org-publish-sitemap-dir-entry-format))
+	      (org-publish-sitemap-preamble
+	       (or (plist-get project-plist :sitemap-preamble)
+		   org-publish-sitemap-preamble))
+	      (org-publish-sitemap-postamble
+	       (or (plist-get project-plist :sitemap-postamble)
+		   org-publish-sitemap-postamble)))
+	  (funcall sitemap-function project sitemap-filename)))
       ;; Publish all files from PROJECT excepted "theindex.org".  Its
       ;; publishing will be deferred until "theindex.inc" is
       ;; populated.
@@ -812,31 +855,47 @@ If `:auto-sitemap' is set, publish the sitemap too.  If
 Optionally set the filename of the sitemap with SITEMAP-FILENAME.
 Default for SITEMAP-FILENAME is `sitemap.org'."
   (let* ((project-plist (cdr project))
-         (dir (file-name-as-directory
-               (plist-get project-plist :base-directory)))
-         (sitemap-filename (concat dir (or sitemap-filename "sitemap.org")))
-         (files (nreverse
-                 ;; Sitemap shouldn't list itself.
-                 (cl-remove sitemap-filename
-                            (org-publish-get-base-files
-                             project
-                             (plist-get project-plist :exclude) t)
-                            :test #'equal)))
-         (sitemap-title (or (plist-get project-plist :sitemap-title)
-                            (concat "Sitemap for project " (car project)))))
+	 (dir (file-name-as-directory
+	       (plist-get project-plist :base-directory)))
+	 (sitemap-filename (concat dir (or sitemap-filename "sitemap.org")))
+	 (files (nreverse
+		 ;; Sitemap shouldn't list itself.
+		 (cl-remove sitemap-filename
+			    (org-publish-get-base-files
+			     project
+			     (plist-get project-plist :exclude) t)
+			    :test #'equal)))
+	 (sitemap-title (or (plist-get project-plist :sitemap-title)
+			    (concat "Sitemap for project " (car project))))
+	 (prepare-pre-or-postamble (lambda (pre-or-postamble)
+				     (cond ((not pre-or-postamble) nil)
+					   ((stringp pre-or-postamble) pre-or-postamble)
+					   ((listp pre-or-postamble)
+					    (mapconcat 'identity preamble "\n"))
+					   ((functionp pre-or-postamble)
+					    (funcall pre-or-postamble project-plist))
+					   (t (error (concat "unknown `:sitemap-preamble' or "
+							     "`:sitemap-postamble' format")))))))
     (with-temp-file sitemap-filename
       (insert
        (mapconcat
-        'identity
-        (list
-         (format "#+TITLE: %s" sitemap-title)
-         ;; Call function to build sitemap based on files and the project-plist.
-         (let* ((style (or (plist-get project-plist :sitemap-style) 'tree))
-                (fun (intern (format "org-publish-org-sitemap-as-%s" style))))
-           (unless (functionp fun)
-             (error "Unknown function %s for :sitemap-style %s."
-                    fun style))
-           (funcall fun files project-plist))) "\n")))))
+	'identity
+	(list
+	 (format "#+TITLE: %s" sitemap-title)
+	 ;; Insert sitemap-preamble.
+	 (funcall prepare-pre-or-postamble
+		  (plist-get project-plist :sitemap-preamble))
+	 ;; Call function to build sitemap based on files and the project-plist.
+	 (let* ((style (or (plist-get project-plist :sitemap-style) 'tree))
+		(fun (intern (format "org-publish-org-sitemap-as-%s" style))))
+	   (unless (functionp fun)
+	     (error "Unknown function %s for :sitemap-style %s."
+		    fun style))
+	   (funcall fun files project-plist))
+	 ;; Insert sitemap-postamble.
+	 (funcall prepare-pre-or-postamble
+		  (plist-get project-plist :sitemap-postamble)))
+        "\n")))))
 
 (defun org-publish-org-sitemap-as-list (files project-plist)
   "Insert FILES as simple list separated by newlines.
-- 
2.8.3


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #7: 0006-org.texi-Document-ox-publish-sitemap-changes.patch --]
[-- Type: text/x-diff, Size: 2398 bytes --]

From f4661b81ab3715e238f52def586a9c89ecb86a08 Mon Sep 17 00:00:00 2001
From: Rasmus <rasmus@gmx.us>
Date: Fri, 27 May 2016 15:16:20 +0200
Subject: [PATCH 6/6] org.texi: Document ox-publish sitemap changes

* doc/org.texi (Sitemap): Update documentation.
---
 doc/org.texi | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/doc/org.texi b/doc/org.texi
index 9d89975..3f96eaf 100644
--- a/doc/org.texi
+++ b/doc/org.texi
@@ -14604,8 +14604,9 @@ becomes @file{sitemap.html}).
 
 @item @code{:sitemap-function}
 @tab Plug-in function to use for generation of the sitemap.
-Defaults to @code{org-publish-org-sitemap}, which generates a plain list
-of links to all files in the project.
+Defaults to @code{org-publish-org-sitemap}, which generates a plain list of
+links to all files in the project.  See further details in
+@code{org-publish-project-alist}.
 
 @item @code{:sitemap-sort-folders}
 @tab Where folders should appear in the sitemap.  Set this to @code{first}
@@ -14624,12 +14625,9 @@ a file is retrieved with @code{org-publish-find-date}.
 @tab Should sorting be case-sensitive?  Default @code{nil}.
 
 @item @code{:sitemap-file-entry-format}
-@tab With this option one can tell how a sitemap's entry is formatted in the
-sitemap.  This is a format string with some escape sequences: @code{%t} stands
-for the title of the file, @code{%a} stands for the author of the file and
-@code{%d} stands for the date of the file.  The date is retrieved with the
-@code{org-publish-find-date} function and formatted with
-@code{org-publish-sitemap-date-format}.  Default @code{%t}.
+@item @code{:sitemap-dir-entry-format}
+@tab With this option one can tell how the entries of the sitemap is
+formatted.  See @code{org-publish-sitemap-file-entry-format} for details.
 
 @item @code{:sitemap-date-format}
 @tab Format string for the @code{format-time-string} function that tells how
@@ -14641,6 +14639,12 @@ a sitemap entry's date is to be formatted.  This property bypasses
 Useful to have cool URIs (see @uref{http://www.w3.org/Provider/Style/URI}).
 Defaults to @code{nil}.
 
+@item @code{:sitemap-preamble}
+@item @code{:sitemap-postamble}
+@tab Preamble and postamble for sitemap.  Useful for inserting
+    @code{#+OPTIONS}, footers etc.  See @code{org-publish-sitemap-preamble}
+    for details.
+
 @end multitable
 
 @node Generating an index
-- 
2.8.3


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [ox-publish, patch] More flexible sitemaps
  2016-05-27 16:41   ` Rasmus
@ 2016-06-01 15:34     ` Nicolas Goaziou
  2016-07-05 11:08       ` Robert Klein
  2016-07-07  9:03       ` Rasmus
  0 siblings, 2 replies; 8+ messages in thread
From: Nicolas Goaziou @ 2016-06-01 15:34 UTC (permalink / raw)
  To: Rasmus; +Cc: emacs-orgmode

Hello,

Rasmus <rasmus@gmx.us> writes:

> This was by far the hardest part...

Thank you. Some comments follow.

> +(defun org-publish-find-property (file property &optional reset)
> +  "Find the PROPERTY of FILE in project.
> +PROPERTY can be a string or a symbol-property."

Could you also document RESET argument?

> +  (unless (or (file-directory-p file) (directory-name-p file))

What is `directory-name-p'?

> +    (let ((prop (cond ((stringp property)
> +                       (intern (downcase (format ":%s" property))))
> +                      (t property))))

The following might be more robust:

  (if (keywordp property) property
    (intern (downcase (format ":%s" property)))

> +      (and (not reset) (org-publish-cache-get-file-property file prop nil t))

(unless reset ...)

Anyway, you don't seem to use the return value. If this is used for
side-effect only, you may want to call `org-publish-initialize-cache'
instead.

> +      (let* ((org-inhibit-startup t)
> +             (visiting (find-buffer-visiting file))
> +             (buffer (or visiting (find-file-noselect file)))
> +             (case-fold-search t))
> +        (with-current-buffer buffer
> +          ;; protect local variables in open buffers
> +          (org-export-with-buffer-copy
> +           (let* ((backends (append (list nil)
> +                                    (mapcar 'org-export-backend-name
> +                                            org-export-registered-backends)))
> +                  (value (cl-loop for backend in backends
> +                                  when (org-no-properties
> +                                        (org-element-interpret-data
> +                                         (plist-get (org-export-get-environment backend)
> +                                                    prop)))
> +                                  return it)))

Return value of `org-element-interpret-data' is never nil so the loop
always returns the first back-end.

In any case, this is sub-optimal since common keywords are retrieved for
each back-end. Also, two back-ends could use the same keyword, with
different values (e.g., using `parsed' or not).

One option could be to change the policy about keywords in ox.el and
move KEYWORDS and SUBTITLE there. Unfortunately, there is still a change
to miss some cases.

Another option would be to provide BACKEND to sitemap-function. I think
it can be backward-compatible if we make it an optional argument.

> +(defcustom org-publish-sitemap-file-entry-format "%- [[file:%link][%TITLE]]"
>    "Format string for site-map file entry.
> -You could use brackets to delimit on what part the link will be.
>  
> -%t is the title.
> -%a is the author.
> -%d is the date formatted using `org-publish-sitemap-date-format'."
> +Most keywords can be accessed using a the name of the keyword
> +prefixed with \"%\", e.g. \"%TITLE\" or \"%KEYWORDS\".  In
> +addition, the following keywords are defined.
> +
> +  %fullpath
> +    The full path of the file.
> +
> +  %relpath
> +    The relative path of the file.
> +
> +  %filename
> +  %f
> +    The filename.
> +
> +  %link
> +  %l
> +    The link.
> +
> +  %*
> +    Leveled headline relative to the base directory.
> +
> +  %*
> +    Leveled item relative to the base directory.
> +
> +  %author
> +    The author of the document, the project, or `user-full-name'.
> +
> +  %date
> +     Date formatted using `org-publish-sitemap-date-format'.
> +
> +See also `org-publish-sitemap-dir-entry-format'."

Could it be possible to use single-char placeholders and `format-spec',
which is more robust than replace-match (e.g., it is possibl to escape
percent signs)...

> +(defcustom org-publish-sitemap-dir-entry-format "%- %f"
> +  "Format string for site-map file entry.
> +
> +The following keywords are defined.
> +
> +  %fullpath
> +    The full path of the file.
> +
> +  %relpath
> +    The relative path of the file.
> +
> +  %filename
> +  %f
> +    The filename.
> +
> +  %link
> +  %l
> +    The link.
> +
> +  %*
> +    Leveled headline relative to the base directory.
> +
> +  %*
> +    Leveled item relative to the base directory.
> +
> +  %author
> +    The author of the document, the project, or `user-full-name'.
> +
> +  %date
> +     Date formatted using `org-publish-sitemap-date-format'.

Ditto.

> @@ -1292,11 +1394,11 @@ Returns value on success, else nil."
>  (defun org-publish-cache-ctime-of-src (file)
>    "Get the ctime of FILE as an integer."
>    (let ((attr (file-attributes
> -	       (expand-file-name (or (file-symlink-p file) file)
> -				 (file-name-directory file)))))
> +               (expand-file-name (or (file-symlink-p file) file)
> +                                 (file-name-directory file)))))

Maybe 

  (file-truename file)

instead.

> +  `:sitemap-preamble'
> +  `:sitemap-postamble'
> +
> +    Preamble and postamble for sitemap.  Useful for inserting
> +    #+OPTIONS: keywords, footers etc.  See
> +    `org-publish-sitemap-preamble' for details.

Note there are not much details in `org-publish-sitemap-preamble' either.

> +(defcustom org-publish-sitemap-preamble nil
> +  "Sitemap preamble.
> +
> +Can be either a string, a list of strings, or a function that
> +takes a project-plist as an argument and return a string."

"returns"

Is allowing both strings and lists of strings useful enough?

> +Can be either a string, a list of strings, or a function that
> +takes a project-plist as an argument and return a string."

See above.

> +	 (sitemap-title (or (plist-get project-plist :sitemap-title)
> +			    (concat "Sitemap for project " (car project))))
> +	 (prepare-pre-or-postamble (lambda (pre-or-postamble)

I suggest to move lambda on the next line and use a less mouthful name
for the argument.

> +				     (cond ((not pre-or-postamble) nil)
> +					   ((stringp pre-or-postamble) pre-or-postamble)
> +					   ((listp pre-or-postamble)
> +					    (mapconcat 'identity preamble "\n"))

#'identity

> +					   ((functionp pre-or-postamble)
> +					    (funcall pre-or-postamble project-plist))
> +					   (t (error (concat "unknown `:sitemap-preamble' or "
> +							     "`:sitemap-postamble' format")))))))

I think `concat' is not necessary with the indentation rule suggested
above. Also: "Unknown".

> +	 ;; Call function to build sitemap based on files and the project-plist.
> +	 (let* ((style (or (plist-get project-plist :sitemap-style) 'tree))
> +		(fun (intern (format "org-publish-org-sitemap-as-%s" style))))

Side note : I think this is a bit too smart. It prevents, e.g., from
grepping for a function name. Maybe 

  (if (eq style 'something) #'... #'....)

would be better.

> +	   (unless (functionp fun)
> +	     (error "Unknown function %s for :sitemap-style %s."
> +		    fun style))

I know this is not directly related to your patch, but it should
probably be `user-error'. Also, error messages are not expected to end
on a period.

> +@item @code{:sitemap-preamble}
> +@item @code{:sitemap-postamble}
> +@tab Preamble and postamble for sitemap.  Useful for inserting
> +    @code{#+OPTIONS}, footers etc.  See @code{org-publish-sitemap-preamble}
> +    for details.

I don't understand the "useful for inserting @code{#+OPTIONS}" part.
Maybe some examples could be useful. This part of the manual is rather
terse.

Regards,

-- 
Nicolas Goaziou

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ox-publish, patch] More flexible sitemaps
  2016-06-01 15:34     ` Nicolas Goaziou
@ 2016-07-05 11:08       ` Robert Klein
  2016-07-06 11:17         ` Rasmus
  2016-07-07  9:03       ` Rasmus
  1 sibling, 1 reply; 8+ messages in thread
From: Robert Klein @ 2016-07-05 11:08 UTC (permalink / raw)
  To: Nicolas Goaziou; +Cc: emacs-orgmode, Rasmus

Hi,

are those Patches still being worked on?

Thanks and best regards
Robert



On Wed, 01 Jun 2016 17:34:56 +0200
Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:

> Hello,
> 
> Rasmus <rasmus@gmx.us> writes:
> 
> > This was by far the hardest part...  
> 
> Thank you. Some comments follow.
> 
> > +(defun org-publish-find-property (file property &optional reset)
> > +  "Find the PROPERTY of FILE in project.
> > +PROPERTY can be a string or a symbol-property."  
> 
> Could you also document RESET argument?
> 

[rest deleted from reply]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ox-publish, patch] More flexible sitemaps
  2016-07-05 11:08       ` Robert Klein
@ 2016-07-06 11:17         ` Rasmus
  0 siblings, 0 replies; 8+ messages in thread
From: Rasmus @ 2016-07-06 11:17 UTC (permalink / raw)
  To: emacs-orgmode

Robert Klein <roklein@roklein.de> writes:

> are those Patches still being worked on?

Yes, but they haven't been pushed yet.

Rasmus

-- 
May the Force be with you

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ox-publish, patch] More flexible sitemaps
  2016-06-01 15:34     ` Nicolas Goaziou
  2016-07-05 11:08       ` Robert Klein
@ 2016-07-07  9:03       ` Rasmus
  2016-07-20  7:56         ` Nicolas Goaziou
  1 sibling, 1 reply; 8+ messages in thread
From: Rasmus @ 2016-07-07  9:03 UTC (permalink / raw)
  To: emacs-orgmode; +Cc: roklein

Hi,

Sorry about the slow reply.  Sometimes there's not enough time.

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

>> +  (unless (or (file-directory-p file) (directory-name-p file))
>
> What is `directory-name-p'?

Oh, it's new in Emacs-25.  Thanks for pointing that out!



The following is about the index function, which is the most "difficult"
part.

>> +      (let* ((org-inhibit-startup t)
>> +             (visiting (find-buffer-visiting file))
>> +             (buffer (or visiting (find-file-noselect file)))
>> +             (case-fold-search t))
>> +        (with-current-buffer buffer
>> +          ;; protect local variables in open buffers
>> +          (org-export-with-buffer-copy
>> +           (let* ((backends (append (list nil)
>> +                                    (mapcar 'org-export-backend-name
>> +                                            org-export-registered-backends)))
>> +                  (value (cl-loop for backend in backends
>> +                                  when (org-no-properties
>> +                                        (org-element-interpret-data
>> + (plist-get (org-export-get-environment backend)
>> +                                                    prop)))
>> +                                  return it)))
>
> Return value of `org-element-interpret-data' is never nil so the loop
> always returns the first back-end.
>
> In any case, this is sub-optimal since common keywords are retrieved for
> each back-end. Also, two back-ends could use the same keyword, with
> different values (e.g., using `parsed' or not).
>
> One option could be to change the policy about keywords in ox.el and
> move KEYWORDS and SUBTITLE there. Unfortunately, there is still a change
> to miss some cases.
>
> Another option would be to provide BACKEND to sitemap-function. I think
> it can be backward-compatible if we make it an optional argument.

I obviously agree with your criticism.  It’s not obvious how to do this
properly.

If we provide a backend we’d have to move the index preparation to
beginning of each export process as :publishing-function can be a list.
Also, I’m not sure how we’d know about the backend.  Before the exporting
starts, we at most know the names of the functions right?  If one of the
publishing functions is anonymous we don’t even have that.

Perhaps the best way is to move keywords back to ox, though I’d rather opt
for something else.

>> +	 ;; Call function to build sitemap based on files and the project-plist.
>> +	 (let* ((style (or (plist-get project-plist :sitemap-style) 'tree))
>> +		(fun (intern (format "org-publish-org-sitemap-as-%s" style))))
>
> Side note : I think this is a bit too smart. It prevents, e.g., from
> grepping for a function name. Maybe 
>
>   (if (eq style 'something) #'... #'....)
>
> would be better.

The point is that it should be easy to supply your own functions.  All
sorts of requirements for the index/sitemap could come up, and it is
important to not limit to the tree-style and the flat-style.  I might want
to provide an index ordered by keywords, say.

Perhaps styles should be stores in an alist with elements like

        (STYLE STYLE-FUNCTION) ? 

>> +@item @code{:sitemap-preamble}
>> +@item @code{:sitemap-postamble}
>> +@tab Preamble and postamble for sitemap.  Useful for inserting
>> +    @code{#+OPTIONS}, footers etc.  See @code{org-publish-sitemap-preamble}
>> +    for details.
>
> I don't understand the "useful for inserting @code{#+OPTIONS}" part.
> Maybe some examples could be useful. This part of the manual is rather
> terse.

It might be useful to suppress the printing of the title or the author or
similar.

Rasmus

-- 
I feel emotional landscapes they puzzle me

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ox-publish, patch] More flexible sitemaps
  2016-07-07  9:03       ` Rasmus
@ 2016-07-20  7:56         ` Nicolas Goaziou
  0 siblings, 0 replies; 8+ messages in thread
From: Nicolas Goaziou @ 2016-07-20  7:56 UTC (permalink / raw)
  To: Rasmus; +Cc: emacs-orgmode, roklein

Hello,

Rasmus <rasmus@gmx.us> writes:

> Sorry about the slow reply.  Sometimes there's not enough time.

So true.

> I obviously agree with your criticism.  It’s not obvious how to do this
> properly.
>
> If we provide a backend we’d have to move the index preparation to
> beginning of each export process as :publishing-function can be a list.
> Also, I’m not sure how we’d know about the backend.  Before the exporting
> starts, we at most know the names of the functions right?  If one of the
> publishing functions is anonymous we don’t even have that.
>
> Perhaps the best way is to move keywords back to ox, though I’d rather opt
> for something else.

You're right, "ox-publish" is mostly back-end agnostic so the
information is not readily available.

However, we're creating an Org file here. We don't need to have parsed
values for keywords. So, what about simply looking for, e.g., #+SUBTITLE
value in the document and use it instead as a template replacement? IOW,
what about eschewing so-called "environment" altogether?

>>> +	 ;; Call function to build sitemap based on files and the project-plist.
>>> +	 (let* ((style (or (plist-get project-plist :sitemap-style) 'tree))
>>> +		(fun (intern (format "org-publish-org-sitemap-as-%s" style))))
>>
>> Side note : I think this is a bit too smart. It prevents, e.g., from
>> grepping for a function name. Maybe 
>>
>>   (if (eq style 'something) #'... #'....)
>>
>> would be better.
>
> The point is that it should be easy to supply your own functions.  All
> sorts of requirements for the index/sitemap could come up, and it is
> important to not limit to the tree-style and the flat-style.

But that's outside the scope of the patch currently considered. My point
is that we should avoid forging function names as a UI.

> I might want to provide an index ordered by keywords, say.
>
> Perhaps styles should be stores in an alist with elements like
>
>         (STYLE STYLE-FUNCTION) ?

Some functions may handle multiple styles, as `org-publish-org-sitemap'
does. Besides, any user-provided sitemap function is likely to handle
its own style without relying on :sitemap-style property.

Therefore, providing :sitemap-function and :sitemap-style is enough,
there's no need to merge them together.

WDYT?


Regards,

-- 
Nicolas Goaziou

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2016-07-20  7:56 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-05-19 15:39 [ox-publish, patch] More flexible sitemaps Rasmus
2016-05-22 22:58 ` Nicolas Goaziou
2016-05-27 16:41   ` Rasmus
2016-06-01 15:34     ` Nicolas Goaziou
2016-07-05 11:08       ` Robert Klein
2016-07-06 11:17         ` Rasmus
2016-07-07  9:03       ` Rasmus
2016-07-20  7:56         ` Nicolas Goaziou

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).