From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Maus Subject: Publish atom feeds based on Org files Date: Fri, 19 Mar 2010 23:00:23 +0100 Message-ID: <87ocikm0p4.wl%dmaus@ictsoc.de> Mime-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: multipart/mixed; boundary="===============0286315033==" Return-path: Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1NskFA-0008VW-Su for emacs-orgmode@gnu.org; Fri, 19 Mar 2010 18:00:36 -0400 Received: from [140.186.70.92] (port=44859 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1NskF9-0008US-2u for emacs-orgmode@gnu.org; Fri, 19 Mar 2010 18:00:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1NskF6-0002Vz-2C for emacs-orgmode@gnu.org; Fri, 19 Mar 2010 18:00:34 -0400 Received: from mysql1.xlhost.de ([213.202.242.106]:37864) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1NskF5-0002Vh-IU for emacs-orgmode@gnu.org; Fri, 19 Mar 2010 18:00:32 -0400 Received: from localhost (localhost [127.0.0.1]) by mysql1.xlhost.de (Postfix) with ESMTP id 81ACD11F26 for ; Fri, 19 Mar 2010 23:00:28 +0100 (CET) Received: from mysql1.xlhost.de ([127.0.0.1]) by localhost (mysql1.xlhost.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1vobXznMMIEx for ; Fri, 19 Mar 2010 23:00:28 +0100 (CET) Received: from thinkpad.ictsoc.de (unknown [88.130.162.190]) by mysql1.xlhost.de (Postfix) with ESMTPSA id A112311F23 for ; Fri, 19 Mar 2010 23:00:25 +0100 (CET) List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Org Mode --===============0286315033== Content-Type: multipart/signed; boundary="pgp-sign-Multipart_Fri_Mar_19_23:00:23_2010-2"; protocol="application/pgp-signature" Content-Transfer-Encoding: 7bit --pgp-sign-Multipart_Fri_Mar_19_23:00:23_2010-2 Content-Type: multipart/mixed; boundary="Multipart_Fri_Mar_19_23:00:23_2010-1" Content-Transfer-Encoding: 7bit --Multipart_Fri_Mar_19_23:00:23_2010-1 Content-Type: text/plain; charset=US-ASCII Hello all, Carving another stone that might fit into the cathedral known as Emacs/Orgmode I am glad to present attached file org-atom.el that provides an exporting and a publishing function to create atom feeds based on Org files. To use org-atom.el you need the library atom-syndication.el, an elisp implementation of the atom syndication format that can be found in it's github repository. http://github.com/dmj/atom-syndication/ Below you'll find a reduced ascii export of org-atom's project page (currently at http://ictsoc.de/code/org-atom.html) that gives a brief explanation of how to setup Org files for feed export. Although both, atom-syndication.el and org-atom.el, are not quite finshed yet, I decided to make the community aware of org-atom in the hope some might try it out and give some feedback and ideas how to extend org-atom's functionality. My personal goal for org-atom is to make use of all functionality that the atom syndication format provides: E.g. what popped into my mind is that having an atom feed for Worg, especially the hacking-section would be incredible. Anyway: If you try org-atom and create feeds I'd like you to ask to validate the feeds even if your feed reader accepts it. Validating the feed can be achived in two ways: - use the w3c feed validator at http://validator.w3.org/feed/ It will not just validate the feed but also provide an short summary why a feed is not valid (if so) and suggestions on how to improve the feed. - use nxml-mode with the Relax NG Compact Syntax Grammar The syntax grammar is shipped with atom-syndication.el in the file atom-syndication.rnc. Just open the feed file, maybe turn on nxml mode (M-x nxml-mode RET) and set the scheme file to atom-syndication.rnc (M-x rng-set-schema-file-and-validate RET /path/to/atom-syndication.rnc RET) And here's the brief summary of the functions: Publish atom feeds based on Org files ===================================== Author: David Maus Date: 2010-03-19 22:23:01 CET Table of Contents ================= 1 About 2 Exporting and publishing 2.1 New infile export options 2.2 Mapping of other Org entities to atom metadata elements 2.3 Exporting a single Org file 2.4 Publishing a feed for multiple files in a publishing project 1 About ~~~~~~~~ This addon for Emacs Org mode, an Emacs Mode for Notes, Project Planning, and Authoring, provides export and publishing functions to create an atom feed based on Org files. This file provides some information on the concept, the state of implementation and a maybe some ideas how to extend the exporter. 2 Exporting and publishing ~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2.1 New infile export options ============================== =#+FEED_URL=: URL of the published feed. This option MUST be present when exporting an Org file using =org-export-as-atom=. =#+FEED_CONTENT_URL=: URL of the published html file. When exporting an Org file using =org-export-as-atom= and the customization variable =org-atom-publish-content= is not set, this option MUST be present. =#+FEED_ID=: Unique identifier of the feed. If not set, the feed url is used as id. =#+FEED_MAP_ENTRIES=: A TAGS/PROP/TODO query that matches entry headlines. If not set, all headlines are exported as atom entries. 2.2 Mapping of other Org entities to atom metadata elements ============================================================ In addition org-atom maps other Org mode entities to atom metadata elements. Following table provides a comprehensive list of all currently supported metadata elements and their mapped Org entities. 2.2.1 atom:feed metadata ------------------------- Element Description Org entity ------------------------+---------------------------------+------------------------------------- atom:author name name of author #+AUTHOR atom:generator name name of generator const: `org-atom-generator-name' atom:generator version version of generator const: `org-atom-generator-version' atom:id unique id of feed #+FEED_ID atom:link rel="self" link to the published feed #+FEED_URL atom:updated time when feed was updated (current-time) on export atom:subtitle subtitle or description of feed #+DESCRIPTION atom:title title of feed #+TITLE 2.2.2 atom:entry metadata -------------------------- Element Description Org entity ---------------------------+-------------------------------+--------------------------------------------- atom:author name name of author #+AUTHOR atom:content content of feed entry subtree of feed headline atom:id unique id of entry ID property atom:link rel="alternate" link to entry content #+FEED_CONTENT_URL atom:published time when entry was published property `org-atom-published-property-name' atom:title entry title headline title w/o TODO keywords and tags atom:updated time when entry was updated property `org-atom-updated-property-name' 2.3 Exporting a single Org file ================================ A minimal setup of an Org file that will be published as atom feed could look like this: ,---- | #+TITLE: Example file for feed export | #+AUTHOR: John Doe | #+EMAIL: john@example.tld | # | #+FEED_URL: http://example.tld/~john/ | #+FEED_CONTENT_URL: http://example.tld/~john/news.html | #+FEED_ID: a20d6ebc-7bab-4307-8b1c-7e29ac49bf66 | #+FEED_MAP_ENTRIES: LEVEL=2 | # | | * This entry is not exported as feed | | * Everything below this is! | | ** Breaking news: Org now publishes atom feeds! | :PROPERTIES: | :ID: ca13acc9-c14c-4707-b0e9-7f35687a4473 | :Created: [2010-03-19 Fr 15:55] | :END: | | Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean | commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus | et magnis dis parturient montes, nascetur ridiculus mus. Donec quam | felis, ultricies nec, pellentesque eu, pretium quis, sem. Nulla | consequat massa quis enim. Donec pede justo, fringilla vel, aliquet | nec, vulputate eget, arcu. In enim justo, rhoncus ut, imperdiet a, | venenatis vitae, justo. Nullam dictum felis eu pede mollis | pretium. `---- Calling =org-export-as-atom= on this file produces an atom feed: ,---- | Example file for feed export | Org/Atom | urn:uuid:a20d6ebc-7bab-4307-8b1c-7e29ac49bf66 | 2010-03-19T22:13:30+01:00 | | John Doe | 2010-03-19T15:55:00+01:00 | <div id="outline-container-1" class="outline-3"> | <h3 id="sec-1"><a name="ID-ca13acc9-c14c-4707-b0e9-7f35687a4473" id="ID-ca13acc9-c14c-4707-b0e9-7f35687a4473"></a>Breaking news: Org now publishes atom feeds! </h3> | <div class="outline-text-3" id="text-1"> | | | <p> | Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean | commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus | et magnis dis parturient montes, nascetur ridiculus mus. Donec quam | felis, ultricies nec, pellentesque eu, pretium quis, sem. Nulla | consequat massa quis enim. Donec pede justo, fringilla vel, aliquet | nec, vulputate eget, arcu. In enim justo, rhoncus ut, imperdiet a, | venenatis vitae, justo. Nullam dictum felis eu pede mollis | pretium. | </p></div> | </div> | | | | <p>Breaking news: Org now publishes atom feeds! | </p> | 2010-03-19T15:55:00+01:00 | urn:uuid:ca13acc9-c14c-4707-b0e9-7f35687a4473 `---- 2.4 Publishing a feed for multiple files in a publishing project ================================================================= To publish a feed for multiple Org files in a publishing project org-atom provides the function =org-atom-publish-feed-index= that can be plugged in a project's publishing alist as the =:index-function=: ,---- | (setq org-publish-project-alist | '( | ... | ("news-feed" | :base-directory "~/website/" | :base-extension "org" | :publishing-directory "~/website/" | ;; new property :publishing-url | :publishing-url "http://example.tld/~john/" | :auto-index t | :index-filename "news.atom" | :index-title "News feed for john@example.tld" | :index-function org-atom-publish-feed-index | ;; feed specific settings | :feed-id "6ea57592-69f2-4ef8-b44d-b7a511bd2fe8") | ... | )) `---- Please note the new publishing property =:publishing-url=: Instead of setting the feed url (=:feed-url=) in this alist and the content urls in each file =:publishing-url= is used to derive the url of the feed (assuming the feed is published under the same url) and the content url of the files. Running =org-publish= lets =org-atom-publish-feed-index= interate over all files in the project alist and create a combined feed for all entries of all processed files. -- David -- OpenPGP... 0x99ADB83B5A4478E6 Jabber.... dmjena@jabber.org Email..... dmaus@ictsoc.de --Multipart_Fri_Mar_19_23:00:23_2010-1 Content-Type: application/octet-stream; type=emacs-lisp Content-Disposition: attachment; filename="org-atom.el" Content-Transfer-Encoding: 7bit ;;; org-atom.el --- Publish atom feeds based on Org files. ;; ;; Author: David Maus ;; Keywords: hypermedia, outline ;; Version: 0.1beta ;; ;; Copyright (C) 2010 by David Maus ;; ;; This file is NOT part of Gnu Emacs. ;; ;; This program is free software: you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation, either version 3 of the License, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public License ;; along with this program. If not, see . ;; ;;; Commentary: ;; ;; This addon for Emacs Org mode, an Emacs Mode for Notes, Project ;; Planning, and Authoring, provides export and publishing functions to ;; create an atom feed based on Org files. ;; ;;; Code: (require 'atom-syndication) (defconst org-atom-uuid-regexp "^[[:xdigit:]]\\{8\\}\\(-[[:xdigit:]]\\{4\\}\\)\\{3\\}-[[:xdigit:]]\\{12\\}$" "Regular expression matching a uuid.") (defconst org-atom-infile-options '(("FEED_MAP_ENTRIES" :feed-map-entries) ("FEED_ID" :feed-id) ("FEED_URL" :feed-url) ("FEED_CONTENT_URL" :feed-content-url))) (defconst org-atom-generator-name "Org/Atom" "Name of the atom generator.") (defconst org-atom-generator-version "0.1beta" "Version string of the atom generator.") (defcustom org-atom-feed-extension "atom" "Extension of feed output file." :type 'string :group 'org-export-atom) (defcustom org-atom-published-property-name "Created" "Name of property for publication date." :type 'string :group 'org-export-atom) (defcustom org-atom-updated-property-name "Updated" "Name of property for date when entry was updated." :type 'string :group 'org-export-atom) (defcustom org-atom-publish-content t "Publish feed content. If unset only publish link to content." :type 'boolean :group 'org-export-atom) (defcustom org-atom-prefer-urn-uuid t "Create iri with urn:uuid prefix when id looks like a uuid." :type 'boolean :group 'org-export-atom) (defun org-export-as-atom (&optional ext-plist to-buffer body-only pub-dir) "Export outline as atom feed. EXT-PLIST is a property list with external parameters overriding org-mode's default settings, but still inferior to file-local settings. When TO-BUFFER is non-nil, create a buffer with that name and export to that buffer. If TO-BUFFER is the symbol `string', don't leave any buffer behind but just return the resulting atom feed as a string. When BODY-ONLY is set, return only the atom:entry elements. When PUB-DIR is set, use this as the publishing directory." (interactive) (let* ((opt-plist (org-combine-plists (org-default-export-plist) ext-plist (org-infile-export-plist))) (feed-url (org-trim (or (plist-get opt-plist :feed-url) ""))) (feed-content-url (org-trim (or (plist-get opt-plist :feed-content-url) ""))) (feed-id (org-trim (or (plist-get opt-plist :feed-id) feed-url))) (feed-map-entries (org-trim (or (plist-get opt-plist :feed-map-entries) ""))) (author (plist-get opt-plist :author)) (email (plist-get opt-plist :email)) (description (or (plist-get opt-plist :feed-description) (plist-get opt-plist :description))) (feed-title (or (plist-get opt-plist :feed-title) (plist-get opt-plist :title))) (feed-file (if (buffer-file-name) (concat (file-name-sans-extension (file-truename (buffer-file-name))) "." org-atom-feed-extension))) (body-only (or body-only (plist-get opt-plist :body-only))) (atom-syndication-construct-text-html-function 'org-atom-htmlize) entries feed) ;; check mandatory options (when (and (not body-only) (string= feed-url "")) (error "Missing url for feed")) ;; atom entry w/o content MUST have link pointing to the content (when (or (not org-atom-publish-content) (string= feed-content-url "")) (error "Missing url for feed content")) (unless to-buffer (setq to-buffer (if feed-file (or (find-buffer-visiting feed-file) (find-file-noselect feed-file)) (error "Need a file name to be able to export"))) (with-current-buffer to-buffer (erase-buffer))) ;; prepare headlines (when (> (length (org-map-entries 'org-atom-prepare-headline feed-map-entries)) 0) (message "Exporting...") ;; there are entries in this file (setq entries (org-map-entries '(lambda () (append (if body-only (list (list 'author author))) (org-atom-export-headline feed-url feed-content-url))) feed-map-entries)) (setq feed (if body-only (mapconcat 'atom-syndication-element-entry entries "\n") (atom-syndication-element-feed (append (unless (string= description "") (list (list 'subtitle (org-trim description)))) (list (list 'title (org-trim feed-title)) (list 'generator org-atom-generator-name org-atom-generator-version) (list 'id (concat (if (and org-atom-prefer-urn-uuid (org-atom-looks-like-uuid-p feed-id)) "urn:uuid:" "") feed-id)) (list 'updated (current-time)) (list 'link feed-url nil "self") (list 'author author)) (mapcar '(lambda (entry) (cons 'entry (list entry))) entries))))) (if (eq to-buffer 'string) feed (with-current-buffer to-buffer (insert feed) (if (buffer-file-name) (save-buffer))))))) (defun org-atom-publish-feed-index (project &optional filename) "Publish feed for set of files in PROJECT. Optional argument FILENAME is name of the output file." (let* ((project-plist (cdr project)) (dir (file-name-as-directory (plist-get project-plist :base-directory))) (exclude-regexp (plist-get project-plist :exclude)) (files (nreverse (org-publish-get-base-files project exclude-regexp))) (index-filename (concat dir (or index-filename (concat "feed." org-atom-feed-extension)))) (index-title (or (plist-get project-plist :index-title) (concat "Index for project " (car project)))) (pub-url (plist-get project-plist :publishing-url)) (feed-url (concat pub-url (if (string-match-p "/$" pub-url) "" "/") index-filename)) (feed-id (or (plist-get project-plist :feed-id) feed-url)) (visiting (find-buffer-visiting index-filename)) file index-buffer) ;; maybe adjust publication url (unless (string-match-p "/$" pub-url) (setq pub-url (concat pub-url "/"))) (setq project-plist (plist-put project-plist :feed-title index-title)) (with-current-buffer (setq index-buffer (or visiting (find-file index-filename))) (erase-buffer) (insert (concat "\n" (atom-syndication-element-feed (append (list (list 'title index-title) (list 'id (concat (if (and org-atom-prefer-urn-uuid (org-atom-looks-like-uuid-p feed-id)) "urn:uuid:" "") feed-id)) (list 'updated (current-time)) (list 'link feed-url nil "self")))))) (re-search-backward "") (while (setq file (pop files)) (let* ((entries-plist (org-combine-plists project-plist (plist-put nil :feed-content-url (concat pub-url (file-relative-name (file-name-sans-extension file) dir) ".html")))) (entries (with-current-buffer (or (find-buffer-visiting file) (find-file-noselect file)) (org-export-as-atom entries-plist 'string t)))) (when entries (insert entries)))) (save-buffer)) (or visiting (kill-buffer index-buffer)))) (defun org-atom-export-headline (id-prefix content-url &optional pom) "Return atom:entry alist for headline. ID-PREFIX is a string that is used as prefix for the atom:id element. CONTENT-URL is a url pointing on the published html file. Optional argument POM is point or marker of headline. If not set, export headline at point." (save-excursion (goto-char (or pom (point))) (let* ((comps (org-heading-components)) (title (nth 4 comps)) (id (org-id-get)) (published (org-entry-get nil org-atom-published-property-name)) (updated (or (org-entry-get nil org-atom-updated-property-name) published)) (elist (append (if published (list (list 'published (org-time-string-to-time published)))) (when org-atom-publish-content (let (beg end content) (save-excursion (org-back-to-heading) (setq beg (point)) (outline-end-of-subtree) (setq end (point)) (setq content (buffer-substring-no-properties beg end)) (list (list 'content content "html"))))) (if (and content-url (not (string= content-url ""))) (list (list 'link (concat content-url "#" id) nil "alternate"))) (list (list 'title title (cons 'type "html")) (list 'updated (org-time-string-to-time updated)) (list 'id (concat (if (and org-atom-prefer-urn-uuid (org-atom-looks-like-uuid-p id)) "urn:uuid:" id-prefix) id)))))) elist))) (defun org-atom-prepare-headline (&optional pom) "Prepare headline at point or marker POM for export. If POM is ommited, preapre headline at point." (save-excursion (goto-char (or pom (point))) (let ((id (org-id-get-create)) (dtime (or (org-entry-get nil org-atom-published-property-name) (org-entry-get nil org-atom-updated-property-name)))) (unless dtime (org-entry-put nil org-atom-published-property-name (concat "[" (substring (format-time-string (cdr org-time-stamp-formats)) 1 -1) "]")))))) (defun org-atom-htmlize (string) "Return sanitized html markup for STRING." (with-temp-buffer (insert string) (org-mode) (atom-syndication-sanitize (org-export-region-as-html (point-min) (point-max) t 'string)))) (defun org-atom-looks-like-uuid-p (string) "Return non-nil if STRING looks like a uuid." (string-match-p org-atom-uuid-regexp string)) ;; add infile options (dolist (opt org-atom-infile-options) (unless (memq opt org-export-inbuffer-options-extra) (setq org-export-inbuffer-options-extra (append (list opt) org-export-inbuffer-options-extra)))) (provide 'org-atom) ;;; org-atom.el ends here --Multipart_Fri_Mar_19_23:00:23_2010-1-- --pgp-sign-Multipart_Fri_Mar_19_23:00:23_2010-2 Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iF4EABEIAAYFAkuj8/cACgkQma24O1pEeOYRfAD/WudhNl0tKe4W+haBFY5Ui+QO WTorWGHx/1TD6k6E2E8A/2P06wY9wqU1gBEjNlkxgnaJC3tmM5sGjup+iOLWphPr =CcXc -----END PGP SIGNATURE----- --pgp-sign-Multipart_Fri_Mar_19_23:00:23_2010-2-- --===============0286315033== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Emacs-orgmode mailing list Please use `Reply All' to send replies to the list. Emacs-orgmode@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-orgmode --===============0286315033==--