From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Eric Schulte" Subject: Re: [PATCH] org-bibtex.el --- convert between Org headings and bibtex entries Date: Sat, 23 Apr 2011 08:07:06 -0600 Message-ID: <878vv112ph.fsf@gmail.com> References: <87y6357q81.fsf@gmail.com> <87liz5aztx.fsf@fastmail.fm> <878vv22xgz.fsf_-_@gmail.com> <87r58uz3ux.fsf@fastmail.fm> <87r58tdeaf.fsf@fastmail.fm> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Return-path: Received: from eggs.gnu.org ([140.186.70.92]:34802) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QDdUU-0006yU-4b for emacs-orgmode@gnu.org; Sat, 23 Apr 2011 10:07:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QDdUR-0000ZZ-Gx for emacs-orgmode@gnu.org; Sat, 23 Apr 2011 10:07:18 -0400 Received: from mail-pz0-f41.google.com ([209.85.210.41]:49176) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QDdUR-0000ZR-0y for emacs-orgmode@gnu.org; Sat, 23 Apr 2011 10:07:15 -0400 Received: by pzk32 with SMTP id 32so1031493pzk.0 for ; Sat, 23 Apr 2011 07:07:13 -0700 (PDT) In-Reply-To: <87r58tdeaf.fsf@fastmail.fm> (Matt Lundin's message of "Fri, 22 Apr 2011 20:03:52 -0400") List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Matt Lundin Cc: Org Mode --=-=-= Content-Type: text/plain Matt Lundin writes: > Matt Lundin writes: > >> "Eric Schulte" writes: >>> As one possible solution to this name conflict, I've folded my >>> org-bibtex.el into the existing org-bibtex.el in the Org-mode core. The >>> attached patch performs this integration adding the functionality in my >>> version of org-bibtex.el into the Org-mode core. >>> >>> Does this seem like an appropriate addition? >> >> +1 (or maybe +.75 -- see below) >> >> I think this could finally tempt me to move all my bibliographical data >> into org-mode files. (I've been keeping all my data in a central bibtex >> file and using reftex to generate links to citations.) > > I've been testing the patch and have a few comments. > > 1. The prompt for the completing-read on line 440 is missing a space. It > shows "Type:" rather than "Type: ". > Fixed, Thanks > > 2. In addition, since the symbols in org-bibtex-types are keywords, the > completions are in the form ":article", ":book", etc. Might I ask why > everything is org-bibtex-type is a keyword? Could we use strings or > plain symbols instead? (Note: I haven't read the code carefully > enough to see if there is a reason.) > I tend to prefer keywords for items which are primarily used as keys (e.g., for alist lookup w/assoc), I like that keywords are only interned once and are clearly not variables or functions, however you're right that it may add some extra complexity to this code. I've changed the code which prompts for fields so that it now strips the leading ":" from the keyword names, leading to nicer prompts. > > 3. The tooltip/help pop-up for org-bibtex-create takes up half a frame. > The pop-up buffer with field information should now only be as large as required for the text (although if you have a vertically split frame the buffer will not shrink). It is using the same code used to minimize the help buffer presented by `org-export'. > > Thanks, > Matt > The updated version is contained in the attached patch. --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename=0001-org-bibtex-translating-between-Org-mode-headings-and.patch Content-Transfer-Encoding: quoted-printable >From 34f5036409fe77e20c0444ccfcc13cc4be250d92 Mon Sep 17 00:00:00 2001 From: Eric Schulte Date: Sat, 23 Apr 2011 08:05:08 -0600 Subject: [PATCH] org-bibtex: translating between Org-mode headings and Bibt= ex entries * lisp/org-bibtex.el: Updating Copyright dates, author information, commentary and history notes. (org-bibtex-types): List of bibtex types with descriptions and required and optional fields. (org-bibtex-fields): List of bibtex fields with descriptions. (*org-bibtex-entries*): Special variable to hold parsed bibtex entries. (org-bibtex-autogen-keys): Custom variable controlling whether bibtex keys are automatically generated (org-bibtex-prefix): Custom variable allowing use of optional prefix for bibtex properties in Org-mode headlines. (org-bibtex-get): Helper function for accessing bibtex elements of a property list. (org-bibtex-put): Helper function for inserting bibtex element into a property list. (org-bibtex-headline): Return a bibtex entry of the given headline as a string. (org-bibtex-ask): Prompt the user to fill in the value of a bibtex field. (org-bibtex-autokey): Generate a bibtex key for the current headline. (org-bibtex-fleshout): Fill in missing bibtex properties of the current headline. (org-bibtex): Export the current Org-mode buffer to a bibtex buffer. (org-bibtex-check): Check that all bibtex properties are present in the current headline. (org-bibtex-check-all): Check all headlines in the current buffer. (org-bibtex-create): Create a new bibtex headline at the current level. (org-bibtex-read): Read the current bibtex entry from a bibtex file. (org-bibtex-write): Write the most recently read bibtex entry into an Org-mode file. --- lisp/org-bibtex.el | 319 ++++++++++++++++++++++++++++++++++++++++++++++++= +-- 1 files changed, 306 insertions(+), 13 deletions(-) diff --git a/lisp/org-bibtex.el b/lisp/org-bibtex.el index b7b7416..9ee30f1 100644 --- a/lisp/org-bibtex.el +++ b/lisp/org-bibtex.el @@ -1,9 +1,10 @@ ;;; org-bibtex.el --- Org links to BibTeX entries ;; -;; Copyright (C) 2007, 2008, 2009, 2010 Free Software Foundation, Inc. +;; Copyright (C) 2007, 2008, 2009, 2010, 2011 Free Software Foundation, In= c. ;; ;; Author: Bastien Guerry ;; Carsten Dominik +;; Eric Schulte ;; Keywords: org, wp, remember ;; Version: 7.5 ;; @@ -66,12 +67,30 @@ ;; =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D ;; * READ <=3D=3D [point here] ;; -;; [[file:/file.bib::dolev83][Dolev & Yao 1983: security of public key pro= tocols]] +;; [[file:file.bib::dolev83][Dolev & Yao 1983: security of public key prot= ocols]] ;; ;; Danny Dolev and Andrew C. Yao (1983): On the security of public-key pro= tocols ;; In IEEE Transaction on Information Theory, 198--208. ;; =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D ;; +;; Additionally, the following functions are now available for storing +;; bibtex entries within Org-mode documents. +;; +;; - Run `org-bibtex' to export the current file to a .bib. +;; +;; - Run `org-bibtex-check' or `org-bibtex-check-all' to check and +;; fill in missing field of either the current, or all headlines +;; +;; - Run `org-bibtex-create' to add a bibtex entry +;; +;; - Use `org-bibtex-read' to read a bibtex entry after `point' or in +;; the active region, then call `org-bibtex-write' in a .org file to +;; insert a heading for the read bibtex entry +;; +;; - All Bibtex information is taken from the document compiled by +;; Andrew Roberts from the Bibtex manual, available at +;; http://www.andy-roberts.net/misc/latex/sessions/bibtex/bibentries.pdf +;; ;;; History: ;; ;; The link creation part has been part of Org-mode for a long time. @@ -80,12 +99,17 @@ ;; of Austin Frank: http://article.gmane.org/gmane.emacs.orgmode/4112 ;; and then implemented by Bastien Guerry. ;; +;; Eric Schulte eventually added the functions for translating between +;; Org-mode headlines and Bibtex entries, and for fleshing out the Bibtex +;; fields of existing Org-mode headlines. +;; ;; Org-mode loads this module by default - if this is not what you want, ;; configure the variable `org-modules'. =20 ;;; Code: =20 (require 'org) +(require 'bibtex) =20 (defvar description nil) ; dynamically scoped from org.el =20 @@ -93,21 +117,209 @@ (declare-function bibtex-generate-autokey "bibtex" ()) (declare-function bibtex-parse-entry "bibtex" (&optional content)) (declare-function bibtex-url "bibtex" (&optional pos no-browse)) +(declare-function longlines-mode "longlines" (&optional arg)) + + +;;; Bibtex data +(defvar org-bibtex-types + '((:article + (:description . "An article from a journal or magazine") + (:required :author :title :journal :year) + (:optional :volume :number :pages :month :note)) + (:book + (:description . "A book with an explicit publisher") + (:required (:editor :author) :title :publisher :year) + (:optional (:volume :number) :series :address :edition :month :note)) + (:booklet + (:description . "A work that is printed and bound, but without a name= d publisher or sponsoring institution.") + (:required :title) + (:optional :author :howpublished :address :month :year :note)) + (:conference + (:description . "") + (:required :author :title :booktitle :year) + (:optional :editor :pages :organization :publisher :address :month :n= ote)) + (:inbook + (:description . "A part of a book, which may be a chapter (or section= or whatever) and/or a range of pages.") + (:required (:author :editor) :title (:chapter :pages) :publisher :yea= r) + (:optional (:volume :number) :series :type :address :edition :month := note)) + (:incollection + (:description . "A part of a book having its own title.") + (:required :author :title :booktitle :publisher :year) + (:optional :editor (:volume :number) :series :type :chapter :pages :a= ddress :edition :month :note)) + (:inproceedings + (:description . "An article in a conference proceedings") + (:required :author :title :booktitle :year) + (:optional :editor (:volume :number) :series :pages :address :month := organization :publisher :note)) + (:manual + (:description . "Technical documentation.") + (:required :title) + (:optional :author :organization :address :edition :month :year :note= )) + (:mastersthesis + (:description . "A Master=E2=80=99s thesis.") + (:required :author :title :school :year) + (:optional :type :address :month :note)) + (:misc + (:description . "Use this type when nothing else fits.") + (:required) + (:optional :author :title :howpublished :month :year :note)) + (:phdthesis + (:description . "A PhD thesis.") + (:required :author :title :school :year) + (:optional :type :address :month :note)) + (:proceedings + (:description . "The proceedings of a conference.") + (:required :title :year) + (:optional :editor (:volume :number) :series :address :month :organiz= ation :publisher :note)) + (:techreport + (:description . "A report published by a school or other institution.= ") + (:required :author :title :institution :year) + (:optional :type :address :month :note)) + (:unpublished + (:description . "A document having an author and title, but not forma= lly published.") + (:required :author :title :note) + (:optional :month :year))) + "Bibtex entry types with required and optional parameters.") + +(defvar org-bibtex-fields + '((:address . "Usually the address of the publisher or other type o= f institution. For major publishing houses, van Leunen recommends omitting = the information entirely. For small publishers, on the other hand, you can= help the reader by giving the complete address.") + (:annote . "An annotation. It is not used by the standard biblio= graphy styles, but may be used by others that produce an annotated bibliogr= aphy.") + (:author . "The name(s) of the author(s), in the format describe= d in the LaTeX book. Remember, all names are separated with the and keywor= d, and not commas.") + (:booktitle . "Title of a book, part of which is being cited. See t= he LaTeX book for how to type titles. For book entries, use the title field= instead.") + (:chapter . "A chapter (or section or whatever) number.") + (:crossref . "The database key of the entry being cross referenced= .") + (:edition . "The edition of a book for example, 'Second'. This sh= ould be an ordinal, and should have the first letter capitalized, as shown = here; the standard styles convert to lower case when necessary.") + (:editor . "Name(s) of editor(s), typed as indicated in the LaTe= X book. If there is also an author field, then the editor field gives the e= ditor of the book or collection in which the reference appears.") + (:howpublished . "How something strange has been published. The first = word should be capitalized.") + (:institution . "The sponsoring institution of a technical report.") + (:journal . "A journal name.") + (:key . "Used for alphabetizing, cross-referencing, and creat= ing a label when the author information is missing. This field should not b= e confused with the key that appears in the \cite command and at the beginn= ing of the database entry.") + (:month . "The month in which the work was published or, for an= unpublished work, in which it was written. You should use the standard thr= ee-letter abbreviation,") + (:note . "Any additional information that can help the reader.= The first word should be capitalized.") + (:number . "Any additional information that can help the reader.= The first word should be capitalized.") + (:organization . "The organization that sponsors a conference or that = publishes a manual.") + (:pages . "One or more page numbers or range of numbers, such a= s 42-111 or 7,41,73-97 or 43+ (the =E2=80=98+=E2=80=99 in this last example= indicates pages following that don=E2=80=99t form simple range). BibTEX re= quires double dashes for page ranges (--).") + (:publisher . "The publisher=E2=80=99s name.") + (:school . "The name of the school where a thesis was written.") + (:series . "The name of a series or set of books. When citing an= entire book, the the title field gives its title and an optional series fi= eld gives the name of a series or multi-volume set in which the book is pub= lished.") + (:title . "The work=E2=80=99s title, typed as explained in the = LaTeX book.") + (:type . "The type of a technical report for example, 'Researc= h Note'.") + (:volume . "The volume of a journal or multi-volume book.") + (:year . "The year of publication or, for an unpublished work,= the year it was written. Generally it should consist of four numerals, su= ch as 1984, although the standard styles can handle any year whose last fou= r nonpunctuation characters are numerals, such as '(about 1984)'")) + "Bibtex fields with descriptions.") + +(defvar *org-bibtex-entries* nil + "List to hold parsed bibtex entries.") + +(defcustom org-bibtex-autogen-keys nil + "Set to a truthy value to use `bibtex-generate-autokey' to generate keys= ." + :group 'org-bibtex + :type 'boolean) + +(defcustom org-bibtex-prefix nil + "Optional prefix for all bibtex property names. +For example setting to 'BIB_' would allow interoperability with fireforg." + :group 'org-bibtex + :type 'string) + + +;;; Utility functions +(defun org-bibtex-get (property) + (or (org-entry-get (point) (upcase property)) + (org-entry-get (point) (concat org-bibtex-prefix (upcase property)))= )) =20 +(defun org-bibtex-put (property value) + (let ((prop (upcase (if (keywordp property) + (substring (symbol-name property) 1) + property)))) + (org-set-property + (concat (unless (string=3D "CUSTOM_ID" prop) org-bibtex-prefix) prop) + value))) + +(defun org-bibtex-headline () + "Return a bibtex entry of the given headline as a string." + (flet ((get (key lst) (cdr (assoc key lst))) + (to-k (string) (intern (concat ":" string))) + (from-k (key) (substring (symbol-name key) 1)) + (flatten (&rest lsts) + (apply #'append (mapcar + (lambda (e) + (if (listp e) (apply #'flatten e) (li= st e))) + lsts)))) + (let ((notes (buffer-string)) + (id (org-bibtex-get "custom_id")) + (type (org-bibtex-get "type"))) + (when type + (let ((entry (format + "@%s{%s,\n%s\n}\n" type id + (mapconcat + (lambda (pair) (format " %s=3D{%s}" (car pair) (cd= r pair))) + (remove nil + (mapcar + (lambda (field) + (let ((value (or (org-bibtex-get (from-k field)) + (and (equal :title field) + (org-get-heading))))) + (when value (cons (from-k field) value)))) + (flatten + (get :required (get (to-k type) org-bibtex-types)) + (get :optional (get (to-k type) org-bibtex-types))))) + ",\n")))) + (with-temp-buffer + (insert entry) + (bibtex-reformat) (buffer-string))))))) + +(defun org-bibtex-ask (field) + (unless (assoc field org-bibtex-fields) + (error "field:%s is not known" field)) + (save-window-excursion + (let* ((name (substring (symbol-name field) 1)) + (buf-name (format "*Bibtex Help %s*" name))) + (with-output-to-temp-buffer buf-name + (princ (cdr (assoc field org-bibtex-fields)))) + (with-current-buffer buf-name (longlines-mode t)) + (org-fit-window-to-buffer (get-buffer-window buf-name)) + ((lambda (result) (when (> (length result) 0) result)) + (read-from-minibuffer (format "%s: " name)))))) + +(defun org-bibtex-autokey () + "Generate an autokey for the current headline" + (org-bibtex-put "CUSTOM_ID" + (if org-bibtex-autogen-keys + (let ((entry (org-bibtex-headline))) + (with-temp-buffer + (insert entry) + (bibtex-generate-autokey))) + (read-from-minibuffer "id: ")))) + +(defun org-bibtex-fleshout (type &optional optional) + "Fleshout the current heading, ensuring that all required fields are pre= sent. +With optional argument OPTIONAL, also prompt for optional fields." + (flet ((get (key lst) (cdr (assoc key lst))) + (keyword (name) (intern (concat ":" (downcase name)))) + (name (keyword) (upcase (substring (symbol-name keyword) 1)))) + (dolist (field (append + (remove :title (get :required (get type org-bibtex-types))) + (when optional (get :optional (get type org-bibtex-types))))) + (when (consp field) ; or'd pair of fields e.g., (:editor :author) + (let ((present (first (remove nil + (mapcar + (lambda (f) (when (org-bibtex-get (name f= )) f)) + field))))) + (setf field (or present (keyword (org-icompleting-read + "Field: " (mapcar #'name field))))))) + (let ((name (name field))) + (unless (org-bibtex-get name) + (let ((prop (org-bibtex-ask field))) + (when prop (org-bibtex-put name prop))))))) + (when (and type (assoc type org-bibtex-types) + (not (org-bibtex-get "CUSTOM_ID"))) + (org-bibtex-autokey))) + + +;;; Bibtex link functions (org-add-link-type "bibtex" 'org-bibtex-open) (add-hook 'org-store-link-functions 'org-bibtex-store-link) =20 -;; (defun org-bibtex-publish (path) -;; "Build the description of the BibTeX entry for publishing." -;; (let* ((search (when (string-match "::\\(.+\\)\\'" path) -;; (match-string 1 path))) -;; (path (substring path 0 (match-beginning 0))) -;; key) -;; (with-temp-buffer -;; (org-open-file path t nil search) -;; (setq key (org-create-file-search-functions))) -;; (or description key))) - (defun org-bibtex-open (path) "Visit the bibliography entry on PATH." (let* ((search (when (string-match "::\\(.+\\)\\'" path) @@ -198,6 +410,87 @@ ;; Finally add the link search function to the right hook. (add-hook 'org-execute-file-search-functions 'org-execute-file-search-in-b= ibtex) =20 + +;;; Bibtex <-> Org-mode headline translation functions +(defun org-bibtex () + "Export each headline in the current file to a bibtex entry. +Headlines are exported using `org-bibtex-export-headline'." + (interactive) + (let ((bibtex-entries (remove nil (org-map-entries #'org-bibtex-headline= )))) + (with-temp-file (concat (file-name-sans-extension (buffer-file-name)) = ".bib") + (insert (mapconcat #'identity bibtex-entries "\n"))))) + +(defun org-bibtex-check (&optional optional) + "Check the current headline for required fields. +With prefix argument OPTIONAL also prompt for optional fields." + (interactive "P") + (save-restriction + (org-narrow-to-subtree) + (let ((type ((lambda (name) (when name (intern (concat ":" name)))) + (org-bibtex-get "TYPE")))) + (when type (org-bibtex-fleshout type optional))))) + +(defun org-bibtex-check-all (&optional optional) + "Check all headlines in the current file. +With prefix argument OPTIONAL also prompt for optional fields." + (interactive) (org-map-entries (lambda () (org-bibtex-check optional)))) + +(defun org-bibtex-create (type) + "Create a new entry at the given level." + (interactive + (list (org-icompleting-read + "Type: " + (mapcar (lambda (type) (symbol-name (car type))) org-bibtex-types)))) + (let ((type (if (keywordp type) type (intern type)))) + (unless (assoc type org-bibtex-types) + (error "type:%s is not known" type)) + (org-insert-heading) + (let ((title (org-bibtex-ask :title))) + (insert title) (org-bibtex-put "TITLE" title)) + (org-bibtex-put "TYPE" (substring (symbol-name type) 1)) + (org-bibtex-fleshout type))) + +(defun org-bibtex-read () + "Read a bibtex entry and save to `*org-bibtex-entries*'. +This uses `bibtex-parse-entry'." + (interactive) + (flet ((keyword (str) (intern (concat ":" (downcase str)))) + (clean-space (str) (replace-regexp-in-string + "[[:space:]\n\r]+" " " str)) + (strip-delim (str) ; strip enclosing "..." and {...} + (dolist (pair '((34 . 34) (123 . 125))) + (when (and (=3D (aref str 0) (car pair)) + (=3D (aref str (1- (length str))) (cdr pair))) + (setf str (subseq str 1 (1- (length str)))))) str)) + (push (mapcar + (lambda (pair) + (cons (let ((field (keyword (car pair)))) + (case field + (:=3Dtype=3D :type) + (:=3Dkey=3D :key) + (otherwise field))) + (clean-space (strip-delim (cdr pair))))) + (save-excursion (bibtex-beginning-of-entry) (bibtex-parse-entry= ))) + *org-bibtex-entries*))) + +(defun org-bibtex-write () + "Insert a heading built from the first element of `*org-bibtex-entries*'= ." + (interactive) + (when (=3D (length *org-bibtex-entries*) 0) + (error "No entries in `*org-bibtex-entries*'.")) + (let ((entry (pop *org-bibtex-entries*))) + (flet ((get (field) (cdr (assoc field entry)))) + (org-insert-heading) + (insert (get :title)) + (org-bibtex-put "TITLE" (get :title)) + (org-bibtex-put "TYPE" (downcase (get :type))) + (dolist (pair entry) + (case (car pair) + (:title nil) + (:type nil) + (:key (org-bibtex-put "CUSTOM_ID" (cdr pair))) + (otherwise (org-bibtex-put (car pair) (cdr pair)))))))) + (provide 'org-bibtex) =20 ;; arch-tag: 83987d5a-01b8-41c7-85bc-77700f1285f5 --=20 1.7.1 --=-=-= Content-Type: text/plain Thanks for your feedback. I hope to hear from Carsten or Bastien (the authors of the current org-bibtex) about whether this is the right place for this functionality or if it should live somewhere else. Best -- Eric -- Eric Schulte http://cs.unm.edu/~eschulte/ --=-=-=--