Re: [ANN] org-bibtex.el --- convert between Org headings and bibtex entries

["Eric Schulte" <schulte.eric@gmail.com>]

Hi Matt,

Matt Lundin <mdl@imapmail.org> writes:

> Hi Eric,
>
> "Eric Schulte" <schulte.eric@gmail.com> writes:
>
>> Matt Lundin <mdl@imapmail.org> writes:
>>
>> [...]
>>>> I understand I may add to the types variable.  When using
>>>> org-bibtex-create, I can enter any arbitrary field as a PROPERTY;
>>>> however, org-bibtex ignores anything outside of the universe it knows
>>>> about.  Would it be bad practice to allow the export of any arbitrary
>>>> field type one has recorded?  I think the emacs bibtex-mode may
>>>> recognize erroneous bibtex entries.   
>>>
>>> Bibtex-mode does indeed allow for arbitrary fields, as do bibtex and
>>> biblatex. AFAIK, they are simply ignored when processing a bib file. One
>>> limitation that arises when storing bibtex data as org properties is
>>> that properties drawers are used for much more. For instance, one would
>>> probably not want to see "logging = {lognoterepeat}," in one's exported
>>> bibtex file.
>>>
>>> But for biblatex users, it would indeed be prohibitively expensive to
>>> have to inform org-mode ahead of time about the innumerable odd fields
>>> that various biblatex backends define.
>>
>> There is already an option for an org-bibtex specific property name
>> prefix, (namely `org-bibtex-prefix').  Perhaps when this prefix is used,
>> and the `org-bibtex' functions is called with a prefix argument (note:
>> entirely different usage of the term "prefix") then only entries which
>> begin with the `org-bibtex-prefix' would be exported...  I believe that
>> should provide a natural way for arbitrary fields to pass through
>> org-bibtex without the user needing to explicitly name them, or there
>> being any danger of contamination from existing org-mode properties.
>
> I went ahead and implemented this. (Alas, it meant cluttering up your
> very elegant org-bibtex-headline with another mapcar.) 
>

This looks great thanks.

>
> Assuming that not all users who use a prefix will want to export
> arbitrary fields, I made the functionality dependent on two variables:
> org-bibtex-prefix and a org-bibtex-export-arbitrary-fields. But this
> could be simplified.
>

I think this is the best approach.

>
> I also made the key property configurable.
>

Nice, I think this is also an improvement.

>
> The patch was created against a patched org-bibtex.el, so I will wait
> until your changes get merged into the repo before sending a formal
> patch. But I thought I'd send it along to see if you think the changes
> are appropriate.
>

Oh, I should have read this last paragraph.

Having received confirmation from Bastien that it was alright to include
this code into org-bibtex in the core, I have applied your patch on top
of my own patches.  The results are now in the Org-mode git repository.
Please do let me know if anything looks amiss --- I mainly just expanded
your patch to take into account my intervening changes, and to avoid
lines longer than 80 characters.

Thanks! -- Eric

>
> Best,
> Matt 
>
> diff --git a/lisp/org-bibtex.el b/lisp/org-bibtex.el
> index 9ee30f1..afa3764 100644
> --- a/lisp/org-bibtex.el
> +++ b/lisp/org-bibtex.el
> @@ -221,6 +221,24 @@ For example setting to 'BIB_' would allow interoperability with fireforg."
>    :group 'org-bibtex
>    :type  'string)
>  
> +(defcustom org-bibtex-export-arbitrary-fields nil
> +  "When converting to bibtex allow fields not defined in `org-bibtex-fields'.
> +This only has effect if org-bibtex-prefix is defined, so as to
> +ensure that other org-properties, such as CATEGORY or LOGGING are
> +not placed in the exported bibtex entry."
> +  :group 'org-bibtex
> +  :type 'boolean)
> +
> +;; TODO if ID, test to make sure ID is unique
> +(defcustom org-bibtex-key-property "CUSTOM_ID"
> +  "Property that holds the bibtex key.
> +By default, this is CUSTOM_ID, which enables easy linking to
> +bibtex headlines from within an org file. This can be set to ID
> +to enable global links, but only with great caution, as global
> +IDs must be unique."
> +  :group 'org-bibtex
> +  :type 'string)
> +
>  \f
>  ;;; Utility functions
>  (defun org-bibtex-get (property)
> @@ -232,7 +250,7 @@ For example setting to 'BIB_' would allow interoperability with fireforg."
>                            (substring (symbol-name property) 1)
>                          property))))
>      (org-set-property
> -     (concat (unless (string= "CUSTOM_ID" prop) org-bibtex-prefix) prop)
> +     (concat (unless (string= org-bibtex-key-property prop) org-bibtex-prefix) prop)
>       value)))
>  
>  (defun org-bibtex-headline ()
> @@ -246,7 +264,7 @@ For example setting to 'BIB_' would allow interoperability with fireforg."
>                                       (if (listp e) (apply #'flatten e) (list e)))
>                                     lsts))))
>      (let ((notes (buffer-string))
> -          (id (org-bibtex-get "custom_id"))
> +          (id (org-bibtex-get org-bibtex-key-property))
>            (type (org-bibtex-get "type")))
>        (when type
>          (let ((entry (format
> @@ -254,15 +272,23 @@ For example setting to 'BIB_' would allow interoperability with fireforg."
>                        (mapconcat
>                         (lambda (pair) (format "  %s={%s}" (car pair) (cdr pair)))
>                         (remove nil
> -			 (mapcar
> -			  (lambda (field)
> -			    (let ((value (or (org-bibtex-get (from-k field))
> -					     (and (equal :title field)
> -						  (org-get-heading)))))
> -			      (when value (cons (from-k field) value))))
> -			  (flatten
> -			   (get :required (get (to-k type) org-bibtex-types))
> -			   (get :optional (get (to-k type) org-bibtex-types)))))
> +			 (if (and org-bibtex-export-arbitrary-fields org-bibtex-prefix)
> +			     (mapcar 
> +			      (lambda (kv)
> +				(when (string-match org-bibtex-prefix (car kv))
> +				  (cons (downcase (replace-regexp-in-string 
> +						   org-bibtex-prefix "" (car kv)))
> +					(cdr kv))))
> +			      (org-entry-properties nil 'standard))
> +			   (mapcar
> +			    (lambda (field)
> +			      (let ((value (or (org-bibtex-get (from-k field))
> +					       (and (equal :title field)
> +						    (org-get-heading)))))
> +				(when value (cons (from-k field) value))))
> +			    (flatten
> +			     (get :required (get (to-k type) org-bibtex-types))
> +			     (get :optional (get (to-k type) org-bibtex-types))))))
>                         ",\n"))))
>            (with-temp-buffer
>              (insert entry)
> @@ -283,7 +309,7 @@ For example setting to 'BIB_' would allow interoperability with fireforg."
>  
>  (defun org-bibtex-autokey ()
>    "Generate an autokey for the current headline"
> -  (org-bibtex-put "CUSTOM_ID"
> +  (org-bibtex-put org-bibtex-key-property
>                    (if org-bibtex-autogen-keys
>                        (let ((entry (org-bibtex-headline)))
>                          (with-temp-buffer
> @@ -312,7 +338,7 @@ With optional argument OPTIONAL, also prompt for optional fields."
>            (let ((prop (org-bibtex-ask field)))
>              (when prop (org-bibtex-put name prop)))))))
>    (when (and type (assoc type org-bibtex-types)
> -             (not (org-bibtex-get "CUSTOM_ID")))
> +             (not (org-bibtex-get org-bibtex-key-property)))
>      (org-bibtex-autokey)))
>  
>  \f
> @@ -488,7 +514,7 @@ This uses `bibtex-parse-entry'."
>          (case (car pair)
>            (:title    nil)
>            (:type     nil)
> -          (:key      (org-bibtex-put "CUSTOM_ID" (cdr pair)))
> +          (:key      (org-bibtex-put org-bibtex-key-property (cdr pair)))
>            (otherwise (org-bibtex-put (car pair)  (cdr pair))))))))
>  
>  (provide 'org-bibtex)

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/