Re: Citation processing via Zotero + zotxt

[John Kitchin <jkitchin@andrew.cmu.edu>]

Martin Yrjölä writes:

> Hi Richard.
>
> Thank you for your work on improving citation support for org-mode. I
> have some experiences from using Zotero, but for the moment I'm using
> org-ref[1] to manage my citations.

This really means you are using bibtex as the reference database
backend, and probably helm-bibtex to insert citations as org-mode links.
Bibliography generation is probably managed by LaTeX/Bibtex. org-ref
mostly provides the functionality of the links, and some utility
functions to help build and maintain bibtex files, and to check all the
cross-reference, label and citation links in the document.

Ideally, something like zotxt would enable Zotero as the reference
database backend with some selection mechanism (it could be helm, or
ido, or some other interface) to insert citation syntax, and at export
time also leverage Zotero to generate the bibliography. I guess this
would not too often be export to LaTeX, since that would require syncing
with a Bibtex file. For quick and dirty work this would be fine, but my
experience is converted bibtex files are never clean enough for
production publication, and require a lot of manual cleaning of
capitalization, journal abbreviations, math, etc... It is ok to do it
once, but not every time you build a document!

A minimal, generalized interface that would probably work with org-ref
would look something like (* is some prefix name like org-cite-):

*-select-citations-from-db opens some selection interface with
  candidates from the db-backend ((org-)bibtex, Zotero, Mendeley, RIS,
  ...), allows user to select some citations with some filtering
  capability then inserts citation syntax into org file. This function
  should allow multiple selections, and allow appending to an existing
  citation. org-ref supports reftex and helm-bibtex for this, and ebib
  would also work.

*-export-citations This is the most difficult part, this code would have
  to functionally do what LaTeX/BibTex accomplishes, and probably with
  multiple passes to collect citations, get formatted bibliographic
  entries, replace citations in the text (eg with numbers, au-year,
  etc...), and insert the formatted bibliography in the right place. If
  Zotero can give the formatted entry for each citation, this might be a
  good start. It would be integrated into the export process. It might
  even be backend-agnostic if the formatting was just plain or in org
  markup text.

*-on-cite-click-func This is technically optional, but it is what makes
  org-ref so great. Figures out what citation was clicked on, and runs
  some action on that citation, e.g. open citation, find related
  articles in google scholar, open browser to citation url, open notes
  to the citation, etc... This is org-ref's most clever code, figuring
  out what was clicked on, and providing multiple actions on that
  citation key.

most other stuff in org-ref covers cross-references, labels and eqn
labels, or is very specific to bibtex.


> Richard Lawrence writes:
>
>> For the past few days, I've been looking more closely at using the
>> combination of Zotero [1] with Erik Hetzner's zotxt plugin [2] as a
>> means of processing citations when exporting to non-LaTeX backends.  I
>> am now thinking that this is probably our best option, but I'd like to
>> know what other people think before I sink a lot of work into it.
>>
>> Here are the reasons I think this is the best option:
>>
>> 1) It is really easy for users.

It is pretty easy to use zotero. I couldn't figure out zotxt though.

>
> I think ease of use is very important. Maybe some day we can talk with
> Zotero directly from Emacs, but installing a browser extension seems to
> be the easiest we can do for now.

It is possible to talk directly to the Zotero sqlite database, but you
need the browser extension or standalone to make it!

>> I know that many people (perhaps especially the `power users' who have
>> been active in the citations discussion so far) prefer to maintain their
>> reference database without the aid of a GUI reference manager like
>> Zotero.

These are people who use bibtex I think. I don't think its practical to
directly edit the sqlite database ;)

> Maybe two way sync between BibTeX and Zotero would be sufficient for
> power users? I know there is already automatic exporters to BibTex for
> Zotero, but some work needs to be done to make it a two-way sync.

I am personally skeptical of this, having "translated" Endnote libraries
to Bibtex and back. The simple stuff mostly works, but accented names,
chemical formulas, and many other things tend to not translate very well.

> I don't mind the GUI for editing the bibliography database, especially
> when the Zotero Translators [2] make such a good job for scraping
> citation metadata directly from web pages.

you should try doi-utils-add-bibtex-entry-from-doi in org-ref ;) it does
a lot of those things too! maybe i will get some good ideas there ;)

> I wouldn't like to use the Zotero GUI for bibliography notes. I think
> the best feature of org-ref is that the bibliography notes are stored in
> an org-mode file, my preferred note-taking tool. Will this be supported
> with the new citation processing you are working on?

This should be independent of the citation processing, and is only
related to the functions available when you click on a citation. Notes
can be accessed via helm-bibtex too.

>
> Best regards,
> Martin Yrjölä
>
> [1] https://github.com/jkitchin/org-ref
> [2] https://www.zotero.org/support/dev/translators

--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu