Re: Citation processing via Zotero + zotxt

[Rasmus <rasmus@gmx.us>]

Hi Richard,

First, thank you for looking into this.  I learned something new from this
post.

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> 1) It is really easy for users.

I generally don’t buy the arguments a reference manager being nice per
see.

Also, last I checked Zotero also existed as a standalone manager.  If this
also works re your points below, using Zotero is a non-issue.  There
exists a plugin to keep a bibtex file up to date.  I don’t know if it’s
two way, though.

> For those unfamiliar, Zotero is a reference manager, and zotxt is a
> Zotero plugin that makes it easier to work with Zotero from plain text
> documents.  Both are Firefox plugins, which means they can be installed
> by a non-technical user with a couple of clicks.  It also means that
> users get updates automatically.  I think this is *really* important.

I don’t think this is relevant.

> Pretty much all the other options we have talked about seem like they
> will require multi-step, non-trivial installation procedures ("First
> install {Node.js/Haskell/JVM ...}, then install
> {citeproc-node/pandoc-citeproc/citeproc-java...}, then install our
> wrapper script...").  Updating could require other manual operations of
> similar complexity.  Avoiding that kind of procedure will make citations
> a lot more usable from Org for everyone.

I think this is *very* important.

> Also, unlike the other options, Zotero is a full-featured reference
> manager, not just a batch processor.  So we as users get a useful piece
> of software with a simple installation procedure; the other options
> require a complex installation procedure for a less-useful program.

I don’t think this is relevant.

> 2) It is quite complete.
>
> Previously, I thought that it would be a similar amount of work to
> communicate with Zotero from Emacs as any of the other CSL
> implementations out there.  However, after looking at zotxt a bit more
> closely, I discovered that it has an (undocumented) API endpoint [3]

This sounds amazing, but also dangerous.  Do you know whether stabilizing
the API has been discussed upstream?

> that pretty much does exactly what we need: it accepts a list of
> citation objects, and returns a list of formatted citations and a
> formatted bibliography, which can be inserted into the exported
> document.

Could you give an example of the sort of input you give?  I.e. is it based
on keys?  From my bibtex-centric world view I imagine something like:

   I send key or pointer @K to a DB entry as well as a CSL-file pointer C,
   and maybe a desired output format F.  I get a string back that is the
   formatting of the data behind @K formatted according to the rules in C,
   adapted to F.

Is that correct?  If so, does it support html, text and odt?

> This endpoint still needs a little bit of work, to generalize it and
> make it easier to get the data in the format we need.  (That is probably
> why it is undocumented in the README.) But it requires much less work
> than I thought it would, and much less work than it would be to get a
> full-featured setup with something like citeproc-node.

This is a very strong argument.

At some point Matt talked about adding support for org citation syntax in
citeproc-js.  Would this be advantageous if going this route?  I guess not.

> Erik has also written a package for communicating with zotxt from Emacs,
> zotxt-emacs [4], which is available on MELPA.  This package already
> contains a lot of useful functions for querying the Zotero database and
> inserting reference data into documents, including links in Org
> documents.  I think it would be pretty straightforward to extend this
> package to provide a nice UI for writers who are inserting citations
> into Org documents, including search-based lookups of keys, etc.
> Perhaps org-ref could also be taught to communicate with zotxt (with or
> without zotxt-emacs) without too much work.

I guess we’d need to convince Erik to move it to GELPA.  Fortunately (from
that point of view), neither zotxt nor zotxt-emacs seems to have too many
contributors.

> 3) It uses citeproc-js.
>
> In previous discussions, I think we agreed that it would be best for us
> to use citeproc-js as a CSL processor, since it is the `canonical' CSL
> implementation, as opposed to pandoc-citeproc or citeproc-java.  Zotero
> just uses citeproc-js internally to process citations, so it meets this
> requirement. 

Indeed.

> I know that many people (perhaps especially the `power users' who have
> been active in the citations discussion so far) prefer to maintain their
> reference database without the aid of a GUI reference manager like
> Zotero.

I think this solves part of that issue.  Again it could be a dangerous
solution.  Alternatively, perhaps it’s possible to feed-in a bibtex file
to Zotero via an API.  This would also limit the damage.

IMO we can leverage zotero as a tool, but we cannot enforce it as a
bibliography manager.

> I still think Zotero + zotxt is the best option for non-LaTeX
> citation processing, even for these folks.  The ease of installation
> (and removal) of the required programs alone makes it worth it, even if
> you never actually populate a Zotero database.  So given what I know at
> the moment, I think our efforts would best be directed at making the
> in-progress org-cite library communicate with Zotero via zotxt.  What do
> you think?

+1, though re zotxt we should make sure Erik would want to move it to
GELPA.

Rasmus

-- 
The right to be left alone is a human right