From mboxrd@z Thu Jan 1 00:00:00 1970 From: Richard Lawrence Subject: Re: Citation processing via Zotero + zotxt Date: Mon, 30 Nov 2015 18:59:05 -0800 Message-ID: <87d1uqyiva.fsf@berkeley.edu> References: <87wpt1yj5k.fsf@berkeley.edu> Mime-Version: 1.0 Content-Type: text/plain Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:41637) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a3b9U-0005d2-Vt for emacs-orgmode@gnu.org; Mon, 30 Nov 2015 21:58:50 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a3b9Q-0007cP-TX for emacs-orgmode@gnu.org; Mon, 30 Nov 2015 21:58:48 -0500 Received: from mail-pa0-x22d.google.com ([2607:f8b0:400e:c03::22d]:35751) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a3b9Q-0007bW-Hd for emacs-orgmode@gnu.org; Mon, 30 Nov 2015 21:58:44 -0500 Received: by pacej9 with SMTP id ej9so205451674pac.2 for ; Mon, 30 Nov 2015 18:58:42 -0800 (PST) In-Reply-To: List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: John Kitchin , Martin =?utf-8?B?WXJqw7Zsw6Q=?= Cc: emacs-orgmode@gnu.org Hi John and all, Thanks for your input! John Kitchin writes: > This really means you are using bibtex as the reference database > backend, and probably helm-bibtex to insert citations as org-mode links. > Bibliography generation is probably managed by LaTeX/Bibtex. org-ref > mostly provides the functionality of the links, and some utility > functions to help build and maintain bibtex files, and to check all the > cross-reference, label and citation links in the document. > > Ideally, something like zotxt would enable Zotero as the reference > database backend with some selection mechanism (it could be helm, or > ido, or some other interface) to insert citation syntax This is pretty much what the zotxt-emacs package does. It communicates with the zotxt API, and it defines a minor mode for use with Org. This minor mode provides functions for querying the zotxt database (by default, it searches by author, title and year) and for inserting and updating Org links that point to items in the Zotero database. It should be straightforward to extend these capabilities to insert citation syntax instead of, or in addition to, links. > , and at export time also leverage Zotero to generate the > bibliography. I guess this would not too often be export to LaTeX, > since that would require syncing with a Bibtex file. For quick and > dirty work this would be fine, but my experience is converted bibtex > files are never clean enough for production publication, and require a > lot of manual cleaning of capitalization, journal abbreviations, math, > etc... It is ok to do it once, but not every time you build a > document! Hmm, that is interesting and a bit concerning. Zotero has the ability to export sets of items to a BibTeX file. Ideally this kind of cleanup should, and in principle could, be dealt with `upstream' in the Zotero database. The CSL representation of items allows for pretty fine-grained and explicit control over things like capitalization and abbreviations. But I do not know how much of that is exposed in Zotero, and I don't know how well it is preserved upon export to BibTeX. Certainly, I'd want export to LaTeX to be just as quick and painless as export to other backends in general. Some manual cleanup of the .bib file seems acceptable to me as a final step before publication, but I think it would count against Zotero if it was too annoying to Zotero can also import BibTeX, so it would also be possible in principle to go in the other direction: keep your database as .bib, but convert it on the fly when exporting to other backends. > A minimal, generalized interface that would probably work with org-ref > would look something like (* is some prefix name like org-cite-): > > *-select-citations-from-db opens some selection interface with > candidates from the db-backend ((org-)bibtex, Zotero, Mendeley, RIS, > ...), allows user to select some citations with some filtering > capability then inserts citation syntax into org file. This function > should allow multiple selections, and allow appending to an existing > citation. org-ref supports reftex and helm-bibtex for this, and ebib > would also work. As I mentioned above, zotxt-emacs already provides an API in this direction. > *-export-citations This is the most difficult part, this code would have > to functionally do what LaTeX/BibTex accomplishes, and probably with > multiple passes to collect citations, get formatted bibliographic > entries, replace citations in the text (eg with numbers, au-year, > etc...), and insert the formatted bibliography in the right place. If > Zotero can give the formatted entry for each citation, this might be a > good start. It would be integrated into the export process. It might > even be backend-agnostic if the formatting was just plain or in org > markup text. Yep, this is the hard one, but there is some more-or-less working code to do this (via my org-citeproc script) in the in-progress org-cite.el. I'm proposing reworking this code to communicate with Zotero, rather than org-citeproc or another citeproc implementation. > *-on-cite-click-func This is technically optional, but it is what makes > org-ref so great. Figures out what citation was clicked on, and runs > some action on that citation, e.g. open citation, find related > articles in google scholar, open browser to citation url, open notes > to the citation, etc... This is org-ref's most clever code, figuring > out what was clicked on, and providing multiple actions on that > citation key. This needs more work, in org-element. Right now, as far as I know, there's no support for making citation-reference objects clickable. On the other hand, it doesn't seem like it should be that difficult. > It is pretty easy to use zotero. I couldn't figure out zotxt though. Hmm, where did you have trouble? I was able to install it via this link https://addons.mozilla.org/en-US/firefox/addon/zotxt/ and that was pretty much all there was to do. (Figuring out how to use the API it exposes is another matter, but that's programming work, not something an ordinary user would have to do.) >> Maybe two way sync between BibTeX and Zotero would be sufficient for >> power users? I know there is already automatic exporters to BibTex for >> Zotero, but some work needs to be done to make it a two-way sync. > > I am personally skeptical of this, having "translated" Endnote libraries > to Bibtex and back. The simple stuff mostly works, but accented names, > chemical formulas, and many other things tend to not translate very well. Me too. I think two-way sync is probably too annoying to try to get right, and there isn't much to be gained by it. I think instead we should just support keeping reference information in a single database---even if we "bless" several different kinds of database (Zotero, org-bibtex, .bib)---and then provide a way to export citations that point to items in that database for all relevant export backends. Eric's question is relevant here: Eric S Fraga writes: > 2. How would I use this starting from an org-bibtex database (which I > typically export to bibtex)? I can envision a couple of possibilities. One simple option would be to switch to managing your reference database with Zotero, by exporting from org-bibtex to .bib, and then importing the .bib into Zotero. I don't want to force that on anyone, though. Another option is to use the org-bibtex to produce .bib at export time, and then use Zotero to read the .bib and process citations when exporting to non-LaTeX formats. This second option is more work, as I don't know of any API for loading items into Zotero's citation processor in BibTeX format. But given that Zotero is able to import .bib files, I imagine this API would not be too much work to build. So, the short answer is: I think you should be able to continue to use org-bibtex as your reference database, and gain citation support when exporting to non-LaTeX backends, perhaps with as little effort as installing Zotero + zotxt and writing #+BIBDB: my-org-bibtex-file.org in your document. Anyway, that would be my goal. Best, Richard