Re: Citation processing via Zotero + zotxt

[John Kitchin <jkitchin@andrew.cmu.edu>]

Matt Lundin writes:

> Hi John,
>
> John Kitchin <jkitchin@andrew.cmu.edu> writes:
>
>> If a reference type is not listed in the CSL, it also will not be
>> supported by CSL I suppose.
>
> How is this different than biblatex or bibtex? A user could just modify
> the style or put in a request with a maintainer.

Its not different. Just to point out CSL has limitations too.

>> I also suppose the CSL must be backend specific to output formats
>> appropriate to org, html, LaTeX, markdown, etc... for any particular
>> style.
>
> AFAIU, CSL styles are backend agnostic (otherwise they wouldn't be of
> much use). It is the processor (citeproc-js, pandoc, etc.) that takes
> the instructions (e.g., font-style="italic" in a CSL file) and adds the
> appropriate markup for a defined backend. So once you add a new output
> format to a processor, it works with all styles.

I believe that. I still don't totally see where font-style="italic" gets
converted to /text/ or <i>text</i> or \it{text} etc... I trust it
happens, I just don't see it. Where does one do something fancier like:
<i title="italics text"><a href="www.italics.org">text</a></i>

It sounds trivial, but imagine a day when we all have an orcid, and our
names are linked to our orcid pages in bibliographies, e.g. my name
would be <a href="http://orcid.org/0000-0003-2625-9232">J. R.
Kitchin</a> or some other format.

See this highly linked bibliography:
http://kitchingroup.cheme.cmu.edu/dept-publications-2014.html
Where almost every part of each entry is linked to something (true to
some stuff that is behind a paywall, but lets not get distracted by
that). Is that something a CSL/citation processor could do? These aren't
reasons not to do CSL, and they aren't mission critical to citations.
They just make them richer and more useful. And to be fair, the
information these pieces are linked to came from a bibliographic record
from Scopus, not a bibtex file I maintain, which only has a doi in it to
access that information.

Clearly I still don't see how a cite-processor actually works. I gather
that
1. you extract citation data from a document.
2. send citation data, style and bibliography data to the citation
processor
3. It returns replacement text, and the bibliography string
4. you substitute the replacements in the text and insert the
bibliography string somewhere.

If the CSL doesn't have the backend information, and the citation
processor doesn't know about org/html/etc... then somewhere between step
3 and 4 you add the formatting right? Does the processor get another
piece of information to tell it how to format the output? For example,
if your CSL says a citation should be superscripted, how does the
citation processor know to output <sup>4</sup> vs. $^4$? or ^{4}.


>> We should not try to support all of these things. We could support a
>> small number of things that could be improved or increased in the
>> future.
>
> I would suggest that tapping into a CSL tool like zotero of citeproc-js
> is in fact *a small thing* we can do right now that would have a big
> payoff for lots of users, even if it does not support 100% of use
> cases.

I have no objection to it. Getting high quality references into Word
documents from org-mode is one barrier to convincing more people
org-mode is a competitive writing tool for publications in my mind.

Clearly we need an external program for this. I looked at pandoc before
http://kitchingroup.cheme.cmu.edu/blog/2015/01/29/Export-org-mode-to-docx-with-citations-via-pandoc/
and it was ok, but had some issues getting to Word with citations.

Would zotero or citeproc-js be any better?

>> The only time-tested, publication quality solutions for citations in
>> my opinion right now are bib(la)tex, MS Word/reference manager, and
>> "by hand". Even these get "edited" in their final print versions by
>> journals.
>
> Is this assessment based on your particular disciplinary experience?
Certainly ;) Those have met 100% of my needs, and about 10% of my
desires in scientific publishing for the past 15 years. org-ref now
meets 99.9% of my desires ;)

Also, by time-tested, I mean I have published papers by those methods
specifically, so I know they work (including using org-mode to make
bibtex/latex files). I have not published any papers using org mode with
export to Word, so I don't know if it is possible to do it. The final
details of formatting may prove too difficult in some cases for direct
export.

> I
> ask because many of us in the humanities have not enjoyed the benefits
> of automated, text-based citation processing until quite recently, so
> *both* biblatex and CSL seem awesome.
So even Word/Endnote\|Papers\|Zotero has not been a
citation/bibliography solution?

>The citation style in my field
> (the Chicago Manual of Style) is more quirky and complex than any
> scientific citation style. Thus, it is likely more feasible to implement
> basic bibtex functionality in lisp than it is to re-implement
> biblatex-chicago.[fn:1] CSL offers the advantage of allowing export to
> backends that can easily be converted to Word (the format that
> humanities publishers require).

Somewhat ironically, CSL won't support some standard citation formats in
chemistry journals! Luckily, those are supported in bibtex.

>> It might start making more sense to think of a lisp based citation
>> processor. It might even address some limitations of bib(la)tex.
>
> That would be very cool, especially if we could import/convert CSL files
> (I don't want to rewrite all 1200+ lines of the
> chicago-fullnote-bibliography CSL style). :)

It is easy to read in the xml files, and they read into a lisp data
structure. That isn't helpful on its own, of course, without a lisp
citation processor to send them too ;)

>
> Matt
>
> Footnotes:
>
> [fn:1] I cloned the CSL repository and did a quick sort by word count.
> Not surprisingly, the longest files were all in the humanities:
>
>     1296     2700    44202 chicago-fullnote-bibliography-fr.csl
>     1273     2590    40935 chicago-fullnote-bibliography.csl
>     1264     2576    40674 chicago-fullnote-bibliography-no-ibid.csl
>     1241     2531    39515 chicago-library-list.csl
>     1241     2530    39535 chicago-annotated-bibliography.csl
>     1240     2524    39445 chicago-note-bibliography.csl
>     1235     2506    40168 zeitschrift-fur-religionswissenschaft-note.csl
>     1227     2508    39077 chicago-note-biblio-no-ibid.csl
>     1132     2620    41990 mcgill-fr.csl
>     1060     2149    34008 moorlands-college.csl
>      998     2175    34470 lluelles.csl
>      927     2066    31551 lluelles-no-ibid.csl
>      911     1862    29828 proinflow.csl
>      906     1961    28990 irish-historical-studies.csl
>      878     1828    29238 universite-laval-faculte-de-theologie-et-de-sciences-religieuses.csl
>      862     1830    29437 chicago-author-date-fr.csl
>      856     1782    28244 oxford-studies-in-ancient-philosophy.csl
>      809     1941    27717 university-college-dublin-school-of-history-and-archives.csl
>      809     1796    26790 turabian-fullnote-bibliography.csl
>      806     1857    27140 wheaton-college-phd-in-biblical-and-theological-studies.csl
>      796     1791    26472 modern-language-association-6th-edition-note.csl
>      793     1671    26453 sheffield-hallam-university-history.csl
>      792     1910    30773 pour-reussir-note.csl
>      788     1792    26377 svensk-exegetisk-arsbok.csl
>      781     1782    26316 early-christianity.csl
>      779     1772    26038 society-of-biblical-literature-fullnote-bibliography.csl
>      775     1808    26461 new-testament-studies.csl
>      768     2350    29928 clio-medica.csl
>      714     1533    22993 iso690-author-date-cs.csl
>      708     1578    25548 chicago-author-date-basque.csl
>      708     1520    22731 iso690-author-date-sk.csl
>      707     1605    24582 melbourne-school-of-theology.csl
>      701     1533    22280 moore-theological-college.csl
>      696     1756    25783 associacao-brasileira-de-normas-tecnicas-ufjf.csl
>      694     1474    22229 podzemna-voda.csl
>      692     2315    29639 foerster-geisteswissenschaft.csl
>      692     1591    24881 oscola.csl

--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu