Re: wip-cite status question and feedback

["András Simonyi" <andras.simonyi@gmail.com>]

Hi,

apologies for reacting that late (it seems that I messed up my email filtering
royally) -- it is very nice to see progress in this area. Thanks to all of you
for trying to bring this forward, and, of course, to Bruce for initiating the
thread.

I think that the syntax that has crystallized is a good starting point
for having
dedicated citation support in Org, so in the following I'll concentrate on some
of the infrastructure issues raised.

(i) Default ("built in") citation processor in Org

I think citeproc-el could be a reasonable solution for this. The core is
well-tested (thanks to the CSL test suite), implements a fairly large subset of
the CSL 1.01 standard, and can output citations in a number of formats including
in Org syntax. If I understood correctly, this would require citeproc-el to be
part of Emacs, and not even an ELPA package would suffice. In principle I'm open
to this, but I'm not familiar with the process and there are potential pitfalls,
e.g. citeproc-el currently depends on some "modern" libraries, e.g., "s" or
"dash" which AFAIK are not in Emacs.

(ii) Citation API

NG == Nicolas Goaziou wrote on 8 Apr 2020

  NG> 2. [ ] Decide about API Org should provide for it to be useful. Here are
  NG> some low hanging fruits:
  NG>    - [ ] List all ".bib" files associated to the document,
  NG>    - [ ] List all citations,
  NG>    - [ ] Return citation key at point, if any.
  NG>    - Anything else?

From the citeproc-el (and CSL) perspective, the list of bibliography database
files, the place where the bibliography should be placed (if it's specified) and
the list of citations would be enough, I think. For the proper handling of
footnote-based styles it would be important to provide footnote information
about each citation: is a citation in a footnote and what is its footnote number
(index in the list of footnotes)? (citeproc-org contains a function to collect
this info.)

  NG> Moreover, we need to decide about how external processors could plug into
  NG> the export framework.
  NG>    - [ ] For example, it could be a simple variable bound to a list of
  NG> functions. Each function accepts three arguments: the export back-end, as
  NG> a symbol, the full citation, as a list of triplets (key, prefix, suffix)
  NG> along with global prefix/suffix, and the usual INFO communication channel.
  NG> Does it need more?

Unfortunately I'm unfamiliar with the Org exporter, so this might not be a
problem at all, but there is a seemingly tricky point during export, related,
again, to footnotes in footnote-based citation styles: If a footnote style is
used and a citation is not already in a footnote then a footnote has to be
dynamically generated around the citation during export. The simplest way of
doing this is to wrap the rendered citation (which can contain backend-specific
font-properties, e.g. smallcaps) into an inline Org footnote block. E.g.,
citeproc-org, which basically acts as an Org->Org preprocessor, generates
dynamical footnote citations for html export along the lines of

[fn:: @@html:<span style="font-variant:small-caps">Gödel</span> 1931@@]

If this scenario can be handled by the proposed mechanism then I don't expect
any other problems.

  NG>    - [ ] Also, the prefix/suffix may contain some Org markup, so this
  NG> needs to be also processed. Should it happen before, or after the external
  NG> processor does its job? I.e., should the function translate into Org or
  NG> target format?

This is a very good (and a bit tricky...) question! Since the proposed Org
citation syntax does not contain locator-related slots, the citation processor
has to locate this information in the affixes (in citeproc-org, in the suffix),
parse and remove it. This part should definitely be done in Org format, so I'd
say the Org markup rendering should be after the citation rendering. (With the
proviso that part of the citation processor's output can already be rendered in
the target format as in the previous example.)

  NG> 3. [ ] Specify the kind of basic translation that be done out of the box?
  NG> Ideally, every non-derived back-end should do something, even basic.
  NG> Therefore, we need to propose some translation pattern for each of the
  NG> following:
  NG>    - [ ] ASCII
  NG>    - [ ] HTML
  NG>    - [ ] LaTeX
  NG>    - [ ] ODT
  NG>    - [ ] Texinfo

As an Org->Org preprocessor, citeproc-org supports all these back-ends in the
sense that for HTML and LaTeX it outputs target specific rendering using the
@@latex and @@html syntax and generic Org for the rest.

(iii) Citation visualization

JK == John Kitchin wrote on 8 Apr 2020

  JK> org-ref relies very heavily on the link functionality to provide actions
  JK> on a cite, e.g. to open the pdf, or url, allow sorting, to change the cite
  JK> color when it is a bad key, etc If the new syntax also has that
  JK> capability

I'd add to this that it would be very useful if citations -- similarly to links
-- had a "descriptive" (as opposed to "literal") rendering mode in the buffer as
well, which would show the rendered citation, as produced by a citation
processor (if available). Similarly to links one could toggle between the two
display modes.

(iv) Bibliography format

This might go without saying but perhaps it's worth specifying which
bibliography format(s) should be supported by the "out of the box"
functionality.
My impression is that despite the clear superiority of biblatex and CSL, BibTeX
is still the most widely used format, and should be the first/main target. Later
on, of course, it would be nice to support others, e.g., using a CSL processor
one basically gets CSL-JSON support for free, and there could be even an Org-CSL
format, by analogy with Org-BibTex. What do you think?

Sorry again for replying only now & best regards,

András

On Sun, 24 May 2020 at 15:18, Bruce D'Arcus <bdarcus@gmail.com> wrote:
>
> Hi Bastian,
>
>
> On Sun, May 24, 2020 at 8:12 AM Bastien <bzg@gnu.org> wrote:
> >
> > Hi Bruce,
> >
> > "Bruce D'Arcus" <bdarcus@gmail.com> writes:
> >
> > > I'm not sure of the value of this sort of question thrown in the
> > > middle of a long-running, many year, conversation. You seem to assume
> > > nobody considered this.
> >
> > Well, this sounded a bit harsh, problably more than what was intended.
>
> Yes, probably true.
>
> I was trying to be direct, to avoid a potentially long-winded diversion.
>
> But it was a bit harsh. Sorry Gustav.
>
> > > But to answer anyway ...
> >
> > And your answer was precisely what I was (also) looking for, so thanks
> > for it.  I haven't followed nor helped developments in this area but I
> > hope this can settle down and be widely available.
>
> Indeed; thanks!
>
> Bruce