Re: (Not so) Short note about citations in Org

[Denis Maier <denismaier@mailbox.org>]

Hi everyone,

great to see that there's been so much progress since this issue has 
last been discussed! Looks really impressive so far.

Two comments/questions:

- I think it should be possible to use some markup in prefixes and suffixes.
- Will this proposal allow multiple bibliographies? I think that's been 
discussed last year or so.

Best,
Denis

Am 22.04.2021 um 01:34 schrieb Nicolas Goaziou:
> Hello,
> 
> I just rebased "wip-cite-new" branch, which now includes citation
> syntax, and an interface between external citation processors and the
> rest of Org. I'll throw in a demo at the end of this message. TL;DR:
> search for "Demo time".
> 
> As a reminder, the full citation syntax is
> 
>    [cite/style:common prefix ;prefix -@key suffix; ... ; common suffix]
> 
> Everything is optional, except the brackets, "cite" and the colon. Also
> the citation must contain at least a key. So its minimal form is:
> 
>    [cite:@key]
> 
> or its "suppress author" variant:
> 
>    [cite:-@key]
> 
> A noteworthy difference with the last merge is that I removed the
> opportunity to add some Org syntax (emphasis, sub/superscript…) in
> prefixes or suffixes. It makes the code a bit simpler. Of course, I'm
> open to discussion about this change.
> 
> The syntax also introduces three keywords (square brackets are for
> optional arguments):
> 
>     #+bibliography: filename
>     #+print_bibliography: [style]
>     #+cite_export: backend-name [bibliography-style] [citation-style]
> 
> Now, the real novelty is the new "oc.el" library, which is an API to
> operate simply on citations. I paste here its commentary section.
> 
> 
>      This library provides tooling to handle citations in Org, e.g,
>      follow, fontify, and export them, respectively called "follow",
>      "activate" and "export" capabilities. Libraries responsible for
>      providing some, or all, of these capabilities are called "citation
>      processors".
>      
>      Such processors are defined using `org-cite-register-processor'.
>      Using this function, it is possible, in addition to giving it
>      a name, to attach functions associated to capabilities.  As such,
>      a processor handling citation export must set the
>      `:export-citation' property to an appropriate function.  Likewise,
>      "activate" capability require an appropriate `:activate' property,
>      and, unsurprisingly, "follow" capability implies `:follow'
>      property.
>      
>      As a user, the first thing to do is setting a bibliography, either
>      globally with `org-cite-global-bibliography', or locally using one
>      ore more "bibliography" keywords.  Then one can select any
>      registered processor for each capability by providing a processor
>      name to the variables `org-cite-activate-processor' and
>      `org-cite-follow-processor'.
>      
>      The "export" capability is slightly more involved as one need to
>      select the processor providing it, but may also provide a default
>      style for citations and bibliography.  These three parameters, or
>      triplet, can be set in `org-cite-export-processor' variable, or in
>      a document, through the "cite_export" keyword.
>      
>      Eventually, this library provides some tools, mainly targeted at
>      processor implementors.  Most are export-specific and are located
>      in the "Tools only available during export" section.  The few
>      others can be used directly from an Org buffer, or operate on
>      processors.  See "Generic tools" section.
> 
> 
> There are two points of view to consider here. As a user, as stated
> above, you first need to provide a bibliography, for all documents using
> the `org-cite-global-bibliography' variable, or for a single document
> (or a set of documents, using "setupfile" keyword) with
> 
>      #+bibliography: one-file.bib
>      #+bibliography: another-file.ext
>      #+bibliography: "some file with spaces"
> 
> You can use both the variable and the keywords, in which case files are
> _accumulated_ in the list.
> 
> Then when you
> 
>    (require 'org-cite-something)
> 
> the "oc-something.el" library, in addition to possibly many other tools,
> registers a "citation processor", for example `something'. That
> processor may operate on any of the three entry points "activate",
> "follow", or "export". If you are not sure about which one it supports,
> you may inspect the processor with, e.g.,
> 
>    (org-cite-processor-has-capability-p 'something 'follow)
> 
> If this is non-nil, you can set `org-cite-follow-processor' to
> `something'. Once done, calling `org-open-at-point' on a citation starts
> whatever action the processor defined. If the variable is nil, nothing
> happens.
> 
> If you need to use a different processor for a given document, consider
> using file local variables.
> 
> Likewise, you can fontify citations according to a given processor using
> `org-cite-activate-processor'. This time, however, Org provides some
> fontification even when the variable is nil. The default set-up merely
> applies new `org-cite' and `org-cite-key' faces on citations.
> 
> The "export" capability introduces the concept of "style", which is an
> _indication_ to the citation processor, which may or may consider
> applying. More precisely, a style can be set for citations and
> bibliography, at three levels from the most general to the most
> specific:
> 
>    (setq org-cite-export-processor '(something "bibstyle" "citestyle"))
> 
>    #+cite_export: something bibstyle citestyle
> 
>    #+print_bibliography: bibstyle
>    [cite/citestyle:...]
> 
> An export processor is required to support at least one default style
> for citations and bibliography called the "nil" style. It may support
> any number of other styles, and should treat any unknown style
> indication as the "nil" style. So
> 
>    [cite/totallyunknownstyle:...]
> 
> may be treated as
> 
>    [cite/nil: ...]
> 
> which, in turn, is strictly equivalent to
> 
>    [cite: ...]
> 
> Now onto the developer point of view. A citation back-end can provide
> many tools, but in order to interact with Org through the three entry
> points listed earlier, it also needs to define a so-called processor,
> using `org-cite-register-processor' function. For reference, here is its
> docstring.
> 
> 
>        Mark citation processor NAME as available.
> 
>      NAME is a symbol.  BODY is a property list, where the following optional keys
>      can be set:
> 
>      `:activate'
> 
>        Function activating a citation.  It is called with a single argument: a
>        citation object extracted from the current buffer.  It may add text
>        properties to the buffer.  If it is not provided, `org-cite-fontify-default'
>        is used.
> 
>      `:export-bibliography'
> 
>        Function rendering a bibliography.  It is called with five arguments: a list
>        of citations, a list of bibliography files, the style, as a string or nil,
>        the export back-end, as a symbol, and the communication channel, as a
>        property list.
> 
>        It is called at each \"print_bibliography\" keyword in the parse tree.
>        It may return a string or nil.  When it returns nil, the keyword is ignored.
>        Otherwise, the string it returns replaces the keyword in the export output.
> 
>      `:export-citation'    (mandatory for \"export\" capability)
> 
>        Function rendering citations.  It is called with four arguments: a citation
>        object, the style, as a string or nil, the export back-end, as a symbol,
>        and the communication channel, as a property list.
> 
>        It is called on each citation object in the parse tree.  It may return
>        a string or nil.  When it returns nil, the citation is ignored.  Otherwise,
>        the string it returns replaces the citation object in the export output.
> 
>      `:export-finalizer'
> 
>        Function called at the end of export process.  It must accept five
>        arguments: the output, as a string, a list of citations, a list of
>        bibliography files, a list of bibliography styles requested by various
>        \"print_bibliography\" keywords in the document, as strings or nil, and the
>        export back-end, as a symbol.
> 
>        It must return a string, which will become the final output from the export
>        process, barring subsequent modifications from export filters.
> 
>      `:follow'
> 
>        Function called to follow a citation.  It accepts two arguments, the
>        citation or citation reference object at point, and any prefix argument
>        received during interactive call of `org-open-at-point'.
> 
> 
> The "follow" and "activate" capabilities are relatively simple to
> implement because both require a single function to operate. On the
> other hand, "export" capability may require up to three functions. Of
> those, only the `export-citation' function cannot be omitted.
> 
> The "oc.el" library provides several tools to help developers writing
> those functions. For example, `org-cite-list-bibliography-files' may be
> useful when providing "follow" or "activate" capability.
> 
> Likewise, `org-cite-get-references' lists all citation-reference objects
> contained in a citation. For each object, you can extract boundaries as
> buffer positions, the key, the prefix and the suffix. This can be very
> useful for activating a citation reference, e.g., when apply some
> specific key-map on a part of the buffer.
> 
> Most important helper functions available for export are
> `org-cite-list-citations', which provides the _ordered_ list of
> citations in the exported document, and `org-cite-inside-footnote-p',
> which returns the closest (inline) footnote reference or footnote
> definition containing a citation, if any. This function can be paired,
> e.g., with `org-export-get-footnote-number'.
> 
> === Demo time ===
> 
> Let's say I want to implement "oc-demo.el", which will provide
> "activate" and "export" capabilities.
> 
> For activation, I want:
> - the same default fontification (`org-cite' and `org-cite-key' faces),
> - to display the cite key when mouse is over a citation reference,
> - hide the unsightly "cite:" part of the citation.
> 
> I therefore write the following function:
> 
>    (defun org-cite-demo-activate (citation)
>      ;; Apply default fontification, the lazy way.
>      (org-cite-fontify-default citation)
>      ;; Hide "cite:" prefix.
>      (let* ((prefix-start (1+ (org-element-property :begin citation)))
>             (prefix-end (+ 5 prefix-start)))
>        (add-text-properties prefix-start prefix-end '(invisible t))
>        (org-rear-nonsticky-at prefix-start)
>        (org-rear-nonsticky-at prefix-end))
>      ;; Apply `help-echo' on each key reference.
>      (org-with-point-at (org-element-property :contents-begin citation)
>        (let ((end (org-element-property :contents-end citation)))
>          (while (re-search-forward org-element-citation-key-re end t)
>            (add-text-properties
>             (match-beginning 0) (match-end 0)
>             ;; Drop the @ symbol.
>             (list 'help-echo
>                   (substring (match-string-no-properties 0) 1)))))))
> 
> I want to export citations as (Doe, 1999) or (1999) when suppress author
> parameter is set. I also want to support the "plain" style, which
> removes the parenthesis.
> 
> The bibliography should be inserted at the very end of the document,
> listing all bibliography files used, but only if there is some citation
> and some "print_bibliography" keyword in the document.
> 
> I won't use :export-bibliography, because it is called at each
> "print_bibliography" keyword, so possibly in the middle of the document.
> I'll use a finalizer instead.
> 
>    (defun org-cite-demo-export-citation (citation style back-end info)
>      (concat
>       ;; Global prefix, if any.  We use `org-export-data' as prefixes or
>       ;; suffixes may contain special characters that need to be escaped
>       ;; by the export process.
>       (org-export-data (org-element-property :prefix citation) info)
>       (and (not (equal style "plain")) "(")
>       (mapconcat (lambda (ref)
>                    (concat
>                     (org-export-data (org-element-property :prefix ref) info)
>                     (if (org-element-property :suppress-author ref)
>                         "1999"
>                       "Doe, 1999")
>                     (org-export-data (org-element-property :suffix ref) info)))
>                  ;; Grab all references within the citation.
>                  (org-cite-get-references citation)
>                  ", ")
>       (and (not (equal style "plain")) ")")
>       ;; Global suffix, if any.
>       (org-export-data (org-element-property :suffix citation) info)))
> 
>    (defun org-cite-demo-finalizer (output citations bibliography styles backend)
>      (concat output
>              ;; Print bibliography only if there are some citations and
>              ;; a "print_bibliography" keyword in the document.
>              (and citations
>                   styles                ;proof of "print_bibliography"
>                   (concat "\nBibliography: "
>                           (mapconcat #'identity bibliography ", ")
>                           "."))))
> 
> All is left to do is to register the "demo" processor.
> 
>    (org-cite-register-processor 'demo
>      :activate #'org-cite-demo-activate
>      :export-citation #'org-cite-demo-export-citation
>      :export-finalizer #'org-cite-demo-finalizer)
> 
> That's about it.
> 
> You can, if you wish so, try out this demo on the following document:
> 
> --8<---------------cut here---------------start------------->8---
> #+cite_export: demo
> 
> #+bibliography: bib.bib
> #+bibliography: bib2.bib
> 
> Simple reference: [cite:@key]
> 
> Multiple references: [cite:@key1;@key2]
> 
> Suppress author: [cite:-@key]
> 
> Plain style: [cite/plain:@key]
> 
> Full syntax: [cite:common prefix ;prefix @key suffix ;@key2; common suffix]
> 
> #+print_bibliography:
> 
> # Local Variables:
> # org-cite-activate-processor: demo
> # End:
> --8<---------------cut here---------------end--------------->8---
> 
> WDYT?
> 
> Regards,
>