Re: [PATCH] oc-csl: Improve LaTeX bibliography formatting

["András Simonyi" <andras.simonyi@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3201 bytes --]

Dear All,

first of all, apologies for the delay, unfortunately, I haven't been
able to work on my WIP patches for a while. Now I've attached a new
version of the patch, which hopefully addresses all issues discussed
earlier.

best wishes,
András

On Tue, 8 Nov 2022 at 06:26, Ihor Radchenko <yantar92@posteo.net> wrote:
>
> András Simonyi <andras.simonyi@gmail.com> writes:
>
> >> Also, it would be nice to describe CSL usage and tweaks in the manual.
> >
> > Time permitting I may try to add something, but wouldn't it be a
> > problem if the CSL export processor was discussed in much more detail
> > than the others?
> > I was also thinking about providing a list of available citation
> > substyles but I do not want to make the manual very unbalanced.
>
> Maybe not in the release, but otherwise we need to finish the citation
> section of the manual one way or another. May as well start from CSL
> part.
>
> >> I have two comments here:
> >> 1. Where are all these new commands coming from? They are not used
> >>    directly in the code. Are you tweaking citeproc.el output this way? May
> >>    it be better to use customizations provided by citeproc.el itself?
> >
> > Yes, the citeproc org-latex formatter, which I added specifically for
> > Org, uses these commands in the LaTeX code produced for the
> > bibliography. As citeproc doesn't have customizable variables by
> > design (if I recall correctly, the only exception is 2 hooks), and
> > oc-csl already had some variables concerned with very similar
> > formatting settings (org-cite-csl-latex-hanging-indent,
> > org-cite-csl-html-hanging-indent,
> > org-cite-csl-html-label-width-per-char) I think it is more consistent
> > to have the new ones also in Org.
>
> Thanks for the clarification. I'd prefer to see a similar explanation
> and the details about what the LaTeX variables/commands do in the
> docstring.
>
> >> 2. You are declaring this variable as defcustom, but it is not clear
> >>    what is going to happen if the user changes it. It is not how to
> >>    change this template in meaningful ways either.
> >
> > Right, I can try to detail a bit in the docstring what type of
> > commands and environments have to be provided by the preamble (are
> > expected by citeproc). I tried to follow Timothy's handling of the
> > ox-latex engraved preamble, but a simpler alternative would be to
> > treat it simply as a constant template, at least for the time being --
> > WDYT?
>
> Note that `org-latex-engraved-preamble' explains which packages need to
> be loaded and which commands need to be defined in the preamble. This at
> least make it more clear what the users may change and not break the
> export.
>
> I see not problem keeping this a defcustom, but we definitely need to
> explain the default value and what is required to be in there. At least,
> to make the code readable for future contributors.
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>

[-- Attachment #2: 0001-oc-csl-Improve-LaTeX-bibliography-formatting.patch --]
[-- Type: text/x-patch, Size: 9670 bytes --]

From 7f02881ff1ef9c3dd3eca0cd63a91936dcb90a45 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Andr=C3=A1s=20Simonyi?= <andras.simonyi@gmail.com>
Date: Sun, 11 Dec 2022 18:03:39 +0100
Subject: [PATCH] oc-csl: Improve LaTeX bibliography formatting

* lisp/oc-csl.el (org-cite-csl--output-format): Use the dedicated
'org-latex' citeproc formatter to export references in LaTeX.
(org-cite-csl-latex-preamble, org-cite-csl--generate-latex-preamble,
org-cite-csl-finalizer): Insert a preamble fragment compatible with
the 'org-latex' citeproc formatter.
(org-cite-csl-latex-label-separator,
org-cite-csl-latex-label-width-per-char): Introduce additional
variables to control bibliography formatting.

* etc/ORG-NEWS: Describe the introduced new options.
---
 etc/ORG-NEWS   |  11 +++++
 lisp/oc-csl.el | 126 ++++++++++++++++++++++++++++++++++++++++++-------
 2 files changed, 120 insertions(+), 17 deletions(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 5d5e726e0..6441cdc1f 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -12,6 +12,17 @@ See the end of the file for license conditions.
 Please send Org bug reports to mailto:emacs-orgmode@gnu.org.
 
 * Version 9.7 (not released yet)
+
+** New options
+*** New custom settings for the "csl" citation export processor's LaTeX output
+
+The settings ~org-cite-csl-latex-label-separator~ and
+~org-cite-csl-latex-label-width-per-char~ allow the user to control
+the horizontal positioning of entry labels for labeled bibliography
+styles, and the setting ~org-cite-csl-latex-preamble~ makes it
+possible to customize the entire fragment that is injected into the
+preamble when the "csl" citation processor is used for LaTeX export.
+
 * Version 9.6
 
 ** Important announcements and breaking changes
diff --git a/lisp/oc-csl.el b/lisp/oc-csl.el
index 1ccb74e92..e3f57918c 100644
--- a/lisp/oc-csl.el
+++ b/lisp/oc-csl.el
@@ -214,6 +214,92 @@ Used only when `second-field-align' is activated by the used CSL style."
   :type 'string
   :safe #'stringp)
 
+(defcustom org-cite-csl-latex-label-separator "0.6em"
+  "Distance between citation label and bibliography item for LaTeX
+output in valid LaTeX units.  Used only when `second-field-align'
+is activated by the used CSL style."
+  :group 'org-cite
+  :package-version '(Org . "9.7")
+  :type 'string
+  :safe #'stringp)
+
+(defcustom org-cite-csl-latex-label-width-per-char "0.45em"
+  "Character width in LaTeX units for calculating entry label widths.
+Used only when `second-field-align' is activated by the used CSL
+style."
+  :group 'org-cite
+  :package-version '(Org . "9.7")
+  :type 'string
+  :safe #'stringp)
+
+;; The following was inspired by and in many details follows how
+;; Pandoc's (<https://github.com/jgm/pandoc>) default LaTeX template
+;; handles CSL output.  Many thanks to the author, John MacFarlane!
+(defcustom org-cite-csl-latex-preamble
+  "\\usepackage{calc}
+\\newlength{\\cslhangindent}
+\\setlength{\\cslhangindent}{[CSL-HANGINDENT]}
+\\newlength{\\csllabelsep}
+\\setlength{\\csllabelsep}{[CSL-LABELSEP]}
+\\newlength{\\csllabelwidth}
+\\setlength{\\csllabelwidth}{[CSL-LABELWIDTH-PER-CHAR] * [CSL-MAXLABEL-CHARS]}
+\\newenvironment{cslbibliography}[2] % 1st arg. is hanging-indent, 2nd entry spacing.
+ {% By default, paragraphs are not indented.
+  \\setlength{\\parindent}{0pt}
+  % Hanging indent is turned on when first argument is 1.
+  \\ifodd #1
+  \\let\\oldpar\\par
+  \\def\\par{\\hangindent=\\cslhangindent\\oldpar}
+  \\fi
+  % Set entry spacing based on the second argument.
+  \\setlength{\\parskip}{\\parskip +  #2\\baselineskip}
+ }%
+ {}
+\\newcommand{\\cslblock}[1]{#1\\hfill\\break}
+\\newcommand{\\cslleftmargin}[1]{\\parbox[t]{\\csllabelsep + \\csllabelwidth}{#1}}
+\\newcommand{\\cslrightinline}[1]
+  {\\parbox[t]{\\linewidth - \\csllabelsep - \\csllabelwidth}{#1}\\break}
+\\newcommand{\\cslindent}[1]{\\hspace{\\cslhangindent}#1}
+\\newcommand{\\cslbibitem}[2]
+  {\\leavevmode\\vadjust pre{\\hypertarget{citeproc_bib_item_#1}{}}#2}
+\\makeatletter
+\\newcommand{\\cslcitation}[2]
+ {\\protect\\hyper@linkstart{cite}{citeproc_bib_item_#1}#2\\hyper@linkend}
+\\makeatother"
+  "LaTeX preamble content inserted by the `csl' citation processor.
+
+This preamble can be anything as long as it provides definitions
+for the environment and commands that Citeproc's `org-latex'
+formatter uses for formatting citations and bibliographies.  In
+particular, it has to define
+- the commands \\cslblock{<text>}, \\cslleftmargin{<text>},
+  \\cslrightinline{<text>} and \\cslindent{<text>} for formatting
+  text that have, respectively, the CSL display attributes
+  `block', `left-margin', `right-inline' and `indent';
+- the commands \\cslcitation{<item_no>}{<item_text>} and
+  \\cslbibitem{<item_no>}{<item_text>}, which are used to
+  format individual citations and bibliography items, including
+  hyperlinking citations to the corresponding bibliography entry
+  using their numerical id, which is passed as the first,
+  <item_no> argument;
+- and the environment \\cslbibliography{<hanging-indent>}{<entry-spacing>},
+  in which bibliographies are wrapped; the value of the
+  <hanging-indent> argument is 1 if hanging indent should be
+  applied and 0 if not, while the <entry-spacing> argument is an
+  integer specifying the number of extra line-heights
+  required between bibliography entries in addition to normal
+  line spacing.
+
+When present, the placeholders [CSL-HANGINDENT], [CSL-LABELSEP],
+[CSL-LABELWIDTH-PER-CHAR] and [CSL-MAXLABEL-CHARS] are replaced,
+respectively, by the contents of the customizable variables
+`org-cite-csl-latex-hanging-indent', `org-cite-csl-latex-label-separator',
+`org-cite-csl-latex-label-width-per-char', and the maximal label length
+in the bibliography measured in characters."
+  :group 'org-cite
+  :type 'string
+  :package-version '(Org . "9.7"))
+
 \f
 ;;; Internal variables
 (defconst org-cite-csl--etc-dir
@@ -413,7 +499,7 @@ corresponding to one of the output formats supported by Citeproc: `html',
   (let ((backend (plist-get info :back-end)))
     (cond
      ((org-export-derived-backend-p backend 'html) 'html)
-     ((org-export-derived-backend-p backend 'latex) 'latex)
+     ((org-export-derived-backend-p backend 'latex) 'org-latex)
      (t 'org))))
 
 (defun org-cite-csl--style-file (info)
@@ -670,6 +756,21 @@ value is the bibliography as rendered by Citeproc."
             (plist-put info :cite-citeproc-rendered-bibliographies result)
             result)))))
 
+(defun org-cite-csl--generate-latex-preamble (info)
+  "Generate the CSL-related part of the LaTeX preamble.
+INFO is the export state, as a property list."
+  (let* ((parameters (cadr (org-cite-csl--rendered-bibliographies info)))
+         (max-offset (cdr (assq 'max-offset parameters)))
+         (result org-cite-csl-latex-preamble))
+    (map-do (lambda (placeholder replacement)
+              (when (string-match placeholder result)
+                (setq result (replace-match replacement t t result))))
+            `("\\[CSL-HANGINDENT\\]" ,org-cite-csl-latex-hanging-indent
+              "\\[CSL-LABELSEP\\]" ,org-cite-csl-latex-label-separator
+              "\\[CSL-LABELWIDTH-PER-CHAR\\]" ,org-cite-csl-latex-label-width-per-char
+              "\\[CSL-MAXLABEL-CHARS\\]" ,(number-to-string max-offset)))
+    result))
+
 \f
 ;;; Export capability
 (defun org-cite-csl-render-citation (citation _style _backend info)
@@ -688,8 +789,8 @@ INFO is the export state, as a property list."
 INFO is the export state, as a property list."
   (org-cite-csl--barf-without-citeproc)
   (pcase-let*  ((format (org-cite-csl--output-format info))
-		(`(,outputs ,parameters) (org-cite-csl--rendered-bibliographies info))
-		(output (cdr (assoc props outputs))))
+                (`(,outputs ,parameters) (org-cite-csl--rendered-bibliographies info))
+                (output (cdr (assoc props outputs))))
     (pcase format
       ('html
        (concat
@@ -714,12 +815,7 @@ INFO is the export state, as a property list."
               org-cite-csl-html-hanging-indent
               org-cite-csl-html-hanging-indent))
         output))
-      ('latex
-       (if (cdr (assq 'hanging-indent parameters))
-           (format "\\begin{hangparas}{%s}{1}\n%s\n\\end{hangparas}"
-                   org-cite-csl-latex-hanging-indent
-                   output)
-         output))
+      ('org-latex output)
       (_
        ;; Parse Org output to re-export it during the regular export
        ;; process.
@@ -730,18 +826,14 @@ INFO is the export state, as a property list."
 OUTPUT is the export document, as a string.  INFO is the export state, as a
 property list."
   (org-cite-csl--barf-without-citeproc)
-  (if (not (eq 'latex (org-cite-csl--output-format info)))
+  (if (not (eq 'org-latex (org-cite-csl--output-format info)))
       output
     (with-temp-buffer
       (save-excursion (insert output))
       (when (search-forward "\\begin{document}" nil t)
-        (goto-char (match-beginning 0))
-        ;; Ensure that \citeprocitem is defined for citeproc-el.
-        (insert "\\makeatletter\n\\newcommand{\\citeprocitem}[2]{\\hyper@linkstart{cite}{citeproc_bib_item_#1}#2\\hyper@linkend}\n\\makeatother\n\n")
-        ;; Ensure there is a \usepackage{hanging} somewhere or add one.
-        (let ((re (rx "\\usepackage" (opt "[" (*? nonl) "]") "{hanging}")))
-          (unless (re-search-backward re nil t)
-            (insert "\\usepackage[notquote]{hanging}\n"))))
+	(goto-char (match-beginning 0))
+	;; Insert the CSL-specific parts of the LaTeX preamble.
+	(insert (org-cite-csl--generate-latex-preamble info)))
       (buffer-string))))
 
 \f
-- 
2.25.1