citations: org-cite vs org-ref 3.0

citations: org-cite vs org-ref 3.0

[Vikas Rawal]

[-- Attachment #1: Type: text/plain, Size: 858 bytes --]

I am sorry I have not followed the discussions recently. I had
happily settled on org-cite but I find that much has changed again.

In particular, John Kitchin has given up the work on org-ref-cite and
switched back to org-ref. There is a new org-ref version 3.0 which is not
compatible with org-cite.

This obviously creates many problems including that two people using
different citation systems cannot share org files.

What is the general view of the community about this? Is there a
comprehensive discussion of pros and cons of each?

What is everyone doing? I was an org-ref user for long before I switched to
org-cite. I can now shift to citar but since I am an ivy user, the switch
is not trivial. Also, I like many helper functions that John has created,
and would have to miss those if I do not use org-ref/org-ref-cite.

This seems messy.

Vikas

[-- Attachment #2: Type: text/html, Size: 1073 bytes --]

Re: citations: org-cite vs org-ref 3.0

[Nicolas Goaziou]

Hello,

Vikas Rawal <vikasrawal@gmail.com> writes:

> This obviously creates many problems including that two people using
> different citation systems cannot share org files.

Indeed.
>
> What is the general view of the community about this?

I don't know about the general view of the community, but, as a data
point, I find it very sad.

That's not helpful, tho.

Regards,
-- 
Nicolas Goaziou

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Sun, Mar 20, 2022 at 8:09 AM Vikas Rawal <vikasrawal@gmail.com> wrote:

> What is the general view of the community about this? Is there a comprehensive discussion of pros and cons of each?

Not really, but there's John's summary here:

https://github.com/jkitchin/org-ref#what-about-org-cite

The high-level discussion at the beginning, including the enumerated
points, seems right to me.

The following point that "org-cite does not meet my citation and
technical document publishing needs, and it was not possible to
integrate it into org-ref without compromising those" is more subject
to debate, particularly the first clause.

I don't see any practical advantage to the org-ref syntax and model,
unless you include cross-references and such.

But that's because I just don't think cross-references and indexes
should be handled in org-cite; I think if we need improvements in
existing cross-references etc support, we should add those there.

I suspect he's also meaning the different ways that citation
commands/styles are handled in the two systems.

Here's an example from org-ref 3:

[[citealp:See &kitchin-2015-examp page 2]]

Note that first piece: "citealp". That's the command, which org-ref
will output directly to LaTeX/Natbib as \citealp. Note that command
only makes sense for natbib.

Org-cite has a different abstraction here: the style/substyle system.
So the above would be ...

[cite/bare:See @kitchin-2015-examp page 2]

... and then whatever export processor would map that "bare" style to
the appropriate output.

So that leaves the last bit; the "not possible" point, which I can't
address. It would obviously be nice if org-ref supported org-cite in
the future.

> What is everyone doing? I was an org-ref user for long before I switched to org-cite. I can now shift to citar but since I am an ivy user, the switch is not trivial. Also, I like many helper functions that John has created, and would have to miss those if I do not use org-ref/org-ref-cite.

A month or so ago, John offered to turn over maintenance of
org-ref-cite to someone else, which might be one good option for
someone who uses ivy, in particular, and interested in developing it
further.

More generally, the modular design of org-cite should result, in time,
with diverse components, including for different completion
frameworks. For example, I think it'd be pretty easy to create an
insert process for helm-bibtex or ivy-bibtex, and a follow processor
for bibtex-completion.

Or alternatively, as citar now no longer requires bibtex-completion,
someone could write a small citar front-end for ivy as an insert
processor.

The whole point of the org-ctite design is it should be easy for users
to mix-and-match different pieces, and for documents to remain
compatible across users and export backends. It also allows developers
to focus on those small components.

Bruce

Re: citations: org-cite vs org-ref 3.0

[Ihor Radchenko]

Vikas Rawal <vikasrawal@gmail.com> writes:

> What is the general view of the community about this? Is there a
> comprehensive discussion of pros and cons of each?

Prof. Kitchin himself provided a summary on why he decided to give up on
using org-cite. See https://github.com/jkitchin/org-ref/issues/892

> What is everyone doing? I was an org-ref user for long before I switched to
> org-cite. I can now shift to citar but since I am an ivy user, the switch
> is not trivial. Also, I like many helper functions that John has created,
> and would have to miss those if I do not use org-ref/org-ref-cite.

I personally used org-ref for a long time simply because no
alternatives. I dislike org-ref because Prof. Kitchin prefers to do
things in very specific way and it is not always possible to customise
them. It is hard to use only a part of org-ref without using everything
and getting surprising behaviour.

Now, I am trying to switch to org-cite + .org files as bibliography. But
the whole thing is an ad-hoc mess.

For citar, why not simply using ivy-bibtex? It supports org-cite, AFAIK.

Best,
Ihor

Re: citations: org-cite vs org-ref 3.0

[Vikas Rawal]

[-- Attachment #1: Type: text/plain, Size: 1034 bytes --]

>
> > What is the general view of the community about this?
>
> I don't know about the general view of the community, but, as a data
> point, I find it very sad.
>

Exactly how I feel. Particularly so because org-cite was indeed inspired
from org-ref and all the great work John has done on this over the years. I
do not see any substantive discussion of the limitations that John has
pointed out with org-cite in the archives of the mailing list. We should
have a conversation about those.

If I understand correctly, the main limitation according to him is that
org-cite does not handle cross-references. Is that the only problem? What
do others think about it? Do we want the citation system to be able to
handle cross-references? If not, do we need a separate system that handles
cross-references beyond what is available in org-mode already? Can that
system not be created in a manner that would not break the org-cite syntax?

I think we should have a conversation and collectively think of what is the
best way forward.

Vikas

[-- Attachment #2: Type: text/html, Size: 1332 bytes --]

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Sun, Mar 20, 2022 at 10:08 AM Vikas Rawal <vikasrawal@gmail.com> wrote:
>>
>> > What is the general view of the community about this?
>>
>> I don't know about the general view of the community, but, as a data
>> point, I find it very sad.
>
>
> Exactly how I feel. Particularly so because org-cite was indeed inspired from org-ref and all the great work John has done on this over the years. I do not see any substantive discussion of the limitations that John has pointed out with org-cite in the archives of the mailing list. We should have a conversation about those.
>
> If I understand correctly, the main limitation according to him is that org-cite does not handle cross-references. Is that the only problem? What do others think about it? Do we want the citation system to be able to handle cross-references? If not, do we need a separate system that handles cross-references beyond what is available in org-mode already? Can that system not be created in a manner that would not break the org-cite syntax?
>
> I think we should have a conversation and collectively think of what is the best way forward.

Yes, and no.

Conversation on org-cite took place on this list for years, with
diverse people, with diverse expertise and POV, cycling in and out
over that time.

John raised the cross-reference question towards the very end, during
the testing period, when it was almost merged.

https://www.mail-archive.com/emacs-orgmode@gnu.org/msg138349.html

I still think the path forward on that is TBD, though as I said, I
don't think it should be in scope for org-cite per se, but org more
generally yes.

If he has some other issues (which I suspect he does; maybe related to
fontification?) that might be addressed, I'm sure he'll raise them.

Bruce

Re: citations: org-cite vs org-ref 3.0

[Thomas S. Dye]

Ihor Radchenko <yantar92@gmail.com> writes:

> Vikas Rawal <vikasrawal@gmail.com> writes:
>
>> What is the general view of the community about this? Is there 
>> a
>> comprehensive discussion of pros and cons of each?
>
> Prof. Kitchin himself provided a summary on why he decided to 
> give up on
> using org-cite. See 
> https://github.com/jkitchin/org-ref/issues/892
>
This claim in Kitchin's summary surprised me:

"It comes down to org-ref using bibtex/biblatex as the predominant 
citation processor, and org-cite using CSL. These two processors 
have different syntaxes, and I don't think it is possible to have 
a single approach that works for both of them without making 
compromises in capability."

I wonder what are the "compromises in capability" required to 
support both?

I haven't found them yet, though I'm just now finishing old 
projects started prior to the release of org-cite and haven't 
worked through a LaTeX export project with it.  My first, and so 
far only, project with org-cite was for a simple html-based 
presentation using CSL, and I was pleased with the results and 
relative ease of setup.

All the best,
Tom

--
Thomas S. Dye
https://tsdye.online/tsdye

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Sun, Mar 20, 2022 at 9:42 AM Ihor Radchenko <yantar92@gmail.com> wrote:

> For citar, why not simply using ivy-bibtex? It supports org-cite, AFAIK.

Not really; or rather minimally.

Ivy-bibtex supports, for example, inserting of org-cite citations, but
not via org-cite-insert.

So there are currently no org-cite processors for ivy-bibtex etc.

Bruce

Re: citations: org-cite vs org-ref 3.0

[Dominik Schrempf]

[-- Attachment #1: Type: text/plain, Size: 1953 bytes --]

“Thomas S. Dye” <tsd@tsdye.online> writes:

> Ihor Radchenko <yantar92@gmail.com> writes:
>
>> Vikas Rawal <vikasrawal@gmail.com> writes:
>>
>>> What is the general view of the community about this? Is there a
>>> comprehensive discussion of pros and cons of each?
>>
>> Prof. Kitchin himself provided a summary on why he decided to give up on
>> using org-cite. See <https://github.com/jkitchin/org-ref/issues/892>
>>
> This claim in Kitchin’s summary surprised me:
>
> “It comes down to org-ref using bibtex/biblatex as the predominant citation
> processor, and org-cite using CSL. These two processors have different syntaxes,
> and I don’t think it is possible to have a single approach that works for both
> of them without making compromises in capability.”
>
> I wonder what are the “compromises in capability” required to support both?
>
> I haven’t found them yet, though I’m just now finishing old projects started
> prior to the release of org-cite and haven’t worked through a LaTeX export
> project with it.  My first, and so far only, project with org-cite was for a
> simple html-based presentation using CSL, and I was pleased with the results and
> relative ease of setup.
>
> All the best,
> Tom

For what it’s worth, I use `org-ref` because fine-grained citation export with
LaTeX (using BibTeX or BibLaTeX) only works with `org-ref`, and not with
`org-cite`.

If I remember correctly, `org-cite` exports the formatted citations into the
LaTeX file, which is a no-go for me (let me know if I am wrong). Then, for
example, it is really difficult to use different style files, etc. With
`org-ref`, I can use BibLaTeX with specific style files. (Journal require
different style files). I can even use my personal LaTeX classes (with specific
bibliography styles) for Org export.

Not sure if this helps…, but this is maybe what John Kitchin is referring to.

Dominik

Re: citations: org-cite vs org-ref 3.0

[Vikas Rawal]

[-- Attachment #1: Type: text/plain, Size: 944 bytes --]

>
>
>
> For what it’s worth, I use `org-ref` because fine-grained citation export
> with
> LaTeX (using BibTeX or BibLaTeX) only works with `org-ref`, and not with
> `org-cite`.
>

 It would help if you can explain what you mean.

If I remember correctly, `org-cite` exports the formatted citations into the
> LaTeX file, which is a no-go for me (let me know if I am wrong).


This is not correct. org-cite exports citation commands to LaTeX, and LaTeX
does rest of the processing. I do not think there is any difference in the
approach between org-cite and org-ref.


> Then, for
> example, it is really difficult to use different style files, etc. With
> `org-ref`, I can use BibLaTeX with specific style files. (Journal require
> different style files). I can even use my personal LaTeX classes (with
> specific
> bibliography styles) for Org export.
>
>
Surely org-cite allows you to do all of this.

Vikas

[-- Attachment #2: Type: text/html, Size: 1514 bytes --]

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Sun, Mar 20, 2022 at 4:21 PM Dominik Schrempf
<dominik.schrempf@gmail.com> wrote:

> For what it’s worth, I use `org-ref` because fine-grained citation export with
> LaTeX (using BibTeX or BibLaTeX) only works with `org-ref`, and not with
> `org-cite`.

Part of the challenge is this isn't an apples-apples comparison;
org-cite a framework, and it's the processors that really provide the
functionality.

> If I remember correctly, `org-cite` exports the formatted citations into the
> LaTeX file ...

This is not true.

Or rather, it's only true if you use the oc-csl export processor; you
can use other export processors (like oc-biblatex) to get whatever
LaTeX output you want, and ...

>  .... it is really difficult to use different style files, etc. With
> `org-ref`, I can use BibLaTeX with specific style files. (Journal require
> different style files). I can even use my personal LaTeX classes (with specific
> bibliography styles) for Org export.

... I believe Nicolas recently added support for customizing the
export styles for biblatex.

Bruce

Re: citations: org-cite vs org-ref 3.0

[chris]

On Sunday, 20 March 2022 20:44:50 CET Bruce D'Arcus wrote:
> On Sun, Mar 20, 2022 at 9:42 AM Ihor Radchenko <yantar92@gmail.com> wrote:
> > For citar, why not simply using ivy-bibtex? It supports org-cite, AFAIK.
> 
> Not really; or rather minimally.

I use `org-cite` with a very minimal configuration, and it works very well. I 
don't use `ivy` at all. I don't use `helm` at all. Those are very large 
framework and should not be forced to people. (I don't use `org-ref`.)

I only use `citar` with minimal configuration. I use `vertico` for the 
completion. `citar` is simple enough for me to be able to read and understand 
a large part of it.

IMO the more layers of code there are, the more difficult it is to have things 
work right. And similarly with the size of the code.

Chris

> 
> Ivy-bibtex supports, for example, inserting of org-cite citations, but
> not via org-cite-insert.

And I have `org-cite-insert` working straight out of the box.

> 
> So there are currently no org-cite processors for ivy-bibtex etc.
> 
> Bruce

Re: citations: org-cite vs org-ref 3.0

[Dominik Schrempf]

[-- Attachment #1: Type: text/plain, Size: 1469 bytes --]

“Bruce D’Arcus” <bdarcus@gmail.com> writes:

> On Sun, Mar 20, 2022 at 4:21 PM Dominik Schrempf
> <dominik.schrempf@gmail.com> wrote:
>
>> For what it’s worth, I use `org-ref` because fine-grained citation export with
>> LaTeX (using BibTeX or BibLaTeX) only works with `org-ref`, and not with
>> `org-cite`.
>
> Part of the challenge is this isn’t an apples-apples comparison;
> org-cite a framework, and it’s the processors that really provide the
> functionality.
>
>> If I remember correctly, `org-cite` exports the formatted citations into the
>> LaTeX file …
>
> This is not true.

Thanks for correcting me on that. I remember having LaTeX files with full
citations in them, but maybe I messed something up.

>
> Or rather, it’s only true if you use the oc-csl export processor; you
> can use other export processors (like oc-biblatex) to get whatever
> LaTeX output you want, and …
>
>>  …. it is really difficult to use different style files, etc. With
>> `org-ref`, I can use BibLaTeX with specific style files. (Journal require
>> different style files). I can even use my personal LaTeX classes (with specific
>> bibliography styles) for Org export.
>
> … I believe Nicolas recently added support for customizing the
> export styles for biblatex.
>
> Bruce

When I tried setting `org-cite` up, it didn’t work. Maybe I should give it
another try at some point. For now, I am happy with `org-ref`.

Re: citations: org-cite vs org-ref 3.0

[John Kitchin]

I do not think it is productive for the community to say or consider it
is a sad situation. Many good things have emerged from these
discussions, even if it is not yet consensus on a solution. It is a
complex problem, with many years of effort by many people on each side.
That is an indication of how ambitious this project is and that there
may be more than one solution that is needed. It pains me quite a bit
there is a sentiment of fractionation, and that I may be contributing to
it.

My regular job workload the past few years has been crushing, and I have
not had the time to participate in this that I wish I had. I am not sure
I can add much here without sounding or feeling defensive about org-ref,
and my decision to continue supporting and developing it. I have thought
about this for most of the day, and in the (very long, apologies in
advance) response that follows I will do my best to provide a balanced
perspective (from my point of view) on the situation.

Some specific context that is important to me is that I wrote org-ref
long ago to solve a specific problem for me in the preparation of
scientific publications that are destined for LaTeX export. I intended
it to provide nearly equivalent bib(la)tex citation export, and as
reasonable an export as possible for everything else. I use org-ref
professionally, and it is a complete solution for me. I simply cannot
compromise on the capability org-ref provides me, or wait for an
alternative complete solution in org-mode. I have work I have to do now,
and org-ref lets me do it. This alone is reason enough for me to
continue using, developing and supporting org-ref. I understand org is
not intended to be a substitute for writing LaTeX, but it is a fact of
my job that I have to do that.

There are more than 8 years of legacy org-ref documents. I have written
40+ scientific papers with it, and countless technical documents with
more than 8000 cite links among them. org-ref has exceeded 190K
downloads from MELPA, so I feel obligated to maintain org-ref for
myself, and those users. org-ref may be heavyweight in bundling a lot of
capability together that could be separated into individual packages,
but it is also convenient for people who need it, and a GitHUB issue or
pull request away from new features. I remain committed to supporting
this, and I do it in a way I can manage, hence the monolithic package
design.

org-cite was also developed to solve some specific citation problems for
others that org-ref did not address well at the time it was started. I
believe those were issues like better pre/post note support, and
integration with CSL.

I think org-ref and org-cite have different priorities, they solve
different problems with different approaches, and they have different
pros and cons. I believe there are mutually incompatible compromises one
must make here because the specific choices you make determine what is
easy and what is possible. There is not a direct mapping of bib(la)tex
and CSL as a citation processor. They are different programs and they
don't share citation commands, or style information. It is inevitable
that something will be lost in the translation between these from a
single source like org, and whether that loss is acceptable depends on
what you need. For many things, close enough is ok for me, but for
manuscripts and proposals, they must be perfect, and in bib(la)tex form
for the journals I publish in. It is not that one thing is possible in
one and not the other; it is that you have to compromise one to do the
other no matter what you choose and that typically makes one thing or
the other easier to do. I am not even sure it is possible to do
everything one can do in bib(la)tex with CSL, for example, I don't know
the equivalent of \citenum in CSL. org-ref sides on making bib(la)tex
easy, and CSL is possible.

With org-cite, CSL can be readily used across export backends, more
bibliography database formats are supported, and it is also possible to
get LaTeX export, although there is no goal to fully support all of
bib(la)tex. A pure org approach (e.g. org-files as bibliography
database) can be used, there are a lot of CSL styles to work with, and
you can develop or adapt your own style if needed. With reasonable
defaults this should be a straightforward introduction to using
citations for new users that works out of the box. I support that.

With org-ref, bib(la)tex export is almost fully supported, and is easy,
and other exports via CSL are possible. Of course, you must have a
working LaTeX installation, only bibtex databases are supported, and
working knowledge of bib(la)tex is required to leverage this. Whether
you want it or not, you get a lot of extra utilities for getting bibtex
entries from a DOI, and support for cross-references, indexes,
glossaries and acronyms. This is a lot to ask of people who don't need
it, and convenient for those who do. I support this.

Which one of these approaches is right depends on what you want to be
easy. Neither is right or wrong, that is determined by what you need at
the time and what you prioritize in your solution. It is even possible
you need both approaches at different times. The two approaches are not
compatible, but it is org-mode after all, and you can certainly convert
back and forth between them for the most part.

Both projects have benefited from this discussion a lot. org has
org-cite now, and org-ref now handles pre/post notes like org-cite does,
it supports CSL much better, and is even a little more modular, lighter
weight, and more easily integrated with other completion backends than
ivy or helm. That should broadly be viewed as a win-win situation.

Here are some factors that have prevented me from deprecating org-ref. I
spent about a month and half trying to get a solution to this at
https://github.com/jkitchin/org-ref-cite, and I don't make these
observations lightly. That solution was complete and highly functional
for org-cite, but as I describe below not a replacement for org-ref at
this time. I still welcome a new maintainer for this code.

Cross-references are critical for me; without them, there is no path
forward for me with org-cite. I did work on a cross-reference approach
that leveraged org-cite syntax
(https://github.com/jkitchin/org-ref-cite/issues/16), but there was not
much appetite for the approach so I abandoned that. There are
cross-reference capabilities in org, that may be suitable for some
applications, but they do not come close to what org-ref offers, and
that the kind of technical documents I write require. For me, any
cross-reference capability would also have to support what is possible
in LaTeX, and preferrably look similar to the LaTeX commands. It has not
been possible to write an orthogonal package that could co-exist with
org-ref to address this; this would require a new syntax for
cross-references in my opinion. My opinion is that practically citations
and cross-references are just links to a place in your document (either
a figure/table/section/etc, or an entry in a bibliography). In org-ref,
both are represented by links; of course, they have different types and
functions, but they both have follow like actions. My opinion seems to
be in the minority on this.

I do not like the abstraction away from LaTeX cite commands in org-cite.
This is an example of a compromise between LaTeX and CSL. They do not
share common citation commands, so you can either choose one or the
other, or make a new abstraction that generalizes them. I strongly favor
the LaTeX commands because I write for LaTeX export, and there is
extensive documentation for how to cite in LaTeX and what to expect.
Clearly org-cite favors using the standardized abstractions in org-cite,
and then mapping them to the LaTeX commands I think. My perspective on
this is partially one of an educator; I have taught a lot of people how
to use org-mode for technical writing. This layer of abstraction adds
additional complexity to documentation, and in teaching people what to
do. As a LaTeX-centric user, that abstraction adds an additional
cognitive load while writing I find unwelcome. I concede that it also
reflects "what I do", that org is not LaTeX, and that others may have a
more CSL centric perspective. org-cite is for them. I can see that the
abstraction away from the LaTeX cite commands strengthens the org is not
LaTeX philosophy, and will serve part of the community well. If I wasn't
required to generate LaTeX documents, and teach others how to do it, I
would not feel so strongly.

It is my opinion that the modularity of org-cite is a challenge. I think
it is too difficult to configure, and difficult to support, even more so
than org-ref. I know my opinion differs from many on the list who want
modularity and configurability. I have supported org-ref since around
2014, and even the modularity there (helm or ivy) has been a challenge
to support. org-ref has always been configurable, but monolithic. Still,
I learned a lot from Bruce (thank you!) that pushed me to redesign parts
of org-ref to be more modular and more easily pluggable to other
completion backends, and less specific on ivy/helm where practical.
There are limitations to this I learned about, compromises one has to
choose in doing it, and consequences in maintenance, support and
documentation from them. We still don't fully agree on some of these
points, but org-ref is closer to that ideal than it was. Maybe one day I
will abandon ivy like I did helm many years ago and feel differently
about this, but that day is sufficiently far away that I don't see it
now. 

Finally, the org-cite code is magnificent, and written at a level well
above my coding skills. I am grateful to those who wrote it, and
especially to Nicolas, for the opportunity to learn from it. The code I
wrote in org-ref-cite was challenging. org-cite uses (IMO) advanced
emacs-lisp techniques, and more complex data structures than I am
accustomed to. I learned a lot studying the org-cite code, but I will be
honest that I find it difficult to make contributions to. That gave me
pause in continuing to develop it. It is fair to say that org-cite
showed me some ways to address limitations of org-ref that I did not see
before, org-ref is better for it, and the writing community that uses
pre/post notes and biblatex is much better served as a result.

Where does this leave me, org-ref and org-cite? I still have differences
of opinion on design choices between them, and those differences are
likely irreconcilable. These differences arise from my experiences in
writing, teaching, using, developing and supporting org-ref. For those
who need high fidelity LaTeX export like I do, I think org-ref is still
a superior solution. For everyone else, and especially if you do not
need sophisticated cross-references and don't want the dependencies of
org-ref, org-cite is likely the better solution. 

I am content to agree to disagree on these points and move forward with
both packages because they solve different problems, are suitable for
different communities, and they continue to benefit each other. I can
see not everyone sees this as a positive situation though, and that has
weighed heavily upon me lately. These times are heavy enough. Anyway,
this turned out much longer than I expected, so thanks everyone who has
contributed to making org better, I hope I got things mostly correct,
you found it a fair assessment, we might still be friends, and thanks
for reading to the end.

j

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Hello,
>
> Vikas Rawal <vikasrawal@gmail.com> writes:
>
>> This obviously creates many problems including that two people using
>> different citation systems cannot share org files.
>
> Indeed.
>>
>> What is the general view of the community about this?
>
> I don't know about the general view of the community, but, as a data
> point, I find it very sad.
>
> That's not helpful, tho.
>
> Regards,


-- 
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu
Pronouns: he/him/his

Re: citations: org-cite vs org-ref 3.0

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 13908 bytes --]

Hi John,

Thanks for your considered response.

When you contrast org-cite and org-ref, you say:

> With org-ref, bib(la)tex export is almost fully supported, and is easy,

I find this odd as org-cite supports bib(la)tex export, and rather easily.

┌────
│ #+bibliography: references.bib
│ #+cite_export: biblatex authortitle/authortitle-ibid
│ 
│ [cite:@key] etc.
│ 
│ #+print_bibliography:
└────

The limitation which I think is on your mind is that not all bib(la)tex commands
are supported, and not in the “usual” form. For instance, to get `pnotecite' one
would use `[cite/locators:]'. However, to get a 1-to-1 name mapping, you can just
customise `org-cite-biblatex-styles'. For instance, `parencite' is not currently
available, but if I just add `("parencite" nil "parencite" nil nil)' I can then do
`[cite/parencite:]' or if I replace the first `"parencite"' with `"paren"', just
`[cite/paren:]'.

A package could be created, say `org-cite-literal-biblatex' which is just a copy
of `oc-biblatex.el' with a different default `org-cite-biblatex-styles' and
`org-cite-biblatex-style-shortcuts' (or just sets those variables in
`org-cite-biblatex'). As far as I can tell this would provide exactly the
functionality you say org-cite can’t provide but org-ref does.

You can already use `.bib' files, and so frankly I cannot myself see the point in
org-ref’s existence beyond bifurcating the community on this. At this point the
only remaining motivation I see is old documents and current users, and for this
a migration tool seems more appropriate.

I don’t mean to be overly critical, however this is my current honest assessment
of the situation.

–
All the best,
Timothy

John Kitchin <jkitchin@andrew.cmu.edu> writes:

> I do not think it is productive for the community to say or consider it
> is a sad situation. Many good things have emerged from these
> discussions, even if it is not yet consensus on a solution. It is a
> complex problem, with many years of effort by many people on each side.
> That is an indication of how ambitious this project is and that there
> may be more than one solution that is needed. It pains me quite a bit
> there is a sentiment of fractionation, and that I may be contributing to
> it.
>
> My regular job workload the past few years has been crushing, and I have
> not had the time to participate in this that I wish I had. I am not sure
> I can add much here without sounding or feeling defensive about org-ref,
> and my decision to continue supporting and developing it. I have thought
> about this for most of the day, and in the (very long, apologies in
> advance) response that follows I will do my best to provide a balanced
> perspective (from my point of view) on the situation.
>
> Some specific context that is important to me is that I wrote org-ref
> long ago to solve a specific problem for me in the preparation of
> scientific publications that are destined for LaTeX export. I intended
> it to provide nearly equivalent bib(la)tex citation export, and as
> reasonable an export as possible for everything else. I use org-ref
> professionally, and it is a complete solution for me. I simply cannot
> compromise on the capability org-ref provides me, or wait for an
> alternative complete solution in org-mode. I have work I have to do now,
> and org-ref lets me do it. This alone is reason enough for me to
> continue using, developing and supporting org-ref. I understand org is
> not intended to be a substitute for writing LaTeX, but it is a fact of
> my job that I have to do that.
>
> There are more than 8 years of legacy org-ref documents. I have written
> 40+ scientific papers with it, and countless technical documents with
> more than 8000 cite links among them. org-ref has exceeded 190K
> downloads from MELPA, so I feel obligated to maintain org-ref for
> myself, and those users. org-ref may be heavyweight in bundling a lot of
> capability together that could be separated into individual packages,
> but it is also convenient for people who need it, and a GitHUB issue or
> pull request away from new features. I remain committed to supporting
> this, and I do it in a way I can manage, hence the monolithic package
> design.
>
> org-cite was also developed to solve some specific citation problems for
> others that org-ref did not address well at the time it was started. I
> believe those were issues like better pre/post note support, and
> integration with CSL.
>
> I think org-ref and org-cite have different priorities, they solve
> different problems with different approaches, and they have different
> pros and cons. I believe there are mutually incompatible compromises one
> must make here because the specific choices you make determine what is
> easy and what is possible. There is not a direct mapping of bib(la)tex
> and CSL as a citation processor. They are different programs and they
> don’t share citation commands, or style information. It is inevitable
> that something will be lost in the translation between these from a
> single source like org, and whether that loss is acceptable depends on
> what you need. For many things, close enough is ok for me, but for
> manuscripts and proposals, they must be perfect, and in bib(la)tex form
> for the journals I publish in. It is not that one thing is possible in
> one and not the other; it is that you have to compromise one to do the
> other no matter what you choose and that typically makes one thing or
> the other easier to do. I am not even sure it is possible to do
> everything one can do in bib(la)tex with CSL, for example, I don’t know
> the equivalent of  in CSL. org-ref sides on making bib(la)tex
> easy, and CSL is possible.
>
> With org-cite, CSL can be readily used across export backends, more
> bibliography database formats are supported, and it is also possible to
> get LaTeX export, although there is no goal to fully support all of
> bib(la)tex. A pure org approach (e.g. org-files as bibliography
> database) can be used, there are a lot of CSL styles to work with, and
> you can develop or adapt your own style if needed. With reasonable
> defaults this should be a straightforward introduction to using
> citations for new users that works out of the box. I support that.
>
> With org-ref, bib(la)tex export is almost fully supported, and is easy,
> and other exports via CSL are possible. Of course, you must have a
> working LaTeX installation, only bibtex databases are supported, and
> working knowledge of bib(la)tex is required to leverage this. Whether
> you want it or not, you get a lot of extra utilities for getting bibtex
> entries from a DOI, and support for cross-references, indexes,
> glossaries and acronyms. This is a lot to ask of people who don’t need
> it, and convenient for those who do. I support this.
>
> Which one of these approaches is right depends on what you want to be
> easy. Neither is right or wrong, that is determined by what you need at
> the time and what you prioritize in your solution. It is even possible
> you need both approaches at different times. The two approaches are not
> compatible, but it is org-mode after all, and you can certainly convert
> back and forth between them for the most part.
>
> Both projects have benefited from this discussion a lot. org has
> org-cite now, and org-ref now handles pre/post notes like org-cite does,
> it supports CSL much better, and is even a little more modular, lighter
> weight, and more easily integrated with other completion backends than
> ivy or helm. That should broadly be viewed as a win-win situation.
>
> Here are some factors that have prevented me from deprecating org-ref. I
> spent about a month and half trying to get a solution to this at
> <https://github.com/jkitchin/org-ref-cite>, and I don’t make these
> observations lightly. That solution was complete and highly functional
> for org-cite, but as I describe below not a replacement for org-ref at
> this time. I still welcome a new maintainer for this code.
>
> Cross-references are critical for me; without them, there is no path
> forward for me with org-cite. I did work on a cross-reference approach
> that leveraged org-cite syntax
> (<https://github.com/jkitchin/org-ref-cite/issues/16>), but there was not
> much appetite for the approach so I abandoned that. There are
> cross-reference capabilities in org, that may be suitable for some
> applications, but they do not come close to what org-ref offers, and
> that the kind of technical documents I write require. For me, any
> cross-reference capability would also have to support what is possible
> in LaTeX, and preferrably look similar to the LaTeX commands. It has not
> been possible to write an orthogonal package that could co-exist with
> org-ref to address this; this would require a new syntax for
> cross-references in my opinion. My opinion is that practically citations
> and cross-references are just links to a place in your document (either
> a figure/table/section/etc, or an entry in a bibliography). In org-ref,
> both are represented by links; of course, they have different types and
> functions, but they both have follow like actions. My opinion seems to
> be in the minority on this.
>
> I do not like the abstraction away from LaTeX cite commands in org-cite.
> This is an example of a compromise between LaTeX and CSL. They do not
> share common citation commands, so you can either choose one or the
> other, or make a new abstraction that generalizes them. I strongly favor
> the LaTeX commands because I write for LaTeX export, and there is
> extensive documentation for how to cite in LaTeX and what to expect.
> Clearly org-cite favors using the standardized abstractions in org-cite,
> and then mapping them to the LaTeX commands I think. My perspective on
> this is partially one of an educator; I have taught a lot of people how
> to use org-mode for technical writing. This layer of abstraction adds
> additional complexity to documentation, and in teaching people what to
> do. As a LaTeX-centric user, that abstraction adds an additional
> cognitive load while writing I find unwelcome. I concede that it also
> reflects “what I do”, that org is not LaTeX, and that others may have a
> more CSL centric perspective. org-cite is for them. I can see that the
> abstraction away from the LaTeX cite commands strengthens the org is not
> LaTeX philosophy, and will serve part of the community well. If I wasn’t
> required to generate LaTeX documents, and teach others how to do it, I
> would not feel so strongly.
>
> It is my opinion that the modularity of org-cite is a challenge. I think
> it is too difficult to configure, and difficult to support, even more so
> than org-ref. I know my opinion differs from many on the list who want
> modularity and configurability. I have supported org-ref since around
> 2014, and even the modularity there (helm or ivy) has been a challenge
> to support. org-ref has always been configurable, but monolithic. Still,
> I learned a lot from Bruce (thank you!) that pushed me to redesign parts
> of org-ref to be more modular and more easily pluggable to other
> completion backends, and less specific on ivy/helm where practical.
> There are limitations to this I learned about, compromises one has to
> choose in doing it, and consequences in maintenance, support and
> documentation from them. We still don’t fully agree on some of these
> points, but org-ref is closer to that ideal than it was. Maybe one day I
> will abandon ivy like I did helm many years ago and feel differently
> about this, but that day is sufficiently far away that I don’t see it
> now.
>
> Finally, the org-cite code is magnificent, and written at a level well
> above my coding skills. I am grateful to those who wrote it, and
> especially to Nicolas, for the opportunity to learn from it. The code I
> wrote in org-ref-cite was challenging. org-cite uses (IMO) advanced
> emacs-lisp techniques, and more complex data structures than I am
> accustomed to. I learned a lot studying the org-cite code, but I will be
> honest that I find it difficult to make contributions to. That gave me
> pause in continuing to develop it. It is fair to say that org-cite
> showed me some ways to address limitations of org-ref that I did not see
> before, org-ref is better for it, and the writing community that uses
> pre/post notes and biblatex is much better served as a result.
>
> Where does this leave me, org-ref and org-cite? I still have differences
> of opinion on design choices between them, and those differences are
> likely irreconcilable. These differences arise from my experiences in
> writing, teaching, using, developing and supporting org-ref. For those
> who need high fidelity LaTeX export like I do, I think org-ref is still
> a superior solution. For everyone else, and especially if you do not
> need sophisticated cross-references and don’t want the dependencies of
> org-ref, org-cite is likely the better solution.
>
> I am content to agree to disagree on these points and move forward with
> both packages because they solve different problems, are suitable for
> different communities, and they continue to benefit each other. I can
> see not everyone sees this as a positive situation though, and that has
> weighed heavily upon me lately. These times are heavy enough. Anyway,
> this turned out much longer than I expected, so thanks everyone who has
> contributed to making org better, I hope I got things mostly correct,
> you found it a fair assessment, we might still be friends, and thanks
> for reading to the end.
>
> j

Re: citations: org-cite vs org-ref 3.0

[Vikas Rawal]

[-- Attachment #1: Type: text/plain, Size: 3442 bytes --]

Dear John,

Thanks very much for taking time to write a detailed reply.

I do not think it is productive for the community to say or consider it
> is a sad situation.


From the perspective of a user, this was only meant to express a sentiment
that one finds oneself in a situation of having to choose between two good
things, and that we have not been able to find a way to make both
compatible with each other. It was in not meant as a disrespect in any way.

My use case is very similar to yours and I have been an org-ref user for a
long time (I was surely one of the earliest beneficiaries of your work),
having written two books and innumerable research papers with org-ref
citation syntax. Being able to export to LaTeX has been my primary use but
the fact that citations were not exported easily to other formats thus far
was a problem I had to struggle with every now and then.


> There are more than 8 years of legacy org-ref documents. I have written
> 40+ scientific papers with it, and countless technical documents with
> more than 8000 cite links among them. org-ref has exceeded 190K
> downloads from MELPA, so I feel obligated to maintain org-ref for
> myself, and those users.


Given that it is not very difficult to convert a document from old org-ref
citation syntax to the org-cite syntax, at least as far as citation is
concerned, this should not be a big problem. Do these documents use
citation commands that are not available in org-cite? Can those not be
added to org-cite?


> I think org-ref and org-cite have different priorities, they solve
> different problems with different approaches, and they have different
> pros and cons.


It might be useful to discuss specific gaps (such as citenum) that need to
be plugged in org-cite for it to be usable. In fact, making org-cite usable
for a heavyweight user like you is a useful goalpost.

I understand that you do not particularly like the modularity and
complexity of org-cite way of specifying styles and variants. But if one is
able to make the two compatible, filling the gaps, they could have a
friendly co-existence with some way of being able to convert a document
between the two styles. And if there are some incompatibilities that cannot
be resolved, it would be good to know exactly what all those are. If
somebody was to write functions to convert from one format to the other,
they could choose how they want to deal with those incompatibilities.


> Cross-references are critical for me; without them, there is no path
> forward for me with org-cite. I did work on a cross-reference approach
> that leveraged org-cite syntax
> (https://github.com/jkitchin/org-ref-cite/issues/16), but there was not
> much appetite for the approach so I abandoned that.


What org-ref seems to do with cross-references is very nice. Unfortunately
this would not be available if a user chooses to use org-cite. Do the
capabilities of cross-referencing have to be wedded to the citation system?

Can this not be resolved?


> I am content to agree to disagree on these points and move forward with
> both packages because they solve different problems, are suitable for
> different communities, and they continue to benefit each other.


Friendly co-existence should be our goal. But can that be a situation in
which one is able to choose between the best of both and, as far as
possible, switch from one to the other.

Thanks again for your time and effort,

Vikas

[-- Attachment #2: Type: text/html, Size: 4675 bytes --]

Re: citations: org-cite vs org-ref 3.0

[Dominik Schrempf]

[-- Attachment #1: Type: text/plain, Size: 14909 bytes --]

Hi,

I am trying out `org-cite' right now. It works much better than the last time (I
am using the `biblatex' backend right now). However, I can not find any
documentation about the available /styles/.

They are mentioned here: <https://orgmode.org/manual/Citations.html>

But no styles are provided.

For example, I need citations in the text (showing up as “Author(s) (Year)”).
With BibTeX, this would be `\citet{}' (or `\textcite{}' with BibLaTeX). But
there is a lot more. See: <http://tug.ctan.org/info/biblatex-cheatsheet/biblatex-cheatsheet.pdf>).

Thank you for working on `org-cite', and `org-ref'!
Dominik

Timothy <tecosaur@gmail.com> writes:

> Hi John,
>
> Thanks for your considered response.
>
> When you contrast org-cite and org-ref, you say:
>
>> With org-ref, bib(la)tex export is almost fully supported, and is easy,
>
> I find this odd as org-cite supports bib(la)tex export, and rather easily.
>
> ┌────
> │ #+bibliography: references.bib
> │ #+cite_export: biblatex authortitle/authortitle-ibid
> │
> │ (NO_ITEM_DATA:key) etc.
> │
> │ #+print_bibliography:
> └────
>
> The limitation which I think is on your mind is that not all bib(la)tex commands
> are supported, and not in the “usual” form. For instance, to get `pnotecite’ one
> would use `[cite/locators:]’. However, to get a 1-to-1 name mapping, you can just
> customise `org-cite-biblatex-styles’. For instance, `parencite’ is not currently
> available, but if I just add `(“parencite” nil “parencite” nil nil)’ I can then do
> `[cite/parencite:]’ or if I replace the first `“parencite”’ with `“paren”’, just
> `[cite/paren:]’.
>
> A package could be created, say `org-cite-literal-biblatex’ which is just a copy
> of `oc-biblatex.el’ with a different default `org-cite-biblatex-styles’ and
> `org-cite-biblatex-style-shortcuts’ (or just sets those variables in
> `org-cite-biblatex’). As far as I can tell this would provide exactly the
> functionality you say org-cite can’t provide but org-ref does.
>
> You can already use `.bib’ files, and so frankly I cannot myself see the point in
> org-ref’s existence beyond bifurcating the community on this. At this point the
> only remaining motivation I see is old documents and current users, and for this
> a migration tool seems more appropriate.
>
> I don’t mean to be overly critical, however this is my current honest assessment
> of the situation.
>
> –
> All the best,
> Timothy
>
> John Kitchin <jkitchin@andrew.cmu.edu> writes:
>
>> I do not think it is productive for the community to say or consider it
>> is a sad situation. Many good things have emerged from these
>> discussions, even if it is not yet consensus on a solution. It is a
>> complex problem, with many years of effort by many people on each side.
>> That is an indication of how ambitious this project is and that there
>> may be more than one solution that is needed. It pains me quite a bit
>> there is a sentiment of fractionation, and that I may be contributing to
>> it.
>>
>> My regular job workload the past few years has been crushing, and I have
>> not had the time to participate in this that I wish I had. I am not sure
>> I can add much here without sounding or feeling defensive about org-ref,
>> and my decision to continue supporting and developing it. I have thought
>> about this for most of the day, and in the (very long, apologies in
>> advance) response that follows I will do my best to provide a balanced
>> perspective (from my point of view) on the situation.
>>
>> Some specific context that is important to me is that I wrote org-ref
>> long ago to solve a specific problem for me in the preparation of
>> scientific publications that are destined for LaTeX export. I intended
>> it to provide nearly equivalent bib(la)tex citation export, and as
>> reasonable an export as possible for everything else. I use org-ref
>> professionally, and it is a complete solution for me. I simply cannot
>> compromise on the capability org-ref provides me, or wait for an
>> alternative complete solution in org-mode. I have work I have to do now,
>> and org-ref lets me do it. This alone is reason enough for me to
>> continue using, developing and supporting org-ref. I understand org is
>> not intended to be a substitute for writing LaTeX, but it is a fact of
>> my job that I have to do that.
>>
>> There are more than 8 years of legacy org-ref documents. I have written
>> 40+ scientific papers with it, and countless technical documents with
>> more than 8000 cite links among them. org-ref has exceeded 190K
>> downloads from MELPA, so I feel obligated to maintain org-ref for
>> myself, and those users. org-ref may be heavyweight in bundling a lot of
>> capability together that could be separated into individual packages,
>> but it is also convenient for people who need it, and a GitHUB issue or
>> pull request away from new features. I remain committed to supporting
>> this, and I do it in a way I can manage, hence the monolithic package
>> design.
>>
>> org-cite was also developed to solve some specific citation problems for
>> others that org-ref did not address well at the time it was started. I
>> believe those were issues like better pre/post note support, and
>> integration with CSL.
>>
>> I think org-ref and org-cite have different priorities, they solve
>> different problems with different approaches, and they have different
>> pros and cons. I believe there are mutually incompatible compromises one
>> must make here because the specific choices you make determine what is
>> easy and what is possible. There is not a direct mapping of bib(la)tex
>> and CSL as a citation processor. They are different programs and they
>> don’t share citation commands, or style information. It is inevitable
>> that something will be lost in the translation between these from a
>> single source like org, and whether that loss is acceptable depends on
>> what you need. For many things, close enough is ok for me, but for
>> manuscripts and proposals, they must be perfect, and in bib(la)tex form
>> for the journals I publish in. It is not that one thing is possible in
>> one and not the other; it is that you have to compromise one to do the
>> other no matter what you choose and that typically makes one thing or
>> the other easier to do. I am not even sure it is possible to do
>> everything one can do in bib(la)tex with CSL, for example, I don’t know
>> the equivalent of  in CSL. org-ref sides on making bib(la)tex
>> easy, and CSL is possible.
>>
>> With org-cite, CSL can be readily used across export backends, more
>> bibliography database formats are supported, and it is also possible to
>> get LaTeX export, although there is no goal to fully support all of
>> bib(la)tex. A pure org approach (e.g. org-files as bibliography
>> database) can be used, there are a lot of CSL styles to work with, and
>> you can develop or adapt your own style if needed. With reasonable
>> defaults this should be a straightforward introduction to using
>> citations for new users that works out of the box. I support that.
>>
>> With org-ref, bib(la)tex export is almost fully supported, and is easy,
>> and other exports via CSL are possible. Of course, you must have a
>> working LaTeX installation, only bibtex databases are supported, and
>> working knowledge of bib(la)tex is required to leverage this. Whether
>> you want it or not, you get a lot of extra utilities for getting bibtex
>> entries from a DOI, and support for cross-references, indexes,
>> glossaries and acronyms. This is a lot to ask of people who don’t need
>> it, and convenient for those who do. I support this.
>>
>> Which one of these approaches is right depends on what you want to be
>> easy. Neither is right or wrong, that is determined by what you need at
>> the time and what you prioritize in your solution. It is even possible
>> you need both approaches at different times. The two approaches are not
>> compatible, but it is org-mode after all, and you can certainly convert
>> back and forth between them for the most part.
>>
>> Both projects have benefited from this discussion a lot. org has
>> org-cite now, and org-ref now handles pre/post notes like org-cite does,
>> it supports CSL much better, and is even a little more modular, lighter
>> weight, and more easily integrated with other completion backends than
>> ivy or helm. That should broadly be viewed as a win-win situation.
>>
>> Here are some factors that have prevented me from deprecating org-ref. I
>> spent about a month and half trying to get a solution to this at
>> <https://github.com/jkitchin/org-ref-cite>, and I don’t make these
>> observations lightly. That solution was complete and highly functional
>> for org-cite, but as I describe below not a replacement for org-ref at
>> this time. I still welcome a new maintainer for this code.
>>
>> Cross-references are critical for me; without them, there is no path
>> forward for me with org-cite. I did work on a cross-reference approach
>> that leveraged org-cite syntax
>> (<https://github.com/jkitchin/org-ref-cite/issues/16>), but there was not
>> much appetite for the approach so I abandoned that. There are
>> cross-reference capabilities in org, that may be suitable for some
>> applications, but they do not come close to what org-ref offers, and
>> that the kind of technical documents I write require. For me, any
>> cross-reference capability would also have to support what is possible
>> in LaTeX, and preferrably look similar to the LaTeX commands. It has not
>> been possible to write an orthogonal package that could co-exist with
>> org-ref to address this; this would require a new syntax for
>> cross-references in my opinion. My opinion is that practically citations
>> and cross-references are just links to a place in your document (either
>> a figure/table/section/etc, or an entry in a bibliography). In org-ref,
>> both are represented by links; of course, they have different types and
>> functions, but they both have follow like actions. My opinion seems to
>> be in the minority on this.
>>
>> I do not like the abstraction away from LaTeX cite commands in org-cite.
>> This is an example of a compromise between LaTeX and CSL. They do not
>> share common citation commands, so you can either choose one or the
>> other, or make a new abstraction that generalizes them. I strongly favor
>> the LaTeX commands because I write for LaTeX export, and there is
>> extensive documentation for how to cite in LaTeX and what to expect.
>> Clearly org-cite favors using the standardized abstractions in org-cite,
>> and then mapping them to the LaTeX commands I think. My perspective on
>> this is partially one of an educator; I have taught a lot of people how
>> to use org-mode for technical writing. This layer of abstraction adds
>> additional complexity to documentation, and in teaching people what to
>> do. As a LaTeX-centric user, that abstraction adds an additional
>> cognitive load while writing I find unwelcome. I concede that it also
>> reflects “what I do”, that org is not LaTeX, and that others may have a
>> more CSL centric perspective. org-cite is for them. I can see that the
>> abstraction away from the LaTeX cite commands strengthens the org is not
>> LaTeX philosophy, and will serve part of the community well. If I wasn’t
>> required to generate LaTeX documents, and teach others how to do it, I
>> would not feel so strongly.
>>
>> It is my opinion that the modularity of org-cite is a challenge. I think
>> it is too difficult to configure, and difficult to support, even more so
>> than org-ref. I know my opinion differs from many on the list who want
>> modularity and configurability. I have supported org-ref since around
>> 2014, and even the modularity there (helm or ivy) has been a challenge
>> to support. org-ref has always been configurable, but monolithic. Still,
>> I learned a lot from Bruce (thank you!) that pushed me to redesign parts
>> of org-ref to be more modular and more easily pluggable to other
>> completion backends, and less specific on ivy/helm where practical.
>> There are limitations to this I learned about, compromises one has to
>> choose in doing it, and consequences in maintenance, support and
>> documentation from them. We still don’t fully agree on some of these
>> points, but org-ref is closer to that ideal than it was. Maybe one day I
>> will abandon ivy like I did helm many years ago and feel differently
>> about this, but that day is sufficiently far away that I don’t see it
>> now.
>>
>> Finally, the org-cite code is magnificent, and written at a level well
>> above my coding skills. I am grateful to those who wrote it, and
>> especially to Nicolas, for the opportunity to learn from it. The code I
>> wrote in org-ref-cite was challenging. org-cite uses (IMO) advanced
>> emacs-lisp techniques, and more complex data structures than I am
>> accustomed to. I learned a lot studying the org-cite code, but I will be
>> honest that I find it difficult to make contributions to. That gave me
>> pause in continuing to develop it. It is fair to say that org-cite
>> showed me some ways to address limitations of org-ref that I did not see
>> before, org-ref is better for it, and the writing community that uses
>> pre/post notes and biblatex is much better served as a result.
>>
>> Where does this leave me, org-ref and org-cite? I still have differences
>> of opinion on design choices between them, and those differences are
>> likely irreconcilable. These differences arise from my experiences in
>> writing, teaching, using, developing and supporting org-ref. For those
>> who need high fidelity LaTeX export like I do, I think org-ref is still
>> a superior solution. For everyone else, and especially if you do not
>> need sophisticated cross-references and don’t want the dependencies of
>> org-ref, org-cite is likely the better solution.
>>
>> I am content to agree to disagree on these points and move forward with
>> both packages because they solve different problems, are suitable for
>> different communities, and they continue to benefit each other. I can
>> see not everyone sees this as a positive situation though, and that has
>> weighed heavily upon me lately. These times are heavy enough. Anyway,
>> this turned out much longer than I expected, so thanks everyone who has
>> contributed to making org better, I hope I got things mostly correct,
>> you found it a fair assessment, we might still be friends, and thanks
>> for reading to the end.
>>
>> j

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Mon, Mar 21, 2022 at 4:24 AM Dominik Schrempf
<dominik.schrempf@gmail.com> wrote:

> I am trying out `org-cite' right now. It works much better than the last time (I
> am using the `biblatex' backend right now). However, I can not find any
> documentation about the available /styles/.
>
> They are mentioned here: <https://orgmode.org/manual/Citations.html>
>
> But no styles are provided.

I have come to believe the best approach to handling available styles
and previews is in the style selection UI. So that one can see what
one is waiting when selecting.

In citar, those styles and preview are currently hard-coded in a
defcustom, but I'd like to figure out how to do that dynamically,
using org-cite-supported-styles etc.

In the meantime, Timothy's article should help (see the table on the styles):

https://blog.tecosaur.com/tmio/2021-07-31-citations.html

> For example, I need citations in the text (showing up as “Author(s) (Year)”).
> With BibTeX, this would be `\citet{}' (or `\textcite{}' with BibLaTeX). But
> there is a lot more. See: <http://tug.ctan.org/info/biblatex-cheatsheet/biblatex-cheatsheet.pdf>).

You want the "text" style; [cite/text:@doe20], or with the shortcut
[cite:/t:@doe20].

Bruce

Re: citations: org-cite vs org-ref 3.0

[John Kitchin]

Vikas Rawal <vikasrawal@gmail.com> writes:

> Dear John,
>
> Thanks very much for taking time to write a detailed reply.
>
>  I do not think it is productive for the community to say or consider it
>  is a sad situation. 
>
> From the perspective of a user, this was only meant to express a
> sentiment that one finds oneself in a situation of having to choose
> between two good things, and that we have not been able to find a way
> to make both compatible with each other. It was in not meant as a
> disrespect in any way.

I don't think you have to choose. You can use org-cite for
citations, and org-ref for cross-references. The citation syntax is
orthogonal, you just should not mix them. You can even wire org-ref to
use org-cite-insert like this:

(setq org-ref-insert-cite-function (lambda () (org-cite-insert nil)))

or use your own key binding if you need the prefix arg functionality of
org-cite-insert.

org-ref is more configurable and modular than it is given credit for.
This is not an either/or choice.

>
> My use case is very similar to yours and I have been an org-ref user
> for a long time (I was surely one of the earliest beneficiaries of
> your work), having written two books and innumerable research papers
> with org-ref citation syntax. Being able to export to LaTeX has been
> my primary use but the fact that citations were not exported easily to
> other formats thus far was a problem I had to struggle with every now
> and then.

with org-ref-3, this is much better, but org-cite also works well,
probably even better if you use footnote styles. Either way, it is not
trivial to get the same output from LaTeX and CSL for arbitrary styles.
They are often close though.

>  There are more than 8 years of legacy org-ref documents. I have written
>  40+ scientific papers with it, and countless technical documents with
>  more than 8000 cite links among them. org-ref has exceeded 190K
>  downloads from MELPA, so I feel obligated to maintain org-ref for
>  myself, and those users.
>
> Given that it is not very difficult to convert a document from old
> org-ref citation syntax to the org-cite syntax, at least as far as
> citation is concerned, this should not be a big problem. Do these
> documents use citation commands that are not available in org-cite?
> Can those not be added to org-cite?

It is possible to convert, I wouldn't say not very difficult. In
principle one maps over the links and citations, and then in reverse
order replace the one format with the converted other format in a
buffer. That basically means change & to @ (easy), and provide the
mapping between org-ref link type and org-cite style. org-ref has 70ish
cite commands that it supports because those are in bib(la)tex. If you
only use the basic types, it "should be easy". If you use others, you
will have to settle for approximately correct in some cases.

>  I think org-ref and org-cite have different priorities, they solve
>  different problems with different approaches, and they have different
>  pros and cons. 
>
> It might be useful to discuss specific gaps (such as citenum) that
> need to be plugged in org-cite for it to be usable. In fact, making
> org-cite usable for a heavyweight user like you is a useful goalpost.

citenum and bibentry are the only two I am not sure have a CSL analog. I
guess that things like citedate and citetitle are not fully implemented
in citeproc. That is fine, org-cite does not need to fully support all
of bib(la)tex, no single user needs that. Lots of users need one part
or another though, and collectively bib(la)tex exists for them.


>  
> I understand that you do not particularly like the modularity and
> complexity of org-cite way of specifying styles and variants. But if
> one is able to make the two compatible, filling the gaps, they could
> have a friendly co-existence with some way of being able to convert a
> document between the two styles. And if there are some
> incompatibilities that cannot be resolved, it would be good to know
> exactly what all those are. If somebody was to write functions to
> convert from one format to the other, they could choose how they want
> to deal with those incompatibilities.

I guess you won't know what all the incompatibilities are until you try
this. In my experience with org-ref, there is a years long tail of
discovery as people try different things and have corner cases of need.


>  
>  Cross-references are critical for me; without them, there is no path
>  forward for me with org-cite. I did work on a cross-reference approach
>  that leveraged org-cite syntax
>  (https://github.com/jkitchin/org-ref-cite/issues/16), but there was not
>  much appetite for the approach so I abandoned that. 
>
> What org-ref seems to do with cross-references is very nice.
> Unfortunately this would not be available if a user chooses to use
> org-cite. Do the capabilities of cross-referencing have to be wedded
> to the citation system?

They don't have to be integrated. Someone could separate out that
functionality. I don't have unlimited time to spend on these things, and
org-ref is designed with the constraints I have so that I am able to
develop and support it. That is all you should read into this. I don't
have the time or energy develop and maintain separate packages that I
use together. I do not want to separate org-ref into separate core
packages for cross-references, indexes, glossary/acronyms, bibtex
utilities, and packages that integrate helm, ivy, etc. 

>
> Can this not be resolved? 
>  
>  I am content to agree to disagree on these points and move forward with
>  both packages because they solve different problems, are suitable for
>  different communities, and they continue to benefit each other. 
>
> Friendly co-existence should be our goal. But can that be a situation
> in which one is able to choose between the best of both and, as far as
> possible, switch from one to the other.

As far as I can tell, you can already do this.

>
> Thanks again for your time and effort,
>
> Vikas


-- 
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu
Pronouns: he/him/his

Re: citations: org-cite vs org-ref 3.0

[John Kitchin]

[-- Attachment #1: Type: text/plain, Size: 14957 bytes --]

On Sun, Mar 20, 2022 at 9:57 PM Timothy <tecosaur@gmail.com> wrote:

> Hi John,
>
> Thanks for your considered response.
>
> When you contrast org-cite and org-ref, you say:
>
> > With org-ref, bib(la)tex export is almost fully supported, and is easy,
>
> I find this odd as org-cite supports bib(la)tex export, and rather easily.
>
> ┌────
> │ #+bibliography: references.bib
> │ #+cite_export: biblatex authortitle/authortitle-ibid
> │
> │ [cite:@key] etc.
> │
> │ #+print_bibliography:
> └────
>
> The limitation which I think is on your mind is that not all bib(la)tex
> commands
> are supported, and not in the “usual” form. For instance, to get
> `pnotecite' one
> would use `[cite/locators:]'. However, to get a 1-to-1 name mapping, you
> can just
> customise `org-cite-biblatex-styles'. For instance, `parencite' is not
> currently
> available, but if I just add `("parencite" nil "parencite" nil nil)' I can
> then do
> `[cite/parencite:]' or if I replace the first `"parencite"' with
> `"paren"', just
> `[cite/paren:]'.
>

Yes, of course I mean the citation commands.


>
> A package could be created, say `org-cite-literal-biblatex' which is just
> a copy
> of `oc-biblatex.el' with a different default `org-cite-biblatex-styles' and
> `org-cite-biblatex-style-shortcuts' (or just sets those variables in
> `org-cite-biblatex'). As far as I can tell this would provide exactly the
> functionality you say org-cite can’t provide but org-ref does.
>

I wrote this package you suggest in org-ref-cite. In discussions during
that development, it was clear the preference was on the more abstracted,
and uniform syntax across backends cite commands in org-cite, and not this
kind of variant. Of course one can do this. It is not that org-cite can't
provide it, it is that it doesn't at this time. Even so, it is only a
partial solution to deprecating org-ref.


>
> You can already use `.bib' files, and so frankly I cannot myself see the
> point in
> org-ref’s existence beyond bifurcating the community on this. At this
> point the
> only remaining motivation I see is old documents and current users, and
> for this
> a migration tool seems more appropriate.


> I don’t mean to be overly critical, however this is my current honest
> assessment
> of the situation.
>
> –
> All the best,
> Timothy
>
> John Kitchin <jkitchin@andrew.cmu.edu> writes:
>
> > I do not think it is productive for the community to say or consider it
> > is a sad situation. Many good things have emerged from these
> > discussions, even if it is not yet consensus on a solution. It is a
> > complex problem, with many years of effort by many people on each side.
> > That is an indication of how ambitious this project is and that there
> > may be more than one solution that is needed. It pains me quite a bit
> > there is a sentiment of fractionation, and that I may be contributing to
> > it.
> >
> > My regular job workload the past few years has been crushing, and I have
> > not had the time to participate in this that I wish I had. I am not sure
> > I can add much here without sounding or feeling defensive about org-ref,
> > and my decision to continue supporting and developing it. I have thought
> > about this for most of the day, and in the (very long, apologies in
> > advance) response that follows I will do my best to provide a balanced
> > perspective (from my point of view) on the situation.
> >
> > Some specific context that is important to me is that I wrote org-ref
> > long ago to solve a specific problem for me in the preparation of
> > scientific publications that are destined for LaTeX export. I intended
> > it to provide nearly equivalent bib(la)tex citation export, and as
> > reasonable an export as possible for everything else. I use org-ref
> > professionally, and it is a complete solution for me. I simply cannot
> > compromise on the capability org-ref provides me, or wait for an
> > alternative complete solution in org-mode. I have work I have to do now,
> > and org-ref lets me do it. This alone is reason enough for me to
> > continue using, developing and supporting org-ref. I understand org is
> > not intended to be a substitute for writing LaTeX, but it is a fact of
> > my job that I have to do that.
> >
> > There are more than 8 years of legacy org-ref documents. I have written
> > 40+ scientific papers with it, and countless technical documents with
> > more than 8000 cite links among them. org-ref has exceeded 190K
> > downloads from MELPA, so I feel obligated to maintain org-ref for
> > myself, and those users. org-ref may be heavyweight in bundling a lot of
> > capability together that could be separated into individual packages,
> > but it is also convenient for people who need it, and a GitHUB issue or
> > pull request away from new features. I remain committed to supporting
> > this, and I do it in a way I can manage, hence the monolithic package
> > design.
> >
> > org-cite was also developed to solve some specific citation problems for
> > others that org-ref did not address well at the time it was started. I
> > believe those were issues like better pre/post note support, and
> > integration with CSL.
> >
> > I think org-ref and org-cite have different priorities, they solve
> > different problems with different approaches, and they have different
> > pros and cons. I believe there are mutually incompatible compromises one
> > must make here because the specific choices you make determine what is
> > easy and what is possible. There is not a direct mapping of bib(la)tex
> > and CSL as a citation processor. They are different programs and they
> > don’t share citation commands, or style information. It is inevitable
> > that something will be lost in the translation between these from a
> > single source like org, and whether that loss is acceptable depends on
> > what you need. For many things, close enough is ok for me, but for
> > manuscripts and proposals, they must be perfect, and in bib(la)tex form
> > for the journals I publish in. It is not that one thing is possible in
> > one and not the other; it is that you have to compromise one to do the
> > other no matter what you choose and that typically makes one thing or
> > the other easier to do. I am not even sure it is possible to do
> > everything one can do in bib(la)tex with CSL, for example, I don’t know
> > the equivalent of  in CSL. org-ref sides on making bib(la)tex
> > easy, and CSL is possible.
> >
> > With org-cite, CSL can be readily used across export backends, more
> > bibliography database formats are supported, and it is also possible to
> > get LaTeX export, although there is no goal to fully support all of
> > bib(la)tex. A pure org approach (e.g. org-files as bibliography
> > database) can be used, there are a lot of CSL styles to work with, and
> > you can develop or adapt your own style if needed. With reasonable
> > defaults this should be a straightforward introduction to using
> > citations for new users that works out of the box. I support that.
> >
> > With org-ref, bib(la)tex export is almost fully supported, and is easy,
> > and other exports via CSL are possible. Of course, you must have a
> > working LaTeX installation, only bibtex databases are supported, and
> > working knowledge of bib(la)tex is required to leverage this. Whether
> > you want it or not, you get a lot of extra utilities for getting bibtex
> > entries from a DOI, and support for cross-references, indexes,
> > glossaries and acronyms. This is a lot to ask of people who don’t need
> > it, and convenient for those who do. I support this.
> >
> > Which one of these approaches is right depends on what you want to be
> > easy. Neither is right or wrong, that is determined by what you need at
> > the time and what you prioritize in your solution. It is even possible
> > you need both approaches at different times. The two approaches are not
> > compatible, but it is org-mode after all, and you can certainly convert
> > back and forth between them for the most part.
> >
> > Both projects have benefited from this discussion a lot. org has
> > org-cite now, and org-ref now handles pre/post notes like org-cite does,
> > it supports CSL much better, and is even a little more modular, lighter
> > weight, and more easily integrated with other completion backends than
> > ivy or helm. That should broadly be viewed as a win-win situation.
> >
> > Here are some factors that have prevented me from deprecating org-ref. I
> > spent about a month and half trying to get a solution to this at
> > <https://github.com/jkitchin/org-ref-cite>, and I don’t make these
> > observations lightly. That solution was complete and highly functional
> > for org-cite, but as I describe below not a replacement for org-ref at
> > this time. I still welcome a new maintainer for this code.
> >
> > Cross-references are critical for me; without them, there is no path
> > forward for me with org-cite. I did work on a cross-reference approach
> > that leveraged org-cite syntax
> > (<https://github.com/jkitchin/org-ref-cite/issues/16>), but there was
> not
> > much appetite for the approach so I abandoned that. There are
> > cross-reference capabilities in org, that may be suitable for some
> > applications, but they do not come close to what org-ref offers, and
> > that the kind of technical documents I write require. For me, any
> > cross-reference capability would also have to support what is possible
> > in LaTeX, and preferrably look similar to the LaTeX commands. It has not
> > been possible to write an orthogonal package that could co-exist with
> > org-ref to address this; this would require a new syntax for
> > cross-references in my opinion. My opinion is that practically citations
> > and cross-references are just links to a place in your document (either
> > a figure/table/section/etc, or an entry in a bibliography). In org-ref,
> > both are represented by links; of course, they have different types and
> > functions, but they both have follow like actions. My opinion seems to
> > be in the minority on this.
> >
> > I do not like the abstraction away from LaTeX cite commands in org-cite.
> > This is an example of a compromise between LaTeX and CSL. They do not
> > share common citation commands, so you can either choose one or the
> > other, or make a new abstraction that generalizes them. I strongly favor
> > the LaTeX commands because I write for LaTeX export, and there is
> > extensive documentation for how to cite in LaTeX and what to expect.
> > Clearly org-cite favors using the standardized abstractions in org-cite,
> > and then mapping them to the LaTeX commands I think. My perspective on
> > this is partially one of an educator; I have taught a lot of people how
> > to use org-mode for technical writing. This layer of abstraction adds
> > additional complexity to documentation, and in teaching people what to
> > do. As a LaTeX-centric user, that abstraction adds an additional
> > cognitive load while writing I find unwelcome. I concede that it also
> > reflects “what I do”, that org is not LaTeX, and that others may have a
> > more CSL centric perspective. org-cite is for them. I can see that the
> > abstraction away from the LaTeX cite commands strengthens the org is not
> > LaTeX philosophy, and will serve part of the community well. If I wasn’t
> > required to generate LaTeX documents, and teach others how to do it, I
> > would not feel so strongly.
> >
> > It is my opinion that the modularity of org-cite is a challenge. I think
> > it is too difficult to configure, and difficult to support, even more so
> > than org-ref. I know my opinion differs from many on the list who want
> > modularity and configurability. I have supported org-ref since around
> > 2014, and even the modularity there (helm or ivy) has been a challenge
> > to support. org-ref has always been configurable, but monolithic. Still,
> > I learned a lot from Bruce (thank you!) that pushed me to redesign parts
> > of org-ref to be more modular and more easily pluggable to other
> > completion backends, and less specific on ivy/helm where practical.
> > There are limitations to this I learned about, compromises one has to
> > choose in doing it, and consequences in maintenance, support and
> > documentation from them. We still don’t fully agree on some of these
> > points, but org-ref is closer to that ideal than it was. Maybe one day I
> > will abandon ivy like I did helm many years ago and feel differently
> > about this, but that day is sufficiently far away that I don’t see it
> > now.
> >
> > Finally, the org-cite code is magnificent, and written at a level well
> > above my coding skills. I am grateful to those who wrote it, and
> > especially to Nicolas, for the opportunity to learn from it. The code I
> > wrote in org-ref-cite was challenging. org-cite uses (IMO) advanced
> > emacs-lisp techniques, and more complex data structures than I am
> > accustomed to. I learned a lot studying the org-cite code, but I will be
> > honest that I find it difficult to make contributions to. That gave me
> > pause in continuing to develop it. It is fair to say that org-cite
> > showed me some ways to address limitations of org-ref that I did not see
> > before, org-ref is better for it, and the writing community that uses
> > pre/post notes and biblatex is much better served as a result.
> >
> > Where does this leave me, org-ref and org-cite? I still have differences
> > of opinion on design choices between them, and those differences are
> > likely irreconcilable. These differences arise from my experiences in
> > writing, teaching, using, developing and supporting org-ref. For those
> > who need high fidelity LaTeX export like I do, I think org-ref is still
> > a superior solution. For everyone else, and especially if you do not
> > need sophisticated cross-references and don’t want the dependencies of
> > org-ref, org-cite is likely the better solution.
> >
> > I am content to agree to disagree on these points and move forward with
> > both packages because they solve different problems, are suitable for
> > different communities, and they continue to benefit each other. I can
> > see not everyone sees this as a positive situation though, and that has
> > weighed heavily upon me lately. These times are heavy enough. Anyway,
> > this turned out much longer than I expected, so thanks everyone who has
> > contributed to making org better, I hope I got things mostly correct,
> > you found it a fair assessment, we might still be friends, and thanks
> > for reading to the end.
> >
> > j
>

[-- Attachment #2: Type: text/html, Size: 17253 bytes --]

Re: citations: org-cite vs org-ref 3.0

[Dominik Schrempf]

[-- Attachment #1: Type: text/plain, Size: 1985 bytes --]

Thank you, I can use `citar-insert-edit` to perform this action.

Now, I failed to create a full citation in the text. This corresponds to the
`\fullcite{}' command in LaTeX. Is there an option for this?

In summary, it is a bit painful to use `org-cite` compared to using `org-ref`.
In my opinion, the main reasons are:

• Documentation is missing for `org-cite`.
• The citation commands seem to be incomplete (or at least not documented).

I am aware that all this is a lot of work, and I am really grateful that we have
`org-cite` and `org-ref` available!

Thank you,
Dominik

“Bruce D’Arcus” <bdarcus@gmail.com> writes:

> On Mon, Mar 21, 2022 at 4:24 AM Dominik Schrempf
> <dominik.schrempf@gmail.com> wrote:
>
>> I am trying out `org-cite’ right now. It works much better than the last time (I
>> am using the `biblatex’ backend right now). However, I can not find any
>> documentation about the available /styles/.
>>
>> They are mentioned here: <https://orgmode.org/manual/Citations.html>
>>
>> But no styles are provided.
>
> I have come to believe the best approach to handling available styles
> and previews is in the style selection UI. So that one can see what
> one is waiting when selecting.
>
> In citar, those styles and preview are currently hard-coded in a
> defcustom, but I’d like to figure out how to do that dynamically,
> using org-cite-supported-styles etc.
>
> In the meantime, Timothy’s article should help (see the table on the styles):
>
> <https://blog.tecosaur.com/tmio/2021-07-31-citations.html>
>
>> For example, I need citations in the text (showing up as “Author(s) (Year)”).
>> With BibTeX, this would be `\citet{}’ (or `\textcite{}’ with BibLaTeX). But
>> there is a lot more. See: <http://tug.ctan.org/info/biblatex-cheatsheet/biblatex-cheatsheet.pdf>).
>
> You want the “text” style; (NO_ITEM_DATA:doe20), or with the shortcut
> (/t: NO_ITEM_DATA:doe20).
>
> Bruce

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Mon, Mar 21, 2022 at 8:23 AM John Kitchin <jkitchin@andrew.cmu.edu> wrote:

>> A package could be created, say `org-cite-literal-biblatex' which is just a copy
>> of `oc-biblatex.el' with a different default `org-cite-biblatex-styles' and
>> `org-cite-biblatex-style-shortcuts' (or just sets those variables in
>> `org-cite-biblatex'). As far as I can tell this would provide exactly the
>> functionality you say org-cite can’t provide but org-ref does.
>
>
> I wrote this package you suggest in org-ref-cite. In discussions during that development, it was clear the preference was on the more abstracted, and uniform syntax across backends cite commands in org-cite, and not this kind of variant. Of course one can do this. It is not that org-cite can't provide it, it is that it doesn't at this time.

Just for some broader context on this particular issue.

The advantage of the org-cite style/variant design reflected in the
included export processors (natbib, biblatex, csl) is that the same
styles will mostly generate the same final output.

But that portability will only work with those styles and variants.

With the new org-cite-biblatex-styles defcustom, however, one can
augment or completely replace all those. But if you care about that
portability, you'd want to be aware of this, and think about it.

So per Timothy's point, you actually don't even need a new processor
for biblatex if you want to include all the extensive list of biblatex
commands.

Natbib AFAIK is already fully covered.

There's another POV on this though:

If one doesn't like to see the org-cite styles, because of familiarity
with LaTeX commands etc., I would argue that can be addressed in the
style part of an insert processor and/or in an activate processor.
E.g. I would argue this is a UI issue; not fundamentally about the
styles names.

Bruce

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Mon, Mar 21, 2022 at 8:41 AM Dominik Schrempf
<dominik.schrempf@gmail.com> wrote:
>
> Thank you, I can use `citar-insert-edit` to perform this action.

You can actually just use org-cite-insert, which is context aware.

> Now, I failed to create a full citation in the text. This corresponds to the
> `\fullcite{}' command in LaTeX. Is there an option for this?

No, but we should add one, as natbib has a similar command.

And, of course, per Timothy's point, you can customize
org-cite-biblatex-styles to include it in the meantime. If you want to
experiment with this, contact me off-list, as there's some nuances
with citar.

> In summary, it is a bit painful to use `org-cite` compared to using `org-ref`.
> In my opinion, the main reasons are:
>
> • Documentation is missing for `org-cite`.
> • The citation commands seem to be incomplete (or at least not documented).

The documentation is admittedly thin. Thoughts on what we should add?

The styles, and how to customize them with oc-biblatex?

Bruce

Re: citations: org-cite vs org-ref 3.0

[juh]

[-- Attachment #1: Type: text/plain, Size: 1234 bytes --]

Am Sun, Mar 20, 2022 at 08:31:29PM -0400 schrieb John Kitchin:
> For those who need high fidelity LaTeX export like I do, I think
> org-ref is still a superior solution. For everyone else, and
> especially if you do not need sophisticated cross-references and don't
> want the dependencies of org-ref, org-cite is likely the better
> solution. 

Thank you very much for the clarification. 

I am new to both and I need something that works together with ConTeXt
export. Currently I am happy with org-cite. I used org-ref before but
never got good results in ConTeXt export. So I think that I will stick
wit org-cite. 

Org-mode is not LaTeX but I wonder if it can be a better replacement for
my Markdown-->Pandoc-->EPUB/ConTeXt single source publication process.
Better as I could make more use of org-roam in my writing process.

Though I am a long time emacs user putting all these blocks together is
a challenge, still.

With your words I now see much much clearer.

juh

-- 
Autoren-Homepage: ......... http://literatur.hasecke.com
Satiren & Essays: ......... http://www.sudelbuch.de
Privater Blog: ............ http://www.hasecke.eu
Netzliteratur-Projekt: .... http://www.generationenprojekt.de



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: citations: org-cite vs org-ref 3.0

[Dominik Schrempf]

[-- Attachment #1: Type: text/plain, Size: 1745 bytes --]

“Bruce D’Arcus” <bdarcus@gmail.com> writes:

> On Mon, Mar 21, 2022 at 8:41 AM Dominik Schrempf
> <dominik.schrempf@gmail.com> wrote:
>>
>> Thank you, I can use `citar-insert-edit` to perform this action.
>
> You can actually just use org-cite-insert, which is context aware.
>
>> Now, I failed to create a full citation in the text. This corresponds to the
>> `\fullcite{}’ command in LaTeX. Is there an option for this?
>
> No, but we should add one, as natbib has a similar command.
>
> And, of course, per Timothy’s point, you can customize
> org-cite-biblatex-styles to include it in the meantime. If you want to
> experiment with this, contact me off-list, as there’s some nuances
> with citar.

Great, that was easy, thank you:

(add-to-list ’org-cite-biblatex-styles ’(“full” nil “fullcite” nil nil))

(This or something similar should be added upstream).

>
>> In summary, it is a bit painful to use `org-cite` compared to using `org-ref`.
>> In my opinion, the main reasons are:
>>
>> • Documentation is missing for `org-cite`.
>> • The citation commands seem to be incomplete (or at least not documented).
>
> The documentation is admittedly thin. Thoughts on what we should add?
>
> The styles, and how to customize them with oc-biblatex?

I know writing documentation is a lot of work. It is easy to complain, but hard
to contribute. I think we need:
• A complete description of styles and variants (for the actual citation
  command, and also for `#+cite_export: ...') or at least a link to an
  up-to-date table.
• An improved description of the capabilities of `org-cite-insert' (i.e., that
  it can change the citation style and variant).

>
> Bruce

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Mon, Mar 21, 2022 at 9:13 AM Dominik Schrempf
<dominik.schrempf@gmail.com> wrote:
>
> “Bruce D’Arcus” <bdarcus@gmail.com> writes:
>
> > On Mon, Mar 21, 2022 at 8:41 AM Dominik Schrempf
> > <dominik.schrempf@gmail.com> wrote:
> >>
> >> Thank you, I can use `citar-insert-edit` to perform this action.
> >
> > You can actually just use org-cite-insert, which is context aware.
> >
> >> Now, I failed to create a full citation in the text. This corresponds to the
> >> `\fullcite{}’ command in LaTeX. Is there an option for this?
> >
> > No, but we should add one, as natbib has a similar command.
> >
> > And, of course, per Timothy’s point, you can customize
> > org-cite-biblatex-styles to include it in the meantime. If you want to
> > experiment with this, contact me off-list, as there’s some nuances
> > with citar.
>
> Great, that was easy, thank you:
>
> (add-to-list ’org-cite-biblatex-styles ’(“full” nil “fullcite” nil nil))

On citar (the "nuance"), until I make the style UI smarter, you'll
want to add this command and a preview to
citar-org-style-preview-alist.

You might want to even completely redefine it to preview using the
biblatex command.

> (This or something similar should be added upstream).

I agree.

We just need to decide what the style name should be (and also add a
mapping from it for bibentry in natbib, and hopefully later csl). We
already have a "full" variant for "author", so another "full" might be
a little confusing.

Bruce

> >> In summary, it is a bit painful to use `org-cite` compared to using `org-ref`.
> >> In my opinion, the main reasons are:
> >>
> >> • Documentation is missing for `org-cite`.
> >> • The citation commands seem to be incomplete (or at least not documented).
> >
> > The documentation is admittedly thin. Thoughts on what we should add?
> >
> > The styles, and how to customize them with oc-biblatex?
>
> I know writing documentation is a lot of work. It is easy to complain, but hard
> to contribute. I think we need:
> • A complete description of styles and variants (for the actual citation
>   command, and also for `#+cite_export: ...') or at least a link to an
>   up-to-date table.
> • An improved description of the capabilities of `org-cite-insert' (i.e., that
>   it can change the citation style and variant).
>
> >
> > Bruce

Re: citations: org-cite vs org-ref 3.0

[John Kitchin]

[-- Attachment #1: Type: text/plain, Size: 3287 bytes --]

Bruce and I looked into this UI approach in
https://github.com/jkitchin/org-ref-cite/issues/9. Bruce and I discussed
and worked on this for almost two weeks. There are 70 comments in this
issue.

There are opportunities now to annotate completion targets, which you can
see in the link above. The annotations are not selectable though during
completion, and this implementation was not too fast as I recall. You
probably should also augment the tooltips like this because you have to be
able to tell what a citation format is in the future too, not just at
insert time, e.g. suppose you are reading work of a collaborator. It was
hard to write, and ambiguous in ways, e.g. what is the export backend you
want? The annotations should differ for LaTeX (where you want to see the
latex command) vs HTML (where you probably want to see the formatted CSL
cite)...

We did not surmount these challenges at the time. Maybe others will succeed
in this another day.

John

-----------------------------------
Professor John Kitchin (he/him/his)
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu



On Mon, Mar 21, 2022 at 8:42 AM Bruce D'Arcus <bdarcus@gmail.com> wrote:

> On Mon, Mar 21, 2022 at 8:23 AM John Kitchin <jkitchin@andrew.cmu.edu>
> wrote:
>
> >> A package could be created, say `org-cite-literal-biblatex' which is
> just a copy
> >> of `oc-biblatex.el' with a different default `org-cite-biblatex-styles'
> and
> >> `org-cite-biblatex-style-shortcuts' (or just sets those variables in
> >> `org-cite-biblatex'). As far as I can tell this would provide exactly
> the
> >> functionality you say org-cite can’t provide but org-ref does.
> >
> >
> > I wrote this package you suggest in org-ref-cite. In discussions during
> that development, it was clear the preference was on the more abstracted,
> and uniform syntax across backends cite commands in org-cite, and not this
> kind of variant. Of course one can do this. It is not that org-cite can't
> provide it, it is that it doesn't at this time.
>
> Just for some broader context on this particular issue.
>
> The advantage of the org-cite style/variant design reflected in the
> included export processors (natbib, biblatex, csl) is that the same
> styles will mostly generate the same final output.
>
> But that portability will only work with those styles and variants.
>
> With the new org-cite-biblatex-styles defcustom, however, one can
> augment or completely replace all those. But if you care about that
> portability, you'd want to be aware of this, and think about it.
>
> So per Timothy's point, you actually don't even need a new processor
> for biblatex if you want to include all the extensive list of biblatex
> commands.
>
> Natbib AFAIK is already fully covered.
>
> There's another POV on this though:
>
> If one doesn't like to see the org-cite styles, because of familiarity
> with LaTeX commands etc., I would argue that can be addressed in the
> style part of an insert processor and/or in an activate processor.
> E.g. I would argue this is a UI issue; not fundamentally about the
> styles names.
>
> Bruce
>

[-- Attachment #2: Type: text/html, Size: 4148 bytes --]

Re: citations: org-cite vs org-ref 3.0

[John Kitchin]

"Bruce D'Arcus" <bdarcus@gmail.com> writes:

> On Sun, Mar 20, 2022 at 9:42 AM Ihor Radchenko <yantar92@gmail.com> wrote:
>
>> For citar, why not simply using ivy-bibtex? It supports org-cite, AFAIK.
>
> Not really; or rather minimally.
>
> Ivy-bibtex supports, for example, inserting of org-cite citations, but
> not via org-cite-insert.
>
> So there are currently no org-cite processors for ivy-bibtex etc.

I think this one in
https://github.com/jkitchin/org-ref-cite/blob/main/org-ref-cite-insert.el

provides an insert processor based on bibtex-completion, and that
provides ivy-bibtex actions. The commentary in this file also discusses some
challenges in relying exclusively on completing-read. My opinion of
course.

I don't think you need a special processor for ivy-bibtex. It could be
wired to insert org-cites on its own.

>
> Bruce


-- 
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu
Pronouns: he/him/his

Re: citations: org-cite vs org-ref 3.0

[John Kitchin]

chris <inkbottle007@gmail.com> writes:

> On Sunday, 20 March 2022 20:44:50 CET Bruce D'Arcus wrote:
>> On Sun, Mar 20, 2022 at 9:42 AM Ihor Radchenko <yantar92@gmail.com> wrote:
>> > For citar, why not simply using ivy-bibtex? It supports org-cite, AFAIK.
>> 
>> Not really; or rather minimally.
>
> I use `org-cite` with a very minimal configuration, and it works very well. I 
> don't use `ivy` at all. I don't use `helm` at all. Those are very large 
> framework and should not be forced to people. (I don't use `org-ref`.)

These days org-ref does not require helm or ivy, and it does not install
them. You can use the core library with any completion backend you want.
It uses vanilla completing-read.

org-ref does rely exclusively on bibtex-completion at this time. I have
not abstracted that out to allow pluggable alternatives like citar.

>
> I only use `citar` with minimal configuration. I use `vertico` for the 
> completion. `citar` is simple enough for me to be able to read and understand 
> a large part of it.
>
> IMO the more layers of code there are, the more difficult it is to have things 
> work right. And similarly with the size of the code.
>
> Chris
>
>> 
>> Ivy-bibtex supports, for example, inserting of org-cite citations, but
>> not via org-cite-insert.
>
> And I have `org-cite-insert` working straight out of the box.
>
>> 
>> So there are currently no org-cite processors for ivy-bibtex etc.
>> 
>> Bruce


-- 
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu
Pronouns: he/him/his

Re: citations: org-cite vs org-ref 3.0

[Max Nikulin]

Disclaimer: I am neither org-cite no org-ref user. In the past I used 
LaTeX and BibTeX directly though, and it is a reason why I am reading 
the discussions.

On 20/03/2022 20:19, Nicolas Goaziou wrote:
> Vikas Rawal writes:
> 
>> This obviously creates many problems including that two people using
>> different citation systems cannot share org files.
> 
> Indeed.
>>
>> What is the general view of the community about this?
> 
> I don't know about the general view of the community, but, as a data
> point, I find it very sad.

Nicolas, I think, a part of problem is that you are not an org-cite 
user. The packages require some *polishing*, but it have to be 
*user-driven*. Even if the feature were perfect, it would face some 
tension from people who already have a working solution.

A bit of routine work will alleviate some user issues:
- add missed styles
- improve documentation, e.g. to make backend choice more conscious.

Another point is more serious. Besides citations there are internal 
cross-references. Org supports them but only in a rudimentary form. 
Actually cross-references are similar to citations in the sense that 
they can have style, prefixes and suffixes, and their appearance depends 
on target properties. Another feature is grouping. However 
cross-references should not be handled by citation backends, they 
require different handlers. Unfortunately there is no way to define 
custom "citation" type e.g. "[ref:...]" in addition to "[cite:...]".

I can not judge if uniform UI issues are really severe and if it would 
be convenient if depending on prefix argument either org-cite or org-ref 
command would be called for a citation or for a reference.

Actually "[cite:...]" construct is a kind of link with additional 
flexibility missed for regular links. Anything besides target and 
description requires some workarounds. Usual approach is proliferation 
of link types. E.g. inline source blocks allows almost arbitrary extra 
parameters. Citation syntax is rather domain specific, it allows more 
than regular links, but for convenience the set of properties is fixed: 
style, prefixes, locators, suffixes. It is impossible to add extra one.

To assign additional properties, info "(org) Links in HTML export" 
https://orgmode.org/manual/Links-in-HTML-export.html recommends usage of 
"#+ATTR_HTML", but such technique has several issues:
- attributes becomes specific to the export backend
- the same attributes are added to the enclosing paragraph
   https://linevi.ch/en/org-link-extra-attrs.html
- a paragraph may have more than one link.
It is possible to use link target similar to form values encoded into 
URI, but it hardly can be considered as convenient for editing.

Custom citation types may alleviate the issue with cross-references. It 
would be great to have more flexible links with arbitrary properties 
(and it would allow to consider citations and cross-references as 
special cases of links), but it does not fit into the Org syntax.

P.S. John has a valid complain but it hardly relates to the "cite vs. 
cross-reference" topic. When some package is not loaded and link type is 
undefined then the link becomes a fuzzy one leading to user confusion.

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Mon, Mar 21, 2022 at 10:41 AM Max Nikulin <manikulin@gmail.com> wrote:

...

> A bit of routine work will alleviate some user issues:
> - add missed styles

The initial list of style-command mappings was pretty comprehensive,
but we left out some of the more obscure biblatex commands because
unsure if they were needed, or how best to add them (conceptually
there's a mix of different kinds of commands in biblatex, which are
hard to fit into a more general style system, for example).

Since then:

- people have occasionally asked to add new mappings, and Nicolas has added them
- he's also added the styles defcustoms for biblatex, so users can do
this themselves

In short, I think we're good on this actually.

> - improve documentation, e.g. to make backend choice more conscious.

This is the bigger user-facing issue that could use attention.

> Another point is more serious. Besides citations there are internal
> cross-references. Org supports them but only in a rudimentary form.

Indeed, the question of how to better support cross-references in org
is an important one.

I don't really use them much, and so am still unsure if this could be
addressed with incremental improvements in existing org link support,
or if it would require more significant enhancements.

Perhaps we need the community to itemize what the gaps and limitations
are there?


Bruce

> Actually cross-references are similar to citations in the sense that
> they can have style, prefixes and suffixes, and their appearance depends
> on target properties. Another feature is grouping. However
> cross-references should not be handled by citation backends, they
> require different handlers. Unfortunately there is no way to define
> custom "citation" type e.g. "[ref:...]" in addition to "[cite:...]".
>
> I can not judge if uniform UI issues are really severe and if it would
> be convenient if depending on prefix argument either org-cite or org-ref
> command would be called for a citation or for a reference.
>
> Actually "[cite:...]" construct is a kind of link with additional
> flexibility missed for regular links. Anything besides target and
> description requires some workarounds. Usual approach is proliferation
> of link types. E.g. inline source blocks allows almost arbitrary extra
> parameters. Citation syntax is rather domain specific, it allows more
> than regular links, but for convenience the set of properties is fixed:
> style, prefixes, locators, suffixes. It is impossible to add extra one.
>
> To assign additional properties, info "(org) Links in HTML export"
> https://orgmode.org/manual/Links-in-HTML-export.html recommends usage of
> "#+ATTR_HTML", but such technique has several issues:
> - attributes becomes specific to the export backend
> - the same attributes are added to the enclosing paragraph
>    https://linevi.ch/en/org-link-extra-attrs.html
> - a paragraph may have more than one link.
> It is possible to use link target similar to form values encoded into
> URI, but it hardly can be considered as convenient for editing.
>
> Custom citation types may alleviate the issue with cross-references. It
> would be great to have more flexible links with arbitrary properties
> (and it would allow to consider citations and cross-references as
> special cases of links), but it does not fit into the Org syntax.
>
> P.S. John has a valid complain but it hardly relates to the "cite vs.
> cross-reference" topic. When some package is not loaded and link type is
> undefined then the link becomes a fuzzy one leading to user confusion.
>
>

Re: citations: org-cite vs org-ref 3.0

[John Kitchin]

"Bruce D'Arcus" <bdarcus@gmail.com> writes:

> On Mon, Mar 21, 2022 at 10:41 AM Max Nikulin <manikulin@gmail.com> wrote:
>
> ...
>
>> A bit of routine work will alleviate some user issues:
>> - add missed styles
>
> The initial list of style-command mappings was pretty comprehensive,
> but we left out some of the more obscure biblatex commands because
> unsure if they were needed, or how best to add them (conceptually
> there's a mix of different kinds of commands in biblatex, which are
> hard to fit into a more general style system, for example).
>
> Since then:
>
> - people have occasionally asked to add new mappings, and Nicolas has added them
> - he's also added the styles defcustoms for biblatex, so users can do
> this themselves
>
> In short, I think we're good on this actually.
>
>> - improve documentation, e.g. to make backend choice more conscious.
>
> This is the bigger user-facing issue that could use attention.
>
>> Another point is more serious. Besides citations there are internal
>> cross-references. Org supports them but only in a rudimentary form.
>
> Indeed, the question of how to better support cross-references in org
> is an important one.
>
> I don't really use them much, and so am still unsure if this could be
> addressed with incremental improvements in existing org link support,
> or if it would require more significant enhancements.

No incremental improvement is required IMO, regular links are
sufficient. org-ref has handled these with regular links from the
beginning. It even has an org-ref-refproc now for non-LaTeX exports that
has some support for things like sorting, grouping and cleveref. You can
find an example org file at
https://github.com/jkitchin/org-ref/blob/master/examples/refproc.org,
and see it in action at https://www.youtube.com/watch?v=rRR-5NSpKyE the
video has some rough spots, but you can get the idea.

You can support all kinds of things in these links (for example, I now
support pre/post note text in the cite link paths), really anything you
want to parse out of the path. 


>
> Perhaps we need the community to itemize what the gaps and limitations
> are there?
>
>
> Bruce
>
>> Actually cross-references are similar to citations in the sense that
>> they can have style, prefixes and suffixes, and their appearance depends
>> on target properties. Another feature is grouping. However
>> cross-references should not be handled by citation backends, they
>> require different handlers. Unfortunately there is no way to define
>> custom "citation" type e.g. "[ref:...]" in addition to "[cite:...]".
>>
>> I can not judge if uniform UI issues are really severe and if it would
>> be convenient if depending on prefix argument either org-cite or org-ref
>> command would be called for a citation or for a reference.
>>
>> Actually "[cite:...]" construct is a kind of link with additional
>> flexibility missed for regular links. Anything besides target and
>> description requires some workarounds. Usual approach is proliferation
>> of link types. E.g. inline source blocks allows almost arbitrary extra
>> parameters. Citation syntax is rather domain specific, it allows more
>> than regular links, but for convenience the set of properties is fixed:
>> style, prefixes, locators, suffixes. It is impossible to add extra one.
>>
>> To assign additional properties, info "(org) Links in HTML export"
>> https://orgmode.org/manual/Links-in-HTML-export.html recommends usage of
>> "#+ATTR_HTML", but such technique has several issues:
>> - attributes becomes specific to the export backend
>> - the same attributes are added to the enclosing paragraph
>>    https://linevi.ch/en/org-link-extra-attrs.html
>> - a paragraph may have more than one link.
>> It is possible to use link target similar to form values encoded into
>> URI, but it hardly can be considered as convenient for editing.
>>
>> Custom citation types may alleviate the issue with cross-references. It
>> would be great to have more flexible links with arbitrary properties
>> (and it would allow to consider citations and cross-references as
>> special cases of links), but it does not fit into the Org syntax.
>>
>> P.S. John has a valid complain but it hardly relates to the "cite vs.
>> cross-reference" topic. When some package is not loaded and link type is
>> undefined then the link becomes a fuzzy one leading to user confusion.
>>
>>


-- 
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu
Pronouns: he/him/his

Re: citations: org-cite vs org-ref 3.0

[Vikas Rawal]

[-- Attachment #1: Type: text/plain, Size: 249 bytes --]

>
>
>
> I don't think you have to choose. You can use org-cite for
> citations, and org-ref for cross-references. The citation syntax is
> orthogonal, you just should not mix them.


That is excellent then.

Thanks for clarifications.

Best,

Vikas

[-- Attachment #2: Type: text/html, Size: 544 bytes --]

Re: citations: org-cite vs org-ref 3.0

[indieterminacy]

Hi Juh,

juh <juh+emacs-orgmoden@mailbox.org> writes:

> [[PGP Signed Part:Undecided]]
> Am Sun, Mar 20, 2022 at 08:31:29PM -0400 schrieb John Kitchin:
>> For those who need high fidelity LaTeX export like I do, I think
>> org-ref is still a superior solution. For everyone else, and
>> especially if you do not need sophisticated cross-references and don't
>> want the dependencies of org-ref, org-cite is likely the better
>> solution. 
>
> Thank you very much for the clarification. 
>
> I am new to both and I need something that works together with ConTeXt
> export. Currently I am happy with org-cite. I used org-ref before but
> never got good results in ConTeXt export. So I think that I will stick
> wit org-cite. 
>

I used to use ConTeXt and have a healthy respect for it.
That feedback is good to hear.

> Org-mode is not LaTeX but I wonder if it can be a better replacement for
> my Markdown-->Pandoc-->EPUB/ConTeXt single source publication process.
> Better as I could make more use of org-roam in my writing process.
>

May I recommend the mininimal Markdown format, GemText?
It has a very simple syntax which is easily convertable into Orgmode
syntax.

It could be useful for projects where the flow starts with simple
written prose.

> Though I am a long time emacs user putting all these blocks together is
> a challenge, still.
>
> With your words I now see much much clearer.
>
> juh


Jonathan

Re: citations: org-cite vs org-ref 3.0

[Max Nikulin]

On 21/03/2022 22:19, Bruce D'Arcus wrote:
> On Mon, Mar 21, 2022 at 10:41 AM Max Nikulin wrote:
> 
>> A bit of routine work will alleviate some user issues:
>> - add missed styles
> 
> The initial list of style-command mappings was pretty comprehensive,
> but we left out some of the more obscure biblatex commands because
> unsure if they were needed, or how best to add them (conceptually
> there's a mix of different kinds of commands in biblatex, which are
> hard to fit into a more general style system, for example).
> 
> Since then:
> 
> - people have occasionally asked to add new mappings, and Nicolas has added them
> - he's also added the styles defcustoms for biblatex, so users can do
> this themselves
> 
> In short, I think we're good on this actually.

John Kitchin, this thread, Sun, 20 Mar 2022 20:31:29 -0400. 
https://list.orgmode.org/m2sfrc149c.fsf@andrew.cmu.edu:

> I don't know the equivalent of \citenum in CSL.

 From my point of view it may be a reason to add a new style to 
defaults. It is important whether a tool works out of the box. Custom 
variables make a document less portable unless they are specified as 
file-local ones. I think, the goal may be formulated as "John can not 
say the following any more" (at least in respect to citations leaving 
aside cross-references):

> I simply cannot
> compromise on the capability org-ref provides me, or wait for an
> alternative complete solution in org-mode.

On the other hand I do not consider the following argument as a strong one

> I do not like the abstraction away from LaTeX cite commands in org-cite.
> This is an example of a compromise between LaTeX and CSL.

despite I believe that convenience and habits are important. Mapping of 
styles to commands is just a piece of knowledge.

I have no particular opinion if enough efforts should be invested from 
both sides to allow mixing on both citation syntax constructs (org-cite 
and org-ref) in the same document. Bruce, you made a lot for support of 
CSL in org-cite, so I will stressed another direction of feature 
comparison since Bib(La)TeX users should feel themselves first-class 
citizens.

The choice between org-ref and org-cite, when the former can do its job, 
should be matter of taste and personal preferences unrelated to 
technical limitations.

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Tue, Mar 22, 2022 at 10:42 AM Max Nikulin <manikulin@gmail.com> wrote:
>
> On 21/03/2022 22:19, Bruce D'Arcus wrote:
> > On Mon, Mar 21, 2022 at 10:41 AM Max Nikulin wrote:
> >
> >> A bit of routine work will alleviate some user issues:
> >> - add missed styles
> >
> > The initial list of style-command mappings was pretty comprehensive,
> > but we left out some of the more obscure biblatex commands because
> > unsure if they were needed, or how best to add them (conceptually
> > there's a mix of different kinds of commands in biblatex, which are
> > hard to fit into a more general style system, for example).
> >
> > Since then:
> >
> > - people have occasionally asked to add new mappings, and Nicolas has added them
> > - he's also added the styles defcustoms for biblatex, so users can do
> > this themselves
> >
> > In short, I think we're good on this actually.
>
> John Kitchin, this thread, Sun, 20 Mar 2022 20:31:29 -0400.
> https://list.orgmode.org/m2sfrc149c.fsf@andrew.cmu.edu:
>
> > I don't know the equivalent of \citenum in CSL.

Right; so John or someone else should send a message to the list
requesting it specifically?

That's how we've done it so far anyway. Not sure what the practical
alternative is.

I will say that much of this discussion is not about org-cite per se,
but rather implementation decisions in the specific bundled
processors. I fear these conversations get hard to track if we aren't
precise about what we're talking about; citation syntax, included
processor functionality or style mappings, documentation, etc.

>  From my point of view it may be a reason to add a new style to
> defaults. It is important whether a tool works out of the box.

But the question here becomes "works for whom?" Citations practices
vary a fair bit by field.

Primarily LaTeX/natbib users like John and other science folks,
biblatex users primarily coming from the humanities as near as I can
tell? Or people coming from markdown and pandoc (which has an
excellent citation system that clearly inspired org-cite) and needing
Word-ready documents for journals etc?

The style system in the bundled processors is designed to work for all
of them, and not privilege one or another. It does this by creating a
new, more general, org-cite style and variant system, which then maps
to different targets.

That approach has obvious advantages: citations are (mostly) decoupled
from particular export processors. One can write a document with these
styles, and get PDF citation and bibliographic output via biblatex,
for example, similar to the HTML or ODT output from the CSL export
processor.

And that also has advantages beyond org. For example, pandoc recently
added support for org-cite, and that includes mapping between the two
style/command systems. So in theory you could convert an org file to
docx, letting pandoc process the citations along the way, and the
result would more than acceptable.

> Custom variables make a document less portable unless they are specified as
> file-local ones.

Indeed!

Or more specifically here, it ties the citations to the export processor.

But maybe for some or many people that's perfectly fine.

One possible idea to consider is to allow two systems in each of
LaTeX-oriented processors: what we might call a default "org-cite"
one, and an optional "literal" one. So if you only use oc-natbib, and
you want the natbib commands directly, you might change a variable to
get that.

I really don't know if that's a good idea or not, but I raise it to
emphasize there's a lot of flexibility with the org-cite design, and
so it's just a question of how we make use of it.

> I think, the goal may be formulated as "John can not
> say the following any more" (at least in respect to citations leaving
> aside cross-references):
>
> > I simply cannot
> > compromise on the capability org-ref provides me, or wait for an
> > alternative complete solution in org-mode.

I don't really see why these are the two choices.

> On the other hand I do not consider the following argument as a strong one
>
> > I do not like the abstraction away from LaTeX cite commands in org-cite.
> > This is an example of a compromise between LaTeX and CSL.
>
> despite I believe that convenience and habits are important. Mapping of
> styles to commands is just a piece of knowledge.

Yes.

> I have no particular opinion if enough efforts should be invested from
> both sides to allow mixing on both citation syntax constructs (org-cite
> and org-ref) in the same document. Bruce, you made a lot for support of
> CSL in org-cite, so I will stressed another direction of feature
> comparison since Bib(La)TeX users should feel themselves first-class
> citizens.

Absolutely! In fact, Denis Meier, who has extensive biblatex
knowledge, helped a lot with the initial mapping there, and some other
biblatex users have contributed feedback along the way.


Bruce

> The choice between org-ref and org-cite, when the former can do its job,
> should be matter of taste and personal preferences unrelated to
> technical limitations.
>
>

Re: citations: org-cite vs org-ref 3.0

[Nicolas Goaziou]

Hello,

Max Nikulin <manikulin@gmail.com> writes:

> Nicolas, I think, a part of problem is that you are not an org-cite
> user. The packages require some *polishing*, but it have to be 
> *user-driven*.

Just to be clear: the development was _user-driven_ from the start.
I read years of discussions about citation, suggested a syntax, waited
for input, amended the syntax… Then I did the same for the whole
library, implementing proof of concepts, waiting for feedback, etc.

> Another point is more serious. Besides citations there are internal
> cross-references. Org supports them but only in a rudimentary form. 
> Actually cross-references are similar to citations in the sense that
> they can have style, prefixes and suffixes, and their appearance
> depends on target properties. Another feature is grouping. However 
> cross-references should not be handled by citation backends, they
> require different handlers. Unfortunately there is no way to define 
> custom "citation" type e.g. "[ref:...]" in addition to "[cite:...]".

There is too little information here for me to understand what
cross-references (or grouping) are, or why they would require different
handlers. In any case, if such thing are deemed necessary in Org Cite,
one can always start a new thread.

The goal of Org Cite is not to do everything, however. Its raison d'être
is to provide building blocks around citations that external packages
can use and extend. Therefore cross-references might belong to an
extension instead.

> To assign additional properties, info "(org) Links in HTML export"
> https://orgmode.org/manual/Links-in-HTML-export.html recommends usage
> of "#+ATTR_HTML", but such technique has several issues:

This is a different issue. Citations are not link, and have a fixed set
of properties.

Regards,
-- 
Nicolas Goaziou

Re: citations: org-cite vs org-ref 3.0

[Max Nikulin]

On 23/03/2022 00:20, Bruce D'Arcus wrote:
> On Tue, Mar 22, 2022 at 10:42 AM Max Nikulin wrote:
>>
>> John Kitchin, this thread, Sun, 20 Mar 2022 20:31:29 -0400.
>> https://list.orgmode.org/m2sfrc149c.fsf@andrew.cmu.edu:
>>
>>> I don't know the equivalent of \citenum in CSL.
> 
> Right; so John or someone else should send a message to the list
> requesting it specifically?

 From my point of view he has already requested support of \citenum by 
that message. I am taking into account that he is the developer org-ref 
and he is happy with this package. John made a lot to adopt org-cite 
syntax to ensure feature parity. Bruce, you should know it better than 
me since you participated in discussions while I am merely and observer. 
John does not really need \citenum namely in org-cite since his tool is 
org-ref, but his words may affect perception of org-cite.

I have an impression that the ball is on the side of the org-cite, and 
next steps may be to ask for real documents (e.g. open access papers) 
that are prepared with such format and to discuss most suitable style 
for CSL.

> One possible idea to consider is to allow two systems in each of
> LaTeX-oriented processors: what we might call a default "org-cite"
> one, and an optional "literal" one. So if you only use oc-natbib, and
> you want the natbib commands directly, you might change a variable to
> get that.

Maybe I will share my idea in response to the message from Nicolas
https://list.orgmode.org/871qytlf49.fsf@nicolasgoaziou.fr
Wed, 23 Mar 2022 00:52:22 +0100, this thread.

Since John explicitly confirmed that org-ref for cross-references may be 
combined with org-cite for citations, it may be great that Org has 2 
independent implementations for citations. When a user meets some corner 
case their has an opportunity to try another package.

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Wed, Mar 23, 2022 at 8:52 AM Max Nikulin <manikulin@gmail.com> wrote:
>
> On 23/03/2022 00:20, Bruce D'Arcus wrote:
> > On Tue, Mar 22, 2022 at 10:42 AM Max Nikulin wrote:
> >>
> >> John Kitchin, this thread, Sun, 20 Mar 2022 20:31:29 -0400.
> >> https://list.orgmode.org/m2sfrc149c.fsf@andrew.cmu.edu:
> >>
> >>> I don't know the equivalent of \citenum in CSL.
> >
> > Right; so John or someone else should send a message to the list
> > requesting it specifically?
>
>  From my point of view he has already requested support of \citenum by
> that message.

I just mean here, in general. If people complain about missing
mappings or performance issues silently, or elsewhere, without ever
raising them on the list, then it's hard to address them.

> I have an impression that the ball is on the side of the org-cite, and
> next steps may be to ask for real documents (e.g. open access papers)
> that are prepared with such format and to discuss most suitable style
> for CSL.

Suffice to say I disagree :-)

Sorry if the below gets wordy, but it's complicated.

Finding and analyzing existing papers again raises the question of
which ones; citation practices look VERY different in chemistry than
in art history or sociology. It also raises the question of who will
do this work, and whether it's the most efficient use of their time.
And finally, your suggestion seems to assume we didn't put a lot of
research and thought into the existing mappings.

Have you actually looked at the table of existing mappings? See table
1 here (which it seems we might want to add to the manual?):

https://blog.tecosaur.com/tmio/2021-07-31-citations.html#cite-syntax

Those mappings merely generalize existing systems (bibtex, natbib,
biblatex, csl) used by millions of users (if you include Zotero,
etc.), and already incorporate the feedback of those users.

So the styles included now ARE a sound starting point, that we can
iterate based on feedback.

We can and should add "number" and "entry" (for "bibentry" and
biblatex "fullcite") styles to those mappings, however, at least for
natbib and biblatex. But the first is one of those lower-level types
of commands, and probably why it's not there now.

On that last point, I do want people to understand that there are
places where you can't easily generalize across those systems, because
the logic of them varies in places. Biblatex, most notably, has IIRC
more than 50 commands, which also vary by style used. And some of
those commands (like autocite) are high-level and appropriate for this
sort of mapping, and others (like footcite) are low-level, and
probably not. Adding every option may make a small number of power
users happy, but at the expense of raising complexity for others.
Which is why the new defcustom is a good compromise.

CSL is different here than the LaTeX alternatives, as Andras can just
add support for some feature to citeproc-el, and add a style for it to
oc-csl. But CSL also has a different design than the LaTeX options,
and so will always be simpler. For example, in effect, all citations
in a CSL systems work like biblatex's "auto" commands. If one uses a
note-based citation style, citations get automatically footnoted, for
example, and so one can seamlessly change between note-based and other
kinds of styles, without having to modify the document source.

So missing mappings aren't necessarily an oversight; it might just be
that how to implement them was unclear, or whether users would need or
want them.



Bruce

Re: citations: org-cite vs org-ref 3.0

[Eric S Fraga]

On Wednesday, 23 Mar 2022 at 10:39, Bruce D'Arcus wrote:
> Those mappings merely generalize existing systems (bibtex, natbib,
> biblatex, csl) used by millions of users (if you include Zotero,
> etc.), and already incorporate the feedback of those users.

I just want to jump in to say that the new cite functionality works very
well, at least for this LaTeX user.  I'm writing a book at the moment
using [cite:...].  No issues encountered and everything I need to do is
there, out-of-the-box!

My thanks to all that put in so much effort into this.

-- 
: Eric S Fraga, with org release_9.5.2-407-gc9c0b9 in Emacs 29.0.50

Re: citations: org-cite vs org-ref 3.0

[Max Nikulin]

On 23/03/2022 06:52, Nicolas Goaziou wrote:
> Max Nikulin writes:
> 
>> Another point is more serious. Besides citations there are internal
>> cross-references. Org supports them but only in a rudimentary form.
>> Actually cross-references are similar to citations in the sense that
>> they can have style, prefixes and suffixes, and their appearance
>> depends on target properties. Another feature is grouping. However
>> cross-references should not be handled by citation backends, they
>> require different handlers. Unfortunately there is no way to define
>> custom "citation" type e.g. "[ref:...]" in addition to "[cite:...]".
> 
> There is too little information here for me to understand what
> cross-references (or grouping) are, or why they would require different
> handlers. In any case, if such thing are deemed necessary in Org Cite,
> one can always start a new thread.

Frankly speaking, for me it was enough just to use available LaTeX 
commands. I never thought about corner cases, but feature request 
requires such analysis.

By grouping I mean the existing feature of org-cite: several keys with 
common prefix and suffix, items within such group may be sorted.

Citation handlers obtain information necessary for formatting from a 
bibliography database. For cross-references such additional info should 
be pulled from other parts of the same document. There is no point to 
mix the code for both cases withing the same handler, it is better to 
combine independent handlers and to allow each type to have alternative 
implementations. I agree with those who say that cross-references and 
citations are rather independent when implementation is considered.

Judging from the description, org-ref (with org-refproc as an optional 
addition) does the job taking advantage of multiple custom link types 
for the same actual standard internal link target.

Sphinx has a feature that is an interesting border case between internal 
cross-references and external links (almost citations). Readthedocs 
sites host mapping tables for python packages. It is possible to fetch a 
file that associates e.g. anchors and section names. In generated docs 
anchors (similar to custom_id values) are replaced by section names and 
link target points to particular location in the external file.

Nicolas, concerning a new thread, I have an impression that you are busy 
with over activities since you are participating in discussions not so 
frequently. So I am unsure at which moment it is appropriate to raise 
such question that otherwise may just be buried in the list archive. 
Sometimes I am in doubts if an idea deserves a dedicated thread or 
initial feedback may be received from a related rather generic topic.

>> To assign additional properties, info "(org) Links in HTML export"
>> https://orgmode.org/manual/Links-in-HTML-export.html recommends usage
>> of "#+ATTR_HTML", but such technique has several issues:
> 
> This is a different issue. Citations are not link, and have a fixed set
> of properties.

Outside of Org, citations are links. Along with cross-references they 
worked before appearance of the web. To be recognizable on paper they 
may have a bit special form. An author may choose what will appear in 
the text: page number, section number, or section text. For HTML links 
page number option is not meaningful.

Within Org citations look like generalized links: a citation may have 
multiple targets, they have more properties: prefixes, suffixes 
(including common ones), style, and locators. However there is no 
description.

Internal links in Org (built-in ones without additional packages) are 
more limited than full-fledged cross-references. When exported they have 
the same form.

I consider fixed set of properties as a problem. At first I tried to 
draw attention to additional link attributes. Then I realized that 
block-level elements may have arbitrary attributes. It would be a great 
feature to have some syntax construct to assign attributes for 
particular inline object. People actively use link types as an 
additional property, but it gives just one degree of freedom. Sometimes 
more parameters are required and abusing link types means combinatorics 
explosion. Encoding extra properties inside link targets sometimes is 
enough (as in org-ref) but some times it is inconvenient.

Advantages of arbitrary attributes for inline objects for links:
- Within the same paragraph only part of links may be marked as 
"nofollow noopener" during exporting to HTML.
- Each link may have its own title.
- Link types similar to org-ref cross-references ("pageref", "nameref", 
"eqref", etc.) becomes an extra attribute while link type and link 
target remains the standard one for Org (heading text, custom id, name 
attribute).

For citations some values may be passed to specific citation backend 
overriding default value derived from style. It should be similar to 
babel header arguments specific to particular language or export options 
for particular format.

Leaving groups aside, attributes for inline objects may be a convenient 
extension.

Re: citations: org-cite vs org-ref 3.0

[Max Nikulin]

On 23/03/2022 21:39, Bruce D'Arcus wrote:
> 
> Finding and analyzing existing papers again raises the question of
> which ones; citation practices look VERY different in chemistry than
> in art history or sociology. It also raises the question of who will
> do this work, and whether it's the most efficient use of their time.
> And finally, your suggestion seems to assume we didn't put a lot of
> research and thought into the existing mappings.

I think that a working solution (Eric just have confirmed it) is a great 
result. I am aware that even in different areas of physics citation 
traditions vary. I am not trying to dispute that you invest a lot of 
efforts. I just consider as normal when some polishing is required after 
the initial release. Really exhaustive research is usually prohibitively 
expensive. I had a hope that it was clear from the beginning that I 
considered a missed style as a minor issue.

My point was that it should not be unconsciously ignored. Since the 
message was long enough, this particular complain may remain unnoticed. 
I can not say that I fully agree with your decision, but I respect it. I 
had no intention to upset you and I beg your pardon if it happened.

> Have you actually looked at the table of existing mappings? See table
> 1 here (which it seems we might want to add to the manual?):
> 
> https://blog.tecosaur.com/tmio/2021-07-31-citations.html#cite-syntax

I have seen this post and I have even checked the org-cite code that the 
example from John's message is not covered by the existing mapping.

Re: citations: org-cite vs org-ref 3.0

[Nicolas Goaziou]

Hello,

John Kitchin <jkitchin@andrew.cmu.edu> writes:

> I do not think it is productive for the community to say or consider it
> is a sad situation. Many good things have emerged from these
> discussions, even if it is not yet consensus on a solution. It is a
> complex problem, with many years of effort by many people on each side.
> That is an indication of how ambitious this project is and that there
> may be more than one solution that is needed.

[...]

> There are more than 8 years of legacy org-ref documents. I have written
> 40+ scientific papers with it, and countless technical documents with
> more than 8000 cite links among them. org-ref has exceeded 190K
> downloads from MELPA, so I feel obligated to maintain org-ref for
> myself, and those users. org-ref may be heavyweight in bundling a lot of
> capability together that could be separated into individual packages,
> but it is also convenient for people who need it, and a GitHUB issue or
> pull request away from new features. I remain committed to supporting
> this, and I do it in a way I can manage, hence the monolithic package
> design.

I think there's a misunderstanding here. Org Cite was never meant as
a replacement for Org Ref. It was designed from the beginning as
a framework other libraries could rely on and extend in their own way.
Org Ref could have been one of them.

It looks like a social problem to me. I remember well asking for
feedback when implementing the various prototypes, i.e., ways to make
Org Cite more useful to other libraries. I don't think I got much of it,
barring the cross-references topic, which is not solved. Maybe I was not
explicit enough about what I was expecting. For example, this is—please
correct me if I'm wrong—the first time I read about the "BibLaTeX is not
fully implemented in Org Cite" and "Org Cite is adding an extra
abstraction layer on top of BibLaTeX commands" issues, which are both
trivial to solve, either on the Org Cite or the Org Ref side. But then
again, it is perfectly fine if Org Cite doesn't provide that, as some
libraries can extend it if needed.

On the other hand, there have been much activity on GitHub repositories,
i.e., out of this mailing list. It seems to me Org Ref project has been
trying to work around possible blockers in Org Cite project the whole
time without the latter knowing about them, and having an opportunity to
lift them.

> Both projects have benefited from this discussion a lot. org has
> org-cite now, and org-ref now handles pre/post notes like org-cite does,
> it supports CSL much better, and is even a little more modular, lighter
> weight, and more easily integrated with other completion backends than
> ivy or helm. That should broadly be viewed as a win-win situation.

But it is not. There are now two, more or less official, citations
syntax. Interoperability is the big loss. Features do not count; it is
perfectly fine to have different packages targeting different needs, as
long as they share the same syntax.

Hopefully, at some point, we'll be able to build a list of blockers that
prevent Org Ref being built on top of Org Cite, and proceed.

Regards,
-- 
Nicolas Goaziou

Re: citations: org-cite vs org-ref 3.0

[Nicolas Goaziou]

Hello,

Dominik Schrempf <dominik.schrempf@gmail.com> writes:

> (add-to-list ’org-cite-biblatex-styles ’(“full” nil “fullcite” nil nil))
>
> (This or something similar should be added upstream).

I can add it, but "full" is already the name of a variant, so
[cite/full: ...] and [cite/style/full: ...] would mean different things.
Is this a problem, or do you think of a better style name? Also, are
there possible variants for this style?

Regards,
-- 
Nicolas Goaziou

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Wed, Mar 23, 2022 at 5:27 PM Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
>
> Hello,
>
> Dominik Schrempf <dominik.schrempf@gmail.com> writes:
>
> > (add-to-list ’org-cite-biblatex-styles ’(“full” nil “fullcite” nil nil))
> >
> > (This or something similar should be added upstream).
>
> I can add it, but "full" is already the name of a variant, so
> [cite/full: ...] and [cite/style/full: ...] would mean different things.
> Is this a problem, or do you think of a better style name?

FWIW, Nicolas, biblatex "fullcite" is equivalent to natbib/bibtex "bibentry".

That might be a reasonable alternative style name?

> Also, are there possible variants for this style?

AFAIK, no.

Bruce

Re: citations: org-cite vs org-ref 3.0

[Nicolas Goaziou]

Hello,

"Bruce D'Arcus" <bdarcus@gmail.com> writes:

> On Wed, Mar 23, 2022 at 5:27 PM Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:

>> I can add it, but "full" is already the name of a variant, so
>> [cite/full: ...] and [cite/style/full: ...] would mean different things.
>> Is this a problem, or do you think of a better style name?
>
> FWIW, Nicolas, biblatex "fullcite" is equivalent to natbib/bibtex "bibentry".
>
> That might be a reasonable alternative style name?
>
>> Also, are there possible variants for this style?
>
> AFAIK, no.

Hmm, OK. What about:

  (“fullcite” nil “fullcite” nil nil)

?

Regards,
-- 
Nicolas Goaziou

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Wed, Mar 23, 2022 at 6:04 PM Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
>
> Hello,
>
> "Bruce D'Arcus" <bdarcus@gmail.com> writes:
>
> > On Wed, Mar 23, 2022 at 5:27 PM Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
>
> >> I can add it, but "full" is already the name of a variant, so
> >> [cite/full: ...] and [cite/style/full: ...] would mean different things.
> >> Is this a problem, or do you think of a better style name?
> >
> > FWIW, Nicolas, biblatex "fullcite" is equivalent to natbib/bibtex "bibentry".
> >
> > That might be a reasonable alternative style name?
> >
> >> Also, are there possible variants for this style?
> >
> > AFAIK, no.
>
> Hmm, OK. What about:
>
>   (“fullcite” nil “fullcite” nil nil)
>
> ?

Seems fine by me, so long as you use the same name for natbib if and
when you add bibentry support?

Bruce

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Wed, Mar 23, 2022 at 1:19 PM Max Nikulin <manikulin@gmail.com> wrote:

> My point was that it should not be unconsciously ignored. Since the
> message was long enough, this particular complain may remain unnoticed.
> I can not say that I fully agree with your decision, but I respect it. I
> had no intention to upset you and I beg your pardon if it happened.

No personal offense taken. I was just a bit frustrated with the
direction this discussion seemed to be going.

Bruce

Re: citations: org-cite vs org-ref 3.0

[Nicolas Goaziou]

Hello,

Max Nikulin <manikulin@gmail.com> writes:

> Nicolas, concerning a new thread, I have an impression that you are
> busy with over activities since you are participating in discussions
> not so frequently. So I am unsure at which moment it is appropriate to
> raise such question that otherwise may just be buried in the list
> archive.

I don't see how my presence (or not) on the list relates to this. If
there's an idea worth a discussion, it should not be buried within
a thread.

> Outside of Org, citations are links.

But we're on an Org mailing list so…

> I consider fixed set of properties as a problem.

I don't think it is a problem for citations.

> At first I tried to draw attention to additional link attributes.

I think link attributes were discussed a couple of times on this ML
already. Nothing was implemented tho.

I'm not convinced Org should generalize this to any inline object,
either, mainly because it sounds messy. Of course, if you have an
idea on the subject, please share it.

In any case, this is another topic, neither related to citations nor to
cross-references.

> For citations some values may be passed to specific citation backend
> overriding default value derived from style.

In that situation, you can define a new style specific to the citation
back-end.

Regards,
-- 
Nicolas Goaziou

Re: citations: org-cite vs org-ref 3.0

[Dominik Schrempf]

[-- Attachment #1: Type: text/plain, Size: 1335 bytes --]

I think `fullcite' is OK, although it will be a bit verbose:

┌────
│ [cite/fullcite:...]
└────


Personally, I don’t mind using `full', and so having a duplicate between a style
and a variant.

But, to be honest, anything is fine with me, as long as it is readily available
and documented.

Thank you!
Dominik

“Bruce D’Arcus” <bdarcus@gmail.com> writes:

> On Wed, Mar 23, 2022 at 6:04 PM Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
>>
>> Hello,
>>
>> “Bruce D’Arcus” <bdarcus@gmail.com> writes:
>>
>> > On Wed, Mar 23, 2022 at 5:27 PM Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
>>
>> >> I can add it, but “full” is already the name of a variant, so
>> >> [cite/full: …] and [cite/style/full: …] would mean different things.
>> >> Is this a problem, or do you think of a better style name?
>> >
>> > FWIW, Nicolas, biblatex “fullcite” is equivalent to natbib/bibtex “bibentry”.
>> >
>> > That might be a reasonable alternative style name?
>> >
>> >> Also, are there possible variants for this style?
>> >
>> > AFAIK, no.
>>
>> Hmm, OK. What about:
>>
>>   (“fullcite” nil “fullcite” nil nil)
>>
>> ?
>
> Seems fine by me, so long as you use the same name for natbib if and
> when you add bibentry support?
>
> Bruce

Re: citations: org-cite vs org-ref 3.0

[Max Nikulin]

On 22/03/2022 00:00, John Kitchin wrote:
> "Bruce D'Arcus" writes:
> 
>> Indeed, the question of how to better support cross-references in org
>> is an important one.
>>
>> I don't really use them much, and so am still unsure if this could be
>> addressed with incremental improvements in existing org link support,
>> or if it would require more significant enhancements.
> 
> No incremental improvement is required IMO, regular links are
> sufficient. org-ref has handled these with regular links from the
> beginning. It even has an org-ref-refproc now for non-LaTeX exports that
> has some support for things like sorting, grouping and cleveref. You can
> find an example org file at
> https://github.com/jkitchin/org-ref/blob/master/examples/refproc.org,

Interesting package. At first I thought you use keywords around links as 
implicit markup, but later I realized that this file is example of 
export, not the original source file (with different link types).

> and see it in action at https://www.youtube.com/watch?v=rRR-5NSpKyE the
> video has some rough spots, but you can get the idea.
> 
> You can support all kinds of things in these links (for example, I now
> support pre/post note text in the cite link paths), really anything you
> want to parse out of the path.

Re: citations: org-cite vs org-ref 3.0

[Max Nikulin]

On 21/03/2022 18:51, John Kitchin wrote:
> Vikas Rawal writes:
>>
>>  From the perspective of a user, this was only meant to express a
>> sentiment that one finds oneself in a situation of having to choose
>> between two good things, and that we have not been able to find a way
>> to make both compatible with each other. It was in not meant as a
>> disrespect in any way.
> 
> I don't think you have to choose. You can use org-cite for
> citations, and org-ref for cross-references. The citation syntax is
> orthogonal, you just should not mix them. You can even wire org-ref to
> use org-cite-insert like this:
> 
> (setq org-ref-insert-cite-function (lambda () (org-cite-insert nil)))

I am glad to read this. John, could you, please, update the README file 
for https://github.com/jkitchin/org-ref to clarify that both packages 
may be used within the same document while org-ref is used solely for 
cross-references (I may create a github issue if your prefer)? From my 
point of view the following phrase may be considered as prohibitive in 
respect to combining the packages for any purpose:

> You can use both org-cite and org-ref (although you should not do 
> that in the same document as they are independent citation tools).
I feel some ambiguity in the starting message in this thread:

Vikas Rawal, Sun, 20 Mar 2022 17:38:30 +0530. 
https://list.orgmode.org/CALtzAB2BHULDOXaamUZFQh2H453Ekb6K7bkOHbU-dHPn98avwg@mail.gmail.com

> This obviously creates many problems including that two people using
> different citation systems cannot share org files.
Accordingly to the org-ref README installed org-ref package does not 
break org-cite, so people are free to share documents. They just should 
have both packages configured and should adhere to a chosen package in 
each document (for *citations*).

Re: citations: org-cite vs org-ref 3.0

[Max Nikulin]

On 24/03/2022 06:04, Nicolas Goaziou wrote:
> Max Nikulin writes:
> 
>> Nicolas, concerning a new thread, I have an impression that you are
>> busy with over activities since you are participating in discussions
>> not so frequently. So I am unsure at which moment it is appropriate to
>> raise such question that otherwise may just be buried in the list
>> archive.
> 
> I don't see how my presence (or not) on the list relates to this. If
> there's an idea worth a discussion, it should not be buried within
> a thread.

It was you who implemented org-cite and org-element parser. John chose a 
direct and practical way. He defined multiple link types and got the 
working solution. A more elegant approach would require some extension 
of syntax, so your opinion is really important. For cross-references it 
might be "[ref/style...]" in additional to "[cite/style...]" or 
attributes specific to links.

>> Outside of Org, citations are links.
> 
> But we're on an Org mailing list so…

In respect to citations and cross-references Org is a tool to prepare 
documents for "outside of Org".

> I think link attributes were discussed a couple of times on this ML
> already. Nothing was implemented tho.

I do not remember such thread during last couple of years. I will try to 
search deeper and maybe will start another thread.

> I'm not convinced Org should generalize this to any inline object,
> either, mainly because it sounds messy. Of course, if you have an
> idea on the subject, please share it.
> 
> In any case, this is another topic, neither related to citations nor to
> cross-references.

"pageref", "nameref", "eqref", "autoref" (see 
https://github.com/jkitchin/org-ref/blob/master/org-ref.org) may be 
values of the style attribute of a "[[name]]" link instead of link type 
as it is done in org-ref.

>> For citations some values may be passed to specific citation backend
>> overriding default value derived from style.
> 
> In that situation, you can define a new style specific to the citation
> back-end.

Sometimes an ad-hoc adjustment at the particular place is more 
convenient than even file-local property.

Re: citations: org-cite vs org-ref 3.0

[Max Nikulin]

On 21/03/2022 18:51, John Kitchin wrote:
> 
> citenum and bibentry are the only two I am not sure have a CSL analog.

I read your messages once more and I should say that I feel some 
disagreement of this one (I removed most of it) and the earlier and 
longer one from Sun, 20 Mar 2022 20:31:29 -0400 
https://list.orgmode.org/m2sfrc149c.fsf@andrew.cmu.edu

I admit that org-ref is carefully tuned to your workflow. I hope, it is 
possible to left aside decomposition of org-cite into modules for some time.

Let's assume org-cite with natbib backend for citations and org-ref for 
cross-references. It seems, a couple of missed styles currently is not a 
problem due to the defcustom for the mapping.

Are there still any technical limitations that prevent getting in the 
exported LaTeX file the same citation commands as for org-ref?

In particular I am worrying concerning 
https://github.com/jkitchin/org-ref README (and the same phrase from the 
earlier message):

> org-cite does not meet my citation and technical document publishing 
> needs, and it was not possible to integrate it into org-ref without 
> compromising those.
Does it refer to exported result or to convenience of working with 
citations? Would it help if it were possible to choose style by its 
natbib command?

I see that you do not like org-cite styles, but I can not figure out 
what are the real blockers that prevent producing documents having the 
same quality.

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

I obviously can't speak for John, but on this:

On Fri, Mar 25, 2022 at 1:11 PM Max Nikulin <manikulin@gmail.com> wrote:
>
> On 21/03/2022 18:51, John Kitchin wrote:
> >
> > citenum and bibentry are the only two I am not sure have a CSL analog.

As I said in an earlier message, it's no problem to add these. Someone
can always add the bibentry/fullcite style to oc-csl later, if and
when Andras adds the functionality to citeproc-el.

Though it's feasible, I doubt you'll ever see citenum in CSL, since
those sorts of low-level commands don't really fit there.

But that also is not a problem.

> I read your messages once more and I should say that I feel some
> disagreement of this one (I removed most of it) and the earlier and
> longer one from Sun, 20 Mar 2022 20:31:29 -0400
> https://list.orgmode.org/m2sfrc149c.fsf@andrew.cmu.edu
>
> I admit that org-ref is carefully tuned to your workflow. I hope, it is
> possible to left aside decomposition of org-cite into modules for some time.
>
> Let's assume org-cite with natbib backend for citations and org-ref for
> cross-references. It seems, a couple of missed styles currently is not a
> problem due to the defcustom for the mapping.

The mapping defcustom is currently only in oc-biblatex. It might be
worth adding it to oc-natbib, and so generalizing how these mappings
are handled in latex?

Bruce

> Are there still any technical limitations that prevent getting in the
> exported LaTeX file the same citation commands as for org-ref?
>
> In particular I am worrying concerning
> https://github.com/jkitchin/org-ref README (and the same phrase from the
> earlier message):
>
> > org-cite does not meet my citation and technical document publishing
> > needs, and it was not possible to integrate it into org-ref without
> > compromising those.
> Does it refer to exported result or to convenience of working with
> citations? Would it help if it were possible to choose style by its
> natbib command?
>
> I see that you do not like org-cite styles, but I can not figure out
> what are the real blockers that prevent producing documents having the
> same quality.
>
>

Re: citations: org-cite vs org-ref 3.0

[M. Pger]

On Wednesday, March 23rd, 2022 at 3:39 PM, Bruce D'Arcus <bdarcus@gmail.com> wrote:

> Have you actually looked at the table of existing mappings? See table 1 here (which it seems we might want to add to the manual?):
>
> https://blog.tecosaur.com/tmio/2021-07-31-citations.html#cite-syntax

Sorry for the late reaction to this. As an user I completely agree. IMO, having these mappings in the doc could clearly convince some org-ref users to give a try to org-cite. Some information about the possibility of using .bst bibliography style files would also be nice. Finally, some info about cross-ref could also be useful, so that users easily get an idea of what would imply a switch from org-ref to org-cite.

For what it's worth, I think that Vanilla org becomes more and more amazing, thanks for that.

Best,

MP

Re: citations: org-cite vs org-ref 3.0

[John Kitchin]

I have updated the readme in org-ref to indicate you can use both
packages.

Max Nikulin <manikulin@gmail.com> writes:

> On 21/03/2022 18:51, John Kitchin wrote:
>> Vikas Rawal writes:
>>>
>>>  From the perspective of a user, this was only meant to express a
>>> sentiment that one finds oneself in a situation of having to choose
>>> between two good things, and that we have not been able to find a way
>>> to make both compatible with each other. It was in not meant as a
>>> disrespect in any way.
>> I don't think you have to choose. You can use org-cite for
>> citations, and org-ref for cross-references. The citation syntax is
>> orthogonal, you just should not mix them. You can even wire org-ref to
>> use org-cite-insert like this:
>> (setq org-ref-insert-cite-function (lambda () (org-cite-insert nil)))
>
> I am glad to read this. John, could you, please, update the README file for
> https://github.com/jkitchin/org-ref to clarify that both packages may be used
> within the same document while org-ref is used solely for cross-references (I
> may create a github issue if your prefer)? From my point of view the following
> phrase may be considered as prohibitive in respect to combining the packages for
> any purpose:
>
>> You can use both org-cite and org-ref (although you should not do that in the
>> same document as they are independent citation tools).
> I feel some ambiguity in the starting message in this thread:
>
> Vikas Rawal, Sun, 20 Mar 2022 17:38:30 +0530.
> https://list.orgmode.org/CALtzAB2BHULDOXaamUZFQh2H453Ekb6K7bkOHbU-dHPn98avwg@mail.gmail.com
>
>> This obviously creates many problems including that two people using
>> different citation systems cannot share org files.
> Accordingly to the org-ref README installed org-ref package does not break
> org-cite, so people are free to share documents. They just should have both
> packages configured and should adhere to a chosen package in each document (for
> *citations*).


-- 
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu
Pronouns: he/him/his

Re: citations: org-cite vs org-ref 3.0

[John Kitchin]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Hello,
>
> Max Nikulin <manikulin@gmail.com> writes:
>
>> Nicolas, concerning a new thread, I have an impression that you are
>> busy with over activities since you are participating in discussions
>> not so frequently. So I am unsure at which moment it is appropriate to
>> raise such question that otherwise may just be buried in the list
>> archive.
>
> I don't see how my presence (or not) on the list relates to this. If
> there's an idea worth a discussion, it should not be buried within
> a thread.
>
>> Outside of Org, citations are links.
>
> But we're on an Org mailing list so…
>
>> I consider fixed set of properties as a problem.
>
> I don't think it is a problem for citations.
>
>> At first I tried to draw attention to additional link attributes.
>
> I think link attributes were discussed a couple of times on this ML
> already. Nothing was implemented tho.

I don't think any further implementation outside the existing link
framework is required to do this. The only limitations of the current
framework are (I think) it is limited to a single line (i.e. not
multiline), and it would be difficult to have nested links.

Otherwise, you can put most things in the path, and then parse it as you
see fit. I think it would be interesting to couple this with a recursive
descent parser one day, and some DSL perhaps.

There is a prototype of this idea at
https://kitchingroup.cheme.cmu.edu/blog/2015/02/05/Extending-the-org-mode-link-syntax-with-attributes/

I wouldn't claim it is the best way to do that, or the right thing to
do.

scimax-editmarks is a step further
(https://github.com/jkitchin/scimax/blob/master/scimax-editmarks.org)
that doesn't use org-links or any org-syntax for something more like an
inline object. It mostly addresses the multiline issue, but it also
doesn't support nested objects, mostly because of my limited knowledge
of recursive parsing. I would not advocate for putting this in org
either though.

If you are interested in this kind of thing, you might find
https://www.emacswiki.org/emacs/linkd.el interesting too. It leverages
an S-expression approach, which makes the parsing and nesting more
straightforward.

I know this is a little off-track of the original thread, but I agree
with Nicolas that it does not seem necessary at this point to add this
into org.

>
> I'm not convinced Org should generalize this to any inline object,
> either, mainly because it sounds messy. Of course, if you have an
> idea on the subject, please share it.
>
> In any case, this is another topic, neither related to citations nor to
> cross-references.
>
>> For citations some values may be passed to specific citation backend
>> overriding default value derived from style.
>
> In that situation, you can define a new style specific to the citation
> back-end.
>
> Regards,


-- 
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu
Pronouns: he/him/his

Re: citations: org-cite vs org-ref 3.0

[Vikas Rawal]

[-- Attachment #1: Type: text/plain, Size: 90 bytes --]

I have updated the readme in org-ref to indicate you can use both
> packages.
>
> Thanks.

[-- Attachment #2: Type: text/html, Size: 400 bytes --]

Re: citations: org-cite vs org-ref 3.0

[John Kitchin]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Hello,
>
> John Kitchin <jkitchin@andrew.cmu.edu> writes:
>
>> I do not think it is productive for the community to say or consider it
>> is a sad situation. Many good things have emerged from these
>> discussions, even if it is not yet consensus on a solution. It is a
>> complex problem, with many years of effort by many people on each side.
>> That is an indication of how ambitious this project is and that there
>> may be more than one solution that is needed.
>
> [...]
>
>> There are more than 8 years of legacy org-ref documents. I have written
>> 40+ scientific papers with it, and countless technical documents with
>> more than 8000 cite links among them. org-ref has exceeded 190K
>> downloads from MELPA, so I feel obligated to maintain org-ref for
>> myself, and those users. org-ref may be heavyweight in bundling a lot of
>> capability together that could be separated into individual packages,
>> but it is also convenient for people who need it, and a GitHUB issue or
>> pull request away from new features. I remain committed to supporting
>> this, and I do it in a way I can manage, hence the monolithic package
>> design.
>
> I think there's a misunderstanding here. Org Cite was never meant as
> a replacement for Org Ref. It was designed from the beginning as
> a framework other libraries could rely on and extend in their own way.
> Org Ref could have been one of them.
>
> It looks like a social problem to me. I remember well asking for
> feedback when implementing the various prototypes, i.e., ways to make
> Org Cite more useful to other libraries. I don't think I got much of it,
> barring the cross-references topic, which is not solved. Maybe I was not
> explicit enough about what I was expecting. For example, this is—please
> correct me if I'm wrong—the first time I read about the "BibLaTeX is not
> fully implemented in Org Cite" and "Org Cite is adding an extra
> abstraction layer on top of BibLaTeX commands" issues, which are both
> trivial to solve, either on the Org Cite or the Org Ref side. But then
> again, it is perfectly fine if Org Cite doesn't provide that, as some
> libraries can extend it if needed.

I don't think it is the first time I have mentioned biblatex is not
fully implemented, but I am not sure. I have mentioned things like
\citenum somewhere, but it is not the main point.

I know it is not that difficult to add support for LaTeX export in
org-cite, e.g. [cite/num:]. I don't think it is all that easy to add
additional backend support though, for something like [cite/num:] in
HTML or other backends. No doubt, it is not impossible, if someone would
just do it, and that might include extending citeproc too, or developing
some pre-processing function to get a citation number, or something
else. Whether cite/num or any other command exists is not the main
point. What is the point is what mechanisms exist to support the
addition, and the exports to various backends.

Regarding that org-cite adds an abstraction layer, how else could one
interpret this (from
https://blog.tecosaur.com/tmio/2021-07-31-citations.html#cite-syntax)
other than abstraction:

[cite/na/b:@key] or [cite/noauthor/bare:@key] to mean \citeyear{key}?

Why wouldn't it be \citetitle? or \citeurl, or \citedate? or even,
\citenum?

I get it, you can define [cite/noauthor/year:] or even [cite/year:] or
[cite/y:] and even [cite/citeyear:] to get the command in there, and
something for each of those other ones. Maybe even the documented
convention will change to some other potentially mnemonic form. 

I also get they are not all that common perhaps, except for a few people
who use them, and maybe should be responsible for supporting it
themselves, either by defcustom or their own library.

This is just trading a proliferation of links for a proliferation of
styles IMO, and it feels a lot like the XKCD standards issue
https://xkcd.com/927/.

As others have argued, it is just a bit of knowledge, maybe a UI can
compensate to help you insert what you want, then know what it means
later. It is my opinion that this will be a technical debt though. I am
content to agree to disagree on this point.

It might be a social problem, and I do not think it is trivial to solve.
It is not just about having a syntax that works. I wrote and shared a
whole set of processors for org-cite that did or tried to do all those
things above. It was fine to use, but it was very difficult code to
write for me. I don't personally want to support it in part because it
was so difficult to write. I don't even want to support it for my own
use at this time. This should not stop anyone who wants to do that
themselves though. Maybe there is a cleaner approach I missed, or others
may be better programmers, and/or have more time to figure this out. At
this time, I do not have time to make for it.

> On the other hand, there have been much activity on GitHub repositories,
> i.e., out of this mailing list. It seems to me Org Ref project has been
> trying to work around possible blockers in Org Cite project the whole
> time without the latter knowing about them, and having an opportunity to
> lift them.

There is nothing nefarious happening here. That work happened in public,
and with some interactions with people on the org-list including Bruce. 

Some motivation for org-cite stemmed from at least perceived limitations
in org-ref, especially related to pre/post notes and CSL support. I
think it is totally reasonable to learn if those were real limitations
or not. 

>> Both projects have benefited from this discussion a lot. org has
>> org-cite now, and org-ref now handles pre/post notes like org-cite does,
>> it supports CSL much better, and is even a little more modular, lighter
>> weight, and more easily integrated with other completion backends than
>> ivy or helm. That should broadly be viewed as a win-win situation.
>
> But it is not. There are now two, more or less official, citations
> syntax. Interoperability is the big loss. Features do not count; it is
> perfectly fine to have different packages targeting different needs, as
> long as they share the same syntax.
>
> Hopefully, at some point, we'll be able to build a list of blockers that
> prevent Org Ref being built on top of Org Cite, and proceed.

You can use the org-cite syntax with org-ref. If all one wants is a
citation syntax for their org-files, and an occasional standard
(cite/citet/citep) LaTeX export export, org-cite will probably meet their
needs. As a few have reported, it works for them.

If you have very large bibtex files, you may find that the basic
processors don't have good performance yet, and you may need to
configure org to use a performant set of processors for activation and
insertion. Yes, this is being worked on in org, and you will need to use
the latest version of org to benefit from it.

If you have very specific needs in biblatex, you may find not every
command has a corresponding org-cite implementation. You may have to add
this to your own setup, or use a specific set of processors that provide
it. You can do that, and still use it with org-ref.

Maybe one day I will have time to try integrating org-cite with org-ref
again. I have been stretched too thin by work for the past two years,
and in the forseeable future to spend much time on org-cite. This is a
me issue, not an org-cite issue.





>
> Regards,


-- 
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu
Pronouns: he/him/his

Re: citations: org-cite vs org-ref 3.0

[John Kitchin]

Max Nikulin <manikulin@gmail.com> writes:

> On 21/03/2022 18:51, John Kitchin wrote:
>> citenum and bibentry are the only two I am not sure have a CSL analog.
>
> I read your messages once more and I should say that I feel some disagreement of
> this one (I removed most of it) and the earlier and longer one from Sun, 20 Mar
> 2022 20:31:29 -0400 https://list.orgmode.org/m2sfrc149c.fsf@andrew.cmu.edu
>
> I admit that org-ref is carefully tuned to your workflow. I hope, it is possible
> to left aside decomposition of org-cite into modules for some time.
>
> Let's assume org-cite with natbib backend for citations and org-ref for
> cross-references. It seems, a couple of missed styles currently is not a problem
> due to the defcustom for the mapping.
>
> Are there still any technical limitations that prevent getting in the exported
> LaTeX file the same citation commands as for org-ref?

If all one wants is LaTeX export, there are not technical limitations.
That is not all people want in general though, they also want other
exports, and that they are reasonably similar, or even possible. It is
easy to get \citenum and \bibentry for LaTeX. I promise though, someone
wants this or some other thing for HTML. 

Rather than rehash a lot of experiences, I really encourage you to try
writing a processor that can support this. Or even, try modifying
org-ref-cite to support it. Not as some thought experiment about what
should be possible, but an actual experiment that is worked out.

> In particular I am worrying concerning https://github.com/jkitchin/org-ref
> README (and the same phrase from the earlier message):
>
>> org-cite does not meet my citation and technical document publishing needs,
>> and it was not possible to integrate it into org-ref without compromising
>> those.

I have taken this out of the readme. I still agree with the sentiment,
but my needs are not the same as others (for example, I include in my
needs include ease of support and development, which is not a user
need), and it is possible to meet some basic needs fully.

> Does it refer to exported result or to convenience of working with citations?
> Would it help if it were possible to choose style by its natbib command?
>
> I see that you do not like org-cite styles, but I can not figure out what are
> the real blockers that prevent producing documents having the same
> quality.

If you spend enough time teaching people how to use these things, you
might feel differently about this. It is so easy to just switch away to
Endnote, Papers, Paperpile, Mendeley, Zotero, Overleaf, etc. I guess I
am not on the prevailing side here though.


-- 
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu
Pronouns: he/him/his

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Sun, Mar 27, 2022 at 3:41 PM John Kitchin <jkitchin@andrew.cmu.edu> wrote:

...

> Regarding that org-cite adds an abstraction layer, how else could one
> interpret this (from
> https://blog.tecosaur.com/tmio/2021-07-31-citations.html#cite-syntax)
> other than abstraction:
>
> [cite/na/b:@key] or [cite/noauthor/bare:@key] to mean \citeyear{key}?
>
> Why wouldn't it be \citetitle? or \citeurl, or \citedate? or even,
> \citenum?

You mean why shouldn't we privilege natbib, as you have in org-ref?

And let me turn the question around: how would you propose to
translate those natbib-derived commands to biblatex, or CSL, so it
works reliably across users and documents? The mapping has to happen
someplace, after all.

And from a UX POV, how well would that work for users who have no
previous experience with natbib or even latex?

...

> It might be a social problem, and I do not think it is trivial to solve.
> It is not just about having a syntax that works. I wrote and shared a
> whole set of processors for org-cite that did or tried to do all those
> things above. It was fine to use, but it was very difficult code to
> write for me. I don't personally want to support it in part because it
> was so difficult to write.

I think what Nicolas is asking is when you have time, to itemize the
pain points that made development difficult for you, so that we might
figure out how to improve them (perhaps new helper functions, etc.).

As another data point, one of the things I've loved about org-cite is
how easy I found it to develop pretty functional processors for citar
with minimal code. Total LOC for citar is just under 2000, with just
under 400 specific to org.

But I'm deliberately developing a small, focused, modular, package.

...

> Some motivation for org-cite stemmed from at least perceived limitations
> in org-ref, especially related to pre/post notes and CSL support. I
> think it is totally reasonable to learn if those were real limitations
> or not.

The org-cite citation syntax and model, I hope you would agree, is
unambiguously better than natbib or the original org-ref syntax. It's
simpler than biblatex in the sense of no difference between single and
multicite citations, but can easily and losslessly map to and from
either.

Org-ref 3.0 adds essentially a copy of that syntax and model, with a
trivial difference.

To me, that's the biggest problem. Aside from users having
incompatible documents, it forces other developers either to dedicate
additional development and maintenance time to supporting both
syntaxes, or to choose one.

Pandoc only supports org-cite.

In the case of citar, I have also decided to only support org-cite
(though I leave the function to generate citations configurable, so
it's easy enough for a user to configure this themselves; but I don't
include this by default because I have other processor code that
relies on parsed citations).

Org-roam supports all three.

It sounds like the biggest hold up was with reconciling the org-ref
command model with more general approach of the oc style and variants.

But as a first step, you could do as you originally planned to: simply
use the same names for styles. If Nicolas were to allow the mapping in
natbib to be handled via the defcustom, you could even do this without
having to write and maintain your own export processor.

And then you could save the other part (how to map those to other
export processors) for another step, if and when you or your users
need it or want it.

Certainly that would address the most fundamental incompatibility.

I guess to be direct: what value does the v3 syntax provide you, your
users, or the org ecosystem in general?

Bruce

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Sun, Mar 27, 2022 at 12:00 PM John Kitchin <jkitchin@andrew.cmu.edu> wrote:
>
>
> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
> > Hello,
> >
> > Max Nikulin <manikulin@gmail.com> writes:
> >
> >> Nicolas, concerning a new thread, I have an impression that you are
> >> busy with over activities since you are participating in discussions
> >> not so frequently. So I am unsure at which moment it is appropriate to
> >> raise such question that otherwise may just be buried in the list
> >> archive.
> >
> > I don't see how my presence (or not) on the list relates to this. If
> > there's an idea worth a discussion, it should not be buried within
> > a thread.
> >
> >> Outside of Org, citations are links.
> >
> > But we're on an Org mailing list so…
> >
> >> I consider fixed set of properties as a problem.
> >
> > I don't think it is a problem for citations.
> >
> >> At first I tried to draw attention to additional link attributes.
> >
> > I think link attributes were discussed a couple of times on this ML
> > already. Nothing was implemented tho.
>
> I don't think any further implementation outside the existing link
> framework is required to do this. The only limitations of the current
> framework are (I think) it is limited to a single line (i.e. not
> multiline), and it would be difficult to have nested links.

I'm glad to hear that.

FWIW, by asking earlier about possible "incremental improvements" for
cross-references, I was including within that possible implementation
improvements: new helper functions or interactive commands, standard
link types, documentation, etc.

I note, for example, that currently local links do export as LaTeX
cross-references, and that if you use cleverref, the results seem
pretty reasonable from my non-expert POV.

But it's not easy from a UX POV to create those cross-references,
particularly with a longer, more complex, document.

And in my coding experiments [1] awhile back, it wasn't that
straightforward to create a UI to make that easier, mostly because it
wasn't unambiguous how to consistently parse common targets (sections,
figures, tables, equations) [2].

Bruce

[1] https://github.com/bdarcus/oxr
[2] https://github.com/bdarcus/oxr/blob/28ec1345e9c8d659afea6c8569479b667697eaa2/oxr.el#L108-L128

> Otherwise, you can put most things in the path, and then parse it as you
> see fit. I think it would be interesting to couple this with a recursive
> descent parser one day, and some DSL perhaps.
>
> There is a prototype of this idea at
> https://kitchingroup.cheme.cmu.edu/blog/2015/02/05/Extending-the-org-mode-link-syntax-with-attributes/
>
> I wouldn't claim it is the best way to do that, or the right thing to
> do.
>
> scimax-editmarks is a step further
> (https://github.com/jkitchin/scimax/blob/master/scimax-editmarks.org)
> that doesn't use org-links or any org-syntax for something more like an
> inline object. It mostly addresses the multiline issue, but it also
> doesn't support nested objects, mostly because of my limited knowledge
> of recursive parsing. I would not advocate for putting this in org
> either though.
>
> If you are interested in this kind of thing, you might find
> https://www.emacswiki.org/emacs/linkd.el interesting too. It leverages
> an S-expression approach, which makes the parsing and nesting more
> straightforward.
>
> I know this is a little off-track of the original thread, but I agree
> with Nicolas that it does not seem necessary at this point to add this
> into org.
>
> >
> > I'm not convinced Org should generalize this to any inline object,
> > either, mainly because it sounds messy. Of course, if you have an
> > idea on the subject, please share it.
> >
> > In any case, this is another topic, neither related to citations nor to
> > cross-references.
> >
> >> For citations some values may be passed to specific citation backend
> >> overriding default value derived from style.
> >
> > In that situation, you can define a new style specific to the citation
> > back-end.
> >
> > Regards,
>
>
> --
> Professor John Kitchin
> Doherty Hall A207F
> Department of Chemical Engineering
> Carnegie Mellon University
> Pittsburgh, PA 15213
> 412-268-7803
> @johnkitchin
> http://kitchingroup.cheme.cmu.edu
> Pronouns: he/him/his
>

Re: citations: org-cite vs org-ref 3.0

[Max Nikulin]

On 28/03/2022 02:40, John Kitchin wrote:
> Max Nikulin writes:
>> On 21/03/2022 18:51, John Kitchin wrote:
>
> Rather than rehash a lot of experiences, I really encourage you to try
> writing a processor that can support this. Or even, try modifying
> org-ref-cite to support it. Not as some thought experiment about what
> should be possible, but an actual experiment that is worked out.

I have some ideas for links, other inline objects and export attributes 
that I should try before.

>> In particular I am worrying concerning https://github.com/jkitchin/org-ref
>> README (and the same phrase from the earlier message):
>>
>>> org-cite does not meet my citation and technical document publishing needs,
>>> and it was not possible to integrate it into org-ref without compromising
>>> those.
> 
> I have taken this out of the readme. I still agree with the sentiment,
> but my needs are not the same as others (for example, I include in my
> needs include ease of support and development, which is not a user
> need), and it is possible to meet some basic needs fully.

John, in another message (Sun, 27 Mar 2022 13:00:40 -0400) 
https://list.orgmode.org/m24k3jnq0k.fsf@andrew.cmu.edu you clearly 
stated a technical limitation that is a real reason why org-cite is not 
an option for you and for some other users: performance has not been 
optimized for large BibTeX databases. It is deserved to be mentioned. 
You even have managed to convince me that, besides adding missing style 
names, some existing ones should be adjusted. noauthor/bare for citeyear 
example makes for me much more sense than general words that org-cite is 
too CSL-oriented. So the opinion that the set of styles has not really 
settled yet may be precious for users choosing between org-cite and org-ref.

 From my point of view such arguments are more clear than your earlier 
message (Sun, 20 Mar 2022 20:31:29 -0400) 
https://list.orgmode.org/m2sfrc149c.fsf@andrew.cmu.edu I had no 
intention to force to remove that phrase before the issues are fixed. I 
was trying to ask you about more details behind your opinion (and now we 
have them).

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Mon, Mar 28, 2022 at 8:37 AM Max Nikulin <manikulin@gmail.com> wrote:
>
> On 28/03/2022 02:40, John Kitchin wrote:
> > Max Nikulin writes:
> >> On 21/03/2022 18:51, John Kitchin wrote:
> >
> > Rather than rehash a lot of experiences, I really encourage you to try
> > writing a processor that can support this. Or even, try modifying
> > org-ref-cite to support it. Not as some thought experiment about what
> > should be possible, but an actual experiment that is worked out.
>
> I have some ideas for links, other inline objects and export attributes
> that I should try before.
>
> >> In particular I am worrying concerning https://github.com/jkitchin/org-ref
> >> README (and the same phrase from the earlier message):
> >>
> >>> org-cite does not meet my citation and technical document publishing needs,
> >>> and it was not possible to integrate it into org-ref without compromising
> >>> those.
> >
> > I have taken this out of the readme. I still agree with the sentiment,
> > but my needs are not the same as others (for example, I include in my
> > needs include ease of support and development, which is not a user
> > need), and it is possible to meet some basic needs fully.
>
> John, in another message (Sun, 27 Mar 2022 13:00:40 -0400)
> https://list.orgmode.org/m24k3jnq0k.fsf@andrew.cmu.edu you clearly
> stated a technical limitation that is a real reason why org-cite is not
> an option for you and for some other users: performance has not been
> optimized for large BibTeX databases. It is deserved to be mentioned.

FWIW, Ihor posted a patch related to this a week or so ago.

> You even have managed to convince me that, besides adding missing style
> names, some existing ones should be adjusted. noauthor/bare for citeyear
> example makes for me much more sense ...

This does need some attention, but there are wrinkles here.

Citeyear is specific to author-date styles, while noauthor is intended
to be more general.

Hence, initially "noauthor".

This indeed was influenced by CSL implementations like Zotero and
pandoc, which have a notion of "suppress author".

But the names are kind of awkward admittedly, and Andras Simonyi
subsequently added a "year" style in oc-csl.

You can see the conversation about it here.

https://github.com/andras-simonyi/oc-csl-ns/issues/1#issuecomment-895440897

I think it's probably a good idea to add "year" to the latex processors too.

I will reproduce Denis' explanation from that linked comment here:

-------
The reasoning behind "noauthor" vs "year" was that our priority was
portability between different classes of citation styles. If we have
this item "John, Doe. /A book/. 2020." the variant "noauthor" can be
used to render this consistently.

In-Text:
a) author-year: "(2020)"
b) author-title: "(/A book/)"

Author-title in footnote: "/A book/. 2020."

Here, it's not so much about what information should be rendered, but
rather what should be ommitted.

"Year" would be much less portable. That doesn't mean that "year"
couldn't be legitimate, in the sense of "I really need the year here".
But that raises another question: How will citeproc-el know where this
year is coming from, and how the year has to be formatted? That will
have to be hardcoded in the processor as styles contain no information
about this.
----

Bruce


Bruce

Re: citations: org-cite vs org-ref 3.0

[Max Nikulin]

On 28/03/2022 20:16, Bruce D'Arcus wrote:
> On Mon, Mar 28, 2022 at 8:37 AM Max Nikulin wrote:
>>
>> John, in another message (Sun, 27 Mar 2022 13:00:40 -0400)
>> https://list.orgmode.org/m24k3jnq0k.fsf@andrew.cmu.edu you clearly
>> stated a technical limitation that is a real reason why org-cite is not
>> an option for you and for some other users: performance has not been
>> optimized for large BibTeX databases. It is deserved to be mentioned.
> 
> FWIW, Ihor posted a patch related to this a week or so ago.

I am optimistic concerning that patch since a couple of users confirmed 
improvement, but it is up to John to decide if it is acceptable in 
comparison to org-ref. I am unsure concerning startup time.

>> You even have managed to convince me that, besides adding missing style
>> names, some existing ones should be adjusted. noauthor/bare for citeyear
>> example makes for me much more sense ...
> 
> This does need some attention, but there are wrinkles here.
> 
> Citeyear is specific to author-date styles, while noauthor is intended
> to be more general.

Anyway citation style is rather specific for a particular CSL style. I 
tried some styles: 
https://github.com/citation-style-language/styles/blob/master/ieee.csl 
https://github.com/citation-style-language/styles/blob/master/american-physics-society.csl 
nature.csl science.csl and for all these styles even "author" is 
meaningless since they are numeric styles.

So it is not more general. Switching CSL style means necessity to update 
styles in each citations (unless it is possible to specify global or 
per-cite mapping).

It seems modifiers are set of boolean flags (positive "year" or negative 
"suppress-author") in citeproc.el, set of values in natbib, and a kind 
of hierarchy in org-cite. From my point of view, set of constrains 
(flags) is the most general variant in this list.

> I think it's probably a good idea to add "year" to the latex processors too.

I agree. Negations are more confusing when an author needs just year.

> "Year" would be much less portable. That doesn't mean that "year"
> couldn't be legitimate, in the sense of "I really need the year here".

I would say that if a citation style has no year than a warning should 
be spat, but year should not appear since it is against the style. 
Unsure if it is useful to have special command or style "query" that can 
insert e.g. year, journal, or any other field disregarding style to 
ensure that the text and the bibliography reference have not diverged, 
but it is rather for descriptive text than for a citation.

> But that raises another question: How will citeproc-el know where this
> year is coming from, and how the year has to be formatted? That will
> have to be hardcoded in the processor as styles contain no information
> about this.

A fallback style (for the particular citation, for the whole document or 
in global user preferences) might be specified for formatters missed in 
the main style.

I have tried the following:

obt.bib:
@ARTICLE{schawlow1958iao,
   author = {Schawlow, Arthur Leonard and Townes, Charles Hard},
   title = {Infrared and Optical Masers},
   journal = {Phys. Rev.},
   year = {1958},
   volume = {112},
   pages = {1940--1949},
   month = {Dec},
   doi = {10.1103/PhysRev.112.1940},
   issue = {6},
   publisher = {American Physical Society},
}

obt.org:
#+bibliography: obt.bib
# #+cite_export: csl
# #+cite_export: csl 
/usr/share/citation-style-language/styles/chicago-author-date.csl
# #+cite_export: csl 
/usr/share/citation-style-language/styles/american-physics-society.csl
- Default: [cite:@schawlow1958iao].
- text: [cite/text:@schawlow1958iao].
- noauthor: [cite/noauthor:@schawlow1958iao].
- year/bare: [cite/year/bare:@schawlow1958iao].

#+print_bibliography:

I am familiar with bst language used by BibTeX and I am surprised that 
initials instead of full names are not enforced by CSL styles. Emphasis 
and bold markers may appear in plain text export. Behavior of styles is 
not uniform in respect to adding (unbreakable?) space before citation.

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Tue, Mar 29, 2022 at 11:23 AM Max Nikulin <manikulin@gmail.com> wrote:
...

> >> You even have managed to convince me that, besides adding missing style
> >> names, some existing ones should be adjusted. noauthor/bare for citeyear
> >> example makes for me much more sense ...
> >
> > This does need some attention, but there are wrinkles here.
> >
> > Citeyear is specific to author-date styles, while noauthor is intended
> > to be more general.
>
> Anyway citation style is rather specific for a particular CSL style. I
> tried some styles:
> https://github.com/citation-style-language/styles/blob/master/ieee.csl
> https://github.com/citation-style-language/styles/blob/master/american-physics-society.csl
> nature.csl science.csl and for all these styles even "author" is
> meaningless since they are numeric styles.

Yes. I think it's more relevant in author-date to note styles. I
believe biblatex has a command relevant here, but Denis knows biblatex
better than I.

> So it is not more general. Switching CSL style means necessity to update
> styles in each citations (unless it is possible to specify global or
> per-cite mapping).

Not really. Arguably the most important style is "text", which applies
to any output style; author-date, note-based, numeric.

When you start getting into some of the others, the range of styles a
given style may apply to shrinks.

But you might say author-date styles are pretty dependent on such
local citation modification. If those are output to a style that has
no such notions (like a numeric one), then a processor can just ignore
it.

> It seems modifiers are set of boolean flags (positive "year" or negative
> "suppress-author") in citeproc.el, set of values in natbib, and a kind
> of hierarchy in org-cite. From my point of view, set of constrains
> (flags) is the most general variant in this list.

I think that's right, and is how it's represented in a GUI app like
Zotero. But that's not so convenient in a plain text format.

But it's a good way to explain the differences.

> > I think it's probably a good idea to add "year" to the latex processors too.
>
> I agree. Negations are more confusing when an author needs just year.

We might as well do that then, along with bibentry/fullcite.

...

> I am familiar with bst language used by BibTeX and I am surprised that
> initials instead of full names are not enforced by CSL styles.

I'm not following here. Certainly one can specify initialization rules
in a CSL style.

WDYM by "enforced"?

> Emphasis and bold markers may appear in plain text export. Behavior of styles is
> not uniform in respect to adding (unbreakable?) space before citation.

Sorry; not following here again. Isn't the space before a citation
determined by the user?

Bruce

Re: citations: org-cite vs org-ref 3.0

[Denis Maier]

Am 29.03.2022 um 18:14 schrieb Bruce D'Arcus:
> On Tue, Mar 29, 2022 at 11:23 AM Max Nikulin <manikulin@gmail.com> wrote:
> ...
> 
>>>> You even have managed to convince me that, besides adding missing style
>>>> names, some existing ones should be adjusted. noauthor/bare for citeyear
>>>> example makes for me much more sense ...
>>>
>>> This does need some attention, but there are wrinkles here.
>>>
>>> Citeyear is specific to author-date styles, while noauthor is intended
>>> to be more general.
>>
>> Anyway citation style is rather specific for a particular CSL style. I
>> tried some styles:
>> https://github.com/citation-style-language/styles/blob/master/ieee.csl
>> https://github.com/citation-style-language/styles/blob/master/american-physics-society.csl
>> nature.csl science.csl and for all these styles even "author" is
>> meaningless since they are numeric styles.
> 
> Yes. I think it's more relevant in author-date to note styles. I
> believe biblatex has a command relevant here, but Denis knows biblatex
> better than I.

I'm not sure I understand the question here. What command should be in 
biblatex? There's \citeauthor if that's what you've meant.

> 
>> So it is not more general. Switching CSL style means necessity to update
>> styles in each citations (unless it is possible to specify global or
>> per-cite mapping).
> 
> Not really. Arguably the most important style is "text", which applies
> to any output style; author-date, note-based, numeric.
> 
> When you start getting into some of the others, the range of styles a
> given style may apply to shrinks.
> 
> But you might say author-date styles are pretty dependent on such
> local citation modification. If those are output to a style that has
> no such notions (like a numeric one), then a processor can just ignore
> it.

Just to add to this: When Bruce and I have worked on that list of styles 
we found that portability can only be ensured when using high-level 
commands (such as biblatex's autocite), but once you start using 
low-level commands like citeyear etc. you really lose that portability 
to a certain degree.

> 
>> It seems modifiers are set of boolean flags (positive "year" or negative
>> "suppress-author") in citeproc.el, set of values in natbib, and a kind
>> of hierarchy in org-cite. From my point of view, set of constrains
>> (flags) is the most general variant in this list.
> 
> I think that's right, and is how it's represented in a GUI app like
> Zotero. But that's not so convenient in a plain text format.
> 
> But it's a good way to explain the differences.
> 
>>> I think it's probably a good idea to add "year" to the latex processors too.
>>
>> I agree. Negations are more confusing when an author needs just year.

Well, negations have the advantage of being more portable. Say you have 
this:

Doe argues X and Y [cite/noauthor:@doe].

It's perfectly clear what this should mean in a author-year, 
author-title or note-based styles, i.e., print the citation without the 
author element. (That's obviously a simplification since some citations 
might not have an author element, but let's just go with it for the moment.)

In a numeric style you can just ignore the noauthor modifier and fall 
back to the default numeric citation.

Now, consider this instead:

Doe argues X and Y [cite/year:@doe].

This might work in author-year styles, but not in author-title, not in 
note-based styles, and not in numeric styles.

Considering the problem that some citations don't have an author element 
I even considered using style names like


[cite/head:@doe]
[cite/tail:@doe]
or
[cite/car:@doe]
[cite/cdr:@doe]
or
[cite/first:@doe]
[cite/rest:@doe]

But that obviously a bit esoteric.

Denis

Re: citations: org-cite vs org-ref 3.0

[John Kitchin]

Max Nikulin <manikulin@gmail.com> writes:

> On 28/03/2022 20:16, Bruce D'Arcus wrote:
>> On Mon, Mar 28, 2022 at 8:37 AM Max Nikulin wrote:
>>>
>>> John, in another message (Sun, 27 Mar 2022 13:00:40 -0400)
>>> https://list.orgmode.org/m24k3jnq0k.fsf@andrew.cmu.edu you clearly
>>> stated a technical limitation that is a real reason why org-cite is not
>>> an option for you and for some other users: performance has not been
>>> optimized for large BibTeX databases. It is deserved to be mentioned.
>> 
>> FWIW, Ihor posted a patch related to this a week or so ago.
>
> I am optimistic concerning that patch since a couple of users confirmed 
> improvement, but it is up to John to decide if it is acceptable in 
> comparison to org-ref. I am unsure concerning startup time.

I think this will be a problem that is processor specific. It is
important that org have a reasonable performance here, but I don't think
it was a goal to have the fastest bibtex/json/etc. parser available,
just a reasonable default that works out of the box. That is, it doesn't
take too long to read the database for insertion, and fontification is
not a performance breaker, on say a moderate sized database and
org-file. Getting high performance from large databases and large files
with lots of citations (say a dissertation or big review article) takes
some work.




-- 
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu
Pronouns: he/him/his

Re: citations: org-cite vs org-ref 3.0

[Max Nikulin]

On 29/03/2022 23:14, Bruce D'Arcus wrote:
> On Tue, Mar 29, 2022 at 11:23 AM Max Nikulin wrote:
>> So it is not more general. Switching CSL style means necessity to update
>> styles in each citations (unless it is possible to specify global or
>> per-cite mapping).
> 
> Not really. Arguably the most important style is "text", which applies
> to any output style; author-date, note-based, numeric.
> 
> When you start getting into some of the others, the range of styles a
> given style may apply to shrinks.

I am from the world of in-text numeric styles. I always considered them 
as almost footnote style since the only difference is brackets vs. 
superscript. Author-date style looks quite distinct from such point of view.

>> I am familiar with bst language used by BibTeX and I am surprised that
>> initials instead of full names are not enforced by CSL styles.
> 
> I'm not following here. Certainly one can specify initialization rules
> in a CSL style.
> 
> WDYM by "enforced"?

My complain was wrong (besides it is unrelated to my statement that it 
is impossible to just specify another style, be ready to edit citations).

Some journals omit even article titles to get more compact bibliography. 
I do not remember whether I have seen papers where names are not 
shortened to initials. Any description of BST BibTeX language explains 
how to use format.names$ function, so I was surprised when I got the 
following. However after your question I have checked that achicago.bst 
formats entry with full names as well.

#+cite_export: csl 
/usr/share/citation-style-language/styles/chicago-author-date.csl

--- >8 ---
• Default (Schawlow and Townes 1958).

Schawlow, Arthur Leonard, and Charles Hard Townes. 1958. “Infrared and
Optical Masers.” /Phys. Rev./ 112 (December). American Physical
Society:1940–49. <https://doi.org/10.1103/PhysRev.112.1940>.
--- 8< ----

>> Emphasis and bold markers may appear in plain text export. Behavior of styles is
>> not uniform in respect to adding (unbreakable?) space before citation.
> 
> Sorry; not following here again. Isn't the space before a citation
> determined by the user?

I was lucky enough to pick a couple of styles having different behavior. 
Notice additional unbreakable space before "[1]" in the second example. 
I have checked a couple of IEEE papers and they have spaces before 
citations, so to switch from IEEE to APS style it is necessary to remove 
spaces before citations.

Examples are generated using text unicode export.

#+cite_export: csl /usr/share/citation-style-language/styles/ieee.csl
- Default: [cite:@schawlow1958iao].

--- >8 ---
• Default: [1].

[1] A. L. Schawlow and C. H. Townes, [“Infrared and optical masers,]”
/Phys. rev./, vol. 112, pp. 1940–1949, Dec. 1958.


[“Infrared and optical masers,]
<https://doi.org/10.1103/PhysRev.112.1940>
--- 8< ---


#+cite_export: csl 
/usr/share/citation-style-language/styles/american-physics-society.csl

--- >8 ---
• Default:  [1].

[1] A. L. Schawlow and C. H. Townes, Phys. Rev. *112*, 1940 (1958).
--- 8< ---

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Thu, Mar 31, 2022 at 11:27 AM Max Nikulin <manikulin@gmail.com> wrote:

....

> >> Emphasis and bold markers may appear in plain text export. Behavior of styles is
> >> not uniform in respect to adding (unbreakable?) space before citation.
> >
> > Sorry; not following here again. Isn't the space before a citation
> > determined by the user?
>
> I was lucky enough to pick a couple of styles having different behavior.
> Notice additional unbreakable space before "[1]" in the second example.
> I have checked a couple of IEEE papers and they have spaces before
> citations, so to switch from IEEE to APS style it is necessary to remove
> spaces before citations.

The latter style includes a non-breaking space in the citation definition ...

https://github.com/citation-style-language/styles/blob/21e2177295be5b98ef49b00dd4b8cc7e68d2143d/american-physics-society.csl#L96

... which strikes me as odd, though would explain what you see.

The former has no space.

Bruce

org-cite, CSL styles and space before citation

[Max Nikulin]

On 01/04/2022 00:27, Bruce D'Arcus wrote:
> On Thu, Mar 31, 2022 at 11:27 AM Max Nikulin wrote:
> 
>>>> Emphasis and bold markers may appear in plain text export. Behavior of styles is
>>>> not uniform in respect to adding (unbreakable?) space before citation.
>>>
>>> Sorry; not following here again. Isn't the space before a citation
>>> determined by the user?
>>
>> I was lucky enough to pick a couple of styles having different behavior.
>> Notice additional unbreakable space before "[1]" in the second example.
>> I have checked a couple of IEEE papers and they have spaces before
>> citations, so to switch from IEEE to APS style it is necessary to remove
>> spaces before citations.
> 
> The latter style includes a non-breaking space in the citation definition ...
> 
> https://github.com/citation-style-language/styles/blob/21e2177295be5b98ef49b00dd4b8cc7e68d2143d/american-physics-society.csl#L96
> 
> ... which strikes me as odd, though would explain what you see.

I grepped the directory with CSL styles for "prefix" and tried 
annales.csl. It removes regular spaces before citations (if they exist) 
but preserves non-breakable one.

Since the project has enormous amount of styles, it is hard to ensure 
that they have uniform quality and behavior e.g. in respect to 
responsibility for a space before citations. A test sample and result of 
rendering for it with a check list for style authors may help, but they 
will evolve in time. It is unlikely that accidental contributors of 
styles will manage to maintain compliance to current edition of tests. 
However it would be great for various tools dealing with styles.

P.S. After all, it is necessary to change style when e.g. a journal 
declined a paper. It may mean that the quality of the paper is not high 
enough. Adjusting the text for another style is a burden that makes it 
harder to just send it to another journal and thus reduces number of low 
quality papers. And such efforts are negligible in comparison with 
attempts to mane the content better. So some oddities with particular 
styles are not a serious issue.

Re: citations: org-cite vs org-ref 3.0

[Bruce D'Arcus]

On Mon, Mar 21, 2022 at 10:06 AM John Kitchin <jkitchin@andrew.cmu.edu> wrote:
>
> Bruce and I looked into this UI approach in https://github.com/jkitchin/org-ref-cite/issues/9. Bruce and I discussed and worked on this for almost two weeks. There are 70 comments in this issue.
>
> There are opportunities now to annotate completion targets, which you can see in the link above. The annotations are not selectable though during completion, and this implementation was not too fast as I recall.

FWIW, an alternative I was playing with is something like this, which
makes use of the new oc-biblatex styles defcustom:

(defcustom style-select-latex-commands nil
  "Whether to use latex commands for style selection."
  :group 'style
  :type '(boolean))

(defun style-latex-alist (&optional swap)
  "Return org-cite-biblatex-styles as alist.
By default, each car is the latex command, and the cdr the
org-cite style with variant. With SWAP, they are reversed."
  (let ((raw-styles org-cite-biblatex-styles))
    (mapcar
     (lambda (s)
       (let* ((style (elt s 0))
              (variant (elt s 1))
              (command (elt s 2))
              (cstyle (concat style (when variant "/") variant)))
         (if swap
             (cons cstyle command)
         (cons command cstyle))))
     raw-styles)))

(defun style-select ()
  "Select oc style."
  (let* ((latex-commands style-select-latex-commands)
         (styles
         (if latex-commands (style-latex-alist)
           (style-latex-alist t)))
         (choice
          (completing-read
           (if latex-commands "Biblatex command: "
             "Style: ") styles)))
    (cdr
     (if style-select-latex-commands (assoc choice (style-latex-alist))
       (rassoc choice (style-latex-alist))))))

> You probably should also augment the tooltips like this because you have to be able to tell what a citation format is in the future too, not just at insert time, e.g. suppose you are reading work of a collaborator. It was hard to write, and ambiguous in ways, e.g. what is the export backend you want? The annotations should differ for LaTeX (where you want to see the latex command) vs HTML (where you probably want to see the formatted CSL cite)...

I was thinking it'd be enough to have a tooltip preview of the
citation, and allow the actual preview to be configurable.

Bruce

> We did not surmount these challenges at the time. Maybe others will succeed in this another day.
>
> John
>
> -----------------------------------
> Professor John Kitchin (he/him/his)
> Doherty Hall A207F
> Department of Chemical Engineering
> Carnegie Mellon University
> Pittsburgh, PA 15213
> 412-268-7803
> @johnkitchin
> http://kitchingroup.cheme.cmu.edu
>
>
>
> On Mon, Mar 21, 2022 at 8:42 AM Bruce D'Arcus <bdarcus@gmail.com> wrote:
>>
>> On Mon, Mar 21, 2022 at 8:23 AM John Kitchin <jkitchin@andrew.cmu.edu> wrote:
>>
>> >> A package could be created, say `org-cite-literal-biblatex' which is just a copy
>> >> of `oc-biblatex.el' with a different default `org-cite-biblatex-styles' and
>> >> `org-cite-biblatex-style-shortcuts' (or just sets those variables in
>> >> `org-cite-biblatex'). As far as I can tell this would provide exactly the
>> >> functionality you say org-cite can’t provide but org-ref does.
>> >
>> >
>> > I wrote this package you suggest in org-ref-cite. In discussions during that development, it was clear the preference was on the more abstracted, and uniform syntax across backends cite commands in org-cite, and not this kind of variant. Of course one can do this. It is not that org-cite can't provide it, it is that it doesn't at this time.
>>
>> Just for some broader context on this particular issue.
>>
>> The advantage of the org-cite style/variant design reflected in the
>> included export processors (natbib, biblatex, csl) is that the same
>> styles will mostly generate the same final output.
>>
>> But that portability will only work with those styles and variants.
>>
>> With the new org-cite-biblatex-styles defcustom, however, one can
>> augment or completely replace all those. But if you care about that
>> portability, you'd want to be aware of this, and think about it.
>>
>> So per Timothy's point, you actually don't even need a new processor
>> for biblatex if you want to include all the extensive list of biblatex
>> commands.
>>
>> Natbib AFAIK is already fully covered.
>>
>> There's another POV on this though:
>>
>> If one doesn't like to see the org-cite styles, because of familiarity
>> with LaTeX commands etc., I would argue that can be addressed in the
>> style part of an insert processor and/or in an activate processor.
>> E.g. I would argue this is a UI issue; not fundamentally about the
>> styles names.
>>
>> Bruce