Citation syntax: Underscore MUST(?) be allowed in cite keys?

Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Vaidheeswaran C]

[-- Attachment #1: Type: text/plain, Size: 517 bytes --]

1. Visit http://www.amazon.in/How-Read-Book-Touchstone-book/dp/0671212095
2. Add it to Zotero.
3. Export My Library to BibTeX format.
4. The attached file -- My Library 1.bib -- is what I get.

When I import the above .bib file to JabRef GUI, the keys that are
reported are \cite{adler_how_1972},
\cite{center_for_history_and_new_media_zotero_????}.

I am a novice comes to citation.  I had a vague impression by that `_'
is not allowed in cite keys.  If we go this way, then the above
workflow will be a nightmare.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: My Library 1.bib --]
[-- Type: text/x-bibtex; name="My Library 1.bib", Size: 1690 bytes --]


@misc{center_for_history_and_new_media_zotero_????,
	title = {Zotero {Quick} {Start} {Guide}},
	url = {http://zotero.org/support/quick_start_guide},
	author = {{Center for History and New Media}},
	annote = {Welcome to Zotero!View the Quick Start Guide to learn how to begin collecting, managing, citing, and sharing your research sources.Thanks for installing Zotero.}
}

@book{adler_how_1972,
	address = {New York},
	edition = {Revised edition edition},
	title = {How to {Read} a {Book}},
	isbn = {9780671212094},
	abstract = {With half a million copies in print, How to Read a Book is the best and most successful guide to reading comprehension for the general reader, completely rewritten and updated with new material.Originally published in 1940, this book is a rare phenomenon, a living classic that introduces and elucidates the various levels of reading and how to achieve them—from elementary reading, through systematic skimming and inspectional reading, to speed reading. Readers will learn when and how to “judge a book by its cover,” and also how to X-ray it, read critically, and extract the author’s message from the text.Also included is instruction in the different techniques that work best for reading particular genres, such as practical books, imaginative literature, plays, poetry, history, science and mathematics, philosophy and social science works.Finally, the authors offer a recommended reading list and supply reading tests you can use measure your own progress in reading skills, comprehension, and speed.},
	language = {English},
	publisher = {Simon \& Schuster},
	author = {Adler, Mortimer J. and Doren, Charles Van},
	month = aug,
	year = {1972}
}

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Rasmus]

Hi,

Vaidheeswaran C <vaidheeswaran.chinnaraju@gmail.com> writes:

> 1. Visit http://www.amazon.in/How-Read-Book-Touchstone-book/dp/0671212095
> 2. Add it to Zotero.
> 3. Export My Library to BibTeX format.
> 4. The attached file -- My Library 1.bib -- is what I get.
>
> When I import the above .bib file to JabRef GUI, the keys that are
> reported are \cite{adler_how_1972},
> \cite{center_for_history_and_new_media_zotero_????}.
>
> I am a novice comes to citation.  I had a vague impression by that `_'
> is not allowed in cite keys.  If we go this way, then the above
> workflow will be a nightmare.

Underscore is fine.  Here's the regexp that bibtex.el uses for keys:

           \\([][[:alnum:].:;?!`'/*@+|()<>&_^$-]+\\)

See bibtex-entry-head.

Jabref is pretty solid (though I'm not at fan of how handles encoding) and
will not produce wrong keys.

Cheers,
Rasmus

-- 
One thing that is clear: it's all down hill from here 

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Vaidheeswaran C]

On Thursday 05 March 2015 12:00 AM, Rasmus wrote:
> Hi,
> 
> Vaidheeswaran C <vaidheeswaran.chinnaraju@gmail.com> writes:
> 
>> 1. Visit http://www.amazon.in/How-Read-Book-Touchstone-book/dp/0671212095
>> 2. Add it to Zotero.
>> 3. Export My Library to BibTeX format.
>> 4. The attached file -- My Library 1.bib -- is what I get.
>>
>> When I import the above .bib file to JabRef GUI, the keys that are
>> reported are \cite{adler_how_1972},
>> \cite{center_for_history_and_new_media_zotero_????}.
>>
>> I am a novice comes to citation.  I had a vague impression by that `_'
>> is not allowed in cite keys.  If we go this way, then the above
>> workflow will be a nightmare.
> 
> Underscore is fine.  Here's the regexp that bibtex.el uses for keys:
> 
>            \\([][[:alnum:].:;?!`'/*@+|()<>&_^$-]+\\)
> 
> See bibtex-entry-head.

I am complaining about how org-element.el behaves.

This [cite:@adler_how_1972] becomes this:

(citation
 (:key
  #("adler" 0 5
    (keymap
     (keymap
      (follow-link . mouse-face)
      (mouse-3 . org-find-file-at-mouse)
      (mouse-2 . org-open-at-mouse))
     face org-link mouse-face highlight htmlize-link
     (:uri "cite:@adler_how_1972")
     fontified t))
  :parentheticalp nil :begin 192 :post-blank 0 :end 214 :suffix
  (#4=(underline
       (:begin 204 :end 209 :contents-begin 205 :contents-end 208
:post-blank 0 :parent #3#)
       #("how" 0 3
	 (:parent #4#)))
      #("1972" 0 4
	(:parent #3#)))
  :parent #5#))

> Jabref is pretty solid (though I'm not at fan of how handles encoding) and
> will not produce wrong keys.
> 
> Cheers,
> Rasmus
> 

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Rasmus]

Vaidheeswaran C <vaidheeswaran.chinnaraju@gmail.com> writes:

> I am complaining about how org-element.el behaves.
>
> This [cite:@adler_how_1972] becomes this:

Oh, you are right.  _ is only allowed as the first character, as you
probably saw.  See http://permalink.gmane.org/gmane.emacs.orgmode/95609.
I would tend to agree that this is problematic, but _ is itself
problematic since it's the subscription character...  From
http://permalink.gmane.org/gmane.emacs.orgmode/95631 I gather that key
support is modeled after Pandoc, but AFAIK Pandoc *does* support _ as part
of the key cf. http://pandoc.org/README.html#citations.

So yeah, I agree with you.

–Rasmus

-- 
ツ

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Vaidheeswaran C]

On Thursday 05 March 2015 12:31 AM, Rasmus wrote:
> Vaidheeswaran C <vaidheeswaran.chinnaraju@gmail.com> writes:
> 
>> I am complaining about how org-element.el behaves.
>>
>> This [cite:@adler_how_1972] becomes this:
> 
> Oh, you are right.  _ is only allowed as the first character, as you
> probably saw.  See http://permalink.gmane.org/gmane.emacs.orgmode/95609.
> I would tend to agree that this is problematic, but _ is itself
> problematic since it's the subscription character...  From
> http://permalink.gmane.org/gmane.emacs.orgmode/95631 I gather that key
> support is modeled after Pandoc, but AFAIK Pandoc *does* support _ as part
> of the key cf. http://pandoc.org/README.html#citations.

My refrain on Citation syntax thread has always been:

    The 3rd-Party Citation Tools are *very much* part of the what is
    being proposed.  Has someone built a prototype that _actually_
    interfaces with these citation tools to understand what the
    practical constraints are?

The workflow question I raised is very much in tune with the above
refrain.

> So yeah, I agree with you.
> 
> –Rasmus
> 

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Christian Moe]

Also, Zotero items are typically identified by [library-ID]_[item-key]
hashes with an underscore separator.

Christian

Rasmus writes:

> Vaidheeswaran C <vaidheeswaran.chinnaraju@gmail.com> writes:
>
>> I am complaining about how org-element.el behaves.
>>
>> This [cite:@adler_how_1972] becomes this:
>
> Oh, you are right.  _ is only allowed as the first character, as you
> probably saw.  See http://permalink.gmane.org/gmane.emacs.orgmode/95609.
> I would tend to agree that this is problematic, but _ is itself
> problematic since it's the subscription character...  From
> http://permalink.gmane.org/gmane.emacs.orgmode/95631 I gather that key
> support is modeled after Pandoc, but AFAIK Pandoc *does* support _ as part
> of the key cf. http://pandoc.org/README.html#citations.
>
> So yeah, I agree with you.
>
> –Rasmus

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Richard Lawrence]

Hi Rasmus and all,

Rasmus <rasmus@gmx.us> writes:

> Vaidheeswaran C <vaidheeswaran.chinnaraju@gmail.com> writes:
>
>> I am complaining about how org-element.el behaves.

> Oh, you are right.  _ is only allowed as the first character, as you
> probably saw.  See http://permalink.gmane.org/gmane.emacs.orgmode/95609.
> I would tend to agree that this is problematic, but _ is itself
> problematic since it's the subscription character...  From
> http://permalink.gmane.org/gmane.emacs.orgmode/95631 I gather that key
> support is modeled after Pandoc, but AFAIK Pandoc *does* support _ as part
> of the key cf. http://pandoc.org/README.html#citations.

Yep, that's my bad, sorry.

Pandoc says: "The citation key must begin with a letter or _, and may
contain alphanumerics, _, and internal punctuation characters
(:.#$%&-+?<>~/)."

I dropped the second underscore when I was writing the grammar.  Thanks,
Vaisheeswaran, for noticing.  Nicolas, IMO we should update the parser
to allow underscores in keys (including at the final character, I
guess).

Best,
Richard

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Nicolas Goaziou]

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> I dropped the second underscore when I was writing the grammar.  Thanks,
> Vaisheeswaran, for noticing.  Nicolas, IMO we should update the parser
> to allow underscores in keys (including at the final character, I
> guess).

Done in ab7ff4034e8cd67ae5b5e2cfddfee87082228801.


Regards,

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Rasmus]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Richard Lawrence <richard.lawrence@berkeley.edu> writes:
>
>> I dropped the second underscore when I was writing the grammar.  Thanks,
>> Vaisheeswaran, for noticing.  Nicolas, IMO we should update the parser
>> to allow underscores in keys (including at the final character, I
>> guess).
>
> Done in ab7ff4034e8cd67ae5b5e2cfddfee87082228801.

This is also not captured: [cite:@Röntgen].  Why limit
org-element--citation-key-re to a-z?

—Rasmus

-- 
Together we'll stand, divided we'll fall

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Nicolas Goaziou]

Rasmus <rasmus@gmx.us> writes:

> This is also not captured: [cite:@Röntgen].  Why limit
> org-element--citation-key-re to a-z?

Done in 8c941008e0aaf958bdc6ae2cc7dfcbe3fd967b92.

While we're at it, as you noticed already, bare citation keys take
precedence over subscript. So, basically, one can never add subscript to
a word starting with "@".

At the bare minimum, we should introduce an entity to generate "@"
(e.g. "\At") so one can always write

  \At_2

WDYT?

Regards,

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Rasmus]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> While we're at it, as you noticed already, bare citation keys take
> precedence over subscript. So, basically, one can never add subscript to
> a word starting with "@".
>
> At the bare minimum, we should introduce an entity to generate "@"
> (e.g. "\At") so one can always write
>
>   \At_2
>
> WDYT?

My gut feeling tells me that scripts are "more important" than citations.
[To me this is certainty not true, I always use subscripts inside latex
fragments/envs].  But I can't think of anything better than your
suggestion and I think it's acceptable.

—Rasmus

-- 
With monopolies the cake is a lie!

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Eric S Fraga]

On Friday,  6 Mar 2015 at 18:55, Rasmus wrote:
> My gut feeling tells me that scripts are "more important" than citations.

My gut agrees with yours on this.

However, the precedence for citations is only for words starting with @
so this should not pose a problem generally.  I cannot remember ever
using @ in normal prose...
-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.1, Org release_8.3beta-843-ga5f1a3.dirty

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Vaidheeswaran C]

On Friday 06 March 2015 04:19 PM, Nicolas Goaziou wrote:
> Richard Lawrence <richard.lawrence@berkeley.edu> writes:
>
>> I dropped the second underscore when I was writing the grammar.  Thanks,
>> Vaisheeswaran, for noticing.  Nicolas, IMO we should update the parser
>> to allow underscores in keys (including at the final character, I
>> guess).
>
> Done in ab7ff4034e8cd67ae5b5e2cfddfee87082228801.

The following combination works when passed through the LaTeX/PDF
exporter.  It doesn't work when the cite syntax is switched to the new
one.

\cite{center_for_history_and_new_media_zotero_????}

@misc{center_for_history_and_new_media_zotero_????,
	title = {Zotero Quick Start Guide},
	url = {http://zotero.org/support/quick_start_guide},
	author = {{Center for History and New Media}},
	annote = {Welcome to {Zotero!View} the Quick Start Guide to learn how
to begin collecting, managing, citing, and sharing your research
{sources.Thanks} for installing Zotero.}
}

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Richard Lawrence]

Hi Vaidheeswaran,

Vaidheeswaran C <vaidheeswaran.chinnaraju@gmail.com> writes:

> The following combination works when passed through the LaTeX/PDF
> exporter.  It doesn't work when the cite syntax is switched to the new
> one.
>
> \cite{center_for_history_and_new_media_zotero_????}

Is that a realistic example of a key?  What program is generating it?
JabRef?

I am not too worried about this case.  I am guessing that the four "?"s
at the end are inserted because the generating program wasn't sure how
to encode some characters.  I think the right thing to do in edge cases
like this is instruct the user that the key is invalid (at least from
Org's perspective) and they should change it.  If key-ending punctuation
turns out to be common, I would revise this opinion, but at the moment I
don't see the need.
  
I do not think we should allow punctuation at the end of keys, because
that will make it too hard to write in-text citations at the end of a
clause, like: "For more, see @Doe99." or "Can we replicate the results
of @Doe99?" or "The following cases have been documented in @Doe99: ..."

Best,
Richard

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Vaidheeswaran C]

On Friday 06 March 2015 11:39 PM, Richard Lawrence wrote:
> Hi Vaidheeswaran,
> 
> Vaidheeswaran C <vaidheeswaran.chinnaraju@gmail.com> writes:
> 
>> The following combination works when passed through the LaTeX/PDF
>> exporter.  It doesn't work when the cite syntax is switched to the new
>> one.
>>
>> \cite{center_for_history_and_new_media_zotero_????}
> 
> Is that a realistic example of a key?

Yes.

> What program is generating it?  JabRef?

It is generated by Zotero.

> I am not too worried about this case.  

Have you tried it with \cite{} and \bibliography{} combination and
seen it works or not.  My argument relies on how existing tool works.


> I am guessing that the four "?"s at the end are inserted because the
> generating program wasn't sure how to encode some characters.

I see that you haven't exporter Zotero libraries to .bib files.  My
guess is the 4 question marks are for "missing" (or "irrelevant") yyyy
digits.

> If key-ending punctuation turns out to be common, I would revise
> this opinion, but at the moment I don't see the need.

I am not imagining things.  I am pointing out how existing tools
behave.

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Richard Lawrence]

Vaidheeswaran C <vaidheeswaran.chinnaraju@gmail.com> writes:

> On Friday 06 March 2015 11:39 PM, Richard Lawrence wrote:
>> Hi Vaidheeswaran,
>> 
>> Vaidheeswaran C <vaidheeswaran.chinnaraju@gmail.com> writes:
>> 
>>> The following combination works when passed through the LaTeX/PDF
>>> exporter.  It doesn't work when the cite syntax is switched to the new
>>> one.
>>>
>>> \cite{center_for_history_and_new_media_zotero_????}
>> 
>> Is that a realistic example of a key?
>
> Yes.
> It is generated by Zotero.

OK.

> Have you tried it with \cite{} and \bibliography{} combination and
> seen it works or not.  My argument relies on how existing tool works.

I don't understand what your argument is, exactly.  So what if Zotero
sometimes produces keys like this?  So what if a LaTeX document will
compile with such keys?  Is it your position that that means Org keys
must allow punctuation at the end?

> I see that you haven't exporter Zotero libraries to .bib files.  My
> guess is the 4 question marks are for "missing" (or "irrelevant") yyyy
> digits.

If such keys represent references with missing data, the document
produced by even `successful' compilation will be incomplete in some
way.  

Like I said, this seems like an edge case, and I don't see that it is
necessarily Org's responsibility to accommodate the keys produced by
Zotero in such edge cases.  And there is a significant benefit to *not*
accommodating such keys: namely, you can use in-text citations at the
end of a sentence.

Again, if there were reason to think that keys which end with
punctuation are common even in the normal case where the data is
complete and correct, that would be reason to re-think the syntax of Org
citation keys.  But Org's citation syntax can't be expected to handle
every tool's behavior in every edge case.
 
Best,
Richard

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Vaidheeswaran C]

On Saturday 07 March 2015 10:39 PM, Richard Lawrence wrote:
> So what if Zotero
> sometimes produces keys like this?  So what if a LaTeX document will
> compile with such keys?  Is it your position that that means Org keys
> must allow punctuation at the end?

Yes.  Nicolas is implementing the parser.  Go or no-go would be his
call though.

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Stefan Nobis]

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> Like I said, this seems like an edge case, and I don't see that it
> is necessarily Org's responsibility to accommodate the keys produced
> by Zotero in such edge cases. And there is a significant benefit to
> *not* accommodating such keys: namely, you can use in-text citations
> at the end of a sentence.

+1

-- 
Until the next mail...,
Stefan.

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Thomas S. Dye]

Aloha all,

Vaidheeswaran C <vaidheeswaran.chinnaraju@gmail.com> writes:

>> If key-ending punctuation turns out to be common, I would revise
>> this opinion, but at the moment I don't see the need.
>
> I am not imagining things.  I am pointing out how existing tools
> behave.

Am I right that key-ending punctuation is a potential problem for the
shorthand part of the syntax and not for the full [cite: ...] syntax?

I'm asking because I haven't fully grasped uses for the shorthand.  What
is the use case?

Won't the Org mode user configure a tool like reftex or ebib (or
something else) to insert citations?

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Nicolas Goaziou]

Hello,

tsd@tsdye.com (Thomas S. Dye) writes:

> Am I right that key-ending punctuation is a potential problem for the
> shorthand part of the syntax and not for the full [cite: ...] syntax?

Exactly. If key-ending punctuation is required, we might have to drop
shortcuts (i.e. [@key] and @key).

At this point, I think this is too early to make a decision anyway.

> I'm asking because I haven't fully grasped uses for the shorthand.  What
> is the use case?

More readable, I guess.

Regards,

-- 
Nicolas Goaziou

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Rasmus]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

>> I'm asking because I haven't fully grasped uses for the shorthand.  What
>> is the use case?
>
> More readable, I guess.

I agree.  In time, "org-reftex" would insert @key if no notes are
requested at the time of insertion.

—Rasmus

-- 
Dung makes an excellent fertilizer

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Thomas S. Dye]

Rasmus <rasmus@gmx.us> writes:

> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
>>> I'm asking because I haven't fully grasped uses for the shorthand.  What
>>> is the use case?
>>
>> More readable, I guess.
>
> I agree.  In time, "org-reftex" would insert @key if no notes are
> requested at the time of insertion.

I think the OP has a valid point.  After we teach org-reftex to insert
@key if no notes are requested, are we going to convince all key
generating software to prohibit keys that end in punctuation?

As I currently understand the problem, that seems like a tall order to
me.

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Rasmus]

tsd@tsdye.com (Thomas S. Dye) writes:

> Rasmus <rasmus@gmx.us> writes:
>
>> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>>
>>>> I'm asking because I haven't fully grasped uses for the shorthand.  What
>>>> is the use case?
>>>
>>> More readable, I guess.
>>
>> I agree.  In time, "org-reftex" would insert @key if no notes are
>> requested at the time of insertion.
>
> I think the OP has a valid point.  After we teach org-reftex to insert
> @key if no notes are requested, are we going to convince all key
> generating software to prohibit keys that end in punctuation?

So just to get it straight: are you advocating for only allowing
[cite:@key]-like constructs to allow punctuation at the end of words?

Perhaps it's a can of worms, but you can also match keys against a
"punctuation at end of word"-regexp and use the fuller cite command then.
I'm not too happy with having the regexps used in [cite:@·] and @· diverge
too much, though...

So /given support for end-of-word punctuation/, we'd either have two
abandon a "single" org-element--citation-key-re (yes that's not entirely
correct) or give up short citations.

> As I currently understand the problem, that seems like a tall order to
> me.

It's also a tall order to support end of word punctuation cf. above.

I think another important question is how easy is it to configure the
citation manager in question not to insert punctuation marks at the end?

—Rasmus

-- 
This message is brought to you by the department of redundant departments

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Thomas S. Dye]

Aloha Rasmus,

Rasmus <rasmus@gmx.us> writes:

> tsd@tsdye.com (Thomas S. Dye) writes:
>
>> Rasmus <rasmus@gmx.us> writes:
>>
>>> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>>>
>>>>> I'm asking because I haven't fully grasped uses for the shorthand.  What
>>>>> is the use case?
>>>>
>>>> More readable, I guess.
>>>
>>> I agree.  In time, "org-reftex" would insert @key if no notes are
>>> requested at the time of insertion.
>>
>> I think the OP has a valid point.  After we teach org-reftex to insert
>> @key if no notes are requested, are we going to convince all key
>> generating software to prohibit keys that end in punctuation?
>
> So just to get it straight: are you advocating for only allowing
> [cite:@key]-like constructs to allow punctuation at the end of words?
>
> Perhaps it's a can of worms, but you can also match keys against a
> "punctuation at end of word"-regexp and use the fuller cite command then.
> I'm not too happy with having the regexps used in [cite:@·] and @· diverge
> too much, though...
>
> So /given support for end-of-word punctuation/, we'd either have two
> abandon a "single" org-element--citation-key-re (yes that's not entirely
> correct) or give up short citations.
>
>> As I currently understand the problem, that seems like a tall order to
>> me.
>
> It's also a tall order to support end of word punctuation cf. above.
>
> I think another important question is how easy is it to configure the
> citation manager in question not to insert punctuation marks at the end?

I'm not an advocate at this point.  I'm just trying to be clear about
a choice that apparently needs to be made.

As I see it, the choice boils down to the relative benefit of citation
shortcuts vs. the limitation of requiring authors to configure the
citation manager so it doesn't produce a key ending in punctuation (or
your solution that uses different regexps for full citations and
shortcuts).

Nicolas guessed that the benefit of citation shortcuts is that they are
more "readable" than a full citation, and you agree with his guess.  The
shortcuts are certainly shorter, so in this sense are more readable.
However, having two different representations of the same thing, a
shortcut and a full citation, means that, for the author (and the
software) recognition is more complex and thus, less readable.  For this
reason, IMHO the readability benefit is not particularly strong.

Richard and Stefan both see keys ending in punctuation marks as corner
cases, so the burden imposed on the author to configure the citation
manager is relatively infrequent.  They know more about this than I do,
so I'm heartened by this information.  However, in the event the
citation manager has to be configured, the author faces a potentially
daunting task.  The algorithm for automatic key generation in
bibtex-mode is summarized in 18 steps, including two near the end that
allow arbitrary input!  I strongly believe Org mode shouldn't send an
author here, unless the corresponding benefits are great.

I'm not capable of forming an opinion about your solution that uses
different regexps.

At this point I think the benefit of citation shortcuts is relatively
modest and the limitation of requiring authors to ensure keys don't end
in punctuation potentially onerous.  On balance, I think strong
consideration should be given to the option of not using shortcuts.

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Rasmus]

tsd@tsdye.com (Thomas S. Dye) writes:

Bibtex.el is not that hard to configure.  I think I have something like
this to configure FIRSTAUTHOR-YY (without the hyphen):

  (setq bibtex-autokey-titlewords 0
        bibtex-autokey-titlewords-stretch 0
        bibtex-autokey-titleword-length 0
        bibtex-autokey-edit-before-use nil)

But this only works on new keys and I wouldn't want "old" .bib file not
working.

> At this point I think the benefit of citation shortcuts is relatively
> modest and the limitation of requiring authors to ensure keys don't end
> in punctuation potentially onerous.  On balance, I think strong
> consideration should be given to the option of not using shortcuts.

But Org is also a format.  I have for instance written limited Org support
for texworks.

For people who do not have the luxury of using Emacs easy syntax matters.
Personally, I think the benefit of short citations is large.

I think allowing different characters if the least bad solution.  Inline
footnotes are also limited compared to footnote-definitions, so perhaps it
is not that bad...

—Rasmus

-- 
Bang bang

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Richard Lawrence]

Hi Tom and all,

tsd@tsdye.com (Thomas S. Dye) writes:

> As I see it, the choice boils down to the relative benefit of citation
> shortcuts vs. the limitation of requiring authors to configure the
> citation manager so it doesn't produce a key ending in punctuation (or
> your solution that uses different regexps for full citations and
> shortcuts).

Yes, that's my understanding of the situation.  I would just add that it
may not even be *possible* to configure how some citation managers
generate keys.  So if there are citation managers that put punctuation
at the end of keys in `normal' cases, that's something serious to
consider.

Another variable to keep in mind here is that we don't have to
`bless'/support every citation manager.  If a citation manager puts
punctuation at the end of keys, and doesn't allow configuring that
behavior or makes it difficult, that's a reason not to bless it, in my
opinion.  But my opinion probably shouldn't count for much on this
point, because I don't use a citation manager myself (I use org-bibtex),
and I write my own keys.

What citation managers are people on this list actually using?  It would
be very helpful to get an idea of what is actually needed before we make
any changes to the syntax of keys.

> Richard and Stefan both see keys ending in punctuation marks as corner
> cases, so the burden imposed on the author to configure the citation
> manager is relatively infrequent.  

Yes, that is my sense.  At any rate, I would like to see clear examples
that are not corner cases before we throw out the shortcut syntax,
because I personally think it is useful and readable.  A large number of
my own citations could be handled by just the shortcut syntax, I think,
so I'd be sad to see it go away without good reason.

> At this point I think the benefit of citation shortcuts is relatively
> modest and the limitation of requiring authors to ensure keys don't end
> in punctuation potentially onerous.  On balance, I think strong
> consideration should be given to the option of not using shortcuts.

I don't disagree, but I think there is an empirical question that needs
to be answered here: within the keys people actually use, how many do
not conform to the syntax?  Of those that don't, do they represent
`normal' cases or not?

Best,
Richard

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Thomas S. Dye]

Aloha Richard,

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> Hi Tom and all,
>
> tsd@tsdye.com (Thomas S. Dye) writes:
>
>> As I see it, the choice boils down to the relative benefit of citation
>> shortcuts vs. the limitation of requiring authors to configure the
>> citation manager so it doesn't produce a key ending in punctuation (or
>> your solution that uses different regexps for full citations and
>> shortcuts).
>
> Yes, that's my understanding of the situation.  I would just add that it
> may not even be *possible* to configure how some citation managers
> generate keys.  So if there are citation managers that put punctuation
> at the end of keys in `normal' cases, that's something serious to
> consider.
>
> Another variable to keep in mind here is that we don't have to
> `bless'/support every citation manager.  If a citation manager puts
> punctuation at the end of keys, and doesn't allow configuring that
> behavior or makes it difficult, that's a reason not to bless it, in my
> opinion.  But my opinion probably shouldn't count for much on this
> point, because I don't use a citation manager myself (I use org-bibtex),
> and I write my own keys.

Oh my.  This is a lot to keep in your head as a bibliographic database
grows.  The one I've created with my colleagues over the last two
decades has more than 5,000 entries.

> What citation managers are people on this list actually using?  It would
> be very helpful to get an idea of what is actually needed before we make
> any changes to the syntax of keys.
>
>> Richard and Stefan both see keys ending in punctuation marks as corner
>> cases, so the burden imposed on the author to configure the citation
>> manager is relatively infrequent.  
>
> Yes, that is my sense.  At any rate, I would like to see clear examples
> that are not corner cases before we throw out the shortcut syntax,
> because I personally think it is useful and readable.  A large number of
> my own citations could be handled by just the shortcut syntax, I think,
> so I'd be sad to see it go away without good reason.
>
>> At this point I think the benefit of citation shortcuts is relatively
>> modest and the limitation of requiring authors to ensure keys don't end
>> in punctuation potentially onerous.  On balance, I think strong
>> consideration should be given to the option of not using shortcuts.
>
> I don't disagree, but I think there is an empirical question that needs
> to be answered here: within the keys people actually use, how many do
> not conform to the syntax?  Of those that don't, do they represent
> `normal' cases or not?

A good friend of mine is a military historian who writes books
describing how the Army habitually plans to fight the last war over
again, then has to adapt hurriedly when the next war turns out to be
different.  It strikes me that basing core features of the citation
syntax on the software users happen to be using today is a bit like
this--at some point the design of the system will prove unprepared for
new developments.

I think Vaidheeswaran C's example of a citation scraped off the internet
with Zotero should carry a lot of weight.  This kind of thing is bound
to happen more and more as authors increasingly harvest citation
information on-line (my generation typically looks on this with horror,
but we'll be swept aside).

I kind of like Rasmus' idea to make the citation insertion routines
aware of punctuation and use a full citation where a shortcut would
introduce ambiguities.

Of course, an old-schooler like me will eventually complain about
wanting a variable =org-citation-always-full= that I can set non-nil.

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Stefan Nobis]

tsd@tsdye.com (Thomas S. Dye) writes:

> It strikes me that basing core features of the citation syntax on
> the software users happen to be using today is a bit like this--at
> some point the design of the system will prove unprepared for new
> developments.

I don't think this is a big problem. We are talking about citation
managers, that already have to interface to different word processors.
They have to be configurable. Also, I don't think it makes any sense
for developers of citation engines to generate keys with random signs.

On the other hand, if we want to be really liberal in terms of keys,
we must allow whitespace, arbitrary unicode values etc. In this case,
its a hard problem to delimit the key because any character we use as
delimiter (like <>, ``, "", etc.) may be used inside the key.

So some constraints for the key are always necessary. I don't know
every citation manager out there but I'm quite confident that all of
them are quite configurable and that keys containing whitespace or
ending in punctuation characters are really corner cases that could
and should be handled in the citation manager.

IMHO keys with lots of ??? in them are a sign of a data problem.
Therefore the author should solve the root cause.

-- 
Until the next mail...,
Stefan.

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Vaidheeswaran C]

On Monday 09 March 2015 02:27 PM, Stefan Nobis wrote:

> IMHO keys with lots of ??? in them are a sign of a data problem.
> Therefore the author should solve the root cause.

@misc{center_for_history_and_new_media_zotero_????,
	title = {Zotero {Quick} {Start} {Guide}},
	url = {http://zotero.org/support/quick_start_guide},
	author = {{Center for History and New Media}},
	annote = {Welcome to Zotero!View the Quick Start Guide to learn how to
begin collecting, managing, citing, and sharing your research
sources.Thanks for installing Zotero.}
}

Not in the specific case that I cited.  The Bib entry is a pointer to
a website.  Insisting on a YEAR field for a website address seems a
bit absurd to me.  The problem is not the "data problem" but the
"problem" could be in the "the inherent nature of the object that is
cited".

My argument is: "Stock LaTeX exporter does it, so Org should also
follow suit".  This argument is difficult to beat without resorting to
vague generalities and hand waves.

When I discuss a problem with a specific example, I would like the
participants to pay attention to the DETAILS of the specific example
and make their comments relevant to the example being discussed.

If you had shared how I can configure Zotero to leave out the question
marks that would have been the most helpful comment from your side.

Anyways....

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Stefan Nobis]

Vaidheeswaran C <vaidheeswaran.chinnaraju@gmail.com> writes:

> On Monday 09 March 2015 02:27 PM, Stefan Nobis wrote:

>> IMHO keys with lots of ??? in them are a sign of a data problem.
>> Therefore the author should solve the root cause.

> Not in the specific case that I cited. The Bib entry is a pointer to
> a website.

I would say, even a website needs a date (in this case: date last
seen). :)

IMHO here you are mixing two different things: We already talked about
direct support for Zotero as a backend, CSL etc. Therefore exporting
to bibtex is not a requirement. If you use bibtex as the primary
source and Zotero only as a tool to fetch references from the web,
then its easy to edit the key in bibtex.

Maybe the best way is to add a new export module to Zotero for even
better org integration and handle correct keys in this module?

IMHO it's the job of the citation manager to generate sane keys, not
the job of org to accept arbitrary keys.

> If you had shared how I can configure Zotero to leave out the
> question marks that would have been the most helpful comment from
> your side.

I'm not a Zotero expert, I even don't use it. But with a quick look at
Google I found this:

  http://curiousjason.com/zoterotobibtex.html

(in the Firefox profile there is a configuration file
zotero/translators/BibTeX.js that needs to be edited; the above source
is from 2010 - maybe today there is a GUI to edit this setting).

-- 
Until the next mail...,
Stefan.

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Vaidheeswaran C]

On Monday 09 March 2015 03:19 PM, Stefan Nobis wrote:
> Vaidheeswaran C <vaidheeswaran.chinnaraju@gmail.com> writes:
> 
>> On Monday 09 March 2015 02:27 PM, Stefan Nobis wrote:
> 
>>> IMHO keys with lots of ??? in them are a sign of a data problem.
>>> Therefore the author should solve the root cause.
> 
>> Not in the specific case that I cited. The Bib entry is a pointer to
>> a website.
> 
> I would say, even a website needs a date (in this case: date last
> seen). :)
> 
> IMHO here you are mixing two different things: We already talked about
> direct support for Zotero as a backend, CSL etc. Therefore exporting
> to bibtex is not a requirement. 

When you say requirement, it is a good idea to say "requirement to
which parties or requirement in which context".

I am running Debian Squeeze.  The pandoc that gets bundled with this
debian version doesn't support citeproc stuff.  IIRC, to upgrade to
recent pandoc I would have to update one of the core haskell
libraries.  I am reluctant to upgrade my debian.  Believe me, if I had
quick access to pandoc-hs, there would be a ox-pandoc-citeproc already
in the tree by now.  (No I am not joking.)

The spririt of Free Software (and definitely Emacs) is to ACCOMODATE
MULTIPLE SIMULTANEOUS implementations of same functionality.  One good
example is a todo-mode.el (which ships by default on Emacs).  I would
venture to say, obsoleting a package is taboo.

By saying "bibtex is not a requirement", I hope you don't mean to
imply that bibtex users are any less blessed or less holy or that
their needs wouldn't be "readily" catered to.

(My intention is NOT to fight with you.  I am merely emphasizing that
pandoc/haskell is a recent development and there are some practical
problems associated with making THIS CUTTING EDGE package a
REQUIREMENT that other ORG USERS MUST depend on.  You can call me a
lazy bum!)

> Maybe the best way is to add a new export module to Zotero for even
> better org integration and handle correct keys in this module?
> 
> IMHO it's the job of the citation manager to generate sane keys, not
> the job of org to accept arbitrary keys.
> 
>> If you had shared how I can configure Zotero to leave out the
>> question marks that would have been the most helpful comment from
>> your side.
> 
> I'm not a Zotero expert, I even don't use it. But with a quick look at
> Google I found this:
> 
>   http://curiousjason.com/zoterotobibtex.html
> 
> (in the Firefox profile there is a configuration file
> zotero/translators/BibTeX.js that needs to be edited; the above source
> is from 2010 - maybe today there is a GUI to edit this setting).

Thanks.

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Rasmus]

Vaidheeswaran C <vaidheeswaran.chinnaraju@gmail.com> writes:

> Believe me, if I had quick access to pandoc-hs, there would be a
> ox-pandoc-citeproc already in the tree by now.

My opinion would be to perhaps not put too much energy into anything
Haskell.  It's extremely annoying when you don't have the luxury of
prebuild packages.  E.g. on Arch there's no prebuild pandoc and ghc is
800MB!  Extrapolating from git-annex getting the right dependencies (down
to version-number, sometimes) seem like a mess with cabal.

—Rasmus

-- 
C is for Cookie

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Stefan Nobis]

Vaidheeswaran C <vaidheeswaran.chinnaraju@gmail.com> writes:

> By saying "bibtex is not a requirement",

I said "exporting to bibtex". You talked about Zotero but showed a
bibtex entry. Therefore exporting from Zotero to bibtex may not
be a requirement, there may be a direct interface to Zotero,
eventually.

We are at the start of the development, we are currently CUTTING EDGE!
Therefore tool support (hopefully including guides to configure
citation managers) will evolve in the future.

My main point is: You found a single example that *may* be a problem
with the current syntax. But there are multiple software packages
involved (Zotero, Zotero to bibtex exporter, org-mode, etc.). The
citation syntax will never be able to handle all of the possible
problems in a longer chain of tools. Sometimes its better to fix a
problem at the start of in the middle of this chain.

IMHO it's a good idea to constrain the syntax for keys a little bit
(no whitespace, no arbitrary unicode character, no punctuation at the
end etc.). If in some cases the default configuration of the involved
tools will create invalid keys, than the configuration should be fixed
instead of dropping constraints for the key syntax.

> I hope you don't mean to imply that bibtex users are any less
> blessed or less holy or that their needs wouldn't be "readily"
> catered to.

I am a bibtex user. :)

I want to say, that it is impossible to accomodate for all the
citations managers with any possible configurations of them. Sometimes
we have to state: This case is not supported, please adjust your
configuration.

-- 
Until the next mail...,
Stefan.

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Richard Lawrence]

Hi Tom and all,

"Thomas S. Dye" <tsd@tsdye.com> writes:

> Richard Lawrence <richard.lawrence@berkeley.edu> writes:
>
>>  But my opinion probably shouldn't count for much on this
>> point, because I don't use a citation manager myself (I use org-bibtex),
>> and I write my own keys.
>
> Oh my.  This is a lot to keep in your head as a bibliographic database
> grows.  The one I've created with my colleagues over the last two
> decades has more than 5,000 entries.

Yes, I realize this method probably isn't going to scale well in the
long run, but it's working for me for now.  The vast majority of my keys
are just the author's last name plus the year.  I just write a key like
that when I add something to my reading list, and fix the rare duplicate
cases as necessary.

(Just to explain why it makes sense to me to do it this way: I used to
work in a psychology lab, where I had to write a lot of little programs
to do data analysis.  The worst part of that job was always dealing with
malformed, missing, and otherwise-corrupt data captured by someone else.
Since then, my attitude has always been that it's much easier to correct
that data at the point where it's captured than figure out what to do
with it somewhere further down the processing pipeline, after the reason
*why* it is malformed has been lost.  In the context of this discussion,
that translates to: a work doesn't get a key in my reading list unless I
have complete citation information for it.  Sometimes I put items on my
reading list that I don't have citation data for yet, but I don't do
org-bibtex-create-in-current-entry on that item until I have the
citation data and can assign it a key.)

>> I don't disagree, but I think there is an empirical question that needs
>> to be answered here: within the keys people actually use, how many do
>> not conform to the syntax?  Of those that don't, do they represent
>> `normal' cases or not?
>
> A good friend of mine is a military historian who writes books
> describing how the Army habitually plans to fight the last war over
> again, then has to adapt hurriedly when the next war turns out to be
> different.  It strikes me that basing core features of the citation
> syntax on the software users happen to be using today is a bit like
> this--at some point the design of the system will prove unprepared for
> new developments.
>
> I think Vaidheeswaran C's example of a citation scraped off the internet
> with Zotero should carry a lot of weight.  This kind of thing is bound
> to happen more and more as authors increasingly harvest citation
> information on-line (my generation typically looks on this with horror,
> but we'll be swept aside).

That's a fair point.  

> I kind of like Rasmus' idea to make the citation insertion routines
> aware of punctuation and use a full citation where a shortcut would
> introduce ambiguities.

That would work for me.  Like Rasmus, I don't particularly like the idea
of letting the syntax of keys vary in the shortcut case and the full
citation case, but if the only difference is whether or not they can end
in clause-ending punctuation, maybe this is the least-bad option.

Another option would be to allow clause-ending punctuation in all keys,
but introduce some kind of optional syntax to express `this key ends
here'.  This could be used to disambiguate the key from any following
punctuation in those cases where this is needed.  Perhaps something like
'{}', since even LaTeX won't allow '}' at the end of a key, or maybe
just '\'.  Thus, in these examples:

  This is an in-text citation, as was shown by @Doe99{}. The next sentence.
  This is an in-text citation, as was shown by @Doe99\. The next sentence.

the key would be parsed as `Doe99', but in this example:

  This is an in-text citation, where @Doe???? is mentioned mid-sentence.

the key would be parsed as `Doe????'.

What do you think?  

Best,
Richard

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Thomas S. Dye]

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> Hi Tom and all,
>
> "Thomas S. Dye" <tsd@tsdye.com> writes:
>
>> Richard Lawrence <richard.lawrence@berkeley.edu> writes:
>>
>>>  But my opinion probably shouldn't count for much on this
>>> point, because I don't use a citation manager myself (I use org-bibtex),
>>> and I write my own keys.
>>
>> Oh my.  This is a lot to keep in your head as a bibliographic database
>> grows.  The one I've created with my colleagues over the last two
>> decades has more than 5,000 entries.
>
> Yes, I realize this method probably isn't going to scale well in the
> long run, but it's working for me for now.  The vast majority of my keys
> are just the author's last name plus the year.  I just write a key like
> that when I add something to my reading list, and fix the rare duplicate
> cases as necessary.
>
> (Just to explain why it makes sense to me to do it this way: I used to
> work in a psychology lab, where I had to write a lot of little programs
> to do data analysis.  The worst part of that job was always dealing with
> malformed, missing, and otherwise-corrupt data captured by someone else.
> Since then, my attitude has always been that it's much easier to correct
> that data at the point where it's captured than figure out what to do
> with it somewhere further down the processing pipeline, after the reason
> *why* it is malformed has been lost.  In the context of this discussion,
> that translates to: a work doesn't get a key in my reading list unless I
> have complete citation information for it.  Sometimes I put items on my
> reading list that I don't have citation data for yet, but I don't do
> org-bibtex-create-in-current-entry on that item until I have the
> citation data and can assign it a key.)
>

We've had a couple dozen contributors to our bibliography over the
years.  Initially, we assigned keys by hand but we found this led to
very many duplicate entries.  Generating keys has helped a lot in this
situation because most duplicates are caught when we merge the
project-specific database, which has already been edited, with the
central one.

>>> I don't disagree, but I think there is an empirical question that needs
>>> to be answered here: within the keys people actually use, how many do
>>> not conform to the syntax?  Of those that don't, do they represent
>>> `normal' cases or not?
>>
>> A good friend of mine is a military historian who writes books
>> describing how the Army habitually plans to fight the last war over
>> again, then has to adapt hurriedly when the next war turns out to be
>> different.  It strikes me that basing core features of the citation
>> syntax on the software users happen to be using today is a bit like
>> this--at some point the design of the system will prove unprepared for
>> new developments.
>>
>> I think Vaidheeswaran C's example of a citation scraped off the internet
>> with Zotero should carry a lot of weight.  This kind of thing is bound
>> to happen more and more as authors increasingly harvest citation
>> information on-line (my generation typically looks on this with horror,
>> but we'll be swept aside).
>
> That's a fair point.  
>
>> I kind of like Rasmus' idea to make the citation insertion routines
>> aware of punctuation and use a full citation where a shortcut would
>> introduce ambiguities.
>
> That would work for me.  Like Rasmus, I don't particularly like the idea
> of letting the syntax of keys vary in the shortcut case and the full
> citation case, but if the only difference is whether or not they can end
> in clause-ending punctuation, maybe this is the least-bad option.
>
> Another option would be to allow clause-ending punctuation in all keys,
> but introduce some kind of optional syntax to express `this key ends
> here'.  This could be used to disambiguate the key from any following
> punctuation in those cases where this is needed.  Perhaps something like
> '{}', since even LaTeX won't allow '}' at the end of a key, or maybe
> just '\'.  Thus, in these examples:
>
>   This is an in-text citation, as was shown by @Doe99{}. The next sentence.
>   This is an in-text citation, as was shown by @Doe99\. The next sentence.
>
> the key would be parsed as `Doe99', but in this example:
>
>   This is an in-text citation, where @Doe???? is mentioned mid-sentence.
>
> the key would be parsed as `Doe????'.
>
> What do you think?  

The {} terminator is used elsewhere in Org mode, so it might be the
least bad option in this instance.

All the best,
Tom

-- 
T.S. Dye & Colleagues, Archaeologists
735 Bishop St, Suite 315, Honolulu, HI 96813
Tel: 808-529-0866, Fax: 808-529-0884
http://www.tsdye.com

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Eric S Fraga]

On Monday,  9 Mar 2015 at 09:05, Richard Lawrence wrote:

[...]

> Another option would be to allow clause-ending punctuation in all keys,
> but introduce some kind of optional syntax to express `this key ends

No, please no!  I would say that the majority of my citations are at the
end of sentences...  I would end up having to use this additional syntax
frequently.  I'd rather use syntax to disambiguate the less frequent
cases and keys with punctuation are those (for me, at least).

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.1, Org release_8.3beta-843-ga5f1a3.dirty

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Thomas S. Dye]

Aloha Eric,

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> On Monday,  9 Mar 2015 at 09:05, Richard Lawrence wrote:
>
> [...]
>
>> Another option would be to allow clause-ending punctuation in all keys,
>> but introduce some kind of optional syntax to express `this key ends
>
> No, please no!  I would say that the majority of my citations are at the
> end of sentences...  I would end up having to use this additional syntax
> frequently.  I'd rather use syntax to disambiguate the less frequent
> cases and keys with punctuation are those (for me, at least).

The problem is limited to the shortcut citations and doesn't affect the
[cite: ...] form, which can be expected to work without modification
wherever it is placed IIUC.

Do you have a use for the shortcuts?  I doubt I'd use them.

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Eric S Fraga]

On Monday,  9 Mar 2015 at 07:49, Thomas S. Dye wrote:

[...]

> Do you have a use for the shortcuts?  I doubt I'd use them.

Wouldn't the shortcut be the most attractive to use generally unless you
have need for the extra capability of the full [cite:] syntax?

The vast majority of my citations, e.g. in a paper I am writing right
now in org, are of the form [[cite:blah-etal-2010a]] and it would be
much easier to type @blah-etal-2010a.  I seldom, if ever, have pre or
post text in my citations.

Or have I misunderstood something in this *very* long thread?  (which I
have been following as a lurker so far... :)

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.1, Org release_8.3beta-843-ga5f1a3.dirty

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Thomas S. Dye]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> On Monday,  9 Mar 2015 at 07:49, Thomas S. Dye wrote:
>
> [...]
>
>> Do you have a use for the shortcuts?  I doubt I'd use them.
>
> Wouldn't the shortcut be the most attractive to use generally unless you
> have need for the extra capability of the full [cite:] syntax?
>
> The vast majority of my citations, e.g. in a paper I am writing right
> now in org, are of the form [[cite:blah-etal-2010a]] and it would be
> much easier to type @blah-etal-2010a.  I seldom, if ever, have pre or
> post text in my citations.
>
> Or have I misunderstood something in this *very* long thread?  (which I
> have been following as a lurker so far... :)

No, you have it right and clearly have a use for shortcuts.  If you want
to type shortcut citations yourself, then the choice is either to accept
some kind of terminator, e.g. {}, or a restriction that citation keys
not end in punctuation characters.

It's been years since I've actually typed in a citation.  Reftex and
Ebib both do a flawless job and I rely on them completely now.
Shortcuts aren't useful in this work flow.

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Eric S Fraga]

On Monday,  9 Mar 2015 at 08:44, Thomas S. Dye wrote:

[...]

> No, you have it right and clearly have a use for shortcuts.  If you want
> to type shortcut citations yourself, then the choice is either to accept
> some kind of terminator, e.g. {}, or a restriction that citation keys
> not end in punctuation characters.

Well, I think it's important to consider user interface issues.  Simple
things should be simple to do and common activities likewise.  Only if
there is a conflict should we need to choose.  Given that most citations
are unlikely to end in punctuation, the easy to use syntax should be for
the majority cases and should not need to cater for punctuation.

I would hate to have to type {} at the end of every simple citation,
especially as I suffer from RSI and those characters are particularly
annoying...

> It's been years since I've actually typed in a citation.  Reftex and
> Ebib both do a flawless job and I rely on them completely now.
> Shortcuts aren't useful in this work flow.

True but this doesn't obviate the need to consider non-automated input.

In any case, none of the decisions need affect me should I dislike the
solution actually implemented in the end.  That's the great thing about
org and plain text!

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.1, Org release_8.3beta-843-ga5f1a3.dirty

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Richard Lawrence]

Hi Tom and all,

"Thomas S. Dye" <tsd@tsdye.com> writes:

> The problem is limited to the shortcut citations and doesn't affect the
> [cite: ...] form, which can be expected to work without modification
> wherever it is placed IIUC.

Actually, it occurs to me now that this might even affect the [cite:
...] form, depending on how you use the prefix and suffix text.

Suppose you often write citations like:

  [cite: See @Doe99, and references therein, for more.]

For readability's sake, I would hope this would be parsed by default as:
  - prefix: 'See '
  - key: 'Doe99'
  - suffix: ', and references therein, for more.'

and rendered like:

  See Doe (1999), and references therein, for more.

I can imagine similar cases at least for periods, colons, and
semi-colons (though those would have to be written as entities here), so
maybe we cannot avoid this problem just by changing or removing the
shortcut syntax.

Best,
Richard

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Rasmus]

[-- Attachment #1: Type: text/plain, Size: 1073 bytes --]

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> Suppose you often write citations like:
>
>   [cite: See @Doe99, and references therein, for more.]
>
> [...]  and rendered like:
>
>   See Doe (1999), and references therein, for more.
 
This is slightly OT, but it comes up frequently enough that it's worth
pointing out.  Assuming we were to handle notes as biblatex (which would
be the better thing to do IMO), the output of:

     [cite: See @Doe99, and references therein, for more.]
Aka: \textcite[See][, and references therein, for more.]{Doe99}

is

    Aksn et al. (See 2006, ,and references therein, for more.)

To get "See Doe (1999), and references therein, for more." one would
write:

      See @doe99, and references therein, for more.
Or:   See \textcite{doe99}, and references therein, for more.

The output of [(cite): See @Doe99, and references therein, for more.] is

      (See Doe 1999, ,and references therein, for more)
or:   \parencite[See][,and references therein, for more]{doe99}

—Rasmus

-- 
This space is left intentionally blank

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: test.tex --]
[-- Type: text/x-tex, Size: 513 bytes --]

\documentclass{article}
\usepackage[style=authoryear]{biblatex}
\addbibresource{biblatex-examples.bib}
\begin{document}
\textcite[See][,and references therein, for more.]{aksin}\par
\textcites(pre)(post)[See][,and references therein, for more.]{aksin}[See][,and references therein, for more.]{angenendt}\par
\parencite[See][,and references therein, for more.]{aksin}\par
\parencites(pre)(post)[See][,and references therein, for more.]{aksin}[See][,and references therein, for more.]{angenendt}\par
\end{document}

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Richard Lawrence]

Rasmus <rasmus@gmx.us> writes:

> Richard Lawrence <richard.lawrence@berkeley.edu> writes:
>
>> Suppose you often write citations like:
>>
>>   [cite: See @Doe99, and references therein, for more.]
>>
>> [...]  and rendered like:
>>
>>   See Doe (1999), and references therein, for more.
>  
> This is slightly OT, but it comes up frequently enough that it's worth
> pointing out.  Assuming we were to handle notes as biblatex (which would
> be the better thing to do IMO), the output of:
>
>      [cite: See @Doe99, and references therein, for more.]
> Aka: \textcite[See][, and references therein, for more.]{Doe99}
>
> is
>
>     Aksn et al. (See 2006, , and references therein, for more.)
>                          ^^

Oh dear, you're right.  Where do that initial comma and space come from?
I guess BibLaTeX inserts them automatically?  Does that happen in all
styles?

From what I can see, Pandoc does not implicitly insert punctuation like
this:

  As @Fenner2012a [, cf. sec. 2] showed ...

renders as

  As Fenner (2012, cf. sec. 2) showed ...
  
Can you turn off the automatic addition of commas in BibLaTeX by setting
something in the preamble?  If so, would that be the right solution
here?  It might be easier to remove punctuation on the LaTeX side than
to get other backends to duplicate LaTeX's implicit punctuating
behavior.

Best,
Richard

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Rasmus]

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> Oh dear, you're right.  Where do that initial comma and space come from?
> I guess BibLaTeX inserts them automatically?  Does that happen in all
> styles?

The default value is determined by \postnotedelim.  So assuming spaces not
stripped \renewcommand{\postnotedelim}{} would probably work in most
cases.  More fancily one could check the next space like xspace.  There's
a mighty cool example using higher level functions here:

  http://tex.stackexchange.com/questions/125358/automating-processing-of-trailing-punctuation-if-any-for-hyperlinks

> From what I can see, Pandoc does not implicitly insert punctuation like
> this:
>
>   As @Fenner2012a [, cf. sec. 2] showed ...
>
> renders as
>
>   As Fenner (2012, cf. sec. 2) showed ...

I prefer the biblatex behavior then.

> Can you turn off the automatic addition of commas in BibLaTeX by setting
> something in the preamble?

Preamble or using \AtNextCite

> If so, would that be the right solution here?  It might be easier to
> remove punctuation on the LaTeX side than to get other backends to
> duplicate LaTeX's implicit punctuating behavior.

Perhaps.  Either looks like the famous can of worms.  I guess we should
promote the option above in the manual and that's it.


-- 
However beautiful the theory, you should occasionally look at the evidence

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Richard Lawrence]

Hi Rasmus,

Thanks, your post was very informative.

Rasmus <rasmus@gmx.us> writes:
> ...
>> Can you turn off the automatic addition of commas in BibLaTeX by setting
>> something in the preamble?
>
> Preamble or using \AtNextCite
>
>> If so, would that be the right solution here?  It might be easier to
>> remove punctuation on the LaTeX side than to get other backends to
>> duplicate LaTeX's implicit punctuating behavior.
>
> Perhaps.  Either looks like the famous can of worms.  I guess we should
> promote the option above in the manual and that's it.

I agree.  Let's make sure to mention this in documentation, but I think
a general solution is too hard right now.

Best,
Richard

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Richard Lawrence]

Hi Eric and all,

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> On Monday,  9 Mar 2015 at 09:05, Richard Lawrence wrote:
>
>> Another option would be to allow clause-ending punctuation in all keys,
>> but introduce some kind of optional syntax to express `this key ends
>
> No, please no!  I would say that the majority of my citations are at the
> end of sentences...  I would end up having to use this additional syntax
> frequently.  I'd rather use syntax to disambiguate the less frequent
> cases and keys with punctuation are those (for me, at least).

I feel the same as you do about this, so here's one more thought for the
thread.  How about this alternative?

We keep the existing syntax for keys, which disallows key-ending
punctuation, but we also allow a second style of key, in which curly
braces surround the key name proper, like:

  @{Doe1999}

Then we can conceivably allow any character at all between the braces,
including punctuation (except `}', which is also a restriction on LaTeX
keys).  That means this type of key should be able to accommodate just
about anything the future may bring.

The opening `{' immediately after the `@' will make this type of key
easy for the parser to distinguish from the first type, and the closing
`}' makes the end of the key unambiguous with respect to following
punctuation.

Tools that insert keys from citation managers can check whether a key
respects the brace-less syntax, and offer to insert it using the second
style when it doesn't.  Or they could be configured to always use this
style, if the user prefers.

And anyone who doesn't need to disambiguate keys from punctuation can
use the brace-less style, which (in my opinion) is easier to read and
type.

What do people think?  

Best,
Richard

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Eric S Fraga]

On Monday,  9 Mar 2015 at 17:36, Richard Lawrence wrote:

[...]

> I feel the same as you do about this, so here's one more thought for the
> thread.  How about this alternative?
>
> We keep the existing syntax for keys, which disallows key-ending
> punctuation, but we also allow a second style of key, in which curly
> braces surround the key name proper, like:
>
>   @{Doe1999}

I like this much better.  

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.1, Org release_8.3beta-843-ga5f1a3.dirty

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Nicolas Goaziou]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

>> We keep the existing syntax for keys, which disallows key-ending
>> punctuation, but we also allow a second style of key, in which curly
>> braces surround the key name proper, like:
>>
>>   @{Doe1999}
>
> I like this much better.

I'd rather have a single syntax for keys. Since this one is not much
more intrusive than the previous one, we could as well drop @key in
favor of @{key}.


Regards,

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Rasmus]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> I'd rather have a single syntax for keys.

Me to.

> Since this one is not much more intrusive than the previous one, we
> could as well drop @key in favor of @{key}.

It seems like a moderately dear price to pay for everyone with "normal"
citation keys...  It's better than @key-with-',?.'{}.

How about other citations?  Are we talking the wholesale solution:

    [cite: @{key}]
    [(cite): @{key}]
    [@{key}]

Otherwise, I don't see how we have come closer to one syntax from a user
point of view.  And I don't like the above.

—Rasmus


-- 
Vote for proprietary math!

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Eric S Fraga]

On Tuesday, 10 Mar 2015 at 09:50, Rasmus wrote:
> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>> Since this one is not much more intrusive than the previous one, we
>> could as well drop @key in favor of @{key}.
>
> It seems like a moderately dear price to pay for everyone with "normal"
> citation keys...  It's better than @key-with-',?.'{}.

I agree.  I would rather type @key >90% of the time instead of
@{key}.  For me, the alternative is more than a moderately high price to
pay!

If we don't want a proliferation of alternative syntax, maybe we need to
impose the restriction already suggested of not allowing punctuation at
the end of a key.  Solve the problem upstream... and not have the tail
wag the dog!

But, of course, don't let this tail (me) wag the dog (the rest of you)
should the consensus be that the design is cleaner with @{key}.  I'll
manage!  :)

Thanks,
eric

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.1, Org release_8.3beta-843-ga5f1a3.dirty

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Matt Price]

[-- Attachment #1: Type: text/plain, Size: 1409 bytes --]

On Tue, Mar 10, 2015 at 6:18 AM, Eric S Fraga <e.fraga@ucl.ac.uk> wrote:

> On Tuesday, 10 Mar 2015 at 09:50, Rasmus wrote:
> > Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
> >> Since this one is not much more intrusive than the previous one, we
> >> could as well drop @key in favor of @{key}.
> >
> > It seems like a moderately dear price to pay for everyone with "normal"
> > citation keys...  It's better than @key-with-',?.'{}.
>
> I agree.  I would rather type @key >90% of the time instead of
> @{key}.  For me, the alternative is more than a moderately high price to
> pay!
>
> If we don't want a proliferation of alternative syntax, maybe we need to
> impose the restriction already suggested of not allowing punctuation at
> the end of a key.  Solve the problem upstream... and not have the tail
> wag the dog!
>
> But, of course, don't let this tail (me) wag the dog (the rest of you)
> should the consensus be that the design is cleaner with @{key}.  I'll
> manage!  :)
>
> Thanks,
> eric
>

I also think a simpler key syntax will make htis much nmore usable.
Citations are in general somewhat distracting from the writing process;
every additional keystroke is gong to add to that distraction.  This is
true even if hand-added citations are likely to be the exception rather
than the norm.



>
> --
> : Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.1, Org
> release_8.3beta-843-ga5f1a3.dirty
>
>

[-- Attachment #2: Type: text/html, Size: 2127 bytes --]

Re: Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Richard Lawrence]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> On Tuesday, 10 Mar 2015 at 09:50, Rasmus wrote:
>> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>>> Since this one is not much more intrusive than the previous one, we
>>> could as well drop @key in favor of @{key}.
>>
>> It seems like a moderately dear price to pay for everyone with "normal"
>> citation keys...  It's better than @key-with-',?.'{}.
>
> I agree.  I would rather type @key >90% of the time instead of
> @{key}.  For me, the alternative is more than a moderately high price to
> pay!

I too agree.  If we only allow one syntax, I much prefer to stick with
the original, and deal with the punctuation restriction elsewhere.

As far as I can tell, the only actual example we've seen of a key that
ends in punctuation is the one Vaidheeswaran sent, and that still seems
like an edge case to me: it should be corrected by adding data to the
reference database, not accommodated by Org's key syntax.

I think Tom's worry that we might see more of that kind of thing in the
future is fair, and deserves more thought.  I suggest we stick with the
original syntax for now, and revisit the issue in the future if it
becomes clear that there are lots of non-conforming keys `in the wild'.
In the meantime, tools that automatically insert keys from citation
managers can warn the user if they don't conform to the syntax.

Best,
Richard

Citation syntax: Underscore MUST(?) be allowed in cite keys?

[Vaidheeswaran C]

[-- Attachment #1: Type: text/plain, Size: 517 bytes --]

1. Visit http://www.amazon.in/How-Read-Book-Touchstone-book/dp/0671212095
2. Add it to Zotero.
3. Export My Library to BibTeX format.
4. The attached file -- My Library 1.bib -- is what I get.

When I import the above .bib file to JabRef GUI, the keys that are
reported are \cite{adler_how_1972},
\cite{center_for_history_and_new_media_zotero_????}.

I am a novice comes to citation.  I had a vague impression by that `_'
is not allowed in cite keys.  If we go this way, then the above
workflow will be a nightmare.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: My Library 1.bib --]
[-- Type: text/x-bibtex; name="My Library 1.bib", Size: 1690 bytes --]


@misc{center_for_history_and_new_media_zotero_????,
	title = {Zotero {Quick} {Start} {Guide}},
	url = {http://zotero.org/support/quick_start_guide},
	author = {{Center for History and New Media}},
	annote = {Welcome to Zotero!View the Quick Start Guide to learn how to begin collecting, managing, citing, and sharing your research sources.Thanks for installing Zotero.}
}

@book{adler_how_1972,
	address = {New York},
	edition = {Revised edition edition},
	title = {How to {Read} a {Book}},
	isbn = {9780671212094},
	abstract = {With half a million copies in print, How to Read a Book is the best and most successful guide to reading comprehension for the general reader, completely rewritten and updated with new material.Originally published in 1940, this book is a rare phenomenon, a living classic that introduces and elucidates the various levels of reading and how to achieve them—from elementary reading, through systematic skimming and inspectional reading, to speed reading. Readers will learn when and how to “judge a book by its cover,” and also how to X-ray it, read critically, and extract the author’s message from the text.Also included is instruction in the different techniques that work best for reading particular genres, such as practical books, imaginative literature, plays, poetry, history, science and mathematics, philosophy and social science works.Finally, the authors offer a recommended reading list and supply reading tests you can use measure your own progress in reading skills, comprehension, and speed.},
	language = {English},
	publisher = {Simon \& Schuster},
	author = {Adler, Mortimer J. and Doren, Charles Van},
	month = aug,
	year = {1972}
}