zotero plain, org-odt, and citations in general

zotero plain, org-odt, and citations in general

[Matt Price]

[-- Attachment #1: Type: text/plain, Size: 851 bytes --]

Hi everyone,

I am about to start experimenting with Erik Hetzner's zotero-plain, which
allows the use of zotero as a bibliographic manager in emacs & especially
org-mode (https://bitbucket.org/egh/zotero-plain as well as a couple of
threads on this list, I think).  I'm wonderinghow other people have used
it, and in particular whether there's any way to preserve the zotero markup
across exports to odt especially. That is, I'd like to keep the original
citations embedded in the final odt, so that I can continue to use zotero
in an odt or doc file after it's been exported from org.

This would take me very , very close to a complete org-centric workflow
that still lets me produce documents for consumption by my MS-using
colleagues.  I really hope it's possible to do! If anyone has done it, I
would really love to hear about it.

Thanks,
Matt

[-- Attachment #2: Type: text/html, Size: 966 bytes --]

Re: zotero plain, org-odt, and citations in general

[Jambunathan K]

> Hi everyone,
>
> I am about to start experimenting with Erik Hetzner's zotero-plain,
> which allows the use of zotero as a bibliographic manager in emacs &
> especially org-mode (https://bitbucket.org/egh/zotero-plain as well
> as a couple of threads on this list, I think).

For Erik's benefit:
===================

org-mode has a native ODT exporter implemented by org-odt.el. The
documentation - org-odt.pdf - can be seen *attached* to this mailing
list post: 
http://article.gmane.org/gmane.emacs.orgmode/48468

The following section - sec. no. 1.1.4 Applying Custom Styles - talks
about where the ODT exporter picks the style from.

Notes to self:
==============

1. styles.odt is here.
   - https://bitbucket.org/egh/zotero-plain/src/66353860ebdd/python
   
   Styles seem to have rststyle as a prefix.

2. example.odt is here.
   - https://bitbucket.org/egh/zotero-plain/src/66353860ebdd/example

   Contains a bibiliographic content presumably created by zrst2odt

3. zrst2odt is here.
   - https://bitbucket.org/egh/zotero-plain/src/66353860ebdd/python/bin

4. Elisp code is here.
   - https://bitbucket.org/egh/zotero-plain/src/66353860ebdd/elisp

   1. org-zotero.el

      This permits insertion of zotero reference links *only* and
      opening them with a browser. 

      Specifically I don't see the translator of these links to
      individual backends like latex, html or more importantly ODT.

      #+begin_src emacs-lisp
        (org-add-link-type TYPE &optional FOLLOW EXPORT)
      #+end_src

   2. zotero.el

      This (apparently) permits download of data from firefox and
      importing it within Emacs.

I have an impression that I have is this:

org->rst->odt

I am not able to make up my mind quickly about whether there is support
for translating org-style zotero links to rst-stlye <whatever>.

Notes to Matt Price or other users of zotero + libreoffice
==========================================================

I see a post from you - titled "zoteroOpenOfficeIntegration doesn't open
port?" - in the zotero forums a few hours ago. Here is a link to
libreoffice related posts on zotero's forums.

http://forums.zotero.org/search/?PostBackAction=Search&Keywords=libreoffice&Type=Comments&btnSubmit=Search

So I believe if you or someone could post a OpenOffice document -
created through zotero - which uses zotero-style citation and
references - to this mailing list or this very thread - these could be
used as a primer to export zotero links to ODT. (Similarly for other Org
supported backends).

It would be wonderful if the example uses as many of the zotero fields -
(as in database fields) - in as many usage contexts as possible.

General comments
================

1. Zotero supports many citation stlyes. So Org/Zotero integration could
   choose one citation style that is widely agreeable for one-off
   colloboration needs.

2. I don't know if importing Zotero stylenames verbatim in to org-odt's
   style file would "pollute" it - thereby preventing org-odt from
   getting in to Emacs proper.

   I am not competent to judge what the copyright and license terms of
   the stylenames used by zotero plugin are. I am seeing that zotero
   itself is AGPL so I believe it is Free as in "Free Software".

3. Even if zotero's ODT style cannot be used by the ODT exporter due
   (2), I can cook up Org's own stlyenames for various citation fields
   and allow the user to remap Org* stlyenames to say Zotero* stlynames.

I see Org already has some(?) support for bibtex. Can anyone comment on
bibtex vs zotero? They seem to serve same purpose - citation
management. Can bibtex be used instead of zotero.

I am not a researcher or academic. So I would need help from potential
users for this to move forward.

>   I'm wonderinghow other people have used it, and in particular
> whether there's any way to preserve the zotero markup across exports
> to odt especially. That is, I'd like to keep the original citations
> embedded in the final odt, so that I can continue to use zotero in an
> odt or doc file after it's been exported from org. 
>
> This would take me very , very close to a complete org-centric
> workflow that still lets me produce documents for consumption by my
> MS-using colleagues.  I really hope it's possible to do! If anyone
> has done it, I would really love to hear about it. 
>
> Thanks,
> Matt
>
>

-- 

Re: zotero plain, org-odt, and citations in general

[Jambunathan K]

[-- Attachment #1: Type: text/plain, Size: 6860 bytes --]


>    1. org-zotero.el
>
>       This permits insertion of zotero reference links *only* and
>       opening them with a browser. 
>
>       Specifically I don't see the translator of these links to
>       individual backends like latex, html or more importantly ODT.
>
>       #+begin_src emacs-lisp
>         (org-add-link-type TYPE &optional FOLLOW EXPORT)
>       #+end_src
>
>    2. zotero.el
>
>       This (apparently) permits download of data from firefox and
>       importing it within Emacs.


I think I now have some understanding of what Erik's library does[1]. 

It creates a bibliographic entry (as in Footnotes) using Quick Copy.  It
doesn not create a footnote reference. One can use the Firefox Zotero
and see how Quick Copy works with both RTF and HTML formats[2].

[Question to Erik: Can one not use "Copy from Clipboard" from elisp
instead of going the mozrepl route to create a biblio.text]

Instead of working with JS libraries, I think the right starting point
would be to work with the underlying zotero.sqlite database itself. It
is not difficult to get a key of the selected item in Zotero
pane[3]. Once this is done the underlying zotero.sqlite database could
be queried for the item properties[4].

Fortunately there is already a GUI/Command line tool - Gnotero[5] -for
accessing the sqlite database.

As for exploiting the power of using zotero within libreoffice - which
is to have the same citation presented in different citation styles -
one need to look at how the reference marks are produced within
OpenDocument XML.

Here is a typical reference mark for an item with local zotero URL[6]:
zotero://report/items/579241_E7232HE2/html/report.html

LIBRARY=>579241
KEY=>E7232HE2


#+begin_src nxml
  <text:p text:style-name="Footnote">
    <text:reference-mark-start text:name="ZOTERO_ITEM {&quot;citationID&quot;:&quot;2lhcdnkups&quot;,&quot;citationItems&quot;:[{&quot;uri&quot;:[&quot;http://zotero.org/groups/51961/items/E7232HE2&quot;]}]} RNDB7mi3QiueI"/>Crew, “Alltagsgeschichte.”
    <text:reference-mark-end text:name="ZOTERO_ITEM {&quot;citationID&quot;:&quot;2lhcdnkups&quot;,&quot;citationItems&quot;:[{&quot;uri&quot;:[&quot;http://zotero.org/groups/51961/items/E7232HE2&quot;]}]} RNDB7mi3QiueI"/>
  </text:p>
#+end_src nxml

One can see the following components:
1. ZOTERO_ITEM 
2. citationID and citationItems
   - The citationItem has the following Zotero URI:
     http://zotero.org/groups/51961/items/E7232HE2
     
     GROUP=>51961
     KEY=>E7232HE2

     Note that the keys for the entry are the same on both the local
     zotero store and the global zotero URI.
3. RNDB7mi3QiueI
   - This is a cookie entered by Zotero which maps the citation entry to
     an entry in the zotero database. Note that it starts with RND.

     This is what is called as a "FieldCode" in zotero parlance.

     Note: Have you seen the "Remove Field Codes" in Zotero menu within
     ODT document. Once you remove the field codes the document becomes
     a simple ODT document and one would no longer be able to move
     between different citation styles.

     In essence, "Field Code" is the MAGIC WAND that zotero uses.

From my little research, I can say with confidence that global zotero
URIs and the above mentioned reference marks are *very* recent
developments.

Exercising the MAGIC WAND within ODT exporter requires diving in to
zotero server inner workings - which is non-trivial as Erik notes - but
would also be futile (as in chasing a shifting sand)

I believe the best course would be to do convention/static footnoting
with the simplest of Chicago styles using a choice of entry types like
Article, Journal, Book or a Website.

Proposal for Org+Zotero integration
===================================

An org document like this

--8<---------------cut here---------------start------------->8---
#+begin_src org
* Headline
  This is a footnote reference[1]

Footnotes:
[1] zotero://report/items/579241_E7232HE2/html/report.html
#+end_src org
--8<---------------cut here---------------end--------------->8---

will be transformed to this on ODT export

--8<---------------cut here---------------start------------->8---
#+begin_odt
* Headline
  This is a footnote reference[1]     

Bibliography:

1. David F. Crew, “Alltagsgeschichte: A New Social History ‘From
below’?,” Central European History 22, no. 3/4 (1989): 394-407.
#+end_odt
--8<---------------cut here---------------end--------------->8---

Note that the footnote link is an org stlye link captured with
org-protocol and has both a LIBRARY_ID and KEY_ID. The bibliography
entry will be generated by "exporting" zotero link to the required
format using a variation of gnotero. The "following/opening" of zotero
link will be done by mapping the local zotero link to
http://zotero.org-based URI (again) using gnotero.


I see that there is a need for bib2odt in much the same way as there is
a bib2html. Can someone post an example of Org file which would use
BibTex for post-processing (either to HTML or LaTeX) The ODT exporter
can also emulate what other backend does.

ps: This mail is more of a note to self or a note to any future
hackers. I am not certain I will be able to hack gnotero and have it
integrate with Emacs - but for a whimsical and a bored hacker like
nothing can be said with certainty.

Btw, gnotero is good. Not sure how usable it is though ...

Jambunathan K.

Footnotes: 

[1]  With Mozrepl 1.1 beta2 on Firefox 3.6.23, I am unable to make
org-zotero.el work. It hangs and I have to C-g out (Christian! you are
not alone). I am able to get as far as creating the LIBRARY_KEY for the
selected item in the zotero pane and not further.

[2] Quick Copy: http://www.zotero.org/support/creating_bibliographies

[3] For producing ITEM keys: One could use zotero.el or simply "Right
Click" on an item and do a "Generate Report from Selected Item". The
resulting url in firefox could be captured using org-protocol and
imported in to the org buffer. A typical URL looks like this:

zotero://report/items/0_6EETTDBE/html/report.html
0 => My Library
6EETTDBE => Key of the selected item

or 
zotero://report/items/579241_E7232HE2/html/report.html
579241 => The local library key for "Digital Humanities UofT group"
E7232HE2 => Key of the selected item.

[4]  For locating zotero.sqlite and querying it with Sqlite Manager see
- https://www.zotero.org/utils/dbfix/
- http://kimmonsdesign.com/node/24

[5]  http://cogsci.nl/software/qnotero
http://www.cogsci.nl/blog/tutorials/97-writing-a-command-line-zotero-client-in-9-lines-of-code
http://files.cogsci.nl/software/gnotero/extras/gnotero_cmdline.py

[6]  See the attached zotero-libreoffice.odt

[-- Attachment #2: zotero-libreoffice.odt --]
[-- Type: application/vnd.oasis.opendocument.text, Size: 13866 bytes --]

[-- Attachment #3: Type: text/plain, Size: 5 bytes --]


-- 

Re: zotero plain, org-odt, and citations in general

[Thomas S. Dye]

Jambunathan K <kjambunathan@gmail.com> writes:

>>    1. org-zotero.el
>>
>>       This permits insertion of zotero reference links *only* and
>>       opening them with a browser. 
>>
>>       Specifically I don't see the translator of these links to
>>       individual backends like latex, html or more importantly ODT.
>>
>>       #+begin_src emacs-lisp
>>         (org-add-link-type TYPE &optional FOLLOW EXPORT)
>>       #+end_src
>>
>>    2. zotero.el
>>
>>       This (apparently) permits download of data from firefox and
>>       importing it within Emacs.
>
>
> I think I now have some understanding of what Erik's library does[1]. 
>
> It creates a bibliographic entry (as in Footnotes) using Quick Copy.  It
> doesn not create a footnote reference. One can use the Firefox Zotero
> and see how Quick Copy works with both RTF and HTML formats[2].
>
> [Question to Erik: Can one not use "Copy from Clipboard" from elisp
> instead of going the mozrepl route to create a biblio.text]
>
> Instead of working with JS libraries, I think the right starting point
> would be to work with the underlying zotero.sqlite database itself. It
> is not difficult to get a key of the selected item in Zotero
> pane[3]. Once this is done the underlying zotero.sqlite database could
> be queried for the item properties[4].
>
> Fortunately there is already a GUI/Command line tool - Gnotero[5] -for
> accessing the sqlite database.
>
> As for exploiting the power of using zotero within libreoffice - which
> is to have the same citation presented in different citation styles -
> one need to look at how the reference marks are produced within
> OpenDocument XML.
>
> Here is a typical reference mark for an item with local zotero URL[6]:
> zotero://report/items/579241_E7232HE2/html/report.html
>
> LIBRARY=>579241
> KEY=>E7232HE2
>
>
> #+begin_src nxml
>   <text:p text:style-name="Footnote">
>     <text:reference-mark-start text:name="ZOTERO_ITEM {&quot;citationID&quot;:&quot;2lhcdnkups&quot;,&quot;citationItems&quot;:[{&quot;uri&quot;:[&quot;http://zotero.org/groups/51961/items/E7232HE2&quot;]}]} RNDB7mi3QiueI"/>Crew, “Alltagsgeschichte.”
>     <text:reference-mark-end text:name="ZOTERO_ITEM {&quot;citationID&quot;:&quot;2lhcdnkups&quot;,&quot;citationItems&quot;:[{&quot;uri&quot;:[&quot;http://zotero.org/groups/51961/items/E7232HE2&quot;]}]} RNDB7mi3QiueI"/>
>   </text:p>
> #+end_src nxml
>
> One can see the following components:
> 1. ZOTERO_ITEM 
> 2. citationID and citationItems
>    - The citationItem has the following Zotero URI:
>      http://zotero.org/groups/51961/items/E7232HE2
>      
>      GROUP=>51961
>      KEY=>E7232HE2
>
>      Note that the keys for the entry are the same on both the local
>      zotero store and the global zotero URI.
> 3. RNDB7mi3QiueI
>    - This is a cookie entered by Zotero which maps the citation entry to
>      an entry in the zotero database. Note that it starts with RND.
>
>      This is what is called as a "FieldCode" in zotero parlance.
>
>      Note: Have you seen the "Remove Field Codes" in Zotero menu within
>      ODT document. Once you remove the field codes the document becomes
>      a simple ODT document and one would no longer be able to move
>      between different citation styles.
>
>      In essence, "Field Code" is the MAGIC WAND that zotero uses.
>
> From my little research, I can say with confidence that global zotero
> URIs and the above mentioned reference marks are *very* recent
> developments.
>
> Exercising the MAGIC WAND within ODT exporter requires diving in to
> zotero server inner workings - which is non-trivial as Erik notes - but
> would also be futile (as in chasing a shifting sand)
>
> I believe the best course would be to do convention/static footnoting
> with the simplest of Chicago styles using a choice of entry types like
> Article, Journal, Book or a Website.
>
> Proposal for Org+Zotero integration
> ===================================
>
> An org document like this
>
> #+begin_src org
> * Headline
>   This is a footnote reference[1]
>
> Footnotes:
> [1] zotero://report/items/579241_E7232HE2/html/report.html
> #+end_src org
>
> will be transformed to this on ODT export
>
> #+begin_odt
> * Headline
>   This is a footnote reference[1]     
>
> Bibliography:
>
> 1. David F. Crew, “Alltagsgeschichte: A New Social History ‘From
> below’?,” Central European History 22, no. 3/4 (1989): 394-407.
> #+end_odt
>
> Note that the footnote link is an org stlye link captured with
> org-protocol and has both a LIBRARY_ID and KEY_ID. The bibliography
> entry will be generated by "exporting" zotero link to the required
> format using a variation of gnotero. The "following/opening" of zotero
> link will be done by mapping the local zotero link to
> http://zotero.org-based URI (again) using gnotero.
>
>
> I see that there is a need for bib2odt in much the same way as there is
> a bib2html. Can someone post an example of Org file which would use
> BibTex for post-processing (either to HTML or LaTeX) The ODT exporter
> can also emulate what other backend does.
>
> ps: This mail is more of a note to self or a note to any future
> hackers. I am not certain I will be able to hack gnotero and have it
> integrate with Emacs - but for a whimsical and a bored hacker like
> nothing can be said with certainty.
>
> Btw, gnotero is good. Not sure how usable it is though ...
>
> Jambunathan K.
>
> Footnotes: 
>
> [1]  With Mozrepl 1.1 beta2 on Firefox 3.6.23, I am unable to make
> org-zotero.el work. It hangs and I have to C-g out (Christian! you are
> not alone). I am able to get as far as creating the LIBRARY_KEY for the
> selected item in the zotero pane and not further.
>
> [2] Quick Copy: http://www.zotero.org/support/creating_bibliographies
>
> [3] For producing ITEM keys: One could use zotero.el or simply "Right
> Click" on an item and do a "Generate Report from Selected Item". The
> resulting url in firefox could be captured using org-protocol and
> imported in to the org buffer. A typical URL looks like this:
>
> zotero://report/items/0_6EETTDBE/html/report.html
> 0 => My Library
> 6EETTDBE => Key of the selected item
>
> or 
> zotero://report/items/579241_E7232HE2/html/report.html
> 579241 => The local library key for "Digital Humanities UofT group"
> E7232HE2 => Key of the selected item.
>
> [4]  For locating zotero.sqlite and querying it with Sqlite Manager see
> - https://www.zotero.org/utils/dbfix/
> - http://kimmonsdesign.com/node/24
>
> [5]  http://cogsci.nl/software/qnotero
> http://www.cogsci.nl/blog/tutorials/97-writing-a-command-line-zotero-client-in-9-lines-of-code
> http://files.cogsci.nl/software/gnotero/extras/gnotero_cmdline.py
>
> [6]  See the attached zotero-libreoffice.odt

Aloha Jambunathan,

For examples of Org-mode files that use bibtex, the following might be
helpful.

A tutorial of sorts:

http://orgmode.org/worg/org-tutorials/org-latex-export.html#sec-17

A piece of reproducible research:

https://github.com/tsdye/LKFS

hth,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: zotero plain, org-odt, and citations in general

[Erik Hetzner]

[-- Attachment #1: Type: text/plain, Size: 3263 bytes --]

Hi Jambunathan,

Thanks for the very detailed reply. Some responses below.

In your previous message, you asked who the author of zotero.el is. I
am. I will add author info.

At Thu, 10 Nov 2011 20:47:30 +0530,
Jambunathan K wrote:
> 
> I think I now have some understanding of what Erik's library does[1]. 
> 
> It creates a bibliographic entry (as in Footnotes) using Quick Copy.  It
> doesn not create a footnote reference. One can use the Firefox Zotero
> and see how Quick Copy works with both RTF and HTML formats[2].

This is correct. I should note that this method is totally inadequate,
as Frank Bennett showed me, for the generation of documents formatting
according to the various style guides. It cannot handle in-text
citations, bibliographies, or cross references. To get some idea of
how this happens in zot4rst (part of zotero-plain), see [1].

I should also note that this process is complicated, involving a
number of steps. Have a look at the python dir in zotero-plain to get
an idea.

> [Question to Erik: Can one not use "Copy from Clipboard" from elisp
> instead of going the mozrepl route to create a biblio.text]

Yes, but then one cannot select an item in Zotero and insert a link
into an org doc from org itself.

I should note that while the org/zotero integration in zotero-plain
“works for me”, I would welcome changes to make it more robust and
feature-full. But I use org for notes and todo lists, not for document
production.

> Instead of working with JS libraries, I think the right starting point
> would be to work with the underlying zotero.sqlite database itself. It
> is not difficult to get a key of the selected item in Zotero
> pane[3]. Once this is done the underlying zotero.sqlite database could
> be queried for the item properties[4].
> 
> Fortunately there is already a GUI/Command line tool - Gnotero[5] -for
> accessing the sqlite database.

This would certainly work. However, one problem would be that gnotero
cannot generate formatted citations from a reference, that is, it
cannot use CSL [2] style rules to generate arbitrary citation
formats. This might be fine; some simple c
 
> As for exploiting the power of using zotero within libreoffice - which
> is to have the same citation presented in different citation styles -
> one need to look at how the reference marks are produced within
> OpenDocument XML.

Not only that, but how the LibreOffice plugin communicates with
Zotero, specifically the citeproc-js [1] part of Zotero, to generate
in-text citations, bibliographies, etc.

> […]
>
> Footnotes: 
> 
> [1]  With Mozrepl 1.1 beta2 on Firefox 3.6.23, I am unable to make
> org-zotero.el work. It hangs and I have to C-g out (Christian! you are
> not alone). I am able to get as far as creating the LIBRARY_KEY for the
> selected item in the zotero pane and not further.
>
> […]

Have you started MozRepl from Firefox? I re-installed mozrepl, checked
that I am using the latest (and unmodified) versions of moz.el and
zotero-plain. Everything seems fine. This is with Emacs 23 - perhaps
Emacs 24 changes things?

best, Erik

1. http://gsl-nagoya-u.net/http/pub/citeproc-doc.html
2. http://www.zotero.org/styles

[-- Attachment #2: Type: text/plain, Size: 53 bytes --]

Sent from my free software system <http://fsf.org/>.

Re: zotero plain, org-odt, and citations in general

[Matt Price]

[-- Attachment #1: Type: text/plain, Size: 2919 bytes --]

Responding to just a few of the excellent points raised by J 7 E

On Thu, Nov 10, 2011 at 12:48 PM, Erik Hetzner <egh@e6h.org> wrote:

> At Thu, 10 Nov 2011 20:47:30 +0530,
> Jambunathan K wrote:
> >
> >
> I should note that while the org/zotero integration in zotero-plain
> “works for me”, I would welcome changes to make it more robust and
> feature-full. But I use org for notes and todo lists, not for document
> production.
>
I'm trying to switch to using org for most of my writing; among other
things, I now use it for posting to wordpress and (hopefully soon) drupal,
where most of my teaching work takes place. So I htink there's a case to
make for org as a person's main workspace.

>
> > Instead of working with JS libraries, I think the right starting point
> > would be to work with the underlying zotero.sqlite database itself. It
> > is not difficult to get a key of the selected item in Zotero
> > pane[3]. Once this is done the underlying zotero.sqlite database could
> > be queried for the item properties[4].
>

Wouldn't the rigt target be, not the sqlite db, but the Zoteor *service*
that runs on port 50001 when Zotero is running?  Aren't there some
higher-level tools for working through that interface -- e.g., erik, your
rst tool uses a python library, does it talk to Zotero that way?


> >
> > Fortunately there is already a GUI/Command line tool - Gnotero[5] -for
> > accessing the sqlite database.
>
> This would certainly work. However, one problem would be that gnotero
> cannot generate formatted citations from a reference, that is, it
> cannot use CSL [2] style rules to generate arbitrary citation
> formats. This might be fine; some simple c
>
> > As for exploiting the power of using zotero within libreoffice - which
> > is to have the same citation presented in different citation styles -
> > one need to look at how the reference marks are produced within
> > OpenDocument XML.
>
> Not only that, but how the LibreOffice plugin communicates with
> Zotero, specifically the citeproc-js [1] part of Zotero, to generate
> in-text citations, bibliographies, etc.
>

so, couldn't one think of the problem as having three parts:
(1) get emacs to talk to the zotero server; and
(2) figure out how org/emacs should interact with citeproc
(3) translate the org zotero links into a syntax that LibreOffice
recognizes and can successfully manipulate

and am I further right that, as an interim measure, solving (1) and (3)
properly and throwing together a quick hack for (2)  -- in which org
supports exactly one, very simple citation style -- would make a workflow
possible in which a person could compose in Org, and translate to ODT for
the final processing of a document, and/or export to HTML using only the
one citaqtion style that is supported by the aforementioned hack?

thanks again for all the help, folks,
matt

[-- Attachment #2: Type: text/html, Size: 3698 bytes --]

Re: zotero plain, org-odt, and citations in general

[Erik Hetzner]

[-- Attachment #1: Type: text/plain, Size: 2780 bytes --]

At Fri, 11 Nov 2011 10:51:22 -0500,
Matt Price wrote:
> 
> Responding to just a few of the excellent points raised by J 7 E
>
> On Thu, Nov 10, 2011 at 12:48 PM, Erik Hetzner <egh@e6h.org> wrote:
> 
> > At Thu, 10 Nov 2011 20:47:30 +0530,
> > Jambunathan K wrote:
> > >
> > >
> > I should note that while the org/zotero integration in zotero-plain
> > “works for me”, I would welcome changes to make it more robust and
> > feature-full. But I use org for notes and todo lists, not for document
> > production.

This is me you are quoting, for the record.

> I'm trying to switch to using org for most of my writing; among other
> things, I now use it for posting to wordpress and (hopefully soon) drupal,
> where most of my teaching work takes place. So I htink there's a case to
> make for org as a person's main workspace.

Definitely! Whatever works for you. I prefer the reStructuredText
syntax for longer documents, so getting real citations working in
org-mode is not a real priority for me. That’s all.

> Wouldn't the rigt target be, not the sqlite db, but the Zoteor *service*
> that runs on port 50001 when Zotero is running?  Aren't there some
> higher-level tools for working through that interface -- e.g., erik, your
> rst tool uses a python library, does it talk to Zotero that way?

the server that runs on port 50001 is undocumented, last I
checked. And there are no tools to connect to it. Frank Bennett put
together a hack of the jsbridge Firefox extension which works quite
well. [1] But that is python only, as far as I can tell.

> […]
> 
> so, couldn't one think of the problem as having three parts:
> (1) get emacs to talk to the zotero server; and
> (2) figure out how org/emacs should interact with citeproc
> (3) translate the org zotero links into a syntax that LibreOffice
> recognizes and can successfully manipulate

I think that is a good summary of what needs to be done. (3) is what
zot4rst does not bother to do.

The task of weaving together the HTML output from citeproc and your
export document can be tricky.

> and am I further right that, as an interim measure, solving (1) and (3)
> properly and throwing together a quick hack for (2)  -- in which org
> supports exactly one, very simple citation style -- would make a workflow
> possible in which a person could compose in Org, and translate to ODT for
> the final processing of a document, and/or export to HTML using only the
> one citaqtion style that is supported by the aforementioned hack?
>
> […]

Yeah, that makes sense. I think (3) is optional as well, depending on
what your workflow looks like, and what your needs are for doing final edits.

best, Erik

1. https://github.com/mozautomation/mozmill

[-- Attachment #2: Type: text/plain, Size: 53 bytes --]

Sent from my free software system <http://fsf.org/>.

Re: zotero plain, org-odt, and citations in general

[Matt Price]

[-- Attachment #1: Type: text/plain, Size: 2325 bytes --]

On Fri, Nov 11, 2011 at 11:12 AM, Erik Hetzner <egh@e6h.org> wrote:

> At Fri, 11 Nov 2011 10:51:22 -0500,
> Matt Price wrote:
> >
> > Responding to just a few of the excellent points raised by J 7 E
> >
> > On Thu, Nov 10, 2011 at 12:48 PM, Erik Hetzner <egh@e6h.org> wrote:
> >
> > > At Thu, 10 Nov 2011 20:47:30 +0530,
> > > Jambunathan K wrote:
> > > >
> > > >
> > > I should note that while the org/zotero integration in zotero-plain
> > > “works for me”, I would welcome changes to make it more robust and
> > > feature-full. But I use org for notes and todo lists, not for document
> > > production.
>
> This is me you are quoting, for the record.
>
> yes sorry, bad formatting on my part!

>
> > Wouldn't the rigt target be, not the sqlite db, but the Zoteor *service*
> > that runs on port 50001 when Zotero is running?  Aren't there some
> > higher-level tools for working through that interface -- e.g., erik, your
> > rst tool uses a python library, does it talk to Zotero that way?
>
> the server that runs on port 50001 is undocumented, last I
> checked. And there are no tools to connect to it. Frank Bennett put
> together a hack of the jsbridge Firefox extension which works quite
> well. [1] But that is python only, as far as I can tell.
>

sorry to hear about the lack of documentation.  that's a bummer actually.


> > […]
> >
> > so, couldn't one think of the problem as having three parts:
> > (1) get emacs to talk to the zotero server; and
> > (2) figure out how org/emacs should interact with citeproc
> > (3) translate the org zotero links into a syntax that LibreOffice
> > recognizes and can successfully manipulate
>
> I think that is a good summary of what needs to be done. (3) is what
> zot4rst does not bother to do.
>
> The task of weaving together the HTML output from citeproc and your
> export document can be tricky.
>
if tricky for you, then likely unmanageable for me.   but I wonder if
Zotero themselves would be interested in helping a little bit with some of
this work?  Seems sort of strange to build this great tool and then limit
its usefulness to two hulking word processors, when really the goal of
portable, reformattable citations applies just as much to text and html as
to word processing documents.

best,
matt

[-- Attachment #2: Type: text/html, Size: 3208 bytes --]

Re: zotero plain, org-odt, and citations in general

[Erik Hetzner]

[-- Attachment #1: Type: text/plain, Size: 908 bytes --]

At Fri, 11 Nov 2011 13:45:27 -0500,
Matt Price wrote:
> 
> On Fri, Nov 11, 2011 at 11:12 AM, Erik Hetzner <egh@e6h.org> wrote:
>
> […]
>
> if tricky for you, then likely unmanageable for me.   but I wonder if
> Zotero themselves would be interested in helping a little bit with some of
> this work?  Seems sort of strange to build this great tool and then limit
> its usefulness to two hulking word processors, when really the goal of
> portable, reformattable citations applies just as much to text and html as
> to word processing documents.

Agreed, but I think that Word, Open/LibreOffice, and LaTeX cover most
use cases. I don’t think the Zotero people have the time to do any
extra work, but they are almost helpful answering questions.

pandoc has similar support for citations and is worth taking a look
at. It requires export from Zotero, but this can be automated.

-Erik

[-- Attachment #2: Type: text/plain, Size: 53 bytes --]

Sent from my free software system <http://fsf.org/>.

Re: zotero plain, org-odt, and citations in general

[Christian Moe]

On 11/11/11 5:12 PM, Erik Hetzner wrote:
> At Fri, 11 Nov 2011 10:51:22 -0500,
> Matt Price wrote:
>> >  Wouldn't the rigt target be, not the sqlite db, but the Zoteor*service*
>> >  that runs on port 50001 when Zotero is running?  Aren't there some
>> >  higher-level tools for working through that interface -- e.g., erik, your
>> >  rst tool uses a python library, does it talk to Zotero that way?
> the server that runs on port 50001 is undocumented, last I
> checked. And there are no tools to connect to it. Frank Bennett put
> together a hack of the jsbridge Firefox extension which works quite
> well. [1] But that is python only, as far as I can tell.
>

A good while back -- before zotero-plain -- I spent some late nights 
eavesdropping on SOAP talk between OpenOffice and Zotero, and writing 
a Perl script to emulate some of it. It wasn't fun, and though I had 
some basic functionality going for a while,

Re: zotero plain, org-odt, and citations in general

[Christian Moe]

On 11/13/11 12:45 AM, Christian Moe wrote:
> A good while back -- before zotero-plain -- I spent some late nights

Sorry about the noise: I didn't mean to send this fragment. Please ignore.

Christian

zotero-cite (A Proposal)

[Jambunathan K]

[-- Attachment #1: Type: text/plain, Size: 3972 bytes --]


Hello Erik

Good news. 

Getting a frugal Zotero-based citations is *definitely* possible. It is
just a matter of time. Your libraries already provide the necessary
plumbing to accomplish the job. 

Long story
==========

For my own understanding, I tried trimming down zot4rst to it's bare
essentials and the attached zotcite.py is the result.

What zotcite does is this: 

For two Zotero Items, it prints their Bibliogrpahic entries and their
Citation Reference, in "Text" format using Chicago-Author-Date style.

I believe a commandline interface could be built around this file along
the following lines. (Will you be interested in building this interface
for us?)

$ zotcite --style chicago --format <text> --items I1, I2 --print biblio
$ zotcite --style chicago --format <text> --items I1, I2 --print citeref

Once this is done, Emacs/Org can do invoke zotcite and get the required
Bibliographic definitions and references using "shell-command".

The assumption is that Emacs/Org somehow has captured zotero keys
through org-protocol or org-zotero.el or someother means.

Here is a output from zotcite.py.

--8<---------------cut here---------------start------------->8---
$ python zotcite.py
C.UTF-8
C.UTF-8
======== KEYS ========
I4AUIZ4S
AKJZBHRW
======== ITEM_IDs ========
[6, 40]
======== BIBDATA ========
[u'Brin, S. 1999. \u201cExtracting patterns and relations from the world wide web.\u201d The World Wide Web and Databases: 172\u2013183.\n', u'Jambunathan, K. On Choice of Connection-Polynomials for LFSR-Based Stream Ciphers. In Progress in Cryptology \u2014INDOCRYPT 2000, ed. Bimal Roy and Eiji Okamoto, 1977:9-18. Berlin, Heidelberg: Springer Berlin Heidelberg. http://www.springerlink.com/content/n27yjr5eqhabux0g/.\n']
===== CITATION REFERENCE ===== 
(This Brin 1999 That)
--8<---------------cut here---------------end--------------->8---

IIRC, I made two changes to the export.js in your Javascript backend.

1. I had trouble understanding the organization of citation clusters. So
   I modified getCitationBlock as below. Note the use of "true" as a
   second param of the appendCitationCluster. This is OK as
   registerItemIds already does updateItems().

--8<---------------cut here---------------start------------->8---
function getCitationBlock (citation) {
    var results;
    var str = "";
    try {
	results = zotero.reStructuredCSL.appendCitationCluster(citation, true);
    } catch (e) {
	zotero.debug("XXX  oops: "+e);
    }
    // var index = citation['properties']['index'];
    // for (var i = 0 ; i <= results.length ; i++) {
    //     // if (results[i][0] == index) {
    //         return escape(str + results[i][1]);
    //     // }
    // }
    return escape(results[0][1]);
}
--8<---------------cut here---------------end--------------->8---

2. There should be way to set the output format form python side of
   things. I had to modify instantiateCiteProc by hand to set the output
   format to "text". You know what output format that LibreOffice plugin
   uses? I see "text", "html" and "rtf" as output formats. But not ODT.
   See https://bitbucket.org/fbennett/citeproc-js/src/tip/src/formats.js

--8<---------------cut here---------------start------------->8---
function instantiateCiteProc (styleid) {
	// Suspenders and a belt.
	try {
		if (!styleid) {
			styleid = "chicago-author-date";
		}
		if (styleid.slice(0,7) !== 'http://') {
			styleid = 'http://www.zotero.org/styles/' + styleid;
		}
		zotero.debug("XXX does this exist?: " + styleid);
		var style = zotero.Styles.get(styleid);
		zotero.reStructuredCSL = style.csl;
		zotero.reStructuredCSL.setOutputFormat("text");
	} catch (e) {
		zotero.debug("XXX instantiateCiteProc oops: " + e);
	}
};
--8<---------------cut here---------------end--------------->8---

Btw, I found that with my 2-day old zotero database and for printing the
above 2 keys, the whole machinery takes a perceivably a lot of time. Is
this your experience as well? I am not a netbook.


[-- Attachment #2: zotcite.py --]
[-- Type: text/plain, Size: 1920 bytes --]

"""
  Module
"""
# -*- coding: utf-8 -*-
import ConfigParser
import json
import os

# Workaround for crashes seen with localename setting
print os.environ["LANG"]
os.environ["LANG"]="C.UTF-8"
print os.environ["LANG"]

import string
import sys

import jsbridge
from zot4rst.util import unquote

DEFAULT_CITATION_FORMAT = "http://www.zotero.org/styles/chicago-author-date"

# placeholder for global bridge to Zotero
zotero_conn = None;

# verbose flag
verbose_flag = False

class ZoteroConn(object):
    def __init__(self, format, **kwargs):
        # connect & setup
        self.back_channel, self.bridge = jsbridge.wait_and_create_network("127.0.0.1", 24242)
        self.back_channel.timeout = self.bridge.timeout = 60
        self.methods = jsbridge.JSObject(self.bridge, "Components.utils.import('resource://csl/export.js')")

if zotero_conn is None:
    zotero_conn = ZoteroConn(DEFAULT_CITATION_FORMAT)


zotero_conn.methods.instantiateCiteProc(DEFAULT_CITATION_FORMAT)

print "======== KEYS ========"
key1 = "I4AUIZ4S"
key2 = "AKJZBHRW"
print key1
print key2

print "======== ITEM_IDs ========"
itemid1 = int(zotero_conn.methods.getItemId(key1))
itemid2 = int(zotero_conn.methods.getItemId(key2))

itemids = []
itemids.append(itemid1)
itemids.append(itemid2)
print itemids

zotero_conn.methods.registerItemIds(itemids)

print "======== BIBDATA ========"
bibdata = unquote(json.loads(zotero_conn.methods.getBibliographyData()))
print bibdata[1]

print "===== CITATION REFERENCE ===== "
from xciterst import CitationInfo

citation = CitationInfo();
citation.id = "40";
citation.prefix="This";
citation.suffix="That";

citation = { 'citationItems' : [citation],
             'properties'    : { 'index'    : 0, # zotero_conn.get_index(cluster)
                                 'noteIndex': 0 # note_index 
                                 }}

res = zotero_conn.methods.getCitationBlock(citation)

print unquote(res)

Re: zotero-cite (A Proposal)

[Matt Price]

[-- Attachment #1: Type: text/plain, Size: 114 bytes --]

just quickly saying thank you, this looks pretty neat, am hoping to try all
of htis code out in a couple of days.

[-- Attachment #2: Type: text/html, Size: 121 bytes --]

Re: zotero-cite (A Proposal)

[Erik Hetzner]

[-- Attachment #1: Type: text/plain, Size: 3291 bytes --]

Hi Jambunathan,

Sorry I took some time to reply. Some answers are below.

At Sat, 12 Nov 2011 02:04:52 +0530,
Jambunathan K wrote:
> 
> Hello Erik
> 
> Good news. 
> 
> Getting a frugal Zotero-based citations is *definitely* possible. It is
> just a matter of time. Your libraries already provide the necessary
> plumbing to accomplish the job. 
> 
> Long story
> ==========
> 
> For my own understanding, I tried trimming down zot4rst to it's bare
> essentials and the attached zotcite.py is the result.

That looks like the essence. Most of the code in zot4rst, as you saw,
deals with interacting with the docutils system and generating output.

> What zotcite does is this: 
> 
> For two Zotero Items, it prints their Bibliogrpahic entries and their
> Citation Reference, in "Text" format using Chicago-Author-Date style.
> 
> I believe a commandline interface could be built around this file along
> the following lines. (Will you be interested in building this interface
> for us?)
> 
> $ zotcite --style chicago --format <text> --items I1, I2 --print biblio
> $ zotcite --style chicago --format <text> --items I1, I2 --print citeref
>
> Once this is done, Emacs/Org can do invoke zotcite and get the required
> Bibliographic definitions and references using "shell-command".

I’ve adapted your script into a command line program, now included
with zotero-plain. The options are slightly different, and --format
does not work yet, but the essence is there.

Of course it requires the user to have python, etc. installed, but it
is probably easier than rewriting everything in emacs lisp.
 
> The assumption is that Emacs/Org somehow has captured zotero keys
> through org-protocol or org-zotero.el or someother means.
> 
> Here is a output from zotcite.py.
> 
> […]
> 
> IIRC, I made two changes to the export.js in your Javascript backend.
> 
> 1. I had trouble understanding the organization of citation clusters. So
>    I modified getCitationBlock as below. Note the use of "true" as a
>    second param of the appendCitationCluster. This is OK as
>    registerItemIds already does updateItems().

Thanks for catching this. That might speed things up a bit.

> […]
> 
> 2. There should be way to set the output format form python side of
>    things. I had to modify instantiateCiteProc by hand to set the output
>    format to "text". You know what output format that LibreOffice plugin
>    uses? I see "text", "html" and "rtf" as output formats. But not ODT.
>    See https://bitbucket.org/fbennett/citeproc-js/src/tip/src/formats.js

Sure, this would be easy.

I don’t know how the LibreOffice plugin works. It doesn’t take much
code, however, to translate the HTML output into something else.

> […]
> 
> Btw, I found that with my 2-day old zotero database and for printing the
> above 2 keys, the whole machinery takes a perceivably a lot of time. Is
> this your experience as well? I am not a netbook.

Yes, it is slow for me too. I wish it were faster. I don’t know if the
issue is jsbridge, or something else.

Ideally I would prefer to simply fetch citations information (as JSON,
for instance) from Zotero and process in an embedded citeproc-js
instance.

best, Erik

[-- Attachment #2: Type: text/plain, Size: 53 bytes --]

Sent from my free software system <http://fsf.org/>.

Re: zotero plain, org-odt, and citations in general

[Jambunathan K]

Erik

>> Footnotes: 
>> 
>> [1]  With Mozrepl 1.1 beta2 on Firefox 3.6.23, I am unable to make
>> org-zotero.el work. It hangs and I have to C-g out (Christian! you are
>> not alone). I am able to get as far as creating the LIBRARY_KEY for the
>> selected item in the zotero pane and not further.
>>
>> […]
>
> Have you started MozRepl from Firefox? I re-installed mozrepl, checked
> that I am using the latest (and unmodified) versions of moz.el and
> zotero-plain. Everything seems fine. This is with Emacs 23 - perhaps
> Emacs 24 changes things?

Here is what I find:

If I put the below snippet in a javascript buffer and do a M-x
send-region, I am able to retrieve the library and key of the selected
item.


#+begin_src javascript
repl.whereAmI()

var ZoteroPane = Components.classes["@mozilla.org/appshell/window-mediator;1"].getService(Components.interfaces.nsIWindowMediator).getMostRecentWindow("navigator:browser").ZoteroPane;
var selected_items = ZoteroPane.getSelectedItems();
new String(selected_items.length);
selected_items[0].libraryID;
selected_items[0].key;
#+end_src

Here is transaction log:

,---- From *MozRepl*
| 
| Welcome to MozRepl.
| 
|  - If you get stuck at the "...>" prompt, enter a semicolon (;) at the beginning of the line to force evaluation.
|  - If you get errors after every character you type, see http://github.com/bard/mozrepl/wikis/troubleshooting (short version: stop using Microsoft telnet, use netcat or putty instead)
| 
| Current working context: chrome://browser/content/browser.xul
| Current input mode: syntax
| 
| repl> [object ChromeWindow] - Document title: "fbennett / citeproc-js / source  Bitbucket - Mozilla Firefox"
| 1 — {0: "1"}
| 
| repl> 579241
| 
| repl> "TPSFKWXK"
| 
| repl> 
`----

The problem is in zotero-js-write-to-file. The problem happens even
after I increase the sleep out time to 1 sec.

See the attached backtrace with With M-x toggle-debug-on-quit RET and a
C-g

I wonder whether the issue is related to we - Matt(?), Christian(?) and
I - using Windows.


As can be seen from the transaction log, the shell interaction is
fine. I am inclined to think that it has something to do with zotero.el.

I am wondering whether there is a way to "capture" the selected item as
a zotero link through the jsbridge interface. Anyways with massaging of
org-protocol we can get a zotero "select" link by viewing the "report"
of the current item.

Debugger entered--Lisp error: (quit)
  sleep-for(0 10)
  (while (not (file-exists-p tmp-file)) (sleep-for 0 10))
  (let ((tmp-file (make-temp-file "zotero"))) (delete-file tmp-file) (zotero-js-write-to-file tmp-file expr) (while (not (file-exists-p tmp-file)) (sleep-for 0 10)) (prog1 (save-excursion (with-temp-buffer (insert-file-contents tmp-file) (buffer-substring-no-properties (point-min) (point-max)))) (delete-file tmp-file)))
  zotero-js-get-string-value("new String(selected_items.length)")
  (error "%s" (zotero-js-get-string-value "new String(selected_items.length)"))
  zotero-get-selected-item-ids()
  (let ((ids (zotero-get-selected-item-ids))) (mapc (lambda (id) (insert (format "[[zotero://select//%s][%s]]\n" id id)) (org-zotero-update-reference-link-at-point) (forward-line 1)) ids))
  org-zotero-insert-reference-link()
  call-interactively(org-zotero-insert-reference-link t nil)
  execute-extended-command(nil)
  call-interactively(execute-extended-command nil nil)
  recursive-edit()

  debug(error (quit))
  sleep-for(1 10)
  (while (not (file-exists-p tmp-file)) (sleep-for 1 10))
  (let ((tmp-file (make-temp-file "zotero"))) (delete-file tmp-file) (zotero-js-write-to-file tmp-file expr) (while (not (file-exists-p tmp-file)) (sleep-for 1 10)) (prog1 (save-excursion (with-temp-buffer (insert-file-contents tmp-file) (buffer-substring-no-properties (point-min) (point-max)))) (delete-file tmp-file)))
  zotero-js-get-string-value("new String(selected_items.length)")
  (error "%s" (zotero-js-get-string-value "new String(selected_items.length)"))
  zotero-get-selected-item-ids()
  (let ((ids (zotero-get-selected-item-ids))) (mapc (lambda (id) (insert (format "[[zotero://select//%s][%s]]\n" id id)) (org-zotero-update-reference-link-at-point) (forward-line 1)) ids))
  org-zotero-insert-reference-link()
  call-interactively(org-zotero-insert-reference-link t nil)
  execute-extended-command(nil)
  call-interactively(execute-extended-command nil nil)

-- 

Re: zotero plain, org-odt, and citations in general

[Christian Moe]

On 11/11/11 10:13 PM, Jambunathan K wrote:
> I wonder whether the issue is related to we - Matt(?), Christian(?) and
> I - using Windows.

You can scratch that possibility, I'm on a Mac (OS X 10.6.8). It 
worked, albeit a little erratically, on my previous (10.5) Mac. That 
one was old and quite slow, and I thought timeouts had something to do 
with the erratic zotero-plain behavior, but that doesn't account for 
the present dead end.

Also, I just tested with Firefox 3.6.7, with the same results, so the 
problem's not with version 8.

Yours,
Christian

Re: zotero plain, org-odt, and citations in general

[Matt Price]

[-- Attachment #1: Type: text/plain, Size: 260 bytes --]

On Sat, Nov 12, 2011 at 2:21 AM, Christian Moe <mail@christianmoe.com>wrote:

> On 11/11/11 10:13 PM, Jambunathan K wrote:
>
>> I wonder whether the issue is related to we - Matt(?), Christian(?) and
>> I - using Windows.
>>
>  not me either -- ubuntu for me.

[-- Attachment #2: Type: text/html, Size: 649 bytes --]

Re: zotero plain, org-odt, and citations in general

[Erik Hetzner]

[-- Attachment #1: Type: text/plain, Size: 1247 bytes --]

At Sat, 12 Nov 2011 02:43:12 +0530,
Jambunathan K wrote:
> 
> 
> Erik
>
> […]
>
> Here is what I find:
> 
> If I put the below snippet in a javascript buffer and do a M-x
> send-region, I am able to retrieve the library and key of the selected
> item.
>
> […]
>
> The problem is in zotero-js-write-to-file. The problem happens even
> after I increase the sleep out time to 1 sec.
> 
> See the attached backtrace with With M-x toggle-debug-on-quit RET and a
> C-g
> 
> I wonder whether the issue is related to we - Matt(?), Christian(?) and
> I - using Windows.
> 
> As can be seen from the transaction log, the shell interaction is
> fine. I am inclined to think that it has something to do with zotero.el.
> 
> I am wondering whether there is a way to "capture" the selected item as
> a zotero link through the jsbridge interface. Anyways with massaging of
> org-protocol we can get a zotero "select" link by viewing the "report"
> of the current item.
Hi,

Thanks for all the info. It appears that the file is not being
created by Zotero, but I cannot figure why.

I can’t really pursue this much, since I am unable to reproduce. If
anybody has any thoughts, I would love to hear them!

best, Erik

[-- Attachment #2: Type: text/plain, Size: 53 bytes --]

Sent from my free software system <http://fsf.org/>.

Re: zotero plain, org-odt, and citations in general

[Erik Hetzner]

[-- Attachment #1: Type: text/plain, Size: 1684 bytes --]

At Tue, 8 Nov 2011 21:06:37 -0500,
Matt Price wrote:
> 
> Hi everyone,
> 
> I am about to start experimenting with Erik Hetzner's zotero-plain, which
> allows the use of zotero as a bibliographic manager in emacs & especially
> org-mode (https://bitbucket.org/egh/zotero-plain as well as a couple of
> threads on this list, I think).  I'm wonderinghow other people have used
> it, and in particular whether there's any way to preserve the zotero markup
> across exports to odt especially. That is, I'd like to keep the original
> citations embedded in the final odt, so that I can continue to use zotero
> in an odt or doc file after it's been exported from org.
> 
> This would take me very , very close to a complete org-centric workflow
> that still lets me produce documents for consumption by my MS-using
> colleagues.  I really hope it's possible to do! If anyone has done it, I
> would really love to hear about it.

Hi Matt,

org-mode support in Zotero plain has not received as much work
recently as the docutils (reStructuredText) support. Basically all
that one can do in org-mode at the moment is insert & update
citations, and following citation links to the entry in Zotero.

zot4rst (docutils) support can transform rst documents into odt
documents with citations generated by Zotero. [1]

It might be possible to implement something similar in the org-mode
Zotero, but I don’t know how, and it might be difficult.

best, Erik

1. However, it should be noted that this support is different from
that provided by the OpenOffice plugin, and one cannot edit a document
generated from a rst file using the Zotero/OpenOffice plugin.


[-- Attachment #2: Type: text/plain, Size: 53 bytes --]

Sent from my free software system <http://fsf.org/>.

Re: zotero plain, org-odt, and citations in general

[Jambunathan K]

> Basically all that one can do in org-mode at the moment is insert &
> update citations, and following citation links to the entry in Zotero.

Can you or someone provide me how such an Org file with citation
definition and reference looks like?

> It might be possible to implement something similar in the org-mode
> Zotero, but I don’t know how, and it might be difficult.

Do you know who the author of zotero.el? The file header in zotero.el
has no author info.
https://bitbucket.org/egh/zotero-plain/src/66353860ebdd/elisp/zotero.el

Assuming that prior work in zotero.el is reasonably complete and usable
- in so far as retrieving all fields associated with a zotero reference
- getting citation sections and references in ODT export shouldn't be a
difficult issue.

I am making these comments as the author of org-odt.el.

Re: zotero plain, org-odt, and citations in general

[Ken Williams]

Jambunathan K <kjambunathan <at> gmail.com> writes:

>  
> > Basically all that one can do in org-mode at the moment is insert &
> > update citations, and following citation links to the entry in Zotero.
> 
> Can you or someone provide me how such an Org file with citation
> definition and reference looks like?

I'd be very interested in this too.  I've just started to work with Zotero and
I'd assumed that to interface with org-mode I'd have to export as BiBTeX first,
but if I can connect them directly that would be awesome.

 -Ken

Re: zotero plain, org-odt, and citations in general

[Christian Moe]

Hi,

Is zotero-plain working for others?

I test drove zotero-plain back when it was just developed. Looked 
promising, but I had uneven success with the MozRepl communication, 
possibly something to do with being on an older and slower machine.

Trying it again today (with the most recent zotero-plain, MozRepl and 
moz.el I could find), I press `C-c z i' to insert a citation and 
nothing happens. Well, the Org buffer hangs until I press C-g, and 
except that the *MozRepl* buffer says "Process MozRepl deleted".

Yours,
Christian

Re: zotero plain, org-odt, and citations in general

[Erik Hetzner]

[-- Attachment #1: Type: text/plain, Size: 741 bytes --]

At Wed, 09 Nov 2011 20:39:18 +0100,
Christian Moe wrote:
> 
> Hi,
> 
> Is zotero-plain working for others?
> 
> I test drove zotero-plain back when it was just developed. Looked 
> promising, but I had uneven success with the MozRepl communication, 
> possibly something to do with being on an older and slower machine.
> 
> Trying it again today (with the most recent zotero-plain, MozRepl and 
> moz.el I could find), I press `C-c z i' to insert a citation and 
> nothing happens. Well, the Org buffer hangs until I press C-g, and 
> except that the *MozRepl* buffer says "Process MozRepl deleted".

Hi Christian,

It works for me, with MozRepl 1.1beta2, and the latest Zotero plain.

MozRepl communication *is* not very good.

best, Erik

[-- Attachment #2: Type: text/plain, Size: 53 bytes --]

Sent from my free software system <http://fsf.org/>.

Re: zotero plain, org-odt, and citations in general

[Christian Moe]

On 11/10/11 5:53 AM, Erik Hetzner wrote:
> It works for me, with MozRepl 1.1beta2, and the latest Zotero plain.

Thanks. If that's with the latest Firefox too (apparently I'm on 8.0, 
Mozilla's new approach versioning makes me dizzy), there must be some 
problem with my setup. I'll try to figure it out.

Yours,
Christian

Re: zotero plain, org-odt, and citations in general

[Matt Price]

[-- Attachment #1: Type: text/plain, Size: 805 bytes --]

On Wed, Nov 9, 2011 at 2:39 PM, Christian Moe <mail@christianmoe.com> wrote:

> Hi,
>
> Is zotero-plain working for others?
>
> I test drove zotero-plain back when it was just developed. Looked
> promising, but I had uneven success with the MozRepl communication,
> possibly something to do with being on an older and slower machine.
>
> Trying it again today (with the most recent zotero-plain, MozRepl and
> moz.el I could find), I press `C-c z i' to insert a citation and nothing
> happens. Well, the Org buffer hangs until I press C-g, and except that the
> *MozRepl* buffer says "Process MozRepl deleted".
>
 Similar for me -- the Org buffer hangs, and MozRepl looks like it's just
been sitting there idly.

also on the most recent MozRepl, emacs 24, fairly recent org-mode, firefox
8, ubuntu 10.10.

[-- Attachment #2: Type: text/html, Size: 1125 bytes --]

Re: zotero plain, org-odt, and citations in general

[Erik Hetzner]

[-- Attachment #1: Type: text/plain, Size: 523 bytes --]

At Fri, 11 Nov 2011 10:37:03 -0500,
Matt Price wrote:
> 
> Similar for me -- the Org buffer hangs, and MozRepl looks like it's
> just been sitting there idly.
> 
> also on the most recent MozRepl, emacs 24, fairly recent org-mode, firefox
> 8, ubuntu 10.10.

I was going to suggest this sounds like a problem with Emacs 24, but I
installed Emacs 24 & it works for me. I am using Firefox 7, but I
doubt that is much of an issue.

I assume that you have started MozRepl and selected an item to insert
in Zotero?

best, Erik


[-- Attachment #2: Type: text/plain, Size: 53 bytes --]

Sent from my free software system <http://fsf.org/>.

Re: zotero plain, org-odt, and citations in general

[Matt Price]

[-- Attachment #1: Type: text/plain, Size: 936 bytes --]

On Fri, Nov 11, 2011 at 12:51 PM, Erik Hetzner <egh@e6h.org> wrote:

> At Fri, 11 Nov 2011 10:37:03 -0500,
> Matt Price wrote:
> >
> > Similar for me -- the Org buffer hangs, and MozRepl looks like it's
> > just been sitting there idly.
> >
> > also on the most recent MozRepl, emacs 24, fairly recent org-mode,
> firefox
> > 8, ubuntu 10.10.
>
> I was going to suggest this sounds like a problem with Emacs 24, but I
> installed Emacs 24 & it works for me. I am using Firefox 7, but I
> doubt that is much of an issue.
>
> I assume that you have started MozRepl and selected an item to insert
> in Zotero?
>
> best, Erik
>

yes, I had done those two things (htough not hte first time round!).  If
MozRepl isn't started I get a diferent, quite helpful error message; in
this case there is just (as for Christian) a message "Process MozRepl
deleted" which shows up before I press C-g to end the hang.  I imagine htis
is a bug in MozRepl

[-- Attachment #2: Type: text/html, Size: 1300 bytes --]

Re: zotero plain, org-odt, and citations in general

[Matt Price]

[-- Attachment #1.1: Type: text/plain, Size: 2586 bytes --]

attached is a sample odt file with Zotero citations included. It's very
simple, actually, because I think the key to making something like this
work would be to offload as much work as possible to zotero itself -- Erik,
correct me if I'm wrong (quite likely).

Both Erik and Jambunathan have raised questions which I'm not al lthat
competent to answer, but doing my best (and sorry for the lack of quotation
markup, not sure how to paste-as-quote in gmail):


1. Zotero supports many citation stlyes. So Org/Zotero integration could
>   choose one citation style that is widely agreeable for one-off
>   colloboration needs.
>

If that's easier, then sure. Since the Openoffice plugin allows you to
reformat citations easily, then if the import result in active zotero
fields that zotero can work with, it shouldn't be hard for the user to
change the citation format easily.

2. I don't know if importing Zotero stylenames verbatim in to org-odt's
>   style file would "pollute" it - thereby preventing org-odt from
>   getting in to Emacs proper.
>

I think what really matters is getting the zotero id's into the ODT in a
way that allows Zotero to see them later on.  Then the user can reformat in
whatever manner she needs to.

  I am not competent to judge what the copyright and license terms of
  the stylenames used by zotero plugin are. I am seeing that zotero
  itself is AGPL so I believe it is Free as in "Free Software".


I think so -- shouldn't be a license problem w/ emacs.

3. Even if zotero's ODT style cannot be used by the ODT exporter due
>   (2), I can cook up Org's own stlyenames for various citation fields
>   and allow the user to remap Org* stlyenames to say Zotero* stlynames.
>
> I see Org already has some(?) support for bibtex. Can anyone comment on
> bibtex vs zotero? They seem to serve same purpose - citation
> management. Can bibtex be used instead of zotero.
>

For me that wouldn't work, becaue bibtex isn't widely used by humanities
scholars and social scientists. Zotero is, right now, by far the best tool
for us.  Also bibtex has, or at least used to have, significant limitations
on styles that make (made?) it unsuitable for humanities scholars.

I wish I could provide an example org-mode file with zotero references in
it -- erik, I'm having trouble making contact with mozrepl from emacs.
given recent developments in Zotero, it would probably be better to have
zotero.el speak directly to the Zotero server instead of working through
firefox -- do you agree, Erik? But I imagine that is a significant
undertaking.

thanks everyone,
Matt

[-- Attachment #1.2: Type: text/html, Size: 3288 bytes --]

[-- Attachment #2: zotero-demo-file.odt --]
[-- Type: application/vnd.oasis.opendocument.text, Size: 11494 bytes --]

Re: zotero plain, org-odt, and citations in general

[Erik Hetzner]

[-- Attachment #1: Type: text/plain, Size: 2933 bytes --]

At Wed, 9 Nov 2011 10:28:21 -0500,
Matt Price wrote:
> 
> attached is a sample odt file with Zotero citations included. It's very
> simple, actually, because I think the key to making something like this
> work would be to offload as much work as possible to zotero itself -- Erik,
> correct me if I'm wrong (quite likely).

Hi,

Thanks for the sample ODT file.

I would like to clarify a few issues that have been raised:

zotero-plain consists of 2 very separate parts: emacs/zotero
integration and docutils/zotero integration (zot4rst). zot4rst is far
more advanced.

zot4rst provides extensions for docutils that allow a user to include
zotero citations in reStructuredText documents. For example:

  :xcite:`[@item1, p. 30]` says blah.

When the user employs the zrst2{pdf,odt,...} tools, docutils connects
to a running Firefox instance (using jsbride & a custom zot4rst
Firefox plugin) and uses Zotero to format citations & bibliographies
(in this case, for @item1, p. 30). When generating an ODT file, this
does *not* use the standard OpenOffice/Zotero integration, but rather
uses Zotero to generate HTML citations, transforms the HTML citations
into docutils internal format, and then uses the rst2odt code to
transform that into an ODT document. So while it might be possible to
include the necessary information that make the standard
OpenOffice/Zotero plugin work, it would probably be difficult.

The Emacs/Zotero integration is far less advanced. It uses MozRepl,
which is flaky, and includes code to insert & update & follow links, &
generate citations into org docs. Here is an example link:

  [[zotero://select//0_83KCW6AU][Rediker, Marcus. The Slave Ship: A Human History. Viking Adult, 2007.]]

This link can be automatically inserted, and can be followed using C-c
C-o, which selects the item in Zotero.

This is *all* the org-mode integration does at present. This serves my
needs (note-taking) and I probably won’t do any enhancements (patches
are welcome!) Keep in mind that citations are tricky: if you want to
transform org citations into publishable documents you want to be able
to handle footnote & in-text citations (it depends on the style) and
bibliographies. Have a look at the python zot4rst code to get an idea
of what is required. If people want org -> odt transformations, it is
going to be difficult.

I hope that helps!

best, Erik

PS: 

> erik, I'm having trouble making contact with mozrepl from emacs.
> given recent developments in Zotero, it would probably be better to have
> zotero.el speak directly to the Zotero server instead of working through
> firefox -- do you agree, Erik? But I imagine that is a significant
> undertaking.

There really is no Zotero server at the moment. The API at zotero.org
does not support bibliography generation, and the internal server that
Zotero provides locally is not documented.

[-- Attachment #2: Type: text/plain, Size: 53 bytes --]

Sent from my free software system <http://fsf.org/>.

Re: zotero plain, org-odt, and citations in general

[Christian Moe]

* Towards a solution for Zotero -> Org -> ODT


I think I have a way to get Zotero references from Org links into ODT 
as working reference marks. Interested parties, please
test if this works for you at all.

1. Create Zotero links such as e.g.:

    : [[zotero:0_RADJMJ8Q][{Bolin, 1958}]]

    Below, I include code for a simple Zotero export translator
    (OrgMode.js) that will create such links at a keystroke/mouse drag,
    since some of us are currently experiencing difficulties with
    zotero-plain.

2. Define a custom link export function to fake a Zotero field code
    for ODT. Code below.

3. Export to ODT and do the things you normally do with Zotero, like
    generating a bibliography. (Note that Zotero won't let you insert a
    bibliography until you've inserted at least one Zotero reference
    from within LibreOffice. You can always delete it afterwards.)

If this works for you, the code (which is a quickly thrown-together 
mess) can probably be improved on.

Yours,
Christian Moe

** OrgMode.js translator

Installation:

1. Tangle or copy the below to a file called "OrgMode.js".
2. Place it in the Zotero translators folder (on Mac, that's

    ~/Library/Application 
Support/Firefox/Profiles/[PROFILENAME]/zotero/translators/

3. Restart Firefox.
4. Open Zotero > Actions (the cogwheel icon) > Preferences. From the
    drop-down menu, select "Org-mode" as the Quick Copy default format.

Use: you can

- drag and drop references from the Zotero panel to the Emacs buffer,
   or
- "Quick Copy" to clipboard with Shift-Cmd-C (Mac) and yank in the
   Emacs buffer

#+name: OrgMode.js
#+begin_src javascript
{
     "translatorID": "b0006c6f-b743-4508-beaf-490bbd68a403",
     "label": "Org-mode",
     "creator": "Christian Moe",
     "target": "org",
     "minVersion": "2.1.9",
     "maxVersion": "",
     "priority": 200,
     "displayOptions": {
	"exportCharset": "UTF-8",
	"exportNotes": true,
	"exportFileData": false
     },
     "inRepository": false,
     "translatorType": 2,
     "browserSupport": "gcs",
     "lastUpdated": "2011-11-12 17:05:00"
}

// Loosely based on Scott Campbell's ZotSelect.js
// posted at 
http://forums.zotero.org/discussion/8306/testing-zoteroselect/

function doExport() {
     // Zotero.write("zotero://select//");
     // Zotero.write("\n");

     var item;
     var notfirst = false;
     while(item = Zotero.nextItem()) {
	
	// Separate multiple links
	if (notfirst) {
	    Zotero.write("; ")
	}

	// Org link path
	Zotero.write("[[zotero:");
	var library_id = item.LibraryID ? item.LibraryID : 0;
	Zotero.write(library_id+"_"+item.key);

	// Org link descr
	Zotero.write("][{");
	// create a unique citation key
	//var descr = buildCiteKey(item, citekeys);
	
	// write citation key
	//Zotero.write("/"+citekey);

	// Authorname
	if(item.creators && item.creators[0] && item.creators[0].lastName) {
	    Zotero.write(item.creators[0].lastName);
	} else {
	    Zotero.write("n.a.")
	}
	// Separator
	Zotero.write(", ")
	// Year
	var numberRe = /^[0-9]+/;
	if(item.date) {
	    var date = Zotero.Utilities.strToDate(item.date);
	    if(date.year && numberRe.test(date.year)) {
		Zotero.write(date.year);
	    }
	} else {
	    Zotero.write("n.d.");
	}

	// Close Org link
	Zotero.write("}]]")
	
	notfirst = true;
     }
}
#+end_src

** Zotero link type

Evaluate the following (or place it in your .emacs for repeated use).

Note that clicking the links does not work on Mac and probably won't 
work on Linux/Windows, either (but please let me know if it does!). 
Consider org-zotero-open a placeholder for now.

#+begin_src elisp
   (org-add-link-type "zotero"
                      'org-zotero-open
                      'org-zotero-export)

   (defun org-zotero-open (path)
     (browse-url (format "zotero://select//%s" path)))

   (defun org-zotero-export (path desc format)
     (cond
      ((eq format 'odt)
       (let
           ((refmark "<text:reference-mark-start 
text:name=\"%s\"/>%s<text:reference-mark-end text:name=\"%s\"/>")
            (zitem "ZOTERO_ITEM 
{&quot;citationID&quot;:&quot;%s&quot;,&quot;citationItems&quot;:[{&quot;uri&quot;:[&quot;http://zotero.org/users/local/%s/items/%s&quot;]}]} 
%s")
            (citation-id (substring (org-id-new) -10)) ; Is this a 
good way to make a unique ID?
            (library-id (car (split-string path "_")))
            (item-key (car (cdr (split-string path "_"))))
            (rnd (concat "RND" (substring (org-id-new) -10))))
         (setq zitem
               (format zitem
                       citation-id
                       library-id
                       item-key
                       rnd))
         (setq desc (format "(%s)" desc))
         (format refmark zitem desc zitem)))
      (t desc)))
#+end_src

Re: zotero plain, org-odt, and citations in general

[Matt Price]

[-- Attachment #1: Type: text/plain, Size: 536 bytes --]

On Sun, Nov 13, 2011 at 5:47 PM, Christian Moe <mail@christianmoe.com>wrote:

> * Towards a solution for Zotero -> Org -> ODT
>
>
> I think I have a way to get Zotero references from Org links into ODT as
> working reference marks. Interested parties, please
> test if this works for you at all.
>
> In a busy patch right now but this looks AWESOME.  Hope to test it out in
a couple of days, sorry to keep you waiting b/c I'm very excited about
this, even if a full-fledged Zotero service interface would, of course, be
cooler...

Matt

[-- Attachment #2: Type: text/html, Size: 841 bytes --]