Re: zotero (or mendeley) integration with org

Re: zotero (or mendeley) integration with org

[Rustom Mody]

Stephen Eglen wrote:
> I've not yet switched to a pdf manager (they're all stuffed into a folder, with a few subfolders, and the only meta-data is
> in the filename!), so I'd appreciate hearing what others to do to look after their pdfs.

Maybe look at tracker [assuming linux] http://projects.gnome.org/tracker/ ?
More generally http://www.wikinfo.org/index.php/Comparison_of_desktop_search_software

Re: Re: zotero (or mendeley) integration with org

[Suvayu Ali]

On Sun, 27 Mar 2011 08:42:19 +0530
Rustom Mody <rustompmody@gmail.com> wrote:

> Stephen Eglen wrote:
> > I've not yet switched to a pdf manager (they're all stuffed into a
> > folder, with a few subfolders, and the only meta-data is in the
> > filename!), so I'd appreciate hearing what others to do to look
> > after their pdfs.
> 
> Maybe look at tracker [assuming linux]
> http://projects.gnome.org/tracker/ ? More generally
> http://www.wikinfo.org/index.php/Comparison_of_desktop_search_software
> 

or Recoll (on GNU/Linux)

-- 
Suvayu

Open source is the future. It sets us free.

zotero (or mendeley) integration with org

[Stephen Eglen]

There was a mail-thread lastyear about zotero and integration with org.
Now that there is an alpha release of 'org-standalone' 
  http://www.zotero.org/blog/2011/02/

has anyone looked at whether this helps integrate org and zotero?

I've not yet switched to a pdf manager (they're all stuffed into a
folder, with a few subfolders, and the only meta-data is in the
filename!), so I'd appreciate hearing what others to do to look after
their pdfs.  Mendeley is a possibility too (although syncing between
machines is a must, and Mendeley doesn't offer that yet.)

Stephen

Re: zotero (or mendeley) integration with org

[Matt Lundin]

Stephen Eglen <S.J.Eglen@damtp.cam.ac.uk> writes:

> There was a mail-thread lastyear about zotero and integration with org.
> Now that there is an alpha release of 'org-standalone' 
>   http://www.zotero.org/blog/2011/02/
>
> has anyone looked at whether this helps integrate org and zotero?
>
> I've not yet switched to a pdf manager (they're all stuffed into a
> folder, with a few subfolders, and the only meta-data is in the
> filename!), so I'd appreciate hearing what others to do to look after
> their pdfs.  Mendeley is a possibility too (although syncing between
> machines is a must, and Mendeley doesn't offer that yet.)

One option is to manage metadata in org-mode itself, relying on
org-attach to store and preserve links to the pdf files. Bibtex source
blocks can used to store bibliographical data for each pdf.

I find the combination of emacs-w3m, google scholar, and org-mode to be
an easier and more transparent way to manage bibtex data than an
indirect route via Zotero or Mendeley. But I also prefer to edit all my
bibtex data by hand. :)

Recoll is great for indexing. I have a mess of spaghetti code I use to
pull recoll results into a temporary org outline. I can then open the
relevant files using org links. I'd be happy to share it if anyone is
interested.

Best,
Matt

Re: zotero (or mendeley) integration with org

[William Gardella]

Matt Lundin <mdl@imapmail.org> writes:

> Stephen Eglen <S.J.Eglen@damtp.cam.ac.uk> writes:
>
>> There was a mail-thread lastyear about zotero and integration with org.
>> Now that there is an alpha release of 'org-standalone' 
>>   http://www.zotero.org/blog/2011/02/
>>
>> has anyone looked at whether this helps integrate org and zotero?
>>
>> I've not yet switched to a pdf manager (they're all stuffed into a
>> folder, with a few subfolders, and the only meta-data is in the
>> filename!), so I'd appreciate hearing what others to do to look after
>> their pdfs.  Mendeley is a possibility too (although syncing between
>> machines is a must, and Mendeley doesn't offer that yet.)
>
> One option is to manage metadata in org-mode itself, relying on
> org-attach to store and preserve links to the pdf files. Bibtex source
> blocks can used to store bibliographical data for each pdf.
>
> I find the combination of emacs-w3m, google scholar, and org-mode to be
> an easier and more transparent way to manage bibtex data than an
> indirect route via Zotero or Mendeley. But I also prefer to edit all my
> bibtex data by hand. :)
>
> Recoll is great for indexing. I have a mess of spaghetti code I use to
> pull recoll results into a temporary org outline. I can then open the
> relevant files using org links. I'd be happy to share it if anyone is
> interested.
>
> Best,
> Matt
>
>

I use something like the setup Matt describes as well (except I haven't
played with Recoll).  I use org-attach to keep documents
organized; usually they're attached to a :noexport: top-level heading
creatively called "Documents," which contains the citation info and
keeps my research/reading notes distinct from the paper I'm writing.
Since my org files are themselves indexed and easily searchable, I've
not felt the need for any collection management software beyond that.
(For legal research it's particularly good, because I can get LEXIS to
email me most legal texts as plaintext, which I then just add inline as
subheadings to the Documents heading.)

For bibliography generation, I use RefTeX to do my BibTeXing for me, and
I use a pretty crude one bibliography database to one paper kind of
system.

Best,
Will

-- 
William Gardella
J.D. Candidate
Class of 2011, University of Pittsburgh School of Law

Re: zotero (or mendeley) integration with org

[Rasmus]

> [Matt and William's setup]

I have looked for a good way to keep track of academic papers (pdfs) and
Bibtex for a long time. I'd love to see a worg page on this topic.

Meanwhile, I have found some sweet Bibtex-search interfaces for
Emacs. These will query a academic search engine and can copy Bibtex
entries directly to a .bib file. I found bibsnarfl[fn:1] being the most
interesting, but a similar code is available for PubMed[fn:2].
Unfortunately, being limited to certain fields, I am personally not able
to adopt either. It would be great to have an interface to a general
academic search engine (Google Scholar, ugh?).

Imagine the combination of a Emacs-powered interface to some search
engine, a university network and some magic snip that would download a 
pdf, add it to a .bib-file (removing annoying entries and adding a
sensible key), and making a nice, easy-to-browse Org-file. 

One day, maybe...

–Rasmus

Footnotes:

[fn:1] http://www.emacswiki.org/emacs/bibsnarf.el

[fn:2] http://www.bioinformatics.org/texmed/

Re: zotero (or mendeley) integration with org

[Matt Lundin]

Rasmus <rasmus.pank@gmail.com> writes:

>> [Matt and William's setup]
>
> I have looked for a good way to keep track of academic papers (pdfs) and
> Bibtex for a long time. I'd love to see a worg page on this topic.
>
> Meanwhile, I have found some sweet Bibtex-search interfaces for
> Emacs. These will query a academic search engine and can copy Bibtex
> entries directly to a .bib file. I found bibsnarfl[fn:1] being the most
> interesting, but a similar code is available for PubMed[fn:2].
> Unfortunately, being limited to certain fields, I am personally not able
> to adopt either. It would be great to have an interface to a general
> academic search engine (Google Scholar, ugh?).

Agreed. Google Scholar citations need very close proofreading, as they
can be erroneous or poorly formatted. I would submit that one should
never use a Google Scholar citation without checking it carefully
against the article or book to which it refers. An advantage of Google
Scholar, however, is that it offers skeletal bibtex entries for books
and articles in a wide variety of fields, whereas many of the databases
accessed by bibsnarf are limited to math and sciences. Since I use
biblatex together with the Chicago Manual of Style, any bibtex entry I
clip has to be edited and tweaked substantially. (Indeed, manual editing
is unavoidable when using biblatex.)

FWIW, here's my workflow:

1. Clip a bibtex citation or unformatted bibliographical data using
   org-capture and conkeror or emacs. (This generates a timestamp and a
   link to where I first found the reference --- very useful data for
   reviewing one's own research habits and sources.)
2. Download the pdf, if possible (or make a todo to get/read it later).
3. While in the capture buffer, use org-attach to move the pdf quickly
   from ~/Downloads to the attachments directory (this is much faster
   than it sounds).
4. Create or correct a bibtex source block within the org entry.
5. Make a TODO to read the pdf. :)

Later, when I read the document, I proofread the bibtex entry once again
and call a function that moves it to a centralized bibtex file, leaving
a link in its place. As a rule, any citation that goes into my official
bibtex file *must* be correct and complete. Reftex, another of Carsten's
ingenious creations, is very handy for creating links to the citation
while taking notes.

For my own purposes, there would be little point in automating this
process, as any pdf I download needs to be inspected manually and any
bibtex entry I clip needs to be proofread and tweaked. The process
itself forces me to check every citation for accuracy.

Best,
Matt

Re: zotero (or mendeley) integration with org

[Stephen Eglen]

> Agreed. Google Scholar citations need very close proofreading, as they
> can be erroneous or poorly formatted. 

Thanks Matt - I'd agree with this, having seen oddities from google
scholar.  I emailed them ages ago about one problem (formatting of
initials in author names), but never heard back... it is a pity that
there is no mechanism for tidying up their references, as it seems to be
the best thing out there that covers all the fields.

Having said that, if google scholar can save me some typing, I'll
happilyuse it as a starting point for a bibtex entry.  I've just started
using pdfmeat -- this is nice, as given a pdf, it outputs the
corresponding bibtex entry from google scholar.  Probably works similar
to the way zotero does it, but can be used straight from the command
line:

  http://code.google.com/p/pdfmeat/

(Warning: I couldn't get one of the python dependencies, unidecode, to
work on mac, but it does work on ubuntu for me.)

> accessed by bibsnarf are limited to math and sciences. Since I use
> biblatex together with the Chicago Manual of Style, any bibtex entry I
> clip has to be edited and tweaked substantially. (Indeed, manual editing
> is unavoidable when using biblatex.)

If its not too tangential, why do you use biblatex -- is it the future
for bibtex?

Thanks for summarising your workflow, very helpful.

Stephen

Re: zotero (or mendeley) integration with org

[Matt Lundin]

Stephen Eglen <S.J.Eglen@damtp.cam.ac.uk> writes:

>> Agreed. Google Scholar citations need very close proofreading, as they
>> can be erroneous or poorly formatted. 
>
> Thanks Matt - I'd agree with this, having seen oddities from google
> scholar.  I emailed them ages ago about one problem (formatting of
> initials in author names), but never heard back... it is a pity that
> there is no mechanism for tidying up their references, as it seems to be
> the best thing out there that covers all the fields.
>
> Having said that, if google scholar can save me some typing, I'll
> happilyuse it as a starting point for a bibtex entry.  I've just started
> using pdfmeat -- this is nice, as given a pdf, it outputs the
> corresponding bibtex entry from google scholar.  Probably works similar
> to the way zotero does it, but can be used straight from the command
> line:
>
>   http://code.google.com/p/pdfmeat/
>

Thanks for the link! That looks like a useful tool.

>> accessed by bibsnarf are limited to math and sciences. Since I use
>> biblatex together with the Chicago Manual of Style, any bibtex entry I
>> clip has to be edited and tweaked substantially. (Indeed, manual editing
>> is unavoidable when using biblatex.)
>
> If its not too tangential, why do you use biblatex -- is it the future
> for bibtex?

I use biblatex because I use citation styles in the humanities
(especially the Chicago Manual of Style). Biblatex and the chicago-notes
package (both now part of TeXLive) handle Chicago Style footnotes and
bibliographies beautifully, with an astounding number of options and
flawless formatting -- but the bibtex entries are a bit fussier than
standard bibtex.

Best,
Matt

Re: Re: zotero (or mendeley) integration with org

[Alan E. Davis]

[-- Attachment #1: Type: text/plain, Size: 2158 bytes --]

Is it possible to use org babel to extract bibtex entries from file of notes
to a *.bib file?

The stumbling point for me in saving bibtex sources is I don't see a way to
use the file as a bibtex *.bib file so as to use that as the direct source
for the publication.  Perhaps this could be automated with babel?

Alan

On Fri, Apr 1, 2011 at 6:13 AM, Matt Lundin <mdl@imapmail.org> wrote:

> Stephen Eglen <S.J.Eglen@damtp.cam.ac.uk> writes:
>
> >> Agreed. Google Scholar citations need very close proofreading, as they
> >> can be erroneous or poorly formatted.
> >
> > Thanks Matt - I'd agree with this, having seen oddities from google
> > scholar.  I emailed them ages ago about one problem (formatting of
> > initials in author names), but never heard back... it is a pity that
> > there is no mechanism for tidying up their references, as it seems to be
> > the best thing out there that covers all the fields.
> >
> > Having said that, if google scholar can save me some typing, I'll
> > happilyuse it as a starting point for a bibtex entry.  I've just started
> > using pdfmeat -- this is nice, as given a pdf, it outputs the
> > corresponding bibtex entry from google scholar.  Probably works similar
> > to the way zotero does it, but can be used straight from the command
> > line:
> >
> >   http://code.google.com/p/pdfmeat/
> >
>
> Thanks for the link! That looks like a useful tool.
>
> >> accessed by bibsnarf are limited to math and sciences. Since I use
> >> biblatex together with the Chicago Manual of Style, any bibtex entry I
> >> clip has to be edited and tweaked substantially. (Indeed, manual editing
> >> is unavoidable when using biblatex.)
> >
> > If its not too tangential, why do you use biblatex -- is it the future
> > for bibtex?
>
> I use biblatex because I use citation styles in the humanities
> (especially the Chicago Manual of Style). Biblatex and the chicago-notes
> package (both now part of TeXLive) handle Chicago Style footnotes and
> bibliographies beautifully, with an astounding number of options and
> flawless formatting -- but the bibtex entries are a bit fussier than
> standard bibtex.
>
> Best,
> Matt
>
>

[-- Attachment #2: Type: text/html, Size: 2882 bytes --]

Re: Re: zotero (or mendeley) integration with org

[Eric S Fraga]

"Alan E. Davis" <lngndvs@gmail.com> writes:

> Is it possible to use org babel to extract bibtex entries from file of notes
> to a *.bib file?
>
> The stumbling point for me in saving bibtex sources is I don't see a way to
> use the file as a bibtex *.bib file so as to use that as the direct source
> for the publication.  Perhaps this could be automated with babel?

Would tangling do what you want?  Seems to work for me:

--8<---------------cut here---------------start------------->8---
#+babel: :tangle hens.bib

* heat exchanger network synthesis
*** Rewriting grammar for HENS with splitting (Fraga, 2009)
#+begin_src bib
@article{fraga-2009a,
	title = {A rewriting grammar for heat exchanger network structure evolution with stream splitting},
	volume = 41,
	issn = {{0305-215X}},
	doi = {10.1080/03052150903070153},
	number = 9,
	journal = {Engineering Optimization},
	author = {Eric S. Fraga},
	year = 2009,
	pages = {813-831}
}
#+end_src
--8<---------------cut here---------------end--------------->8---

Put this in a file, t.org say, and hit =C-c C-v t= (=org-babel-tangle=)
and it should create =hens.bib=.  Sorry for the self-citation here ;-)

You could also put the actual tangle destination on each src block, in
case you want to tangle to more than one file from the same org file.

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.128.ga9e6)

Re: Re: zotero (or mendeley) integration with org

[Alan E. Davis]

[-- Attachment #1: Type: text/plain, Size: 99 bytes --]

I have been enlightened.  This closes a loop for my handling of literature.

Thank you both.

alan

[-- Attachment #2: Type: text/html, Size: 116 bytes --]

Re: Re: zotero (or mendeley) integration with org

[Alan E. Davis]

[-- Attachment #1: Type: text/plain, Size: 631 bytes --]

It's in my mind to find a way to use orgmode for organizing pdfs and BibTex
data.  I haven't untangled storage of PDFs and linking to BibTeX, and I
haven't found a solution to organizing it all through orgmode.

An important piece of the puzzle, though, needs mention: cb2bib helps
semi-automate making a BibTeX entry from a citation, or Google Scholar
BibTeX output.

I wonder if it would help to use orgmode for bibtex *.bib files.  I think
comments can be included in those files.  Or does it also work the other way
around, that any file can be used as a bibtex source database?

Matt's workflow makes sense.

Alan Davis
>
>
>

[-- Attachment #2: Type: text/html, Size: 892 bytes --]

Re: Re: zotero (or mendeley) integration with org

[Cian]

I just use refmode to insert bib citations into my org-files. Both
Zotero and Mendeley can export bibtex files, so that's one method of
semi-automating the process (Mendeley will automatically update the
file for you - not sure about Zotero).

On Wed, Mar 30, 2011 at 1:54 AM, Alan E. Davis <lngndvs@gmail.com> wrote:
> It's in my mind to find a way to use orgmode for organizing pdfs and BibTex
> data.  I haven't untangled storage of PDFs and linking to BibTeX, and I
> haven't found a solution to organizing it all through orgmode.
>
> An important piece of the puzzle, though, needs mention: cb2bib helps
> semi-automate making a BibTeX entry from a citation, or Google Scholar
> BibTeX output.
>
> I wonder if it would help to use orgmode for bibtex *.bib files.  I think
> comments can be included in those files.  Or does it also work the other way
> around, that any file can be used as a bibtex source database?
>
> Matt's workflow makes sense.
>
> Alan Davis
>
>
>

Re: Re: zotero (or mendeley) integration with org

[Joost Kremers]

On Wed, Mar 30, 2011 at 10:54:58AM +1000, Alan E. Davis wrote:
> I wonder if it would help to use orgmode for bibtex *.bib files.  I think
> comments can be included in those files.

yes. there is a @comment command, but bibtex ignores everything that's not
inside an @<entry>, @string or @preamble command, so that you could create an
org file that contains bibtex entries and use that as your bibliography file.

however, i don't know if biblatex+biber is equally forgiving.

>  Or does it also work the other way
> around, that any file can be used as a bibtex source database?

in essence yes, as long as it's a text file and contains valid bibtex entries.
(note that bibtex isn't unicode-aware; i have no idea how bibtex reacts to files
containing unicode characters in text it's ignoring.)


-- 
Joost Kremers
Life has its moments