* contributing to work on citations @ 2015-10-30 20:17 Matt Price 2015-11-04 15:18 ` Aaron Ecay 0 siblings, 1 reply; 11+ messages in thread From: Matt Price @ 2015-10-30 20:17 UTC (permalink / raw) To: Org Mode [-- Attachment #1: Type: text/plain, Size: 1878 bytes --] So, from the "some projects" thread I have the sense there is a group of users with some interest in improving citation support. What is the best way to kickstart that process? it seems to me that, at a minimum, the following is needed: 1. Finalization of the citation syntax. IIUC, This involves refining and merging the work in the wip-cite git branch. 2. Choice of a citation creation tool IIUC, the two main candidates are pandoc-citeproc and citeproc-js/node. Pandoc-citeproc works well but requires users to install the whole pandoc infrastructure, and needs to be wrapped in a thin tool for use with org. Citeproc-node has fewer dependencies (I guess), but needs to be updated to support additional output formats (e.g. latex, odt, and org) and is a little more elaborate to run from the command line (runs as a service, can only be communicated with over http). 3. Handling of citation links in the various export engines Functions must be written to handle citations in, at minimum, latex, html, and org 4. Interface with backends Org should be able to talk to bibtex and zotero databases, at least (and maybe more). A lot of this work has already been done. Richard has written org-citeproc as a wrapper for pandoc-citeproc https://github.com/wyleyr/org-citeproc . Aaron has written some functions for org-citeproc in the wip-cite-awe branch. Nicolas has developed the syntax in wip-cite. I even have a fork of citeproc-node that could be used to develop the missing features we need. And of course John has org-ref ( http://kitchingroup.cheme.cmu.edu/blog/2014/05/13/Using-org-ref-for-citations-and-references/) while Erik H has zotxt-emacs for Zotero integration ( https://github.com/egh/zotxt-emacs). What would be the best next move? It seems to me there are enough moving parts that the process will be quite a bit easier if we co-ordinate. thanks, Matt [-- Attachment #2: Type: text/html, Size: 2356 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: contributing to work on citations 2015-10-30 20:17 contributing to work on citations Matt Price @ 2015-11-04 15:18 ` Aaron Ecay 2015-11-04 19:37 ` Will Monroe ` (2 more replies) 0 siblings, 3 replies; 11+ messages in thread From: Aaron Ecay @ 2015-11-04 15:18 UTC (permalink / raw) To: Matt Price, Org Mode Hi Matt, Thanks for your willingness to help with the project. 2015ko urriak 30an, Matt Price-ek idatzi zuen: > > So, from the "some projects" thread I have the sense there is a group of > users with some interest in improving citation support. What is the best > way to kickstart that process? it seems to me that, at a minimum, the > following is needed: > > 1. Finalization of the citation syntax. > > IIUC, This involves refining and merging the work in the wip-cite git > branch. There was a very voluminous discussion on this last time. Lots of people proposed various additions. I’d be inclined to go with whatever syntax is implemented currently, and let any needed extensions sort themselves out through real-world testing. > > 2. Choice of a citation creation tool > > IIUC, the two main candidates are pandoc-citeproc and citeproc-js/node. > Pandoc-citeproc works well but requires users to install the whole pandoc > infrastructure, and needs to be wrapped in a thin tool for use with org. > Citeproc-node has fewer dependencies (I guess), but needs to be updated to > support additional output formats (e.g. latex, odt, and org) and is a > little more elaborate to run from the command line (runs as a service, can > only be communicated with over http). IIUC, citeproc-node just wraps citeproc-js, which has a “normal” API. So it should be possible to write a CLI wrapper for citeproc-js. (But I have not studied citeproc-js at all, so I could be wrong). My plan is to create preliminary citation support based on Richard’s haskell tool, with the intention of moving to a JS-based tool when possible. So implementing a work-alike of Richard’s tool in JS would be a good development. > > 3. Handling of citation links in the various export engines > Functions must be written to handle citations in, at minimum, latex, html, > and org > > 4. Interface with backends > Org should be able to talk to bibtex and zotero databases, at least (and > maybe more). The code I wrote should have enough hooks to enable this. But I haven’t looked at zotero at all. If zotero support is important to you, you could investigate the “lookup types” in the citation code I wrote, see if they are adequate, and try to use them to implement zotero lookup. From the docstring: "Types of citation lookup backends. Alist from type to list of: - Function called at the beginning of export, with the rest of the keyword line after #+BIBDB: type, and the info plist. Should cache whatever it needs in the info plist. - Function to lookup a citation. Called with the key and the info plist. Will be memoized by `org-export-cite--lookup' (TODO). Should return an alist of keys and values about the citation (author, year, title, etc.) - A boolean; non-nil = this lookup type is remote. All local lookups will be tried before any remote one is. TODO: not yet implemented" > > A lot of this work has already been done. Richard has written org-citeproc > as a wrapper for pandoc-citeproc https://github.com/wyleyr/org-citeproc . > Aaron has written some functions for org-citeproc in the wip-cite-awe > branch. Nicolas has developed the syntax in wip-cite. I even have a fork of > citeproc-node that could be used to develop the missing features we need. > And of course John has org-ref ( > http://kitchingroup.cheme.cmu.edu/blog/2014/05/13/Using-org-ref-for-citations-and-references/) The org-ref features provide a good aspirational horizon for us, I think. I.e. org’s core should aim to support almost all of them eventually. But the support for citation syntax is not so far along that creating such support is an immediate prospect. > while Erik H has zotxt-emacs for Zotero integration ( > https://github.com/egh/zotxt-emacs). > > What would be the best next move? It seems to me there are enough moving > parts that the process will be quite a bit easier if we co-ordinate. Based on the interests you’ve expressed, I think working on the JS citation processor and/or zotero integration would be good contributions you can make. But you can work on other things as well. I think the best way to coordinate will be to keep the list apprised of your progress. If you have push access, you can push WIP code as branches in the org-mode repository. (Otherwise, you can use github or some other git host, or ask for push access.) Thanks again, -- Aaron Ecay ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: contributing to work on citations 2015-11-04 15:18 ` Aaron Ecay @ 2015-11-04 19:37 ` Will Monroe 2015-11-04 19:49 ` William Denton 2015-11-04 20:20 ` John Wiegley 2015-11-04 22:25 ` Matt Price 2015-11-06 1:41 ` Matt Price 2 siblings, 2 replies; 11+ messages in thread From: Will Monroe @ 2015-11-04 19:37 UTC (permalink / raw) To: Aaron Ecay; +Cc: Org Mode Aaron, Matt: > Based on the interests you’ve expressed, I think working on the JS > citation processor and/or zotero integration would be good contributions > you can make. But you can work on other things as well. I think the > best way to coordinate will be to keep the list apprised of your > progress. As someone who uses Zotero and org-mode quite a bit, I'd be glad to help with testing, if that would be useful. I'm learning Elisp but I'm still a beginner. Nevertheless, I have a keen interest in bringing together these tools. All the best, Will ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: contributing to work on citations 2015-11-04 19:37 ` Will Monroe @ 2015-11-04 19:49 ` William Denton 2015-11-04 20:20 ` John Wiegley 1 sibling, 0 replies; 11+ messages in thread From: William Denton @ 2015-11-04 19:49 UTC (permalink / raw) To: Org Mode [-- Attachment #1: Type: TEXT/PLAIN, Size: 398 bytes --] On 4 November 2015, Will Monroe wrote: > As someone who uses Zotero and org-mode quite a bit, I'd be glad to help > with testing, if that would be useful. I'm learning Elisp but I'm still > a beginner. Nevertheless, I have a keen interest in bringing together > these tools. Same here---I'm glad to test. Bill -- William Denton ↔ Toronto, Canada ↔ https://www.miskatonic.org/ ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: contributing to work on citations 2015-11-04 19:37 ` Will Monroe 2015-11-04 19:49 ` William Denton @ 2015-11-04 20:20 ` John Wiegley 1 sibling, 0 replies; 11+ messages in thread From: John Wiegley @ 2015-11-04 20:20 UTC (permalink / raw) To: Will Monroe; +Cc: Aaron Ecay, Org Mode >>>>> Will Monroe <wtmonroe.ls@gmail.com> writes: > As someone who uses Zotero and org-mode quite a bit, I'd be glad to help > with testing, if that would be useful. I'm learning Elisp but I'm still a > beginner. Nevertheless, I have a keen interest in bringing together these > tools. Me too, I'm a fan and active user of both tools as well. John ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: contributing to work on citations 2015-11-04 15:18 ` Aaron Ecay 2015-11-04 19:37 ` Will Monroe @ 2015-11-04 22:25 ` Matt Price 2015-11-06 1:41 ` Matt Price 2 siblings, 0 replies; 11+ messages in thread From: Matt Price @ 2015-11-04 22:25 UTC (permalink / raw) To: Matt Price, Org Mode [-- Attachment #1: Type: text/plain, Size: 5665 bytes --] On Wed, Nov 4, 2015 at 10:18 AM, Aaron Ecay <aaronecay@gmail.com> wrote: > Hi Matt, > > Thanks for your willingness to help with the project. > > 2015ko urriak 30an, Matt Price-ek idatzi zuen: > > > > So, from the "some projects" thread I have the sense there is a group of > > users with some interest in improving citation support. What is the best > > way to kickstart that process? it seems to me that, at a minimum, the > > following is needed: > > > > 1. Finalization of the citation syntax. > > > > IIUC, This involves refining and merging the work in the wip-cite git > > branch. > > There was a very voluminous discussion on this last time. Lots of > people proposed various additions. I’d be inclined to go with whatever > syntax is implemented currently, and let any needed extensions sort > themselves out through real-world testing. > OK, great. Does this mean just checking out the org-wip-cite branch? Or has someone perhaps merged more recent changes into wip-cite somewhere else? > > > > > 2. Choice of a citation creation tool > > > > IIUC, the two main candidates are pandoc-citeproc and citeproc-js/node. > > Pandoc-citeproc works well but requires users to install the whole pandoc > > infrastructure, and needs to be wrapped in a thin tool for use with org. > > Citeproc-node has fewer dependencies (I guess), but needs to be updated > to > > support additional output formats (e.g. latex, odt, and org) and is a > > little more elaborate to run from the command line (runs as a service, > can > > only be communicated with over http). > > IIUC, citeproc-node just wraps citeproc-js, which has a “normal” API. > So it should be possible to write a CLI wrapper for citeproc-js. (But I > have not studied citeproc-js at all, so I could be wrong). > I am not sure but I don't think it's trivial; and alas i am slow at eveyr kind of coding, but I will certainly look into this. > > My plan is to create preliminary citation support based on Richard’s > haskell tool, with the intention of moving to a JS-based tool when > possible. So implementing a work-alike of Richard’s tool in JS would be > a good development. > > OK, I'll pay more attention to Richard's tool and how it works, with the goal of making something like a drop-in replacement. > > > > 3. Handling of citation links in the various export engines > > Functions must be written to handle citations in, at minimum, latex, > html, > > and org > > > > 4. Interface with backends > > Org should be able to talk to bibtex and zotero databases, at least (and > > maybe more). > > The code I wrote should have enough hooks to enable this. But I haven’t > looked at zotero at all. If zotero support is important to you, you > could investigate the “lookup types” in the citation code I wrote, see > if they are adequate, and try to use them to implement zotero lookup. > From the docstring: > > "Types of citation lookup backends. > > Alist from type to list of: > > - Function called at the beginning of export, with the rest of > the keyword line after #+BIBDB: type, and the info plist. > Should cache whatever it needs in the info plist. > > - Function to lookup a citation. Called with the key and the > info plist. Will be memoized by > `org-export-cite--lookup' (TODO). Should return an alist of > keys and values about the citation (author, year, title, etc.) > > - A boolean; non-nil = this lookup type is remote. All local > lookups will be tried before any remote one is. > TODO: not yet implemented" > > I'll start here. Thanks for the pointer. > > > > A lot of this work has already been done. Richard has written > org-citeproc > > as a wrapper for pandoc-citeproc https://github.com/wyleyr/org-citeproc > . > > Aaron has written some functions for org-citeproc in the wip-cite-awe > > branch. Nicolas has developed the syntax in wip-cite. I even have a fork > of > > citeproc-node that could be used to develop the missing features we need. > > And of course John has org-ref ( > > > http://kitchingroup.cheme.cmu.edu/blog/2014/05/13/Using-org-ref-for-citations-and-references/ > ) > > The org-ref features provide a good aspirational horizon for us, I think. > I.e. org’s core should aim to support almost all of them eventually. But > the support for citation syntax is not so far along that creating such > support is an immediate prospect. > > > while Erik H has zotxt-emacs for Zotero integration ( > > https://github.com/egh/zotxt-emacs). > > > > What would be the best next move? It seems to me there are enough moving > > parts that the process will be quite a bit easier if we co-ordinate. > > Based on the interests you’ve expressed, I think working on the JS > citation processor and/or zotero integration would be good contributions > you can make. But you can work on other things as well. I think the > best way to coordinate will be to keep the list apprised of your > progress. If you have push access, you can push WIP code as branches in > the org-mode repository. (Otherwise, you can use github or some other > git host, or ask for push access.) > > OK, will do. I think it would be great if you, Nicolas, Richard, and Erik H if he's still interested, could also keep the rest of us up to date with your efforts -- in all likelihood you will be moving faster than me (though I can speak for Will, Bill and John). I'll try to stay current with what you're doing and make what contributtions I can. thanks Aaron! > Thanks again, > > -- > Aaron Ecay > [-- Attachment #2: Type: text/html, Size: 7595 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: contributing to work on citations 2015-11-04 15:18 ` Aaron Ecay 2015-11-04 19:37 ` Will Monroe 2015-11-04 22:25 ` Matt Price @ 2015-11-06 1:41 ` Matt Price 2015-11-06 13:10 ` Aaron Ecay 2015-11-08 11:17 ` Rasmus 2 siblings, 2 replies; 11+ messages in thread From: Matt Price @ 2015-11-06 1:41 UTC (permalink / raw) To: Matt Price, Org Mode [-- Attachment #1: Type: text/plain, Size: 1901 bytes --] On Wed, Nov 4, 2015 at 10:18 AM, Aaron Ecay <aaronecay@gmail.com> wrote: > > > > 3. Handling of citation links in the various export engines > > Functions must be written to handle citations in, at minimum, latex, > html, > > and org > > > > 4. Interface with backends > > Org should be able to talk to bibtex and zotero databases, at least (and > > maybe more). > > The code I wrote should have enough hooks to enable this. But I haven’t > looked at zotero at all. If zotero support is important to you, you > could investigate the “lookup types” in the citation code I wrote, see > if they are adequate, and try to use them to implement zotero lookup. > From the docstring: > > "Types of citation lookup backends. > > Alist from type to list of: > > - Function called at the beginning of export, with the rest of > the keyword line after #+BIBDB: type, and the info plist. > Should cache whatever it needs in the info plist. > > - Function to lookup a citation. Called with the key and the > info plist. Will be memoized by > `org-export-cite--lookup' (TODO). Should return an alist of > keys and values about the citation (author, year, title, etc.) > > - A boolean; non-nil = this lookup type is remote. All local > lookups will be tried before any remote one is. > TODO: not yet implemented" > Hey, Aaron, Am I right that I need to check out your wip-cite-awe branch from ~8 months ago? Would it make sense to try to merge recent changes on master into that branch first? Anyway, I figure I will try to modify zotxt-emacs so it provides these functions and generates conformant cite: links. That seems the easiest solution, and maybe Erik will give me some help once I've made some initial progress. Meanwhile I will try to learn enough about javascript to get started on the citeproc-js ocmmand-line tool. thx, matt [-- Attachment #2: Type: text/html, Size: 2424 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: contributing to work on citations 2015-11-06 1:41 ` Matt Price @ 2015-11-06 13:10 ` Aaron Ecay 2015-11-06 17:14 ` Matt Price 2015-11-08 11:17 ` Rasmus 1 sibling, 1 reply; 11+ messages in thread From: Aaron Ecay @ 2015-11-06 13:10 UTC (permalink / raw) To: Matt Price, Org Mode Hi Matt, 2015ko azaroak 6an, Matt Price-ek idatzi zuen: > Hey, Aaron, > Am I right that I need to check out your wip-cite-awe branch from ~8 months > ago? Richard Lawrence forked that branch, and made some changes on top of it, so you should probably start with his branch. His changes aren’t in the org-mode repo but rather github: <https://github.com/wyleyr/org-mode.git> (That should be a url suitable for git clone or git remote add; the branch with his work in it is called wip-cite-org-citeproc and not master). > Would it make sense to try to merge recent changes on master into that > branch first? That’s a good first step. Hopefully any conflicts will be minor, but don’t hesitate to shout for help if there are problems! > > Anyway, I figure I will try to modify zotxt-emacs so it provides these > functions and generates conformant cite: links. NB the new citation support doesn’t use links à la org-ref, but rather a dedicated syntax object (maybe that’s what you meant, but calling the new citations “links” is not technically correct). > That seems the easiest solution, and maybe Erik will give me some help > once I've made some initial progress. Meanwhile I will try to learn > enough about javascript to get started on the citeproc-js ocmmand-line > tool. thx, matt Thanks, -- Aaron Ecay ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: contributing to work on citations 2015-11-06 13:10 ` Aaron Ecay @ 2015-11-06 17:14 ` Matt Price 2015-11-22 20:32 ` Richard Lawrence 0 siblings, 1 reply; 11+ messages in thread From: Matt Price @ 2015-11-06 17:14 UTC (permalink / raw) To: Matt Price, Org Mode [-- Attachment #1: Type: text/plain, Size: 1473 bytes --] On Fri, Nov 6, 2015 at 8:10 AM, Aaron Ecay <aaronecay@gmail.com> wrote: > Hi Matt, > > 2015ko azaroak 6an, Matt Price-ek idatzi zuen: > > > Hey, Aaron, > > Am I right that I need to check out your wip-cite-awe branch from ~8 > months > > ago? > > Richard Lawrence forked that branch, and made some changes on top of it, > so you should probably start with his branch. His changes aren’t in the > org-mode repo but rather github: <https://github.com/wyleyr/org-mode.git> > (That should be a url suitable for git clone or git remote add; the branch > with his work in it is called wip-cite-org-citeproc and not master). > > > Would it make sense to try to merge recent changes on master into that > > branch first? > > That’s a good first step. Hopefully any conflicts will be minor, but > don’t hesitate to shout for help if there are problems! > > OK, I tried rebasing on current master, is online here & pull request has been sent ot richard: https://github.com/titaniumbones/org-mode > > > > Anyway, I figure I will try to modify zotxt-emacs so it provides these > > functions and generates conformant cite: links. > > NB the new citation support doesn’t use links à la org-ref, but rather a > dedicated syntax object (maybe that’s what you meant, but calling the > new citations “links” is not technically correct). > > Right. Of course. Thank you! I'll try to get started on this in the next few days. m [-- Attachment #2: Type: text/html, Size: 2326 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: contributing to work on citations 2015-11-06 17:14 ` Matt Price @ 2015-11-22 20:32 ` Richard Lawrence 0 siblings, 0 replies; 11+ messages in thread From: Richard Lawrence @ 2015-11-22 20:32 UTC (permalink / raw) To: emacs-orgmode Hi Matt and all, Matt Price <moptop99@gmail.com> writes: >> OK, I tried rebasing on current master, is online here & pull request has >> been sent ot richard: > > https://github.com/titaniumbones/org-mode Sorry it took me so long to get around to this. Because there have been commits to master in the meantime, and because I wanted to clean up the (unnecessarily complicated) history of my branch, I have performed another rebase of my branch onto master and put the result in a new branch, here: https://github.com/wyleyr/org-mode/tree/wip-cite-rebase This branch linearizes the history of the various changes that went into org-element (parsing citations) and org-cite (handling citations), and cleans up my commit messages, which were produced by an old magit and pretty ugly to deal with. Hopefully that will make it easier for someone to continue this work! Best, Richard ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: contributing to work on citations 2015-11-06 1:41 ` Matt Price 2015-11-06 13:10 ` Aaron Ecay @ 2015-11-08 11:17 ` Rasmus 1 sibling, 0 replies; 11+ messages in thread From: Rasmus @ 2015-11-08 11:17 UTC (permalink / raw) To: emacs-orgmode Hi, Thanks for looking at this, Matt and Aaron. Matt Price <moptop99@gmail.com> writes: > Anyway, I figure I will try to modify zotxt-emacs so it provides these > functions and generates conformant cite: links. That seems the easiest > solution, and maybe Erik will give me some help once I've made some initial > progress. Meanwhile I will try to learn enough about javascript to get > started on the citeproc-js ocmmand-line tool. thx, I use JS sometimes, and FWIW I thinking your plan of altering citeproc-js upstream is the best approach proposed thus far. ATM I don’t have much time, and I having a hard time forming a overview of where to aid wrt. adding citation support. Rasmus -- And when I’m finished thinking, I have to die a lot ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2015-11-22 20:32 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-10-30 20:17 contributing to work on citations Matt Price 2015-11-04 15:18 ` Aaron Ecay 2015-11-04 19:37 ` Will Monroe 2015-11-04 19:49 ` William Denton 2015-11-04 20:20 ` John Wiegley 2015-11-04 22:25 ` Matt Price 2015-11-06 1:41 ` Matt Price 2015-11-06 13:10 ` Aaron Ecay 2015-11-06 17:14 ` Matt Price 2015-11-22 20:32 ` Richard Lawrence 2015-11-08 11:17 ` Rasmus
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).