* Fwd: [RFC] Self-configuring Org-mode files [not found] <t5xd3p8qlct.fsf@rgc.damtp.cam.ac.uk> @ 2010-12-12 3:49 ` Thomas S. Dye 2012-01-12 12:38 ` Rsquared for reproductible research Stephen Eglen 0 siblings, 1 reply; 9+ messages in thread From: Thomas S. Dye @ 2010-12-12 3:49 UTC (permalink / raw) To: emacs-orgmode ml [-- Attachment #1.1: Type: text/plain, Size: 5085 bytes --] Forwarding off-list replies from Stephen Eglen, most recent at the top. > One caveat of course with the idea: what software other than emacs > should be assumed to be available? e.g. if someone has some R code in > the doc, you'll need R on the system, likewise, perl, python. (I > considered whether it was worth setting up a system for processing > Sweave documents -- but a colleague pointed out that unless you can > preload R with most of the popular R packages that are likely to be > needed in a Sweave document, some form of intervention/curation is > always needed.) Begin forwarded message: > From: Stephen Eglen <S.J.Eglen@damtp.cam.ac.uk> > Date: December 11, 2010 7:18:26 AM HST > To: "Thomas S. Dye" <tsd@tsdye.com> > Cc: S.J.Eglen@damtp.cam.ac.uk > Subject: Re: [RFC] Self-configuring Org-mode files > > Hi Tom, > > I think you have hit the nail on the head -- it took me ages to get > org > up and running with all the extra config to generate a beamer doc. > part > of the trouble in that case was that i was following emails that > were a > year old, but still I think the problem you outline is clear. As to > the > solution, your ideas sound sensible. > > To add one extra suggestion: how about a web-service of some sort > where > you upload your .org file, and it returns the 'output' (e.g. pdf doc) > produced by a native Emacs 23.X session running a stable version of > org > with zero extra configuration? I realise that it may be a lot of work > to produce such a web service and expensive to maintain, but those > issues aside, having a "vanilla" system to test RR org documents on > would > be great. > > best wishes, Stephen > > "Thomas S. Dye" <tsd@tsdye.com> writes: > >> Aloha all, >> >> I'm eager to learn how to build a self-configuring Org-mode file. >> >> I think Org-mode is uniquely positioned to produce reproducible >> research >> documents, but frankly speaking, the author of a reproducible Org- >> mode >> research document would be foolish to release it. The probability >> that it >> would run out-of-the-box on a reader's computer is quite a bit less >> than 1. >> Multiply this fraction by the number of potential readers and the >> probability >> that the author will receive one or more emails entitled something >> like "Your >> reproducible research isn't reproducible" rapidly approaches 1. >> >> A desire to produce reproducible research with Org-mode shouldn't >> reduce to an >> open-ended commitment to solve configuration problems on users' >> computers >> around the world. An Org-mode file that reliably self-configures >> might solve >> this problem. >> >> Based on the kind comments of several list members, I've come up >> with what I >> hope are the rudiments of such a self-configuring Org-mode file. I'm >> requesting your comments because I believe, individually and >> collectively, you >> know way more about this than I do. >> >> 1) At the end of the Org-mode file are some instructions saying >> something like >> "To use this document, first >> evaluate this code block", as suggested by Dan Davison: >> >> # Local variables: >> # eval:(sbe "essential-document-config") >> # End: >> >> 2) the source code block "essential-document-config" would consist >> of noweb >> calls to Library of Babel functions: >> >> #+source: essential-document-config >> #+begin_src emacs-lisp :noweb yes >> <<lob-set-this-variable(x=t)>> >> <<lob-set-that-variable(x=nil)>> >> #+end_src >> >> 3) The Library of Babel would be >> populated with little functions, each of which sets a single >> variable in a buffer-local way >> (as suggested several times to me on the list): >> >> #+source: lob-set-this-variable >> #+begin_src emacs-lisp :var x=nil >> (set (make-local-variable 'this-variable) x) >> #+end_src >> >> 4) A super-function in the Library of Babel would set the buffer- >> local >> instance of every relevant Org-mode variable to its default state: >> >> #+source: lob-set-local-defaults >> #+begin_src emacs-lisp >> ... >> #+end_src >> >> 5) >> The Org-mode community would be responsible for populating the >> Library of Babel with the relevant little functions--whenever a new >> configuration variable is introduced, then a little function for it >> would be deposited in the Library of Babel. >> The name of the configuration function would be part of the >> variable's >> docstring. >> >> 6) The author of a reproducible Org-mode document could reasonably >> expect to >> produce a document that self-configures on computers around the >> world, in a >> way that is safe and effective. The author would call the super- >> function, >> then add noweb calls to little functions until the reproducible >> research >> document works as expected. >> >> I'm happy for any comments, positive or negative. >> >> All the best, >> Tom >> >> _______________________________________________ >> Emacs-orgmode mailing list >> Please use `Reply All' to send replies to the list. >> Emacs-orgmode@gnu.org >> http://lists.gnu.org/mailman/listinfo/emacs-orgmode [-- Attachment #1.2: Type: text/html, Size: 10021 bytes --] [-- Attachment #2: Type: text/plain, Size: 201 bytes --] _______________________________________________ Emacs-orgmode mailing list Please use `Reply All' to send replies to the list. Emacs-orgmode@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-orgmode ^ permalink raw reply [flat|nested] 9+ messages in thread
* Rsquared for reproductible research 2010-12-12 3:49 ` Fwd: [RFC] Self-configuring Org-mode files Thomas S. Dye @ 2012-01-12 12:38 ` Stephen Eglen 2012-01-12 15:27 ` brian powell 0 siblings, 1 reply; 9+ messages in thread From: Stephen Eglen @ 2012-01-12 12:38 UTC (permalink / raw) To: emacs-orgmode Following on from an old thread about self-configuring org files for reproducible research, R users might be interested to see the following web site which is exactly what I was thinking of for org mode (but of course, works "only" for R packages.) http://rsquared.stat.uni-muenchen.de/index.rhtml Stephen ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Rsquared for reproductible research 2012-01-12 12:38 ` Rsquared for reproductible research Stephen Eglen @ 2012-01-12 15:27 ` brian powell 2012-01-12 15:40 ` Stephen Eglen 0 siblings, 1 reply; 9+ messages in thread From: brian powell @ 2012-01-12 15:27 UTC (permalink / raw) To: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 1436 bytes --] Wow! http://rsquared.stat.uni-muenchen.de/index.rhtml looks great--very interesting--thanks for the heads-up on that link. Worked with R/S/S-PLUS in grad school--easily my favorite language/system--especially like its ease of extensibility--reminds me of EMACS LISP! I recognized your name, Stephen, as the long-time maintainer of the "Emacs Lisp List"--thanks for that too. Are you envisioning a repository beyond "Emacs Lisp List" for OrgMode implementations and/or Babel examples. Or maybe optional extensions to OrgMode itself? Both? Also, I very much agree that a "near exact replica" of the http://rsquared.stat.uni-muenchen.de/index.rhtml for OrgMode would be great. Read the 3 papers on the site and came across this reference that may be interesting to OrgMode/R/Literate Programming persons: http://cran.r-project.org/web/packages/roxygen/index.html Maybe that could be a focus of such a site if it were made?: OrgMode <=> LiterateProgramming <=> R/Bable/whatever other languages. --where such topics intersect. On Thu, Jan 12, 2012 at 7:38 AM, Stephen Eglen <sje30@cam.ac.uk> wrote: > Following on from an old thread about self-configuring org files for > reproducible research, R users might be interested to see the following > web site > which is exactly what I was thinking of for org mode (but of course, works > "only" for R packages.) > > http://rsquared.stat.uni-muenchen.de/index.rhtml > > Stephen > > > > [-- Attachment #2: Type: text/html, Size: 2296 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Rsquared for reproductible research 2012-01-12 15:27 ` brian powell @ 2012-01-12 15:40 ` Stephen Eglen 2012-01-12 16:29 ` Eric Schulte 0 siblings, 1 reply; 9+ messages in thread From: Stephen Eglen @ 2012-01-12 15:40 UTC (permalink / raw) To: emacs-orgmode; +Cc: Stephen Eglen > I recognized your name, Stephen, as the long-time maintainer of the "Emacs > Lisp List"--thanks for that too. > Are you envisioning a repository beyond "Emacs Lisp List" for OrgMode > implementations and/or Babel examples. I am interested in maintaining a collection of nice org-babel-R examples, given my interest in R and Emacs; I made a small start last year for my tutorial notes on ESS for the useR meeting, but I'd like to update that. The key problem I found with learning org-babel was worked examples, given that the syntax was changing rapidly. I hope that has now stabilised, and I should followup with Eric about this. (I also hope to kill off the Emacs Lisp List once the ELPA has taken off.) > Also, I very much agree that a "near exact replica" of the http:// > rsquared.stat.uni-muenchen.de/index.rhtml for OrgMode would be great. Yes! Any takers?!? > http://cran.r-project.org/web/packages/roxygen/index.html Note that ESS has limited support for editing roxygen tags. Stephen ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Rsquared for reproductible research 2012-01-12 15:40 ` Stephen Eglen @ 2012-01-12 16:29 ` Eric Schulte 2012-01-12 18:24 ` Thomas S. Dye 2012-01-12 20:54 ` Stephen Eglen 0 siblings, 2 replies; 9+ messages in thread From: Eric Schulte @ 2012-01-12 16:29 UTC (permalink / raw) To: Stephen Eglen; +Cc: emacs-orgmode Stephen Eglen <S.J.Eglen@damtp.cam.ac.uk> writes: >> I recognized your name, Stephen, as the long-time maintainer of the "Emacs >> Lisp List"--thanks for that too. >> Are you envisioning a repository beyond "Emacs Lisp List" for OrgMode >> implementations and/or Babel examples. > > I am interested in maintaining a collection of nice org-babel-R > examples, given my interest in R and Emacs; I made a small start last > year for my tutorial notes on ESS for the useR meeting, but I'd like to > update that. The key problem I found with learning org-babel was worked > examples, given that the syntax was changing rapidly. I hope that has > now stabilised, and I should followup with Eric about this. > The syntax is now stabilized (we wanted to get this sorted before the final Emacs24 merge). That which is currently described in the manual is and should remain the proper Org-mode code block syntax. For many small examples, please see [1] which I (unfortunately) haven't been adding to recently, but I will try to once again start using for all of my small generally-mailing-list-inspired Babel one-offs. > > (I also hope to kill off the Emacs Lisp List once the ELPA has taken > off.) > >> Also, I very much agree that a "near exact replica" of the http:// >> rsquared.stat.uni-muenchen.de/index.rhtml for OrgMode would be great. > Yes! Any takers?!? > From looking at the fairly terse web site for R^2 it is not clear to me exactly what the system includes (I'm sure I'm missing something obvious). It seems to be the addition of a packaging system over-top of R source files. What would a potential Org-mode based system provide which is not already possible with Org-mode text files, Org-mode publishing and a version control repository. Perhaps the benefit would simply be a system which eases the integration of Org, publishing, version control, and possibly automatic Makefile creation with tasks like "publish", "clean" etc... Best, > >> http://cran.r-project.org/web/packages/roxygen/index.html > > Note that ESS has limited support for editing roxygen tags. > > Stephen > > Footnotes: [1] http://eschulte.me/org-scraps/ -- Eric Schulte http://cs.unm.edu/~eschulte/ ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Rsquared for reproductible research 2012-01-12 16:29 ` Eric Schulte @ 2012-01-12 18:24 ` Thomas S. Dye 2012-01-12 20:54 ` Stephen Eglen 1 sibling, 0 replies; 9+ messages in thread From: Thomas S. Dye @ 2012-01-12 18:24 UTC (permalink / raw) To: Eric Schulte; +Cc: emacs-orgmode, Stephen Eglen Eric Schulte <eric.schulte@gmx.com> writes: > Stephen Eglen <S.J.Eglen@damtp.cam.ac.uk> writes: > >>> I recognized your name, Stephen, as the long-time maintainer of the "Emacs >>> Lisp List"--thanks for that too. >>> Are you envisioning a repository beyond "Emacs Lisp List" for OrgMode >>> implementations and/or Babel examples. >> >> I am interested in maintaining a collection of nice org-babel-R >> examples, given my interest in R and Emacs; I made a small start last >> year for my tutorial notes on ESS for the useR meeting, but I'd like to >> update that. The key problem I found with learning org-babel was worked >> examples, given that the syntax was changing rapidly. I hope that has >> now stabilised, and I should followup with Eric about this. >> > > The syntax is now stabilized (we wanted to get this sorted before the > final Emacs24 merge). That which is currently described in the manual > is and should remain the proper Org-mode code block syntax. > > For many small examples, please see [1] which I (unfortunately) haven't > been adding to recently, but I will try to once again start using for > all of my small generally-mailing-list-inspired Babel one-offs. > >> >> (I also hope to kill off the Emacs Lisp List once the ELPA has taken >> off.) >> >>> Also, I very much agree that a "near exact replica" of the http:// >>> rsquared.stat.uni-muenchen.de/index.rhtml for OrgMode would be great. >> Yes! Any takers?!? >> > > From looking at the fairly terse web site for R^2 it is not clear to me > exactly what the system includes (I'm sure I'm missing something > obvious). It seems to be the addition of a packaging system over-top of > R source files. What would a potential Org-mode based system provide > which is not already possible with Org-mode text files, Org-mode > publishing and a version control repository. One thing that caught my eye was the facility that compared results across operating systems. Tom > > Perhaps the benefit would simply be a system which eases the integration > of Org, publishing, version control, and possibly automatic Makefile > creation with tasks like "publish", "clean" etc... > > Best, > >> >>> http://cran.r-project.org/web/packages/roxygen/index.html >> >> Note that ESS has limited support for editing roxygen tags. >> >> Stephen >> >> > > > Footnotes: > [1] http://eschulte.me/org-scraps/ -- Thomas S. Dye http://www.tsdye.com ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Rsquared for reproductible research 2012-01-12 16:29 ` Eric Schulte 2012-01-12 18:24 ` Thomas S. Dye @ 2012-01-12 20:54 ` Stephen Eglen 1 sibling, 0 replies; 9+ messages in thread From: Stephen Eglen @ 2012-01-12 20:54 UTC (permalink / raw) To: Eric Schulte; +Cc: emacs-orgmode, Stephen Eglen > The syntax is now stabilized (we wanted to get this sorted before the > final Emacs24 merge). That which is currently described in the manual > is and should remain the proper Org-mode code block syntax. Thanks Eric, this is great news. I'll ensure my examples from last year still work with the current org-mode syntax. > For many small examples, please see [1] which I (unfortunately) haven't That's a nice site, thanks! > From looking at the fairly terse web site for R^2 it is not clear to me > exactly what the system includes (I'm sure I'm missing something > obvious). It seems to be the addition of a packaging system over-top of > R source files. What would a potential Org-mode based system provide > which is not already possible with Org-mode text files, Org-mode > publishing and a version control repository. The accompanying paper from 2011 goes into detail as to what it does. But in essence, the website is supposed to be 'neutral', in that it is not your local system. A document may compile on my system, but not for others, because it depends on my local configuration. Having a neutral system will avoid such problems. A neutral system could also have support for all babel languages, so that I can use it to compile someone's document without e.g. having a common-lisp compiler on my system. > Perhaps the benefit would simply be a system which eases the integration > of Org, publishing, version control, and possibly automatic Makefile > creation with tasks like "publish", "clean" etc... yes, that too! best wishes, Stephen ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Rsquared for reproductible research @ 2012-01-12 19:44 brian powell 2012-01-12 20:33 ` Eric Schulte 0 siblings, 1 reply; 9+ messages in thread From: brian powell @ 2012-01-12 19:44 UTC (permalink / raw) To: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 3297 bytes --] >> Also, I very much agree that a "near exact replica" of the http:// >> rsquared.stat.uni-muenchen.de/index.rhtml for OrgMode would be great. > Yes! Any takers?!? > ... Eric questioned: "From looking at the fairly terse web site for R^2 it is not clear to me exactly what the system includes (I'm sure I'm missing something obvious). It seems to be the addition of a packaging system over-top of R source files. What would a potential Org-mode based system provide which is not already possible with Org-mode text files, Org-mode publishing and a version control repository." ... * I mostly agree with your statements. Good challenges. I did more investigation: This link to the paper that "Friedrich Leischa, , Manuel Eugsterb, Torsten Hothornb" put together may make things clearer--this paper really seems to be the justification/impetus for the R^2 website--it has made things clearer and more exciting for me: http://www.sciencedirect.com/science/article/pii/S1877050911001232 <=> "Executable Papers for the R Community: The R2 Platform for Reproducible Research" ** So papers in R (and maybe other languages--maybe languages run thru Babel in OrgMode) could be "executed" by people in the community--one could verify research studies and papers interactively, ad hoc. *** They mention in the paper that they use several key tools: "R: the lingua franca of statistics and data analysis Sweave: the most popular format for executable papers in the R community CRAN: package building and checking system has been developed for more then a decade and copes successfully with the exponential growth of the number of packages" **** Weave/CWEB/CWEAVE/CTANGLE=>NOWEB (Knuth) comes to mind here--i.e. Literate Programming ***** http://www-cs-staff.stanford.edu/~uno/cweb.html => CTANGLE converts a source file foo.w to a compilable program file foo.c; CWEAVE converts a source file foo.w to a prettily-printable and cross-indexed document file foo.tex. * Exactly the paper can be found at: http://www.sciencedirect.com/science\ ?_ob=MiamiImageURL&_cid=280203&_user=10&_pii=S1877050911001232&\ _check=y&_origin=article&_zone=toolbar&_coverDate=\ 31-Dec-2011&view=c&originContentFamily=serial&wchp=\ dGLbVlS-zSkWb&md5=4681e5babd7822f321d2a0dd3a9f11cf/\ 1-s2.0-S1877050911001232-main.pdf * I agree Eric that the website is a bit terse; but, for the most part I was excited about (and I think Stephen is interested in--he suggested it is something the community might do) the general ideas, the structure of the website's process "OrgMode=>TeX paper in"-->...process...-->Executable/verifiable code interaction a user might experience/stored on-line for researchers (one thing I always enjoy a lot when working with e.g. R/S-PLUS and PYTHON's interactive CLI, etc.) ** They even publish the minute details of the settings on the machines--the local environment variables, etc.--the devil is in the details! * Thanks for the link Eric to your OrgMode "scraps"--they could be very useful: http://eschulte.me/org-scraps/ * Lastly, most importantly I'll repeat the link and query to the community: ... >> Also, I very much agree that a "near exact replica" of the http://rsquared.stat.uni-muenchen.de/index.rhtml >>for OrgMode would be great. > Yes! Any takers?!? ---(Stephen Eglen) [-- Attachment #2: Type: text/html, Size: 4636 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Rsquared for reproductible research 2012-01-12 19:44 brian powell @ 2012-01-12 20:33 ` Eric Schulte 0 siblings, 0 replies; 9+ messages in thread From: Eric Schulte @ 2012-01-12 20:33 UTC (permalink / raw) To: brian powell; +Cc: emacs-orgmode brian powell <briangpowellms@gmail.com> writes: >>> Also, I very much agree that a "near exact replica" of the http:// >>> rsquared.stat.uni-muenchen.de/index.rhtml for OrgMode would be great. >> Yes! Any takers?!? >> > ... > Eric questioned: > "From looking at the fairly terse web site for R^2 it is not clear to me > exactly what the system includes (I'm sure I'm missing something > obvious). It seems to be the addition of a packaging system over-top of > R source files. What would a potential Org-mode based system provide > which is not already possible with Org-mode text files, Org-mode > publishing and a version control repository." > ... > > * I mostly agree with your statements. Good challenges. I did more > investigation: This link to the paper that "Friedrich Leischa, , Manuel > Eugsterb, Torsten Hothornb" put together may make things clearer--this > paper really seems to be the justification/impetus for the R^2 website--it > has made things clearer and more exciting for me: > > http://www.sciencedirect.com/science/article/pii/S1877050911001232 > Ah, thank you for linking to this paper. It seems I was missing was the package-management aspect provided by R2 through CRAN. The instillation of all software dependencies is a huge benefit exactly as installing software with apt-get or pacman is simpler than running "./configure && make" and manually resolving dependencies. While such a tool makes sense for a single language system like R, I fear an Org-mode version of such a system would have too wide of a scope. Given that code blocks may contain arbitrary languages, and that sh blocks can freely call any command-line executable such a system would turn into a system-wide package management tool. Perhaps there already exists a portable package management system designed for local installs which could handle most of the heavy lifting. As another option, distributing Virtual Machine images are one solution which I think work well and are increasingly realistic. Or similarly providing the research environment as a cloud server image (e.g., Amazon EC2). Certainly an interesting area for further work! Best, -- Eric Schulte http://cs.unm.edu/~eschulte/ ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2012-01-12 20:54 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <t5xd3p8qlct.fsf@rgc.damtp.cam.ac.uk>
2010-12-12 3:49 ` Fwd: [RFC] Self-configuring Org-mode files Thomas S. Dye
2012-01-12 12:38 ` Rsquared for reproductible research Stephen Eglen
2012-01-12 15:27 ` brian powell
2012-01-12 15:40 ` Stephen Eglen
2012-01-12 16:29 ` Eric Schulte
2012-01-12 18:24 ` Thomas S. Dye
2012-01-12 20:54 ` Stephen Eglen
2012-01-12 19:44 brian powell
2012-01-12 20:33 ` Eric Schulte
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).