From: "Thomas S. Dye" <tsd@tsdye.com>
To: Andy Choens <andy.choens@gmail.com>
Cc: Org-mode ml <emacs-orgmode@gnu.org>
Subject: Re: Reproducible Research Template
Date: Wed, 5 Jan 2011 06:24:01 -1000 [thread overview]
Message-ID: <F8C7EAD6-3983-4C5E-9853-5027800E14D6@tsdye.com> (raw)
In-Reply-To: <1294170927.2599.110.camel@Yates>
On Jan 4, 2011, at 9:55 AM, Andy Choens wrote:
> I am developing a reproducible research template for R. I am trying
> to implement most of a research "compendium" in org. I say "most"
> because I am going to allow the actual data to exist outside of org,
> simply because most of the data I work with is relational or very
> large, which makes storage in plain text problematic or impossible.
>
> Has anyone ever implemented a reproducible research template in org
> that I can look at? I looked at the stuff on Worg and there are
> examples of a project, but not a template.
>
> In a nutshell, I am trying to do two things.
> 1) Provide a structure for reproducible research / programming ;
> 2) Provide a small set of helper functions.
>
> But, I don't want the helper functions to get in the way. I have
> considered two options:
> 1) Store the example code / template stuff in subheadings
> 2) Store the example code / template stuff in an external file.
>
> Using subheadings is tempting, but I'm afraid that org-babel-execute-
> buffer would cause problems for users who don't use all of the
> template functions.
>
> Using an external file, similar to the Lobrary of Babel is also
> tempting. It would allow me to make a cleaner template for structure
> and still allow users to access any helper functions. Is there a way
> to link to an external file, other than the Library of Babel? If so,
> how do I do this?
>
> Does anyone have any opinions about hiding/linking/importing example
> code in a template?
>
> I certainly appreciate any thoughts.
>
> --andy
Aloha Andy,
Great idea. I'll be interested to see where you go with this.
You can link to an external file other than the one holding the
library of babel using the library of babel facility. I use this in
my config file:
#+source: load-local-lob
#+begin_src emacs-lisp :tangle yes
(org-babel-lob-ingest "~/org/local-lob.org")
#+end_src
I put functions there that are useful to me in more than one Org-mode
buffer, but that are not likely to be useful to other Org-mode users
(and therefore fit for the library of babel).
Most of my projects store data in a MySQL database. My projects
define queries that relate tables to one another, but the results are
typically something that Org-mode understands---a flat table or a
single value. The reproducible research functions that I write break
the analysis workflow down into separate tasks so that the results of
a SQL query are written to the Org-mode buffer. Subsequent steps in
the workflow refer to the named results block. This way, the Org-mode
project contains the actual data used in the analysis without the need
to reproduce a relational structure. The functions that access the
SQL database go in the local library of babel because I don't want to
give users of a RR document direct access to my database.
If I manipulate query results after they are written into the Org-mode
buffer, I also like to write out intermediate results in many
situations. My goal in writing RR code is not speed or compactness,
but maximum transparency. Because Org-mode is language agnostic it is
frequently the case that the files created by others contain code in
languages I don't understand. If these are long and complex, then I'm
at a loss to what is actually going on. If they are short, and
intermediate results are written out in the buffer, then it is easier
for me to follow the analysis.
I look forward to learning from your work with RR templates.
All the best,
Tom
next prev parent reply other threads:[~2011-01-05 16:24 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-01-04 19:55 Reproducible Research Template Andy Choens
2011-01-05 16:24 ` Thomas S. Dye [this message]
2011-01-05 17:24 ` Andy Choens
2011-01-05 16:31 ` Charles C. Berry
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=F8C7EAD6-3983-4C5E-9853-5027800E14D6@tsdye.com \
--to=tsd@tsdye.com \
--cc=andy.choens@gmail.com \
--cc=emacs-orgmode@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).