Re: Exploring data that is in org-mode format

[John Hendy <jw.hendy@gmail.com>]

On Fri, Sep 27, 2013 at 3:18 AM, Alan Schmitt
<alan.schmitt@polytechnique.org> wrote:
> Hello,
>
> This question is slightly off-topic, but it may be of interest to people
> who have a lot of data entered in org-mode.
>
> The short version: what tools are available to explore data, typically
> stored in org-mode tables?

[snip]

> So I collect all this data because it's something I enjoy doing, and I
> would really like to explore it, from the comfortable position of my own
> computer. All of this data is in org-mode tables (or can be easily
> converted to org-mode table). Hence my questions: are there tools you
> would recommend? I'm not afraid of programming (I suspect an answer will
> be 'R'), but I would like pointers to tutorials to do these kind of
> things. The kind of things I would like to do are:
> - extract weekly or monthly tallies or estimation from data collected at
> irregular intervals;
> - compare data sources against each other;
> - estimate future trends based on past data (how much will my gas bill be?);
> - display the result in some kind of dashboard.
>

For simple exploration, you might have a look at ggobi? [1] It allows
you to do some really quick/easy exploration by plotting and being
able to check which variables to use for X and Y, coloring, filtering,
changing plot type, and so on. There's an R package which allows you
to call ggobi on an R data object, which you could easily create with
babel and your existing org tables with the #+name option for the
table and :var specification in the babel block header.

I'd also highly recommend taking a look at shiny via R-Studio.[2] Not
sure if you can call it from Org-mode, but even if you can't... not
*everything* has to be done with Org. You could use the file to do
some data munging/summarization/etc., save it as a new data set (.csv
or similar), and then read that into Shiny. It could be *awesome* for
something like this. I only recently started playing with it but it's
just fantastic and would make for the ability to subset, change
scales/time ranges, and much more in an interactive web app.

I applied for a public server account with RStudio and got it so that
as I learn and do more, I can make them public. As a result of an SO
question, I repaid the answerer by creating an app from his reply:
- Post: http://stackoverflow.com/questions/17958730/faceting-a-set-of-contour-plots-in-ggplot-r
- Shiny app: http://spark.rstudio.com/jwhendy/interactive-contour/


Good luck!
John

[1] http://www.ggobi.org/
[2] http://www.rstudio.com/shiny/


> Thanks a lot,
>
> Alan
>