From mboxrd@z Thu Jan 1 00:00:00 1970 From: John Hendy Subject: Re: Exploring data that is in org-mode format Date: Fri, 27 Sep 2013 13:42:15 -0500 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:37485) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VPcza-0002Ic-57 for emacs-orgmode@gnu.org; Fri, 27 Sep 2013 14:42:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VPczY-00050T-UZ for emacs-orgmode@gnu.org; Fri, 27 Sep 2013 14:42:18 -0400 Received: from mail-ob0-x22a.google.com ([2607:f8b0:4003:c01::22a]:37146) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VPczY-000509-Q9 for emacs-orgmode@gnu.org; Fri, 27 Sep 2013 14:42:16 -0400 Received: by mail-ob0-f170.google.com with SMTP id va2so3475613obc.15 for ; Fri, 27 Sep 2013 11:42:15 -0700 (PDT) In-Reply-To: List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Alan Schmitt Cc: emacs-org list On Fri, Sep 27, 2013 at 3:18 AM, Alan Schmitt wrote: > Hello, > > This question is slightly off-topic, but it may be of interest to people > who have a lot of data entered in org-mode. > > The short version: what tools are available to explore data, typically > stored in org-mode tables? [snip] > So I collect all this data because it's something I enjoy doing, and I > would really like to explore it, from the comfortable position of my own > computer. All of this data is in org-mode tables (or can be easily > converted to org-mode table). Hence my questions: are there tools you > would recommend? I'm not afraid of programming (I suspect an answer will > be 'R'), but I would like pointers to tutorials to do these kind of > things. The kind of things I would like to do are: > - extract weekly or monthly tallies or estimation from data collected at > irregular intervals; > - compare data sources against each other; > - estimate future trends based on past data (how much will my gas bill be?); > - display the result in some kind of dashboard. > For simple exploration, you might have a look at ggobi? [1] It allows you to do some really quick/easy exploration by plotting and being able to check which variables to use for X and Y, coloring, filtering, changing plot type, and so on. There's an R package which allows you to call ggobi on an R data object, which you could easily create with babel and your existing org tables with the #+name option for the table and :var specification in the babel block header. I'd also highly recommend taking a look at shiny via R-Studio.[2] Not sure if you can call it from Org-mode, but even if you can't... not *everything* has to be done with Org. You could use the file to do some data munging/summarization/etc., save it as a new data set (.csv or similar), and then read that into Shiny. It could be *awesome* for something like this. I only recently started playing with it but it's just fantastic and would make for the ability to subset, change scales/time ranges, and much more in an interactive web app. I applied for a public server account with RStudio and got it so that as I learn and do more, I can make them public. As a result of an SO question, I repaid the answerer by creating an app from his reply: - Post: http://stackoverflow.com/questions/17958730/faceting-a-set-of-contour-plots-in-ggplot-r - Shiny app: http://spark.rstudio.com/jwhendy/interactive-contour/ Good luck! John [1] http://www.ggobi.org/ [2] http://www.rstudio.com/shiny/ > Thanks a lot, > > Alan >