From mboxrd@z Thu Jan 1 00:00:00 1970 From: Nick Dokos Subject: Re: Exploring data that is in org-mode format Date: Fri, 27 Sep 2013 08:42:24 -0400 Message-ID: <874n96h1vz.fsf@gmail.com> References: Mime-Version: 1.0 Content-Type: text/plain Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:37914) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VPXNa-0005J9-V4 for emacs-orgmode@gnu.org; Fri, 27 Sep 2013 08:42:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VPXNW-00010K-GO for emacs-orgmode@gnu.org; Fri, 27 Sep 2013 08:42:42 -0400 Received: from plane.gmane.org ([80.91.229.3]:45871) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VPXNW-00010E-9p for emacs-orgmode@gnu.org; Fri, 27 Sep 2013 08:42:38 -0400 Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1VPXNT-0006VC-99 for emacs-orgmode@gnu.org; Fri, 27 Sep 2013 14:42:35 +0200 Received: from pool-108-7-96-134.bstnma.fios.verizon.net ([108.7.96.134]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 27 Sep 2013 14:42:35 +0200 Received: from ndokos by pool-108-7-96-134.bstnma.fios.verizon.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 27 Sep 2013 14:42:35 +0200 List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: emacs-orgmode@gnu.org Alan Schmitt writes: > Hello, > > This question is slightly off-topic, but it may be of interest to people > who have a lot of data entered in org-mode. > > The short version: what tools are available to explore data, typically > stored in org-mode tables? > > The long version: I've tried an interesting website > (https://tictrac.com/) whose goal is to gain some insight about > ourselves by exploring some data we collect (think quantified self). I'm > not happy with this site for three reasons: > - I need to send it the data; > - it focuses on health / activity data whereas there is much more that > interests me (I for instance have weekly records of natural gas use in my > gas-heated house and daily record of temperature average outside which I > would love to compare); > - it won't let you input arbitrary data (I asked about importing a CSV > of my daily coffee consumption, they answered they require an external > service to integrate the data). > > So I collect all this data because it's something I enjoy doing, and I > would really like to explore it, from the comfortable position of my own > computer. All of this data is in org-mode tables (or can be easily > converted to org-mode table). Hence my questions: are there tools you > would recommend? I'm not afraid of programming (I suspect an answer will > be 'R'), but I would like pointers to tutorials to do these kind of > things. The kind of things I would like to do are: > - extract weekly or monthly tallies or estimation from data collected at > irregular intervals; > - compare data sources against each other; > - estimate future trends based on past data (how much will my gas bill be?); > - display the result in some kind of dashboard. > Not org-related and not even emacs-related (sorry Marcin!) but applicable to the question: Apart from R and Matlab, there is also ... Python: I'm currently reading a very nice book that uses Python, Numpy, Pandas and Matplotlib for data exploration. It is called "Python for Data Analysis", by Wes McKinney (the original developer of Pandas). I'm about a third of the way through it and I can recommend it. You can find a link to the book at the Pandas site: http://pandas.pydata.org/ Just in case the question arises: no, I'm not a paid endorser - just a satisfied customer :-) -- Nick