Re: Exploring data that is in org-mode format

[Nick Dokos <ndokos@gmail.com>]

Alan Schmitt <alan.schmitt@polytechnique.org> writes:

> Hello,
>
> This question is slightly off-topic, but it may be of interest to people
> who have a lot of data entered in org-mode.
>
> The short version: what tools are available to explore data, typically
> stored in org-mode tables?
>
> The long version: I've tried an interesting website
> (https://tictrac.com/) whose goal is to gain some insight about
> ourselves by exploring some data we collect (think quantified self). I'm
> not happy with this site for three reasons:
> - I need to send it the data;
> - it focuses on health / activity data whereas there is much more that
> interests me (I for instance have weekly records of natural gas use in my
> gas-heated house and daily record of temperature average outside which I
> would love to compare);
> - it won't let you input arbitrary data (I asked about importing a CSV
> of my daily coffee consumption, they answered they require an external
> service to integrate the data).
>
> So I collect all this data because it's something I enjoy doing, and I
> would really like to explore it, from the comfortable position of my own
> computer. All of this data is in org-mode tables (or can be easily
> converted to org-mode table). Hence my questions: are there tools you
> would recommend? I'm not afraid of programming (I suspect an answer will
> be 'R'), but I would like pointers to tutorials to do these kind of
> things. The kind of things I would like to do are:
> - extract weekly or monthly tallies or estimation from data collected at
> irregular intervals;
> - compare data sources against each other;
> - estimate future trends based on past data (how much will my gas bill be?);
> - display the result in some kind of dashboard.
>

Not org-related and not even emacs-related (sorry Marcin!) but
applicable to the question:

Apart from R and Matlab, there is also ... Python: I'm currently reading 
a very nice book that uses Python, Numpy, Pandas and Matplotlib for
data exploration. It is called "Python for Data Analysis", by Wes
McKinney (the original developer of Pandas). I'm about a third of the
way through it and I can recommend it. You can find a link to the book
at the Pandas site:

   http://pandas.pydata.org/

Just in case the question arises: no, I'm not a paid endorser - just a
satisfied customer :-)

-- 
Nick