Re: using gnuplot's "splot" and "every" commands on org-mode table data

[Eric Schulte <schulte.eric@gmail.com>]

Paul Stansell <paulstansell@gmail.com> writes:

> Hi Eric,
>
> In your last example are you illustrating the case of a data file with
> missing data?

Yes.

> Replacing ""s with blank space in your example doesn't really break
> the plot, it's just that gnuplot interprets the first column of data
> as
>
> 1
> 2
> 3
> 16
>
> 6
> 7
> 4
> 9
>
> and the second as
>
> 2
> 4
> 8
>
>
> 16
> 8
> 4
> 2
>

I would consider this breaking the plot, in that gnuplot no longer lines
up the two columns of data.

>
> Gnuplot has a specific setting for handling missing values.  It's
> common in gnuplot to use '?' to signify a missing value (type "help
> missing" at the gnuplot prompt).  If, in your last example, you
> replace the missing values with '?' (without the quotes) and then type
> in gnuplot
>
>   set datafile missing "?"
>
> and then plot the data with
>
>   plot "data.txt" u 0:1, '' u 0:2
>
> you'll see that gnuplot handles it properly.  Putting '?' in the
> blanks cells of the org mode table in your last attachment and
> plotting from within org mode gives a temporary data file containing
> '"?"', which gnuplot also handles correctly as it must recognise "?"
> (with the quotes) as being a missing value.
>
> But org mode's behaviour of inserting "" in blank cells (instead of
> actual blanks) does break the use of "every" in gnuplot which is
> required by splot when plotting surfaces from data files (unless the
> data files are in "matrix" format).  Also, inserting "" breaks the use
> of gnuplot's "index" which expects two blank lines to separate data
> sets in a single file, but I've never used this facility.
>
> I noticed that org mode writes table values of 'nan' and 'NaN' without
> the double quotes, but, for example, 'MaM' is written with double
> quotes, (as ''"MaM"') in the temporary data file.
>

I didn't know that '?' was the standard gnuplot missing value.  In that
case I think the `org-babel-gnuplot-table-to-data' should be updated to
replace missing values with '?' by default.  If we find a common need to
specialize this value then we can add a gnuplot header argument which
can specify missing values for particular code blocks.

>
> Also, I think it's generally better to remove the double quotes from
> around all strings exported by org mode for plotting by gnuplot as
> they're generally not needed (exceptions are strings which contain
> spaces which can be quoted so they are treated as a single entity by
> gnuplot).  For example, there is a gnuplot demonstration script called
> datastrings.dem which plots data from a file called ctg-y2.dat.  This
> data file has unquoted month names in the first column that are
> plotted as labels on the x-axis.
>

I agree with your reasoning here.  This and the above should be fairly
straightforward changes to the `org-babel-gnuplot-table-to-data'
function.  If I find any free time I'll make these changes.  Although I
certainly wouldn't mind if someone else gets to them first.

Best regards (and thanks for the clear explanations),

>
> Paul

-- 
Eric Schulte
http://cs.unm.edu/~eschulte