Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[Major A]

Hi,

I'd like to use asymptote to plot the values in an Org table.  The table
has cells with numbers but also cells with strings in them.  This table
gets converted to an array of strings in the resulting asymptote file,
with the strings escaped with double-quotes but not the numbers.  In
asymptote, this is an error, so that no plot is produced.

Here's the code:

  #+data: values
  | 0 | 0 | a |
  | 1 | 1 | b |
  | 2 | 3 | c |
  
  #+begin_src asymptote :file asy.png :var v=values
    size(100);
    draw((0,0)--(1,1));
  #+end_src

The temporary asymptote file created during evaluation or export looks
like this:

  string[][] v={
  {0,0,"a"},
  {1,1,"b"},
  {2,3,"c"}
  };

"asy" complains about this file like this:

  asymptote-12652XAo: 2.2: cannot cast 'int' to 'string'
  asymptote-12652XAo: 2.4: cannot cast 'int' to 'string'
  asymptote-12652XAo: 3.2: cannot cast 'int' to 'string'
  asymptote-12652XAo: 3.4: cannot cast 'int' to 'string'
  asymptote-12652XAo: 4.2: cannot cast 'int' to 'string'
  asymptote-12652XAo: 4.4: cannot cast 'int' to 'string'

Am I doing something wrong here?

I actually need this to work ASAP, so if anyone has any workable
solution, then please let me know.

  András




------------------------------------------------------------------------



Emacs  : GNU Emacs 23.3.1 (i486-pc-linux-gnu, GTK+ Version 2.24.3)
 of 2011-04-10 on raven, modified by Debian
Package: Org-mode version 7.7 (release_7.7.205.gce02a)

current state:
==============
(setq
 org-export-latex-after-initial-vars-hook '(org-beamer-after-initial-vars)
 org-speed-command-hook '(org-speed-command-default-hook org-babel-speed-command-hook)
 org-babel-load-languages '((asymptote . t) (ditaa . t) (dot . t) (gnuplot . t) (haskell . t) (latex . t) (octave . t)
 			  	           (R . t) (ruby . t) (scheme . t) (sh . t))
 org-metaup-hook '(org-babel-load-in-session-maybe)
 org-after-todo-state-change-hook '(org-clock-out-if-current)
 org-babel-tangle-lang-exts '(("ruby" . "rb") ("latex" . "tex") ("haskell" . "hs") ("asymptote" . "asy")
 			    	            ("emacs-lisp" . "el"))
 org-export-blocks-postblock-hook '(org-exp-res/src-name-cleanup)
 org-export-latex-format-toc-function 'org-export-latex-format-toc-default
 org-tab-first-hook '(org-hide-block-toggle-maybe org-src-native-tab-command-maybe org-babel-hide-result-toggle-maybe)
 org-src-mode-hook '(org-src-babel-configure-edit-buffer org-src-mode-configure-edit-buffer)
 org-confirm-shell-link-function 'yes-or-no-p
 org-export-first-hook '(org-beamer-initialize-open-trackers)
 org-agenda-before-write-hook '(org-agenda-add-entry-text)
 org-blank-before-new-entry nil
 org-babel-pre-tangle-hook '(save-buffer)
 org-cycle-hook '(org-cycle-hide-archived-subtrees org-cycle-hide-drawers org-cycle-show-empty-lines
 		  org-optimize-window-after-visibility-change)
 org-export-preprocess-before-normalizing-links-hook '(org-remove-file-link-modifiers)
 org-mode-hook '(#[nil "\300\301\302\303\304$\207" [org-add-hook change-major-mode-hook org-show-block-all append local]
 	          5]
			 #[nil "\300\301\302\303\304$\207"
			          [org-add-hook change-major-mode-hook org-babel-show-result-all append local] 5]
				  		 org-babel-result-hide-spec org-babel-hide-all-hashes)
 org-ctrl-c-ctrl-c-hook '(org-babel-hash-at-point org-babel-execute-safely-maybe)
 org-confirm-elisp-link-function 'yes-or-no-p
 org-export-interblocks '((lob org-babel-exp-lob-one-liners) (src org-babel-exp-inline-src-blocks))
 org-clock-out-hook '(org-clock-remove-empty-clock-drawer)
 org-occur-hook '(org-first-headline-recenter)
 org-from-is-user-regexp nil
 org-export-preprocess-before-selecting-backend-code-hook '(org-beamer-select-beamer-code)
 org-confirm-babel-evaluate nil
 org-export-latex-final-hook '(org-beamer-amend-header org-beamer-fix-toc org-beamer-auto-fragile-frames
 			     			              org-beamer-place-default-actions-for-lists)
 org-metadown-hook '(org-babel-pop-to-session-maybe)
 org-export-blocks '((src org-babel-exp-src-block nil) (comment org-export-blocks-format-comment t)
 		        (ditaa org-export-blocks-format-ditaa nil) (dot org-export-blocks-format-dot nil))
 )

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[Nicolas Goaziou]

Hello,

Major A <andras.g.major@gmail.com> writes:

> I'd like to use asymptote to plot the values in an Org table.  The table
> has cells with numbers but also cells with strings in them.  This table
> gets converted to an array of strings in the resulting asymptote file,
> with the strings escaped with double-quotes but not the numbers.  In
> asymptote, this is an error, so that no plot is produced.

I'm not sure to understand. Your array must have all its elements of the
same type. In this case, it detects that common type should be
"string". What else should it do in that case?

That being said, you can have rownames and colnames even in an array of
ints (see :rownames and :colnames headers arguments for your source
block).

In others words, you can plot, for example, the following table without
any problem:

| x | 1 | 2 | 3 |
| y | 1 | 4 | 9 |

Regards,

-- 
Nicolas Goaziou

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[Nick Dokos]

[-- Attachment #1: Type: text/plain, Size: 2679 bytes --]

Major A <andras.g.major@gmail.com> wrote:

> 
> Hi,
> 
> I'd like to use asymptote to plot the values in an Org table.  The table
> has cells with numbers but also cells with strings in them.  This table
> gets converted to an array of strings in the resulting asymptote file,
> with the strings escaped with double-quotes but not the numbers.  In
> asymptote, this is an error, so that no plot is produced.
> 
> Here's the code:
> 
>   #+data: values
>   | 0 | 0 | a |
>   | 1 | 1 | b |
>   | 2 | 3 | c |
>  =20
>   #+begin_src asymptote :file asy.png :var v=3Dvalues
>     size(100);
>     draw((0,0)--(1,1));
>   #+end_src
> 
> The temporary asymptote file created during evaluation or export looks
> like this:
> 
>   string[][] v=3D{
>   {0,0,"a"},
>   {1,1,"b"},
>   {2,3,"c"}
>   };
> 
> "asy" complains about this file like this:
> 
>   asymptote-12652XAo: 2.2: cannot cast 'int' to 'string'
>   asymptote-12652XAo: 2.4: cannot cast 'int' to 'string'
>   asymptote-12652XAo: 3.2: cannot cast 'int' to 'string'
>   asymptote-12652XAo: 3.4: cannot cast 'int' to 'string'
>   asymptote-12652XAo: 4.2: cannot cast 'int' to 'string'
>   asymptote-12652XAo: 4.4: cannot cast 'int' to 'string'
> 
> Am I doing something wrong here?
> 

I don't know anything about asymptote and I am not sure whether this
will help: it does produce a temp file with everything quoted and
running asy on the temp file produces an .eps file that contains the
diagonal line, but it produces a png file which seems somewhat peculiar
to me but maybe it'll work for you.

You have to do

(setq org-babel-min-lines-for-block-output 0)

in order to get the example block result.

I just added a python block to produce another table with everything
quoted.  The second table is then give to asymptote:

--8<---------------cut here---------------start------------->8---
#+data: values
| 0 | 0 | a |
| 1 | 1 | b |
| 2 | 3 | c |

#+begin_src python :results output :var table=values
  print "#+data: qvalues"
  for row in table:
      print "|%s|" % ("|".join(map(lambda x: "\"%s\"" % (x), row)))
  
#+end_src

#+results:
#+begin_example
#+data: qvalues
|"0"|"0"|"a"|
|"1"|"1"|"b"|
|"2"|"3"|"c"|
#+end_example

#+begin_src asymptote :file asy.png :var v=qvalues
   size(100);
   draw((0,0)--(1,1));

#+end_src

#+results:
[[file:asy.png]]
--8<---------------cut here---------------end--------------->8---


The temp file looks like this:


--8<---------------cut here---------------start------------->8---
string[][] v={
{"0","0","a"},
{"1","1","b"},
{"2","3","c"}
};
size(100);
draw((0,0)--(1,1));
--8<---------------cut here---------------end--------------->8---

I attach the asy.png file I get.

Nick


[-- Attachment #2: asy.png --]
[-- Type: image/png, Size: 680 bytes --]

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[András Major]

Hi Nicolas,

> > I'd like to use asymptote to plot the values in an Org table.  The table
> > has cells with numbers but also cells with strings in them.  This table
> > gets converted to an array of strings in the resulting asymptote file,
> > with the strings escaped with double-quotes but not the numbers.  In
> > asymptote, this is an error, so that no plot is produced.
> 
> I'm not sure to understand. Your array must have all its elements of the
> same type. In this case, it detects that common type should be
> "string". What else should it do in that case?
> 
> That being said, you can have rownames and colnames even in an array of
> ints (see :rownames and :colnames headers arguments for your source
> block).
> 
> In others words, you can plot, for example, the following table without
> any problem:
> 
> | x | 1 | 2 | 3 |
> | y | 1 | 4 | 9 |

This isn't what I need.  What I want is to make a graph of certain columns
of a table which contains both ints and strings.  Ideally, there should be
no constraint about which columns contain ints and which contain strings.

Since the table usually comes first (you collect your research in such,
for instance), reformatting afterwards only so that asymptote works is
not an option.

Why isn't it possible to force ob-asymptote to make all cells strings?
Then one could easily convert the strings back to floats or so within
asymptote as needed.

  András

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[András Major]

Hi Nick,

> I don't know anything about asymptote and I am not sure whether this
> will help: it does produce a temp file with everything quoted and
> running asy on the temp file produces an .eps file that contains the
> diagonal line, but it produces a png file which seems somewhat peculiar
> to me but maybe it'll work for you.

Thanks for the example, good idea.  Yes, the diagonal line is OK,
that's what my asymptote example creates, as the code I posted doesn't
actually make use of the array (its mere presence is enough to trigger
the error).

> You have to do
> 
> (setq org-babel-min-lines-for-block-output 0)
> 
> in order to get the example block result.

#+begin_rant

Is it just me, or are there other people who think that this kind of
approach is wrong?  In my view, per-user configuration can affect
the look-and-feel of org-mode, the paths, etc., but I think having
options that affect the result in any way (most babel options, in
fact) should be configurable from within the Org file itself.

After all, most work I do (now in Org) is supposed to be passed
to other people who can then continue doing this work.  I can't
expect them to change "global" settings just so they can reproduce
my output.

I love most software I use exactly because it doesn't rely too
much on global (per-user) configuration.  I think Org should also
reduce the set of global options to the absolute minimum.

#+end_rant

  András

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[Nicolas Goaziou]

András Major <andras.g.major@gmail.com> writes:

> This isn't what I need.  What I want is to make a graph of certain
> columns of a table which contains both ints and strings.  Ideally,
> there should be no constraint about which columns contain ints and
> which contain strings.

Again, this is a limitation of Asymptote. An array _must_ contain
elements of a single type. That beast you want doesn't exist (unless you
typedef it, but then you cannot pass arguments anymore).

> Since the table usually comes first (you collect your research in
> such, for instance), reformatting afterwards only so that asymptote
> works is not an option.

Why? You can always write an intermediary step to "stringify" every
cell. Choose your language. Nick Dokos showed you one way.

> Why isn't it possible to force ob-asymptote to make all cells strings?

It is possible to force ob-asymptote to make all cells strings. But
I don't think ob-asymptote should try that hard to compensate users'
misuses of data types.

Now, I may be totally wrong.

Regards,

-- 
Nicolas Goaziou

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[András Major]

> Why? You can always write an intermediary step to "stringify" every
> cell. Choose your language. Nick Dokos showed you one way.

Apparently, only if you set a global/per-user option in .emacs or suchlike,
which I think is a bad way of doing it.

> > Why isn't it possible to force ob-asymptote to make all cells strings?
> 
> It is possible to force ob-asymptote to make all cells strings. But
> I don't think ob-asymptote should try that hard to compensate users'
> misuses of data types.

I don't think this is misuse in any way.  Consider a table which is a
result of your research:  the columns are the maker (e.g., "Mazda"), the
type (e.g., "MX-5"), engine displacement (a number), the mileage/fuel
consumption (a number), etc., and I want to create some asymptote plot
from this data.

If there is no way of doing this without requiring global/per-user
settings, then I think that something must be changed.

  András

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[Nicolas Goaziou]

András Major <andras.g.major@gmail.com> writes:

>> Why? You can always write an intermediary step to "stringify" every
>> cell. Choose your language. Nick Dokos showed you one way.
>
> Apparently, only if you set a global/per-user option in .emacs or suchlike,
> which I think is a bad way of doing it.
-----
#+tblname: mixed-types
|1|x|5|
|3|y|4|

#+source: stringify(table=mixed-types)
#+begin_src emacs-lisp
(concat "string[][] myvar={{"
        (mapconcat
         (lambda (row)
           (when (listp row)
             (concat (mapconcat (lambda (el) (format "\"%s\"" el)) row ",") "}")))
         table
         ",{")
        "};")
#+end_src

#+source: myplot
#+begin_src asymptote :noweb yes :file myplot.png
<<stringify(mixed-types)>>
unitsize(3cm);
draw((0,0)--(1,1));
#+end_src

#+results: myplot
[[file:myplot.png]]
----

It may be ugly, but it works.

> I don't think this is misuse in any way.  Consider a table which is a
> result of your research:  the columns are the maker (e.g., "Mazda"), the
> type (e.g., "MX-5"), engine displacement (a number), the mileage/fuel
> consumption (a number), etc., and I want to create some asymptote plot
> from this data.

How would you would you specify it to Asymptote? You can't.

You would probably use:

                string[][] cars={{"Mazda","MX-5","1.5"}};

And that's exactly my point: engine displacement may be a number, but
you would have to enter it as a string.

Though, you insist on being able to enter it as a number anyway, hoping
ob-asymptote will do the magic behind. How could it, since the language
can't itself?

Perhaps there should be a way in Babel (not specifically in Asymptote)
to output a table with raw strings. Because want you really want to use
is:

|"1"|"x"|"5"|
|"3"|"y"|"4"|


Regards,

-- 
Nicolas Goaziou

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[Nicolas Goaziou]

[-- Attachment #1: Type: text/plain, Size: 477 bytes --]

Completing myself.

> Though, you insist on being able to enter it as a number anyway,
> hoping ob-asymptote will do the magic behind. How could it, since the
> language can't itself?

Actually, the attached patch does that magic: if there's any string in
the table, every cell will be turned into a string and the array will be
of type string.

I still think a more general way would be better. I'll let Eric Schulte
decide what to do about it.

Regards,

-- 
Nicolas Goaziou

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-asymptote-mixed-types-tables-default-to-string-ar.patch --]
[-- Type: text/x-patch, Size: 4165 bytes --]

From 72dfd3c98c535ea595be053fe321e195cd08c070 Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou <n.goaziou@gmail.com>
Date: Mon, 29 Aug 2011 17:24:07 +0200
Subject: [PATCH] ob-asymptote: mixed-types tables default to string arrays

* lisp/ob-asymptote.el (org-babel-asymptote-table-to-array): Require a
new argument TYPE specifying the detected type of array.  If it's a
string array, make sure every element is returned as a string. Also
improve doc-string.
(org-babel-asymptote-var-to-asymptote): Fill new argument.  Small
refactoring.
(org-babel-asymptote-define-type): Rewrite to avoid stopping search at
first float found, as strings have precedence over floats.
---
 lisp/ob-asymptote.el |   50 ++++++++++++++++++++++++++++----------------------
 1 files changed, 28 insertions(+), 22 deletions(-)

diff --git a/lisp/ob-asymptote.el b/lisp/ob-asymptote.el
index 1cf6db4..0e9b449 100644
--- a/lisp/ob-asymptote.el
+++ b/lisp/ob-asymptote.el
@@ -108,23 +108,29 @@ a variable of the same value."
      ((stringp val)
       (format "string %S=\"%s\";" var val))
      ((listp val)
-      (let* ((dimension-2-p (not (null (cdr val))))
+      (let* ((dimension-2-p (cdr val))
              (dim (if dimension-2-p "[][]" "[]"))
              (type (org-babel-asymptote-define-type val))
              (array (org-babel-asymptote-table-to-array
-                     val
+                     val type
                      (if dimension-2-p '(:lstart "{" :lend "}," :llend "}")))))
         (format "%S%s %S=%s;" type dim var array))))))
 
-(defun org-babel-asymptote-table-to-array (table params)
-  "Convert values of an elisp table into a string of an asymptote array.
+(defun org-babel-asymptote-table-to-array (table type params)
+  "Convert values of TABLE into a string of an asymptote array.
+
+TABLE is a list whose atoms are assumed to be of type
+TYPE. PARAMS is a plist of parameters that can influence the
+conversion.
+
 Empty cells are ignored."
   (labels ((atom-to-string (table)
                            (cond
                             ((null table) '())
                             ((not (listp (car table)))
-                             (cons (if (and (stringp (car table))
-                                            (not (string= (car table) "")))
+                             (cons (if (or (eq type 'string)
+                                           (and (stringp (car table))
+                                                (not (string= (car table) ""))))
                                        (format "\"%s\"" (car table))
                                      (format "%s" (car table)))
                                    (atom-to-string (cdr table))))
@@ -140,22 +146,22 @@ Empty cells are ignored."
 
 (defun org-babel-asymptote-define-type (data)
   "Determine type of DATA.
-DATA is a list. Type symbol is returned as 'symbol. The type is
-usually the type of the first atom encountered, except for arrays
-of int, where every cell must be of int type."
-  (labels ((anything-but-int (el)
-                             (cond
-                              ((null el) nil)
-                              ((not (listp (car el)))
-                               (cond
-                                ((floatp (car el)) 'real)
-                                ((stringp (car el)) 'string)
-                                (t
-                                 (anything-but-int (cdr el)))))
-                              (t
-                               (or (anything-but-int (car el))
-                                   (anything-but-int (cdr el)))))))
-    (or (anything-but-int data) 'int)))
+
+DATA is a list.  Return type as a symbol.
+
+The type is `string' if any element in DATA is
+a string. Otherwise, it is either `float', if some elements are
+floats, or `int'."
+  (let* ((type 'int)
+	 (find-type
+	  (lambda (row)
+	    (catch 'exit
+	      (mapc (lambda (el)
+		      (cond ((listp el) (funcall find-type el))
+			    ((stringp el) (throw 'exit (setq type 'string)))
+			    ((floatp el) (setq type 'float))))
+		    row)))))
+    (funcall find-type data) type))
 
 (provide 'ob-asymptote)
 
-- 
1.7.6.1

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[Eric Schulte]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Completing myself.
>
>> Though, you insist on being able to enter it as a number anyway,
>> hoping ob-asymptote will do the magic behind. How could it, since the
>> language can't itself?
>
> Actually, the attached patch does that magic: if there's any string in
> the table, every cell will be turned into a string and the array will be
> of type string.
>
> I still think a more general way would be better. I'll let Eric Schulte
> decide what to do about it.
>

I don't know asymptote well enough to know if you would /always/ want a
heterogeneous table to be converted to all strings, or if there would
ever be a case where you would want mixed types in a table.  If a later
then perhaps this behavior should be tucked behind a header argument,
e.g., :mixed-table-all-strings (or somesuch) which could default to
either true or false depending on which use cases is more common.

Let me know which would be most useful and least surprising and I'll
gladly apply the patch.

Thanks -- Eric

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[Nicolas Goaziou]

Hello,

Eric Schulte <schulte.eric@gmail.com> writes:

> I don't know asymptote well enough to know if you would /always/ want a
> heterogeneous table to be converted to all strings, or if there would
> ever be a case where you would want mixed types in a table.

There are no mixed types arrays in Asymptote, hence the problem of the
OP. You can choose between int[][], real[][] or string[][] (and other
types specific to Asymptote and, as such, irrelevant here), but you
never want an heterogeneous table.

So, the real question is: what should Org do when an user tries to fit
a mixed array into an Asymptote var? Until now, it was throwing an
error. With this patch, _one_ string in the table will trigger the
string type for every other cell.

But this kind of an hidden feature. The user has to be aware that, under
some circumstances, ints in his table will become strings.

On the other hand, an user knowing he can't mix types in Asymptote (as
he should) can't do much about it, as far as I know.

As I said in this thread, an elegant solution would be to have a way to
specify Babel to output tables of strings easily:

| x | "1" | "2" | "3" |
| y | "1" | "4" | "9" |

That would be the less surprising, as the user would get the type he
sees in his table.


Regards,

-- 
Nicolas Goaziou

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[Eric Schulte]

[-- Attachment #1: Type: text/plain, Size: 1907 bytes --]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> I don't know asymptote well enough to know if you would /always/ want a
>> heterogeneous table to be converted to all strings, or if there would
>> ever be a case where you would want mixed types in a table.
>
> There are no mixed types arrays in Asymptote, hence the problem of the
> OP. You can choose between int[][], real[][] or string[][] (and other
> types specific to Asymptote and, as such, irrelevant here), but you
> never want an heterogeneous table.
>
> So, the real question is: what should Org do when an user tries to fit
> a mixed array into an Asymptote var? Until now, it was throwing an
> error. With this patch, _one_ string in the table will trigger the
> string type for every other cell.
>
> But this kind of an hidden feature. The user has to be aware that, under
> some circumstances, ints in his table will become strings.
>
> On the other hand, an user knowing he can't mix types in Asymptote (as
> he should) can't do much about it, as far as I know.
>
> As I said in this thread, an elegant solution would be to have a way to
> specify Babel to output tables of strings easily:
>
> | x | "1" | "2" | "3" |
> | y | "1" | "4" | "9" |
>
> That would be the less surprising, as the user would get the type he
> sees in his table.
>

Given that asymptote can not make use of heterogeneous tables, it seems
that it would be easiest to simply silently converted any table
containing a single string element to a table of all strings.  I've just
applied your previous patch (thanks for the patch!).  If this proves
confounding in the future we can always revisit the decision.

Additionally, I've just added a new function to the library of babel
which can be used to convert all elements of a table to strings.  It is
not needed here, but it could be useful in the future.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: all-to-string.org --]
[-- Type: text/x-org, Size: 714 bytes --]

** Convert every element of a table to a string

#+tblname: hetero-table
| 1 | 2 | 3 |
| a | b | c |

#+source: all-to-string
#+begin_src emacs-lisp :var tbl='()
  (defun all-to-string (tbl)
    (if (listp tbl)
        (mapcar #'all-to-string tbl)
      (if (stringp tbl)
          tbl
        (format "%s" tbl))))
  (all-to-string tbl)
#+end_src

#+begin_src emacs-lisp :var tbl=hetero-table
  (mapcar (lambda (row) (mapcar (lambda (cell) (stringp cell)) row)) tbl)
#+end_src

#+results:
| nil | nil | nil |
| t   | t   | t   |

#+begin_src emacs-lisp :var tbl=all-to-string(hetero-table)
  (mapcar (lambda (row) (mapcar (lambda (cell) (stringp cell)) row)) tbl)
#+end_src

#+results:
| t | t | t |
| t | t | t |

[-- Attachment #3: Type: text/plain, Size: 79 bytes --]


Cheers -- Eric

>
>
> Regards,

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[András Major]

Hi Eric,

> Given that asymptote can not make use of heterogeneous tables, it seems
> that it would be easiest to simply silently converted any table
> containing a single string element to a table of all strings.  I've just
> applied your previous patch (thanks for the patch!).  If this proves
> confounding in the future we can always revisit the decision.

I've been away since my last post and now you've already applied a patch
-- wow!  Here's another thought though: change the behaviour of the :var
header argument such that you can specify a range of rows, columns, or
a rectangle just like in table references.  I agree that wildly mixing
cell types in a table should be a felony, but "mixing" them the way I'd
like to makes perfect sense.  You would then simply make more than one
:var to specify multiple subtables, each with its own type.  I think
this might be better than brute-forcing everything to string just so
that the asymptote program has to parse the values back to numbers.

What do you think?

  András

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[Eric Schulte]

András Major <andras.g.major@gmail.com> writes:

> Hi Eric,
>
>> Given that asymptote can not make use of heterogeneous tables, it seems
>> that it would be easiest to simply silently converted any table
>> containing a single string element to a table of all strings.  I've just
>> applied your previous patch (thanks for the patch!).  If this proves
>> confounding in the future we can always revisit the decision.
>
> I've been away since my last post and now you've already applied a patch
> -- wow!  Here's another thought though: change the behaviour of the :var
> header argument such that you can specify a range of rows, columns, or
> a rectangle just like in table references.

This is already possible, see "Indexable variable values" [1].

> I agree that wildly mixing cell types in a table should be a felony,
> but "mixing" them the way I'd like to makes perfect sense.  You would
> then simply make more than one :var to specify multiple subtables,
> each with its own type.  I think this might be better than
> brute-forcing everything to string just so that the asymptote program
> has to parse the values back to numbers.
>
> What do you think?
>

I must admit by the time I got to this thread is was already many
messages deep, and I haven't read the initial messages, so I don't know
what your way of mixing was, but from my current understanding of
asymptote the behavior implemented by Nick's patch seems to make the
most sense, in that it allows tables of ints and floats, but when a
single string is present it converts the table to all strings.

Best -- Eric

>
>   András
>
>
>


Footnotes: 
[1]  http://orgmode.org/manual/var.html

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[Nick Dokos]

Eric Schulte <schulte.eric@gmail.com> wrote:

> András Major <andras.g.major@gmail.com> writes:
> 
> > Hi Eric,
> >
> >> Given that asymptote can not make use of heterogeneous tables, it seems
> >> that it would be easiest to simply silently converted any table
> >> containing a single string element to a table of all strings.  I've just
> >> applied your previous patch (thanks for the patch!).  If this proves
> >> confounding in the future we can always revisit the decision.
> >
> > I've been away since my last post and now you've already applied a patch
> > -- wow!  Here's another thought though: change the behaviour of the :var
> > header argument such that you can specify a range of rows, columns, or
> > a rectangle just like in table references.
> 
> This is already possible, see "Indexable variable values" [1].
> 
> > I agree that wildly mixing cell types in a table should be a felony,
> > but "mixing" them the way I'd like to makes perfect sense.  You would
> > then simply make more than one :var to specify multiple subtables,
> > each with its own type.  I think this might be better than
> > brute-forcing everything to string just so that the asymptote program
> > has to parse the values back to numbers.
> >
> > What do you think?
> >
> 
> I must admit by the time I got to this thread is was already many
> messages deep, and I haven't read the initial messages, so I don't know
> what your way of mixing was, but from my current understanding of
> asymptote the behavior implemented by Nick's patch seems to make the
                                     ^^^Nicolas Goaziou's patch^^^

Nick

> most sense, in that it allows tables of ints and floats, but when a
> single string is present it converts the table to all strings.
> 
> Best -- Eric
> 
> >
> >   András
> >
> >
> >
> 
> 
> Footnotes: 
> [1]  http://orgmode.org/manual/var.html
> 
> -- 
> Eric Schulte
> http://cs.unm.edu/~eschulte/
> 

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[András Major]

Hi Eric,

> > I've been away since my last post and now you've already applied a patch
> > -- wow!  Here's another thought though: change the behaviour of the :var
> > header argument such that you can specify a range of rows, columns, or
> > a rectangle just like in table references.
> 
> This is already possible, see "Indexable variable values" [1].

Oh, sorry that I missed that.  I'll try it with indexed variables then.
Still the question arises why the syntax here is different from usual
table indexing -- why can't I use something along the lines of
":var t=table(@II$2..@>$4)"?

That would be really really nice.  Would it be hard to implement?

  András

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[Eric Schulte]

András Major <andras.g.major@gmail.com> writes:

> Hi Eric,
>
>> > I've been away since my last post and now you've already applied a patch
>> > -- wow!  Here's another thought though: change the behaviour of the :var
>> > header argument such that you can specify a range of rows, columns, or
>> > a rectangle just like in table references.
>> 
>> This is already possible, see "Indexable variable values" [1].
>
> Oh, sorry that I missed that.  I'll try it with indexed variables then.
> Still the question arises why the syntax here is different from usual
> table indexing -- why can't I use something along the lines of
> ":var t=table(@II$2..@>$4)"?
>

The table indexing syntax is specifically used for 2-dimensional arrays
while the variable indexing can be used to index into any number of
dimensions.

>
> That would be really really nice.  Would it be hard to implement?
>

It would be nice to be able to use the familiar table syntax in the case
of 2-D arrays.  This shouldn't be hard to implement, the indexing
behavior is defined in the `org-babel-ref-index-list' function which
could be extended to read in org-table type syntax, or it may not be
difficult to write a function which translates the table syntax to the
variable syntax.

Although ultimately it may be simpler just to keep the two separate.

Best -- Eric

>
>   András
>
>
>

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[András Major]

Hi Eric,

> >> This is already possible, see "Indexable variable values" [1].

Just tried, without success.  Using the example from my original post,
I appended "[,0]" as per the manual to use the first column only, and
when I execute the code block, org throws an error

  "Wrong type argument: listp, "0"".

Any ideas?

  András

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[Eric Schulte]

[-- Attachment #1: Type: text/plain, Size: 462 bytes --]

András Major <andras.g.major@gmail.com> writes:

> Hi Eric,
>
>> >> This is already possible, see "Indexable variable values" [1].
>
> Just tried, without success.  Using the example from my original post,
> I appended "[,0]" as per the manual to use the first column only, and
> when I execute the code block, org throws an error
>
>   "Wrong type argument: listp, "0"".
>
> Any ideas?
>

Can you post an example?  Here is a working example.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: simple-indexing.org --]
[-- Type: text/x-org, Size: 176 bytes --]

** simple indexing
#+data: list-o-numbers
| 1 | 2 | 3 |
| 4 | 5 | 6 |
| 7 | 8 | 9 |

#+begin_src sh :var column=list-o-numbers[,0]
  echo $column
#+end_src

#+results:
: 1 4 7

[-- Attachment #3: Type: text/plain, Size: 94 bytes --]


Best -- Eric

>
>   András
>
>
>

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[András Major]

Hi Eric,

> Can you post an example?  Here is a working example.

In your example, simply write "asymptote" in place of "sh" and replace the
code by "size(100);" just to make sure it's valid asymptote (though the
error occurs even if you don't).  In fact, I'm quite sure that asy never
gets executed in this case.

  András

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[Nick Dokos]

András Major <andras.g.major@gmail.com> wrote:

> Hi Eric,
> 
> > Can you post an example?  Here is a working example.
> 
> In your example, simply write "asymptote" in place of "sh" and replace the
> code by "size(100);" just to make sure it's valid asymptote (though the
> error occurs even if you don't).  In fact, I'm quite sure that asy never
> gets executed in this case.
> 

Yes, even without any asymptote code, this breaks. But there seem to be
multiple problems. One is fix-empty-lines, a local routine defines
inside org-babel-asymptote-table-to-array: it seems to assume that the table
is a list of lists, whereas in this case it's a list of strings. The
following patch (which probably is wrong, in that it cures the symptom
rather than the disease):

diff --git a/lisp/ob-asymptote.el b/lisp/ob-asymptote.el
index 89aecb7..39156d2 100644
--- a/lisp/ob-asymptote.el
+++ b/lisp/ob-asymptote.el
@@ -139,7 +139,7 @@ (defun org-babel-asymptote-table-to-array (table type params)
                                    (atom-to-string (cdr table))))))
            ;; Remove any empty row
            (fix-empty-lines (table)
-                            (delq nil (mapcar (lambda (l) (delq "" l)) table))))
+                            (delq nil (mapcar (lambda (l) (if (listp l) (delq "" l) l)) table))))
     (orgtbl-to-generic
      (fix-empty-lines (atom-to-string table))
      (org-combine-plists '(:hline nil :sep "," :tstart "{" :tend "}") params))))


lets things go a bit further, but there is mayhem in orgtbl-format-line,
which is called by orgtbl-format-section, which is called by
orgtbl-to-generic above, and I didn't go any further: I just don't know
what the preconditions are on these routines.

Nick

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[Nicolas Goaziou]

[-- Attachment #1: Type: text/plain, Size: 1273 bytes --]

Hello,

Nick Dokos <nicholas.dokos@hp.com> writes:

> András Major <andras.g.major@gmail.com> wrote:
>
>> Hi Eric,
>> 
>> > Can you post an example?  Here is a working example.
>> 
>> In your example, simply write "asymptote" in place of "sh" and replace the
>> code by "size(100);" just to make sure it's valid asymptote (though the
>> error occurs even if you don't).  In fact, I'm quite sure that asy never
>> gets executed in this case.
>> 
>
> Yes, even without any asymptote code, this breaks. But there seem to be
> multiple problems. One is fix-empty-lines, a local routine defines
> inside org-babel-asymptote-table-to-array: it seems to assume that the table
> is a list of lists, whereas in this case it's a list of strings. The
> following patch (which probably is wrong, in that it cures the symptom
> rather than the disease):

Yes, the problem is more general: uni-dimensional lists are not handled
correctly.

I attach a patch that should fix the problem (and simplify a lot that
whole process).

One thing, though. Now,

| 1 | 2 |

and (but this was already the case)

| 1 |
| 2 |

are treated as bi-dimensional arrays, because that's how they really are
passed to ob-asymptote.

Regards,

-- 
Nicolas Goaziou

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-asymptote-full-support-for-uni-dimensional-lists.patch --]
[-- Type: text/x-patch, Size: 2172 bytes --]

From 82ae786a8ed1248a62912272edec6f8fa7df4d12 Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou <n.goaziou@gmail.com>
Date: Wed, 31 Aug 2011 11:11:31 +0200
Subject: [PATCH 1/2] ob-asymptote: full support for uni-dimensional lists

* lisp/ob-asymptote.el (org-babel-asymptote-var-to-asymptote):
  recognize non-nested lists as uni-dimensional arrays.
---
 lisp/ob-asymptote.el |   18 ++++++++++--------
 1 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/lisp/ob-asymptote.el b/lisp/ob-asymptote.el
index 89aecb7..f90bb85 100644
--- a/lisp/ob-asymptote.el
+++ b/lisp/ob-asymptote.el
@@ -97,9 +97,8 @@ Asymptote does not support sessions"
 The elisp value PAIR is converted into Asymptote code specifying
 a variable of the same value."
   (let ((var (car pair))
-        (val (if (symbolp (cdr pair))
-                 (symbol-name (cdr pair))
-               (cdr pair))))
+        (val (let ((v (cdr pair)))
+	       (if (symbolp v) (symbol-name v) v))))
     (cond
      ((integerp val)
       (format "int %S=%S;" var val))
@@ -107,14 +106,17 @@ a variable of the same value."
       (format "real %S=%S;" var val))
      ((stringp val)
       (format "string %S=\"%s\";" var val))
+     ((and (listp val) (not (listp (car val))))
+      (let* ((type (org-babel-asymptote-define-type val))
+	     (fmt (if (eq 'string type) "\"%s\"" "%s"))
+	     (vect (mapconcat (lambda (e) (format fmt e)) val ", ")))
+	(format "%s[] %S={%s};" type var vect)))
      ((listp val)
-      (let* ((dimension-2-p (cdr val))
-             (dim (if dimension-2-p "[][]" "[]"))
-             (type (org-babel-asymptote-define-type val))
+      (let* ((type (org-babel-asymptote-define-type val))
              (array (org-babel-asymptote-table-to-array
                      val type
-                     (if dimension-2-p '(:lstart "{" :lend "}," :llend "}")))))
-        (format "%S%s %S=%s;" type dim var array))))))
+                     '(:lstart "{" :lend "}," :llend "}"))))
+        (format "%S[][] %S=%s;" type var array))))))
 
 (defun org-babel-asymptote-table-to-array (table type params)
   "Convert values of TABLE into a string of an asymptote array.
-- 
1.7.6.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-ob-asymptote-simplify-table-to-array-conversion-proc.patch --]
[-- Type: text/x-patch, Size: 2731 bytes --]

From 2e3b7579649ead40128e3c8e31ca75ced139285d Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou <n.goaziou@gmail.com>
Date: Wed, 31 Aug 2011 11:26:29 +0200
Subject: [PATCH 2/2] ob-asymptote: simplify table to array conversion process

* lisp/ob-asymptote.el (org-babel-asymptote-var-to-asymptote):
  refactor code.
(org-babel-asymptote-table-to-array): removed function.
---
 lisp/ob-asymptote.el |   40 ++++++++--------------------------------
 1 files changed, 8 insertions(+), 32 deletions(-)

diff --git a/lisp/ob-asymptote.el b/lisp/ob-asymptote.el
index f90bb85..2219e03 100644
--- a/lisp/ob-asymptote.el
+++ b/lisp/ob-asymptote.el
@@ -113,38 +113,14 @@ a variable of the same value."
 	(format "%s[] %S={%s};" type var vect)))
      ((listp val)
       (let* ((type (org-babel-asymptote-define-type val))
-             (array (org-babel-asymptote-table-to-array
-                     val type
-                     '(:lstart "{" :lend "}," :llend "}"))))
-        (format "%S[][] %S=%s;" type var array))))))
-
-(defun org-babel-asymptote-table-to-array (table type params)
-  "Convert values of TABLE into a string of an asymptote array.
-
-TABLE is a list whose atoms are assumed to be of type
-TYPE. PARAMS is a plist of parameters that can influence the
-conversion.
-
-Empty cells are ignored."
-  (labels ((atom-to-string (table)
-                           (cond
-                            ((null table) '())
-                            ((not (listp (car table)))
-                             (cons (if (or (eq type 'string)
-                                           (and (stringp (car table))
-                                                (not (string= (car table) ""))))
-                                       (format "\"%s\"" (car table))
-                                     (format "%s" (car table)))
-                                   (atom-to-string (cdr table))))
-                            (t
-                             (cons (atom-to-string (car table))
-                                   (atom-to-string (cdr table))))))
-           ;; Remove any empty row
-           (fix-empty-lines (table)
-                            (delq nil (mapcar (lambda (l) (delq "" l)) table))))
-    (orgtbl-to-generic
-     (fix-empty-lines (atom-to-string table))
-     (org-combine-plists '(:hline nil :sep "," :tstart "{" :tend "}") params))))
+	     (fmt (if (eq 'string type) "\"%s\"" "%s"))
+             (array (mapconcat (lambda (row)
+				 (concat "{"
+					 (mapconcat (lambda (e) (format fmt e))
+						    row ", ")
+					 "}"))
+			       val ",")))
+        (format "%S[][] %S={%s};" type var array))))))
 
 (defun org-babel-asymptote-define-type (data)
   "Determine type of DATA.
-- 
1.7.6.1

Re: Bug: Babel: asymptote: erroneous conversion of heterogeneous-typed table

[Eric Schulte]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> Nick Dokos <nicholas.dokos@hp.com> writes:
>
>> András Major <andras.g.major@gmail.com> wrote:
>>
>>> Hi Eric,
>>> 
>>> > Can you post an example?  Here is a working example.
>>> 
>>> In your example, simply write "asymptote" in place of "sh" and replace the
>>> code by "size(100);" just to make sure it's valid asymptote (though the
>>> error occurs even if you don't).  In fact, I'm quite sure that asy never
>>> gets executed in this case.
>>> 
>>
>> Yes, even without any asymptote code, this breaks. But there seem to be
>> multiple problems. One is fix-empty-lines, a local routine defines
>> inside org-babel-asymptote-table-to-array: it seems to assume that the table
>> is a list of lists, whereas in this case it's a list of strings. The
>> following patch (which probably is wrong, in that it cures the symptom
>> rather than the disease):
>
> Yes, the problem is more general: uni-dimensional lists are not handled
> correctly.
>
> I attach a patch that should fix the problem (and simplify a lot that
> whole process).
>
> One thing, though. Now,
>
> | 1 | 2 |
>
> and (but this was already the case)
>
> | 1 |
> | 2 |
>
> are treated as bi-dimensional arrays, because that's how they really are
> passed to ob-asymptote.
>

Thanks Nicolas, both of these patches are now applied.

-- Eric

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/