[PATCH] Fix error prone babel table output format detection

[PATCH] Fix error prone babel table output format detection

[Ian Kelling]

From dc0b727328266785528fe160046ae1aa8df8a993 Mon Sep 17 00:00:00 2001
Message-ID: <87zjj2ous9.fsf@treetowl.lan>
MIME-Version: 1.0
Content-Type: text/plain

* lisp/ob-core.el: Test that all elements are in a list are lists
instead of just the first.

org-babel table output uses different formatting for a list of lists,
but detects it incorrectly causing an error, as in this example:
#+begin_src emacs-lisp 
'((1) 2)
#+end_src
---
 lisp/ob-core.el |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 1348f04..5872b68 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -2185,8 +2185,7 @@ code ---- the results are extracted in the syntax of the source
 		  (goto-char beg)
 		  (insert (concat (orgtbl-to-orgtbl
 				   (if (or (eq 'hline (car result))
-					   (and (listp (car result))
-						(listp (cdr (car result)))))
+					   (cl-every 'listp result))
 				       result (list result))
 				   '(:fmt (lambda (cell) (format "%s" cell)))) "\n"))
 		  (goto-char beg) (when (org-at-table-p) (org-table-align)))
-- 
1.7.10.4

Re: [PATCH] Fix error prone babel table output format detection

[Achim Gratz]

Ian Kelling writes:
> org-babel table output uses different formatting for a list of lists,
> but detects it incorrectly causing an error, as in this example:
> #+begin_src emacs-lisp 
> '((1) 2)
> #+end_src

So this isn't a proper table, what do you expect to happen?

> -					   (and (listp (car result))
> -						(listp (cdr (car result)))))
> +					   (cl-every 'listp result))

This is wrong, because a table can come with any number of 'hline
symbols, so ostensibly not every element will be a listp.  Besides, you
can't use cl-every unless Org drops backwards compatibility with older
Emacsen or mandates cl-lib to be present.  Even then, you'd also need to
require cl-extra.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [PATCH] Fix error prone babel table output format detection

[Ian Kelling]

Achim Gratz <Stromeko@nexgo.de> writes:

> Ian Kelling writes:
>> org-babel table output uses different formatting for a list of lists,
>> but detects it incorrectly causing an error, as in this example:
>> #+begin_src emacs-lisp 
>> '((1) 2)
>> #+end_src
>
> So this isn't a proper table, what do you expect to happen?

I expect to be able to emacs -q, make an org mode buffer containing 
that block, do ctrl-c c on it, and not have an error message pop up and
fail to have any output. What is improper about it? Would it be better
if org did not try to format it as a table?

>
>> -					   (and (listp (car result))
>> -						(listp (cdr (car result)))))
>> +					   (cl-every 'listp result))
>
> This is wrong, because a table can come with any number of 'hline
> symbols, so ostensibly not every element will be a listp.

Yes, I agree. It will have to test if elements are listp or hline. 


> Besides, you
> can't use cl-every unless Org drops backwards compatibility with older
> Emacsen or mandates cl-lib to be present.  Even then, you'd also need to
> require cl-extra.

I forgot about the multiple parts of cl. I originally wrote it without,
and can change it.

Re: [PATCH] Fix error prone babel table output format detection

[Ian Kelling]

Ian Kelling <ian@iankelling.org> writes:

> Achim Gratz <Stromeko@nexgo.de> writes:
>
>> Ian Kelling writes:
>>> org-babel table output uses different formatting for a list of lists,
>>> but detects it incorrectly causing an error, as in this example:
>>> #+begin_src emacs-lisp 
>>> '((1) 2)
>>> #+end_src
>>
>> So this isn't a proper table, what do you expect to happen?
>
> I expect to be able to emacs -q, make an org mode buffer containing 
> that block, do ctrl-c c on it, and not have an error message pop up and
> fail to have any output. What is improper about it? Would it be better
> if org did not try to format it as a table?

Actually, I see what you mean by improper.

>
>>
>>> -					   (and (listp (car result))
>>> -						(listp (cdr (car result)))))
>>> +					   (cl-every 'listp result))
>>
>> This is wrong, because a table can come with any number of 'hline
>> symbols, so ostensibly not every element will be a listp.
>
> Yes, I agree. It will have to test if elements are listp or hline. 
>
>
>> Besides, you
>> can't use cl-every unless Org drops backwards compatibility with older
>> Emacsen or mandates cl-lib to be present.  Even then, you'd also need to
>> require cl-extra.
>
> I forgot about the multiple parts of cl. I originally wrote it without,
> and can change it.

Below is a patch that addresses the 2 previously mentioned
problems. 

-- >8 --
Subject: [PATCH] Fix error prone babel table output format detection

* lisp/ob-core.el: Test that all elements are in a list are lists or
'hline instead of just the first.

org-babel table output uses different formatting for a list of lists,
but detects it incorrectly causing an error, as in this example:
#+begin_src emacs-lisp 
'((1) 2)
#+end_src
---
 lisp/ob-core.el |   13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 1348f04..b5b0bc7 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -2184,9 +2184,16 @@ code ---- the results are extracted in the syntax of the source
 		 ((funcall proper-list-p result)
 		  (goto-char beg)
 		  (insert (concat (orgtbl-to-orgtbl
-				   (if (or (eq 'hline (car result))
-					   (and (listp (car result))
-						(listp (cdr (car result)))))
+				     (if (let ((len (length result))
+					       (proper t)
+					       (i 0)
+					       elem)
+					   (while (and proper (< i len ))
+					     (setq elem (nth i result))
+					     (unless (or (listp elem) (eq elem 'hline))
+					       (setq proper nil))
+					     (setq i (1+ i)))
+					   proper)
 				       result (list result))
 				   '(:fmt (lambda (cell) (format "%s" cell)))) "\n"))
 		  (goto-char beg) (when (org-at-table-p) (org-table-align)))
-- 
1.7.10.4

Re: [PATCH] Fix error prone babel table output format detection

[Ian Kelling]

Ian Kelling <ian@iankelling.org> writes:
> Below is a patch that addresses the 2 previously mentioned
> problems. 

It's a bit late.  here is the same patch with correct indentation.

> -- >8 --
Subject: [PATCH] Fix error prone babel table output format detection

* lisp/ob-core.el: Test that all elements are in a list are lists or
'hline instead of just the first.

org-babel table output uses different formatting for a list of lists,
but detects it incorrectly causing an error. An example of a block
causing an error is an emacs lisp source block containing just 1 line:
'((1) 2)
---
 lisp/ob-core.el |   13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 1348f04..9eb2c7a 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -2184,9 +2184,16 @@ code ---- the results are extracted in the syntax of the source
 		 ((funcall proper-list-p result)
 		  (goto-char beg)
 		  (insert (concat (orgtbl-to-orgtbl
-				   (if (or (eq 'hline (car result))
-					   (and (listp (car result))
-						(listp (cdr (car result)))))
+				   (if (let ((len (length result))
+					     (proper t)
+					     (i 0)
+					     elem)
+					 (while (and proper (< i len ))
+					   (setq elem (nth i result))
+					   (unless (or (listp elem) (eq elem 'hline))
+					     (setq proper nil))
+					   (setq i (1+ i)))
+					 proper)
 				       result (list result))
 				   '(:fmt (lambda (cell) (format "%s" cell)))) "\n"))
 		  (goto-char beg) (when (org-at-table-p) (org-table-align)))
-- 
1.7.10.4

Re: [PATCH] Fix error prone babel table output format detection

[Ian Kelling]

Ian Kelling <ian@iankelling.org> writes:
> It's a bit late.  here is the same patch with correct indentation.

That patch went out of it's way not to check more of the list than was
necessary, but after sending it, I kept thinking that it does extra
things which possibly negate any performance benefit of not checking the
whole list. So here is a simpler patch does the same thing, but goes
over the whole list. I'd love to hear a more experienced emacs lisper
weigh in on which is better.

-- >8 --
Subject: [PATCH] Fix error prone babel table output format detection

* lisp/ob-core.el: Test that all elements are in a list are lists or
'hline instead of just the first.

org-babel table output uses different formatting for a list of lists,
but detects it incorrectly causing an error. An example of a block
causing an error is an emacs lisp source block containing just 1 line:
'((1) 2)
---
 lisp/ob-core.el |    7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 1348f04..05ccb00 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -2184,9 +2184,10 @@ code ---- the results are extracted in the syntax of the source
 		 ((funcall proper-list-p result)
 		  (goto-char beg)
 		  (insert (concat (orgtbl-to-orgtbl
-				   (if (or (eq 'hline (car result))
-					   (and (listp (car result))
-						(listp (cdr (car result)))))
+				   (if (let ((proper t))
+					 (dolist (elem result proper)
+					   (unless (or (listp elem) (eq elem 'hline))
+					     (setq proper nil))))
 				       result (list result))
 				   '(:fmt (lambda (cell) (format "%s" cell)))) "\n"))
 		  (goto-char beg) (when (org-at-table-p) (org-table-align)))
-- 
1.7.10.4

Re: [PATCH] Fix error prone babel table output format detection

[Eric Schulte]

Ian Kelling <ian@iankelling.org> writes:

> Ian Kelling <ian@iankelling.org> writes:
>> It's a bit late.  here is the same patch with correct indentation.
>
> That patch went out of it's way not to check more of the list than was
> necessary, but after sending it, I kept thinking that it does extra
> things which possibly negate any performance benefit of not checking the
> whole list. So here is a simpler patch does the same thing, but goes
> over the whole list. I'd love to hear a more experienced emacs lisper
> weigh in on which is better.
>

Hi Ian,

You should use the `org-every' function here.  Look at the source of
that function to see code to efficiently perform this sort of check.

Best,

>
> -- >8 --
> Subject: [PATCH] Fix error prone babel table output format detection
>
> * lisp/ob-core.el: Test that all elements are in a list are lists or
> 'hline instead of just the first.
>
> org-babel table output uses different formatting for a list of lists,
> but detects it incorrectly causing an error. An example of a block
> causing an error is an emacs lisp source block containing just 1 line:
> '((1) 2)
> ---
>  lisp/ob-core.el |    7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/lisp/ob-core.el b/lisp/ob-core.el
> index 1348f04..05ccb00 100644
> --- a/lisp/ob-core.el
> +++ b/lisp/ob-core.el
> @@ -2184,9 +2184,10 @@ code ---- the results are extracted in the syntax of the source
>  		 ((funcall proper-list-p result)
>  		  (goto-char beg)
>  		  (insert (concat (orgtbl-to-orgtbl
> -				   (if (or (eq 'hline (car result))
> -					   (and (listp (car result))
> -						(listp (cdr (car result)))))
> +				   (if (let ((proper t))
> +					 (dolist (elem result proper)
> +					   (unless (or (listp elem) (eq elem 'hline))
> +					     (setq proper nil))))
>  				       result (list result))
>  				   '(:fmt (lambda (cell) (format "%s" cell)))) "\n"))
>  		  (goto-char beg) (when (org-at-table-p) (org-table-align)))

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: [PATCH] Fix error prone babel table output format detection

[Ian Kelling]

Eric Schulte <schulte.eric@gmail.com> writes:

> Hi Ian,
>
> You should use the `org-every' function here.  Look at the source of
> that function to see code to efficiently perform this sort of check.
>
> Best,
>

Brilliant. Thank you. The updated patch below should be good.

-- >8 --
Subject: [PATCH] Fix error prone babel table output format detection

* lisp/ob-core.el: Test that all elements are in a list are lists or
'hline instead of just the first.

org-babel table output uses different formatting for a list of lists,
but detects it incorrectly causing an error. An example of a block
causing an error is an emacs lisp source block containing just 1 line:
'((1) 2)
---
 lisp/ob-core.el |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 1348f04..4ddafaf 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -2184,9 +2184,9 @@ code ---- the results are extracted in the syntax of the source
 		 ((funcall proper-list-p result)
 		  (goto-char beg)
 		  (insert (concat (orgtbl-to-orgtbl
-				   (if (or (eq 'hline (car result))
-					   (and (listp (car result))
-						(listp (cdr (car result)))))
+				   (if (org-every
+					(lambda (el) (or (listp el) (eq el 'hline)))
+					result)
 				       result (list result))
 				   '(:fmt (lambda (cell) (format "%s" cell)))) "\n"))
 		  (goto-char beg) (when (org-at-table-p) (org-table-align)))
-- 
1.7.10.4

Re: [PATCH] Fix error prone babel table output format detection

[Eric Schulte]

This looks good to me.  Could you reformat the patch with

  git format-patch

and attach the results (this will be easier to apply).  Also, this patch
is small enough to apply without any sort of copyright attribution, but
if you think you might make larger contributions in the future, please
look over the org contribution instructions [1].

Thanks!
Eric


Footnotes: 
[1]  http://orgmode.org/worg/org-contribute.html

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: [PATCH] Fix error prone babel table output format detection

[Ian Kelling]

[-- Attachment #1: Type: text/plain, Size: 697 bytes --]

Eric Schulte <schulte.eric@gmail.com> writes:

> This looks good to me.  Could you reformat the patch with
>
>   git format-patch
>
> and attach the results (this will be easier to apply).


Done. I was actually following the instructions in man git-format-patch by
not attaching it before, but I will avoid that in future.

> Also, this patch
> is small enough to apply without any sort of copyright attribution, but
> if you think you might make larger contributions in the future, please
> look over the org contribution instructions [1].>

I've completed emacs fsf copyright assignment. I also have another patch
that has been on the mailing list for about 2 weeks or so waiting for a reply.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0002-Fix-error-prone-babel-table-output-format-detection.patch --]
[-- Type: text/x-diff, Size: 1298 bytes --]

From 4122d20f1253c2cbf5e73070ea00665bbc802275 Mon Sep 17 00:00:00 2001
From: Ian Kelling <ian@iankelling.org>
Date: Wed, 30 Apr 2014 21:56:52 -0700
Subject: [PATCH] Fix error prone babel table output format detection

* lisp/ob-core.el: Test that all elements are in a list are lists or
'hline instead of just the first.

org-babel table output uses different formatting for a list of lists,
but detects it incorrectly causing an error. An example of a block
causing an error is an emacs lisp source block containing just 1 line:
'((1) 2)
---
 lisp/ob-core.el |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 1348f04..4ddafaf 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -2184,9 +2184,9 @@ code ---- the results are extracted in the syntax of the source
 		 ((funcall proper-list-p result)
 		  (goto-char beg)
 		  (insert (concat (orgtbl-to-orgtbl
-				   (if (or (eq 'hline (car result))
-					   (and (listp (car result))
-						(listp (cdr (car result)))))
+				   (if (org-every
+					(lambda (el) (or (listp el) (eq el 'hline)))
+					result)
 				       result (list result))
 				   '(:fmt (lambda (cell) (format "%s" cell)))) "\n"))
 		  (goto-char beg) (when (org-at-table-p) (org-table-align)))
-- 
1.7.10.4

Re: [PATCH] Fix error prone babel table output format detection

[Bastien]

Hi Ian,

Ian Kelling <ian@iankelling.org> writes:

> Done. I was actually following the instructions in man git-format-patch by
> not attaching it before, but I will avoid that in future.

Applied, thanks.  I slightly rewrote the changelog message, please
have a look for further patches.

Thanks!

-- 
 Bastien