emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* [Bug] #+call does not respect :colnames argument
@ 2013-07-26 15:12 Rick Frankel
  2013-07-26 17:53 ` Eric Schulte
  0 siblings, 1 reply; 9+ messages in thread
From: Rick Frankel @ 2013-07-26 15:12 UTC (permalink / raw)
  To: Eric Schulte; +Cc: emacs-orgmode

Eric-

I have debugged why the :colnames argument is not respected as to
removing the header from a table var.

* Given
#+name: with-hline
| A | B | C |
|---+---+---|
| 1 | 2 | 3 |
| 4 | 5 | 6 |

#+name: with-hline2
| B | C | D |
|---+---+---|
| 2 | 3 | 4 |
| 5 | 6 | 7 |

#+name: emacs-echo
#+BEGIN_SRC emacs-lisp :var table=with-hline :colnames yes
(mapcar (lambda (x) (mapcar '1+ x)) table)
#+END_SRC

#+call: emacs-echo(table=with-hline2)[:colnames yes]

* Evaluation
When the `#+call' line is executed, `org-babel-process-params' is
called multiple times. When it is called the last time to process
the actual input (here, calling =emacs-echo= with the param
=with-hline2=), the `:colname-names' parameter has already been (to
=(table "A" "B" "C")=, taken from the variable in the source blocks
header) as a side-effect of the call to
`org-babel-get-src-block-info'. So, when the block is executed later
in the function, then next call to `org-babel-process-params' does
not call disassemble table (taking into account the value of
`:colnames') but uses =processed-vars= directly instead, causing the 
full
table to be used without calling `org-babel-disassemble-tables'.

Here's the relevant section of code from `org-babel-process-params':

#+BEGIN_SRC emacs-lisp
(if (and (assoc :colname-names params)
(assoc :rowname-names params))
(list processed-vars)
(org-babel-disassemble-tables
processed-vars
(cdr (assoc :hlines params))
(cdr (assoc :colnames params))
(cdr (assoc :rownames params))))
#+END_SRC

And here's a cleaned-up debug trace:

#+BEGIN_EXAMPLE
(list processed-vars)
;; called again later in execute-src-block, does not properly parse 
=with-hline2= var
org-babel-process-params(((:comments . "") (:shebang . "") (:cache . 
"no") (:padline . "") (:noweb . "no") (:tangle . "no") (:exports . 
"results") (:results . "silent") (:var . "table=with-hline2") (:hlines . 
"yes") (:padnewline . "yes") (:session . "none") (:colnames . "yes") 
(:result-type . value) (:result-params "replace") (:rowname-names) 
(:colname-names (table "A" "B" "C"))))
;; calls disassemble-tables and returns with :colname-names set
org-babel-process-params(((:comments . "") (:shebang . "") (:cache . 
"no") (:padline . "") (:noweb . "no") (:tangle . "no") (:exports . 
"results") (:results . "replace") (:var . "table=with-hline") (:colnames 
. "yes") (:session . "none") (:padnewline . "yes") (:hlines . "yes")))
org-babel-get-src-block-info()
org-babel-execute-src-block(nil nil ((:var . "table=with-hline2") 
(:results . "silent")))
org-babel-ref-resolve("emacs-echo(table=with-hline2)[:colnames yes]")
org-babel-ref-parse("results=emacs-echo(table=with-hline2)[:colnames 
yes]")
org-babel-process-params(((:comments . "") (:shebang . "") (:cache . 
"no") (:padline . "") (:noweb . "no") (:tangle . "no") (:exports . 
"results") (:results . "replace") (:var . 
"results=emacs-echo(table=with-hline2)[:colnames yes]") (:colnames . 
"no") (:hlines . "yes") (:padnewline . "yes") (:session . "none")))
org-babel-lob-execute(("emacs-echo(table=with-hline2)[:colnames yes]" 
nil 0 nil))
#+END_EXAMPLE

rick

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [Bug] #+call does not respect :colnames argument
  2013-07-26 15:12 [Bug] #+call does not respect :colnames argument Rick Frankel
@ 2013-07-26 17:53 ` Eric Schulte
  2013-07-26 20:52   ` Rick Frankel
  0 siblings, 1 reply; 9+ messages in thread
From: Eric Schulte @ 2013-07-26 17:53 UTC (permalink / raw)
  To: Rick Frankel; +Cc: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 3694 bytes --]

Rick Frankel <rick@rickster.com> writes:

> Eric-
>
> I have debugged why the :colnames argument is not respected as to
> removing the header from a table var.
>
> * Given
> #+name: with-hline
> | A | B | C |
> |---+---+---|
> | 1 | 2 | 3 |
> | 4 | 5 | 6 |
>
> #+name: with-hline2
> | B | C | D |
> |---+---+---|
> | 2 | 3 | 4 |
> | 5 | 6 | 7 |
>
> #+name: emacs-echo
> #+BEGIN_SRC emacs-lisp :var table=with-hline :colnames yes
> (mapcar (lambda (x) (mapcar '1+ x)) table)
> #+END_SRC
>
> #+call: emacs-echo(table=with-hline2)[:colnames yes]
>
> * Evaluation
> When the `#+call' line is executed, `org-babel-process-params' is
> called multiple times. When it is called the last time to process
> the actual input (here, calling =emacs-echo= with the param
> =with-hline2=), the `:colname-names' parameter has already been (to
> =(table "A" "B" "C")=, taken from the variable in the source blocks
> header) as a side-effect of the call to
> `org-babel-get-src-block-info'. So, when the block is executed later
> in the function, then next call to `org-babel-process-params' does
> not call disassemble table (taking into account the value of
> `:colnames') but uses =processed-vars= directly instead, causing the 
> full
> table to be used without calling `org-babel-disassemble-tables'.
>
> Here's the relevant section of code from `org-babel-process-params':
>
> #+BEGIN_SRC emacs-lisp
> (if (and (assoc :colname-names params)
> (assoc :rowname-names params))
> (list processed-vars)
> (org-babel-disassemble-tables
> processed-vars
> (cdr (assoc :hlines params))
> (cdr (assoc :colnames params))
> (cdr (assoc :rownames params))))
> #+END_SRC
>
> And here's a cleaned-up debug trace:
>
> #+BEGIN_EXAMPLE
> (list processed-vars)
> ;; called again later in execute-src-block, does not properly parse 
> =with-hline2= var
> org-babel-process-params(((:comments . "") (:shebang . "") (:cache . 
> "no") (:padline . "") (:noweb . "no") (:tangle . "no") (:exports . 
> "results") (:results . "silent") (:var . "table=with-hline2") (:hlines . 
> "yes") (:padnewline . "yes") (:session . "none") (:colnames . "yes") 
> (:result-type . value) (:result-params "replace") (:rowname-names) 
> (:colname-names (table "A" "B" "C"))))
> ;; calls disassemble-tables and returns with :colname-names set
> org-babel-process-params(((:comments . "") (:shebang . "") (:cache . 
> "no") (:padline . "") (:noweb . "no") (:tangle . "no") (:exports . 
> "results") (:results . "replace") (:var . "table=with-hline") (:colnames 
> . "yes") (:session . "none") (:padnewline . "yes") (:hlines . "yes")))
> org-babel-get-src-block-info()
> org-babel-execute-src-block(nil nil ((:var . "table=with-hline2") 
> (:results . "silent")))
> org-babel-ref-resolve("emacs-echo(table=with-hline2)[:colnames yes]")
> org-babel-ref-parse("results=emacs-echo(table=with-hline2)[:colnames 
> yes]")
> org-babel-process-params(((:comments . "") (:shebang . "") (:cache . 
> "no") (:padline . "") (:noweb . "no") (:tangle . "no") (:exports . 
> "results") (:results . "replace") (:var . 
> "results=emacs-echo(table=with-hline2)[:colnames yes]") (:colnames . 
> "no") (:hlines . "yes") (:padnewline . "yes") (:session . "none")))
> org-babel-lob-execute(("emacs-echo(table=with-hline2)[:colnames yes]" 
> nil 0 nil))
> #+END_EXAMPLE
>
> rick

Hi Rick,

Thanks for taking the time to find the root of this problem.  I believe
I've fixed this by change the `org-babel-merge-params' function so that
when the value of a variable is update, then colname-names and
rowname-names values saved for that variable are removed.

In my local tests the attached patch fixes this issue.  If it works for
you as well then I'll apply it.

Thanks,


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-delete-colnames-rownames-for-replaced-variables.patch --]
[-- Type: text/x-patch, Size: 2940 bytes --]

From e789367e7ec4badd74e9aafb0249aa8798842f65 Mon Sep 17 00:00:00 2001
From: Eric Schulte <schulte.eric@gmail.com>
Date: Fri, 26 Jul 2013 11:48:51 -0600
Subject: [PATCH] delete colnames/rownames for replaced variables

  Thanks to Rick Frankel for help debugging this problem.

* lisp/ob-core.el (org-babel-merge-params): When merging parameters, if
  a variable is replaced with a new value, then delete colnames/rownames
  for the original value of that variable.
---
 lisp/ob-core.el | 40 +++++++++++++++++++++++++++++-----------
 1 file changed, 29 insertions(+), 11 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index b213c2a..c2722db 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -2269,7 +2269,8 @@ parameters when merging lists."
 				    new-params))
 			    result-params)
 		      output)))
-	 params results exports tangle noweb cache vars shebang comments padline)
+	 params results exports tangle noweb cache vars shebang comments padline
+	 clearnames)
 
     (mapc
      (lambda (plist)
@@ -2286,21 +2287,25 @@ parameters when merging lists."
 		   (setq vars
 			 (append
 			  (if (member name (mapcar #'car vars))
-			      (delq nil
-				    (mapcar
-				     (lambda (p)
-				       (unless (equal (car p) name) p))
-				     vars))
+			      (progn
+				(push name clearnames)
+				(delq nil
+				      (mapcar
+				       (lambda (p)
+					 (unless (equal (car p) name) p))
+				       vars)))
 			    vars)
 			  (list (cons name pair))))
 		 ;; if no name is given and we already have named variables
 		 ;; then assign to named variables in order
 		 (if (and vars (nth variable-index vars))
-		     (prog1 (setf (cddr (nth variable-index vars))
-				  (concat (symbol-name
-					   (car (nth variable-index vars)))
-					  "=" (cdr pair)))
-		       (incf variable-index))
+		     (let ((name (car (nth variable-index vars))))
+		       (push name clearnames) ; clear out colnames
+					      ; and rownames
+					      ; for replace vars
+		       (prog1 (setf (cddr (nth variable-index vars))
+				    (concat (symbol-name name) "=" (cdr pair)))
+			 (incf variable-index)))
 		   (error "Variable \"%s\" must be assigned a default value"
 			  (cdr pair))))))
 	    (:results
@@ -2347,6 +2352,19 @@ parameters when merging lists."
      plists)
     (setq vars (reverse vars))
     (while vars (setq params (cons (cons :var (cddr (pop vars))) params)))
+    ;; clear out col-names and row-names for replaced variables
+    (mapc
+     (lambda (name)
+       (mapc
+	(lambda (param)
+	  (setf (cdr (assoc param params))
+		(remove-if (lambda (pair) (equal (car pair) name))
+			   (cdr (assoc param params))))
+	  (setf params (remove-if (lambda (pair) (and (equal (car pair) param)
+						 (null (cdr pair))))
+				  params)))
+	(list :colname-names :rowname-names)))
+     clearnames)
     (mapc
      (lambda (hd)
        (let ((key (intern (concat ":" (symbol-name hd))))
-- 
1.8.3.3


[-- Attachment #3: Type: text/plain, Size: 46 bytes --]


-- 
Eric Schulte
http://cs.unm.edu/~eschulte

^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [Bug] #+call does not respect :colnames argument
  2013-07-26 17:53 ` Eric Schulte
@ 2013-07-26 20:52   ` Rick Frankel
  2013-07-27  0:54     ` Eric Schulte
  0 siblings, 1 reply; 9+ messages in thread
From: Rick Frankel @ 2013-07-26 20:52 UTC (permalink / raw)
  To: Eric Schulte; +Cc: emacs-orgmode

On Fri, Jul 26, 2013 at 11:53:33AM -0600, Eric Schulte wrote:
> Rick Frankel <rick@rickster.com> writes:
> 
> > I have debugged why the :colnames argument is not respected as to
> > removing the header from a table var.

> Thanks for taking the time to find the root of this problem.  I believe
> I've fixed this by change the `org-babel-merge-params' function so that
> when the value of a variable is update, then colname-names and
> rowname-names values saved for that variable are removed.
> 
> In my local tests the attached patch fixes this issue.  If it works for
> you as well then I'll apply it.

Works for me. Thanx, this will simplify a lot of code (e.g., the
example dot-from-tables that was floating around last week)

Aside... It's difficult for me to follow the code, so can you explain
the why the different results between:

#+call: emacs-echo(table=with-hline2) :colnames yes

#+results:
| B | C | D |
|---+---+---|
| 3 | 4 | 5 |
| 6 | 7 | 8 |

and

#+call: emacs-echo(table=with-hline2)[:colnames yes]

#+results:
| B | C | D |


thanx again,
rick

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [Bug] #+call does not respect :colnames argument
  2013-07-26 20:52   ` Rick Frankel
@ 2013-07-27  0:54     ` Eric Schulte
  2013-07-29  1:10       ` Rick Frankel
  2013-07-29  4:18       ` Achim Gratz
  0 siblings, 2 replies; 9+ messages in thread
From: Eric Schulte @ 2013-07-27  0:54 UTC (permalink / raw)
  To: emacs-orgmode

Rick Frankel <rick@rickster.com> writes:

> On Fri, Jul 26, 2013 at 11:53:33AM -0600, Eric Schulte wrote:
>> Rick Frankel <rick@rickster.com> writes:
>> 
>> > I have debugged why the :colnames argument is not respected as to
>> > removing the header from a table var.
>
>> Thanks for taking the time to find the root of this problem.  I believe
>> I've fixed this by change the `org-babel-merge-params' function so that
>> when the value of a variable is update, then colname-names and
>> rowname-names values saved for that variable are removed.
>> 
>> In my local tests the attached patch fixes this issue.  If it works for
>> you as well then I'll apply it.
>
> Works for me. Thanx, this will simplify a lot of code (e.g., the
> example dot-from-tables that was floating around last week)
>
> Aside... It's difficult for me to follow the code, so can you explain
> the why the different results between:
>
> #+call: emacs-echo(table=with-hline2) :colnames yes
>
> #+results:
> | B | C | D |
> |---+---+---|
> | 3 | 4 | 5 |
> | 6 | 7 | 8 |
>
> and
>
> #+call: emacs-echo(table=with-hline2)[:colnames yes]
>
> #+results:
> | B | C | D |
>

The later is not valid call line syntax, see [1] for a full description
of the call line syntax.  In effect what happens in the latter case, is
you set :colnames to "yes]".

I've just pushed up that patch.

Thanks for you help debugging this.

>
>
> thanx again,
> rick


Footnotes: 
[1]  (info "(org)Evaluating Code Blocks")

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [Bug] #+call does not respect :colnames argument
  2013-07-27  0:54     ` Eric Schulte
@ 2013-07-29  1:10       ` Rick Frankel
  2013-07-29  4:18       ` Achim Gratz
  1 sibling, 0 replies; 9+ messages in thread
From: Rick Frankel @ 2013-07-29  1:10 UTC (permalink / raw)
  To: Eric Schulte; +Cc: emacs-orgmode

On Fri, Jul 26, 2013 at 06:54:08PM -0600, Eric Schulte wrote:
> Rick Frankel <rick@rickster.com> writes:
> The later is not valid call line syntax, see [1] for a full description
> of the call line syntax.  In effect what happens in the latter case, is
> you set :colnames to "yes]".

Oh, ok. I just saw other people using that syntax on the  list so wondered.

> Thanks for you help debugging this.

my pleasure. ust wish i understood the code well enought to provide
patches instead :{.

rick

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [Bug] #+call does not respect :colnames argument
  2013-07-27  0:54     ` Eric Schulte
  2013-07-29  1:10       ` Rick Frankel
@ 2013-07-29  4:18       ` Achim Gratz
  2013-07-29 14:00         ` Eric Schulte
  1 sibling, 1 reply; 9+ messages in thread
From: Achim Gratz @ 2013-07-29  4:18 UTC (permalink / raw)
  To: emacs-orgmode

Eric Schulte writes:
> I've just pushed up that patch.

…which breaks testing:

10 unexpected results:
   FAILED  ob-exp/use-case-of-reading-entry-properties
   FAILED  test-ob-header-arg-defaults/global/call
   FAILED  test-ob-header-arg-defaults/global/noweb
   FAILED  test-ob-header-arg-defaults/tree/accumulate/call
   FAILED  test-ob-header-arg-defaults/tree/accumulate/noweb
   FAILED  test-ob-header-arg-defaults/tree/complex/call
   FAILED  test-ob-header-arg-defaults/tree/complex/noweb
   FAILED  test-ob-header-arg-defaults/tree/overwrite/call
   FAILED  test-ob-header-arg-defaults/tree/overwrite/noweb
   FAILED  test-ob-lob/call-with-header-arguments

with 

    (wrong-type-argument consp nil)

None of these tests deal with table arguments AFAICS.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [Bug] #+call does not respect :colnames argument
  2013-07-29  4:18       ` Achim Gratz
@ 2013-07-29 14:00         ` Eric Schulte
  2013-07-29 17:01           ` Achim Gratz
  0 siblings, 1 reply; 9+ messages in thread
From: Eric Schulte @ 2013-07-29 14:00 UTC (permalink / raw)
  To: Achim Gratz; +Cc: emacs-orgmode

Achim Gratz <Stromeko@nexgo.de> writes:

> Eric Schulte writes:
>> I've just pushed up that patch.
>
> …which breaks testing:
>
> 10 unexpected results:
>    FAILED  ob-exp/use-case-of-reading-entry-properties
>    FAILED  test-ob-header-arg-defaults/global/call
>    FAILED  test-ob-header-arg-defaults/global/noweb
>    FAILED  test-ob-header-arg-defaults/tree/accumulate/call
>    FAILED  test-ob-header-arg-defaults/tree/accumulate/noweb
>    FAILED  test-ob-header-arg-defaults/tree/complex/call
>    FAILED  test-ob-header-arg-defaults/tree/complex/noweb
>    FAILED  test-ob-header-arg-defaults/tree/overwrite/call
>    FAILED  test-ob-header-arg-defaults/tree/overwrite/noweb
>    FAILED  test-ob-lob/call-with-header-arguments
>
> with 
>
>     (wrong-type-argument consp nil)
>
> None of these tests deal with table arguments AFAICS.
>

I just pushed up a fix, I now only get 1 failing test locally, but it is
related to table alignment so I believe it must be unrelated to this
commit.

Best,

>
>
> Regards,
> Achim.

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [Bug] #+call does not respect :colnames argument
  2013-07-29 14:00         ` Eric Schulte
@ 2013-07-29 17:01           ` Achim Gratz
  2013-07-29 17:20             ` Eric Schulte
  0 siblings, 1 reply; 9+ messages in thread
From: Achim Gratz @ 2013-07-29 17:01 UTC (permalink / raw)
  To: emacs-orgmode

Eric Schulte writes:
> I just pushed up a fix, I now only get 1 failing test locally, but it is
> related to table alignment so I believe it must be unrelated to this
> commit.

I confirm the fix.  As to your oither test fail, it doesn't show up for
me and it most certainly shouldn't show up for you if you started the
test suite via "make test-dirty".  Is that not working for you?


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [Bug] #+call does not respect :colnames argument
  2013-07-29 17:01           ` Achim Gratz
@ 2013-07-29 17:20             ` Eric Schulte
  0 siblings, 0 replies; 9+ messages in thread
From: Eric Schulte @ 2013-07-29 17:20 UTC (permalink / raw)
  To: Achim Gratz; +Cc: emacs-orgmode

Achim Gratz <Stromeko@nexgo.de> writes:

> Eric Schulte writes:
>> I just pushed up a fix, I now only get 1 failing test locally, but it is
>> related to table alignment so I believe it must be unrelated to this
>> commit.
>
> I confirm the fix.  As to your oither test fail, it doesn't show up for
> me and it most certainly shouldn't show up for you if you started the
> test suite via "make test-dirty".  Is that not working for you?
>

With "make test-dirty" I get all tests passing as expected.  The table
alignment error was from running the test suite interactively with the
`org-test-run-all-tests' function, so it is probably due to something
specific to my config.

Thanks,

>
>
> Regards,
> Achim.

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2013-07-29 17:23 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-07-26 15:12 [Bug] #+call does not respect :colnames argument Rick Frankel
2013-07-26 17:53 ` Eric Schulte
2013-07-26 20:52   ` Rick Frankel
2013-07-27  0:54     ` Eric Schulte
2013-07-29  1:10       ` Rick Frankel
2013-07-29  4:18       ` Achim Gratz
2013-07-29 14:00         ` Eric Schulte
2013-07-29 17:01           ` Achim Gratz
2013-07-29 17:20             ` Eric Schulte

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).