2 Org tests failing

2 Org tests failing

[Sebastien Vauban]

Hello,

I dunno if this must be signalled but, when runnning:

--8<---------------cut here---------------start------------->8---
emacs -Q --batch -L lisp/ -L testing/ -l org-test.el
      --eval '(setq org-confirm-babel-evaluate nil)' -f org-test-run-batch-tests
--8<---------------cut here---------------end--------------->8---

on the fresh Org-mode version 8.2.6 (release_8.2.6-1010-g1ca86f), I do get:

--8<---------------cut here---------------start------------->8---
Running org_test, please wait (this can take a while)...
   FAILED  test-org-table/org-table-calc-current-TBLFM
   FAILED  test-org-table/org-table-calc-current-TBLFM-when-stop-because-of-error
--8<---------------cut here---------------end--------------->8---

Best regards,
  Seb

-- 
Sebastien Vauban

Re: 2 Org tests failing

[Bastien]

Sebastien Vauban <sva-news-D0wtAvR13HarG/iDocfnWg@public.gmane.org>
writes:

> Running org_test, please wait (this can take a while)...
>    FAILED  test-org-table/org-table-calc-current-TBLFM
>    FAILED  test-org-table/org-table-calc-current-TBLFM-when-stop-because-of-error

I can't reproduce this.  Can you run the tests manually and report
the backtrace?

-- 
 Bastien

Re: 2 Org tests failing

[Sebastien Vauban]

Bastien wrote:
> Sebastien Vauban <sva-news-D0wtAvR13HarG/iDocfnWg-XMD5yJDbdMReXY1tMh2IBg@public.gmane.org>
> writes:
>
>> Running org_test, please wait (this can take a while)...
>>    FAILED  test-org-table/org-table-calc-current-TBLFM
>>    FAILED  test-org-table/org-table-calc-current-TBLFM-when-stop-because-of-error
>
> I can't reproduce this.  Can you run the tests manually and report
> the backtrace?

--8<---------------cut here---------------start------------->8---
Test test-org-table/org-table-calc-current-TBLFM condition:
    (file-error "Opening output file" "No such file or directory" "/tmp/org_test/org-test8984aQq")
   FAILED  354/427  test-org-table/org-table-calc-current-TBLFM
Test test-org-table/org-table-calc-current-TBLFM-when-stop-because-of-error backtrace:
  write-region("" nil "/tmp/org_test/org-test8984naw" nil silent nil e
  byte-code("\305\30G!\203\307  !\202\310       \"!
                                                   \203\n
                                                         P\f\203
  make-temp-file("org-test")
  (let ((file (make-temp-file "org-test")) (kill-buffer-query-function
  (lambda nil (let ((file (make-temp-file "org-test")) (kill-buffer-qu
  byte-code("\306\307!q\210\310\216\311 \312\216\313\314\315\316\3
  ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
  byte-code("\306\307!\211\211r\310\311!q\210\312 d\313\223)L\210)\3
  ert-run-test([cl-struct-ert-test test-org-table/org-table-calc-curre
  ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st
  ert-run-tests("\\(org\\|ob\\)" #[(event-type &rest event-args) "\306
  ert-run-tests-batch("\\(org\\|ob\\)")
  ert-run-tests-batch-and-exit("\\(org\\|ob\\)")
  (let ((org-id-track-globally t) (org-test-selector (if org-test-sele
  org-test-run-batch-tests()
  call-interactively(org-test-run-batch-tests nil nil)
  command-execute(org-test-run-batch-tests)
  command-line-1(("-L" "lisp/" "-L" "testing/" "-l" "org-test.el" "--e
  command-line()
  normal-top-level()
--8<---------------cut here---------------end--------------->8---

These were run successfully from within Emacs itself (on Windows).

I've solved the reported problem by creating the missing TMPDIR (set to
/tmp/org_test) for the invocation from my Zsh (Cygwin) terminal...

Though, now, I do have another test failing when testing from the shell:

--8<---------------cut here---------------start------------->8---
Test ob-shell/bash-uses-assoc-arrays backtrace:
  signal(ert-test-failed (((should (equal "20 cm" (org-babel-execute-s
  ert-fail(((should (equal "20 cm" (org-babel-execute-src-block))) :fo
  (if (unwind-protect (setq value-502 (apply fn-500 args-501)) (setq f
  (let (form-description-504) (if (unwind-protect (setq value-502 (app
  (let ((value-502 (quote ert-form-evaluation-aborted-503))) (let (for
  (let ((fn-500 (function equal)) (args-501 (list "20 cm" (org-babel-e
  (save-restriction (org-babel-next-src-block 2) (let ((fn-500 (functi
  (progn (org-id-goto "82320a48-3409-49d7-85c9-5de1c6d3ff87") (setq to
  (unwind-protect (progn (org-id-goto "82320a48-3409-49d7-85c9-5de1c6d
  (let ((save-match-data-internal (match-data))) (unwind-protect (prog
  (progn (let ((save-match-data-internal (match-data))) (unwind-protec
  (unwind-protect (progn (let ((save-match-data-internal (match-data))
  (let ((wconfig (current-window-configuration))) (unwind-protect (pro
  (unwind-protect (let ((wconfig (current-window-configuration))) (unw
  (let* ((id-location (org-id-find "82320a48-3409-49d7-85c9-5de1c6d3ff
  (lambda nil (let* ((id-location (org-id-find "82320a48-3409-49d7-85c
  byte-code("\306\307!q\210\310\216\311 \312\216\313\314\315\316\3
  ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
  byte-code("\306\307!\211\211r\310\311!q\210\312 d\313\223)L\210)\3
  ert-run-test([cl-struct-ert-test ob-shell/bash-uses-assoc-arrays "Ba
  ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st
  ert-run-tests("\\(org\\|ob\\)" #[(event-type &rest event-args) "\306
  ert-run-tests-batch("\\(org\\|ob\\)")
  ert-run-tests-batch-and-exit("\\(org\\|ob\\)")
  (let ((org-id-track-globally t) (org-test-selector (if org-test-sele
  org-test-run-batch-tests()
  call-interactively(org-test-run-batch-tests nil nil)
  command-execute(org-test-run-batch-tests)
  command-line-1(("-L" "lisp/" "-L" "testing/" "-l" "org-test.el" "--e
  command-line()
  normal-top-level()
Test ob-shell/bash-uses-assoc-arrays condition:
    (ert-test-failed
     ((should
       (equal "20 cm"
              (org-babel-execute-src-block)))
      :form
      (equal "20 cm" nil)
      :value nil :explanation
      (different-types "20 cm" nil)))
   FAILED   24/427  ob-shell/bash-uses-assoc-arrays
--8<---------------cut here---------------end--------------->8---

Best regards,
  Seb

-- 
Sebastien Vauban

Re: 2 Org tests failing

[Achim Gratz]

Sebastien Vauban writes:
> I've solved the reported problem by creating the missing TMPDIR (set to
> /tmp/org_test) for the invocation from my Zsh (Cygwin) terminal...

The first thing the build system does before testing is

install -m 755 -d /tmp/tmp-orgtest

so you either pick up the wrong install command via path or have
otherwise misconfigured MKDIR in local.mk.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

Re: 2 Org tests failing

[Sebastien Vauban]

Achim Gratz wrote:
> Sebastien Vauban writes:
>> I've solved the reported problem by creating the missing TMPDIR (set to
>> /tmp/org_test) for the invocation from my Zsh (Cygwin) terminal...
>
> The first thing the build system does before testing is
>
> install -m 755 -d /tmp/tmp-orgtest
>
> so you either pick up the wrong install command via path or have
> otherwise misconfigured MKDIR in local.mk.

No, I was using my own function:

--8<---------------cut here---------------start------------->8---
org_test () {
    (
        cd $HOME/Public/Repositories/org-mode;
        TMPDIR=/tmp/org_test emacs -Q --batch -L lisp/ -L testing/ -l org-test.el --eval '(setq org-confirm-babel-evaluate nil)' -f org-test-run-batch-tests
    )
}
--8<---------------cut here---------------end--------------->8---

I just ran `make test' and got the same error for
`ob-shell/bash-uses-assoc-arrays'.

BTW, how can I avoid compiling the source directory before launching the
tests?  I prefer having only .el files, as I was mistaken multiple times
when trying to understand executions of code I modified.

Best regards,
  Seb

-- 
Sebastien Vauban

Re: 2 Org tests failing

[Sebastien Vauban]

Sebastien Vauban wrote:
> I just ran `make test' and got the same error for
> `ob-shell/bash-uses-assoc-arrays'.

Now that I updated to Org-mode version 8.2.7
(release_8.2.7-1114-g6c2838), I have an extra test failing:

--8<---------------cut here---------------start------------->8---
Test test-org-element/headline-properties backtrace:
  signal(ert-test-failed (((should-not (org-test-with-temp-text "* Hea
  ert-fail(((should-not (org-test-with-temp-text "* Headline\n#+BEGIN_
  (if (not (unwind-protect (setq value-1904 (let ((inside-text (if ...
  (let (form-description-1905) (if (not (unwind-protect (setq value-19
  (let ((value-1904 (ert--gensym "ert-form-evaluation-aborted-"))) (le
  (lambda nil (let ((value-1900 (ert--gensym "ert-form-evaluation-abor
  byte-code("\306\307!q\210\310\216\311 \312\216\313\314\315\316\3
  ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
  byte-code("\306\307!\211\211r\310\311!q\210\312 d\313\223)L\210)\3
  ert-run-test([cl-struct-ert-test test-org-element/headline-propertie
  ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st
  ert-run-tests("\\(org\\|ob\\)" #[(event-type &rest event-args) "\306
  ert-run-tests-batch("\\(org\\|ob\\)")
  ert-run-tests-batch-and-exit("\\(org\\|ob\\)")
  (let ((org-id-track-globally t) (org-test-selector (if org-test-sele
  org-test-run-batch-tests()
  call-interactively(org-test-run-batch-tests nil nil)
  command-execute(org-test-run-batch-tests)
  command-line-1(("-L" "lisp/" "-L" "testing/" "-l" "org-test.el" "--e
  command-line()
  normal-top-level()
Test test-org-element/headline-properties condition:
    (ert-test-failed
     ((should-not
       (org-test-with-temp-text "* Headline\n#+BEGIN_EXAMPLE\n:PROPERTIES:\n:foo: bar\n:END:\n#+END_EXAMPLE"
         (org-element-property :FOO ...)))
      :form
      (let
          ((inside-text ...))
        (with-temp-buffer
          (org-mode)
          (let ... ...)
          (org-element-property :FOO ...)))
      :value "bar"))
   FAILED  169/427  test-org-element/headline-properties
--8<---------------cut here---------------end--------------->8---

Best regards,
  Seb

-- 
Sebastien Vauban

Re: 2 Org tests failing

[Sebastien Vauban]

Sebastien Vauban wrote:
> Sebastien Vauban wrote:
>> I just ran `make test' and got the same error for
>> `ob-shell/bash-uses-assoc-arrays'.
>
> Now that I updated to Org-mode version 8.2.7
> (release_8.2.7-1114-g6c2838), I have an extra test failing:
>
> Test test-org-element/headline-properties backtrace:
>   signal(ert-test-failed (((should-not (org-test-with-temp-text "* Hea
>   ert-fail(((should-not (org-test-with-temp-text "* Headline\n#+BEGIN_
>   (if (not (unwind-protect (setq value-1904 (let ((inside-text (if ...
>   (let (form-description-1905) (if (not (unwind-protect (setq value-19
>   (let ((value-1904 (ert--gensym "ert-form-evaluation-aborted-"))) (le
>   (lambda nil (let ((value-1900 (ert--gensym "ert-form-evaluation-abor
>   byte-code("\306\307!q\210\310\216\311 \312\216\313\314\315\316\3
>   ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
>   byte-code("\306\307!\211\211r\310\311!q\210\312 d\313\223)L\210)\3
>   ert-run-test([cl-struct-ert-test test-org-element/headline-propertie
>   ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st
>   ert-run-tests("\\(org\\|ob\\)" #[(event-type &rest event-args) "\306
>   ert-run-tests-batch("\\(org\\|ob\\)")
>   ert-run-tests-batch-and-exit("\\(org\\|ob\\)")
>   (let ((org-id-track-globally t) (org-test-selector (if org-test-sele
>   org-test-run-batch-tests()
>   call-interactively(org-test-run-batch-tests nil nil)
>   command-execute(org-test-run-batch-tests)
>   command-line-1(("-L" "lisp/" "-L" "testing/" "-l" "org-test.el" "--e
>   command-line()
>   normal-top-level()
> Test test-org-element/headline-properties condition:
>     (ert-test-failed
>      ((should-not
>        (org-test-with-temp-text "* Headline\n#+BEGIN_EXAMPLE\n:PROPERTIES:\n:foo: bar\n:END:\n#+END_EXAMPLE"
>          (org-element-property :FOO ...)))
>       :form
>       (let
>           ((inside-text ...))
>         (with-temp-buffer
>           (org-mode)
>           (let ... ...)
>           (org-element-property :FOO ...)))
>       :value "bar"))
>    FAILED  169/427  test-org-element/headline-properties

False positive: was given by my test function. After using `make test'
(hence recompiling the source), that one disappeared.

Only one staying:

1 unexpected results:
   FAILED  ob-shell/bash-uses-assoc-arrays

Best regards,
  Seb

-- 
Sebastien Vauban

Re: 2 Org tests failing

[Achim Gratz]

Sebastien Vauban writes:
> I just ran `make test' and got the same error for
> `ob-shell/bash-uses-assoc-arrays'.

Yes, that's because not all versions of bash that have associative
arrays can parse the bizarre quoting style that goes through a
sub-process and here-document that is used to fill in the parameters.

A better quoting function would be

(defun org-babel-sh-var-to-sh (var &optional sep hline)
  "Convert an elisp value to a shell variable.
Convert an elisp var into a string of shell commands specifying a
var of the same value."
  (concat "'" (replace-regexp-in-string
	       "'" "'\"'\"'"
	       (org-babel-sh-var-to-string var sep hline))
	  "'"))

(and remove the double quotes around the places where the value is going
to be used). It'd have the advantage of working with more types of
shells, but I'm not 100% certain that there aren't any corner cases.

> BTW, how can I avoid compiling the source directory before launching the
> tests?  I prefer having only .el files, as I was mistaken multiple times
> when trying to understand executions of code I modified.

Take a few minutes and read:
http://orgmode.org/worg/dev/org-build-system.html


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: [PATCH] ob-shell (was: 2 Org tests failing)

[Achim Gratz]

[-- Attachment #1: Type: text/plain, Size: 446 bytes --]

Achim Gratz writes:
> Sebastien Vauban writes:
>> I just ran `make test' and got the same error for
>> `ob-shell/bash-uses-assoc-arrays'.
>
> Yes, that's because not all versions of bash that have associative
> arrays can parse the bizarre quoting style that goes through a
> sub-process and here-document that is used to fill in the parameters.

Here's a patch that implements the suggestion and tested to work
correctly with Cygwin and Linux.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-shell-stratify-shell-variable-quoting.patch --]
[-- Type: text/x-patch, Size: 3322 bytes --]

From a79aff65d562e59ed4e01e550224eb96a665c1ae Mon Sep 17 00:00:00 2001
From: Achim Gratz <Stromeko@Stromeko.DE>
Date: Thu, 19 Jun 2014 21:23:28 +0200
Subject: [PATCH] ob-shell: stratify shell variable quoting

* lisp/ob-shell.el: Remove unused defcustom
  `org-babel-sh-var-quote-fmt'.
  (org-babel-variable-assignments:bash_array):
  (org-babel-variable-assignments:bash_assoc): Remove superfluous
  `mapcar' and double quotes around parameters.
  (org-babel-sh-var-to-sh): Single-quote the whole string and escape
  all single quotes in the original string.
---
 lisp/ob-shell.el | 42 ++++++++++++++++++------------------------
 1 file changed, 18 insertions(+), 24 deletions(-)

diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el
index 474a8f2..7d87026 100644
--- a/lisp/ob-shell.el
+++ b/lisp/ob-shell.el
@@ -45,12 +45,6 @@ (defcustom org-babel-sh-command shell-file-name
   :group 'org-babel
   :type 'string)
 
-(defcustom org-babel-sh-var-quote-fmt
-  "$(cat <<'BABEL_TABLE'\n%s\nBABEL_TABLE\n)"
-  "Format string used to escape variables when passed to shell scripts."
-  :group 'org-babel
-  :type 'string)
-
 (defcustom org-babel-shell-names
   '("sh" "bash" "csh" "ash" "dash" "ksh" "mksh" "posh")
   "List of names of shell supported by babel shell code blocks."
@@ -113,28 +107,26 @@ (defun org-babel-variable-assignments:sh-generic
 (defun org-babel-variable-assignments:bash_array
     (varname values &optional sep hline)
   "Returns a list of statements declaring the values as a bash array."
-  (format "unset %s\ndeclare -a %s=( \"%s\" )"
-     varname varname
-     (mapconcat 'identity
-       (mapcar
-         (lambda (value) (org-babel-sh-var-to-sh value sep hline))
-         values)
-       "\" \"")))
+  (format "unset %s\ndeclare -a %s=( %s )"
+	  varname varname
+	  (mapconcat
+	   (lambda (value) (org-babel-sh-var-to-sh value sep hline))
+	   values
+	   " ")))
 
 (defun org-babel-variable-assignments:bash_assoc
     (varname values &optional sep hline)
   "Returns a list of statements declaring the values as bash associative array."
   (format "unset %s\ndeclare -A %s\n%s"
     varname varname
-    (mapconcat 'identity
-      (mapcar
-        (lambda (items)
-          (format "%s[\"%s\"]=%s"
-            varname
-            (org-babel-sh-var-to-sh (car items) sep hline)
-            (org-babel-sh-var-to-sh (cdr items) sep hline)))
-        values)
-      "\n")))
+    (mapconcat
+     (lambda (items)
+       (format "%s[%s]=%s"
+	       varname
+	       (org-babel-sh-var-to-sh (car items) sep hline)
+	       (org-babel-sh-var-to-sh (cdr items) sep hline)))
+     values
+     "\n")))
 
 (defun org-babel-variable-assignments:bash (varname values &optional sep hline)
   "Represents the parameters as useful Bash shell variables."
@@ -163,8 +155,10 @@ (defun org-babel-sh-var-to-sh (var &optional sep hline)
   "Convert an elisp value to a shell variable.
 Convert an elisp var into a string of shell commands specifying a
 var of the same value."
-  (format org-babel-sh-var-quote-fmt
-	  (org-babel-sh-var-to-string var sep hline)))
+  (concat "'" (replace-regexp-in-string
+	       "'" "'\"'\"'"
+	       (org-babel-sh-var-to-string var sep hline))
+	  "'"))
 
 (defun org-babel-sh-var-to-string (var &optional sep hline)
   "Convert an elisp value to a string."
-- 
2.0.0


[-- Attachment #3: Type: text/plain, Size: 201 bytes --]



Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

Re: [PATCH] ob-shell

[Eric Schulte]

If this maintains existing functionality, please go ahead and apply it.

Thanks,

Achim Gratz <Stromeko@nexgo.de> writes:

> Achim Gratz writes:
>> Sebastien Vauban writes:
>>> I just ran `make test' and got the same error for
>>> `ob-shell/bash-uses-assoc-arrays'.
>>
>> Yes, that's because not all versions of bash that have associative
>> arrays can parse the bizarre quoting style that goes through a
>> sub-process and here-document that is used to fill in the parameters.
>
> Here's a patch that implements the suggestion and tested to work
> correctly with Cygwin and Linux.
>
> From a79aff65d562e59ed4e01e550224eb96a665c1ae Mon Sep 17 00:00:00 2001
> From: Achim Gratz <Stromeko@Stromeko.DE>
> Date: Thu, 19 Jun 2014 21:23:28 +0200
> Subject: [PATCH] ob-shell: stratify shell variable quoting
>
> * lisp/ob-shell.el: Remove unused defcustom
>   `org-babel-sh-var-quote-fmt'.
>   (org-babel-variable-assignments:bash_array):
>   (org-babel-variable-assignments:bash_assoc): Remove superfluous
>   `mapcar' and double quotes around parameters.
>   (org-babel-sh-var-to-sh): Single-quote the whole string and escape
>   all single quotes in the original string.
> ---
>  lisp/ob-shell.el | 42 ++++++++++++++++++------------------------
>  1 file changed, 18 insertions(+), 24 deletions(-)
>
> diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el
> index 474a8f2..7d87026 100644
> --- a/lisp/ob-shell.el
> +++ b/lisp/ob-shell.el
> @@ -45,12 +45,6 @@ (defcustom org-babel-sh-command shell-file-name
>    :group 'org-babel
>    :type 'string)
>  
> -(defcustom org-babel-sh-var-quote-fmt
> -  "$(cat <<'BABEL_TABLE'\n%s\nBABEL_TABLE\n)"
> -  "Format string used to escape variables when passed to shell scripts."
> -  :group 'org-babel
> -  :type 'string)
> -
>  (defcustom org-babel-shell-names
>    '("sh" "bash" "csh" "ash" "dash" "ksh" "mksh" "posh")
>    "List of names of shell supported by babel shell code blocks."
> @@ -113,28 +107,26 @@ (defun org-babel-variable-assignments:sh-generic
>  (defun org-babel-variable-assignments:bash_array
>      (varname values &optional sep hline)
>    "Returns a list of statements declaring the values as a bash array."
> -  (format "unset %s\ndeclare -a %s=( \"%s\" )"
> -     varname varname
> -     (mapconcat 'identity
> -       (mapcar
> -         (lambda (value) (org-babel-sh-var-to-sh value sep hline))
> -         values)
> -       "\" \"")))
> +  (format "unset %s\ndeclare -a %s=( %s )"
> +	  varname varname
> +	  (mapconcat
> +	   (lambda (value) (org-babel-sh-var-to-sh value sep hline))
> +	   values
> +	   " ")))
>  
>  (defun org-babel-variable-assignments:bash_assoc
>      (varname values &optional sep hline)
>    "Returns a list of statements declaring the values as bash associative array."
>    (format "unset %s\ndeclare -A %s\n%s"
>      varname varname
> -    (mapconcat 'identity
> -      (mapcar
> -        (lambda (items)
> -          (format "%s[\"%s\"]=%s"
> -            varname
> -            (org-babel-sh-var-to-sh (car items) sep hline)
> -            (org-babel-sh-var-to-sh (cdr items) sep hline)))
> -        values)
> -      "\n")))
> +    (mapconcat
> +     (lambda (items)
> +       (format "%s[%s]=%s"
> +	       varname
> +	       (org-babel-sh-var-to-sh (car items) sep hline)
> +	       (org-babel-sh-var-to-sh (cdr items) sep hline)))
> +     values
> +     "\n")))
>  
>  (defun org-babel-variable-assignments:bash (varname values &optional sep hline)
>    "Represents the parameters as useful Bash shell variables."
> @@ -163,8 +155,10 @@ (defun org-babel-sh-var-to-sh (var &optional sep hline)
>    "Convert an elisp value to a shell variable.
>  Convert an elisp var into a string of shell commands specifying a
>  var of the same value."
> -  (format org-babel-sh-var-quote-fmt
> -	  (org-babel-sh-var-to-string var sep hline)))
> +  (concat "'" (replace-regexp-in-string
> +	       "'" "'\"'\"'"
> +	       (org-babel-sh-var-to-string var sep hline))
> +	  "'"))
>  
>  (defun org-babel-sh-var-to-string (var &optional sep hline)
>    "Convert an elisp value to a string."
> -- 
> 2.0.0
>
>
>
> Regards,
> Achim.

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D (see https://u.fsf.org/yw)

Re: [PATCH] ob-shell

[Pascal Fleury]

[-- Attachment #1: Type: text/plain, Size: 5225 bytes --]

Hi Achim,

I was wondering how it would behave if the string that is put into a
variable contains newlines, backslashes and other things that bash (and
other shells) treats specially. The trick with cat and BABEL_TABLE is
resistant to this. As in many cases (when I use it at least) the variable
come from other parts of the org file and are computed through another
language (python in my case) I have such cases that would break.

Also, I think the test coverage for such things is limited for ob-shell.el
in the org test suite.
We should probably add some test cases for this, and then make sure it
works with your implementation as well.

--paf


On Sun, Jun 22, 2014 at 2:50 PM, Eric Schulte <schulte.eric@gmail.com>
wrote:

> If this maintains existing functionality, please go ahead and apply it.
>
> Thanks,
>
> Achim Gratz <Stromeko@nexgo.de> writes:
>
> > Achim Gratz writes:
> >> Sebastien Vauban writes:
> >>> I just ran `make test' and got the same error for
> >>> `ob-shell/bash-uses-assoc-arrays'.
> >>
> >> Yes, that's because not all versions of bash that have associative
> >> arrays can parse the bizarre quoting style that goes through a
> >> sub-process and here-document that is used to fill in the parameters.
> >
> > Here's a patch that implements the suggestion and tested to work
> > correctly with Cygwin and Linux.
> >
> > From a79aff65d562e59ed4e01e550224eb96a665c1ae Mon Sep 17 00:00:00 2001
> > From: Achim Gratz <Stromeko@Stromeko.DE>
> > Date: Thu, 19 Jun 2014 21:23:28 +0200
> > Subject: [PATCH] ob-shell: stratify shell variable quoting
> >
> > * lisp/ob-shell.el: Remove unused defcustom
> >   `org-babel-sh-var-quote-fmt'.
> >   (org-babel-variable-assignments:bash_array):
> >   (org-babel-variable-assignments:bash_assoc): Remove superfluous
> >   `mapcar' and double quotes around parameters.
> >   (org-babel-sh-var-to-sh): Single-quote the whole string and escape
> >   all single quotes in the original string.
> > ---
> >  lisp/ob-shell.el | 42 ++++++++++++++++++------------------------
> >  1 file changed, 18 insertions(+), 24 deletions(-)
> >
> > diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el
> > index 474a8f2..7d87026 100644
> > --- a/lisp/ob-shell.el
> > +++ b/lisp/ob-shell.el
> > @@ -45,12 +45,6 @@ (defcustom org-babel-sh-command shell-file-name
> >    :group 'org-babel
> >    :type 'string)
> >
> > -(defcustom org-babel-sh-var-quote-fmt
> > -  "$(cat <<'BABEL_TABLE'\n%s\nBABEL_TABLE\n)"
> > -  "Format string used to escape variables when passed to shell scripts."
> > -  :group 'org-babel
> > -  :type 'string)
> > -
> >  (defcustom org-babel-shell-names
> >    '("sh" "bash" "csh" "ash" "dash" "ksh" "mksh" "posh")
> >    "List of names of shell supported by babel shell code blocks."
> > @@ -113,28 +107,26 @@ (defun org-babel-variable-assignments:sh-generic
> >  (defun org-babel-variable-assignments:bash_array
> >      (varname values &optional sep hline)
> >    "Returns a list of statements declaring the values as a bash array."
> > -  (format "unset %s\ndeclare -a %s=( \"%s\" )"
> > -     varname varname
> > -     (mapconcat 'identity
> > -       (mapcar
> > -         (lambda (value) (org-babel-sh-var-to-sh value sep hline))
> > -         values)
> > -       "\" \"")))
> > +  (format "unset %s\ndeclare -a %s=( %s )"
> > +       varname varname
> > +       (mapconcat
> > +        (lambda (value) (org-babel-sh-var-to-sh value sep hline))
> > +        values
> > +        " ")))
> >
> >  (defun org-babel-variable-assignments:bash_assoc
> >      (varname values &optional sep hline)
> >    "Returns a list of statements declaring the values as bash
> associative array."
> >    (format "unset %s\ndeclare -A %s\n%s"
> >      varname varname
> > -    (mapconcat 'identity
> > -      (mapcar
> > -        (lambda (items)
> > -          (format "%s[\"%s\"]=%s"
> > -            varname
> > -            (org-babel-sh-var-to-sh (car items) sep hline)
> > -            (org-babel-sh-var-to-sh (cdr items) sep hline)))
> > -        values)
> > -      "\n")))
> > +    (mapconcat
> > +     (lambda (items)
> > +       (format "%s[%s]=%s"
> > +            varname
> > +            (org-babel-sh-var-to-sh (car items) sep hline)
> > +            (org-babel-sh-var-to-sh (cdr items) sep hline)))
> > +     values
> > +     "\n")))
> >
> >  (defun org-babel-variable-assignments:bash (varname values &optional
> sep hline)
> >    "Represents the parameters as useful Bash shell variables."
> > @@ -163,8 +155,10 @@ (defun org-babel-sh-var-to-sh (var &optional sep
> hline)
> >    "Convert an elisp value to a shell variable.
> >  Convert an elisp var into a string of shell commands specifying a
> >  var of the same value."
> > -  (format org-babel-sh-var-quote-fmt
> > -       (org-babel-sh-var-to-string var sep hline)))
> > +  (concat "'" (replace-regexp-in-string
> > +            "'" "'\"'\"'"
> > +            (org-babel-sh-var-to-string var sep hline))
> > +       "'"))
> >
> >  (defun org-babel-sh-var-to-string (var &optional sep hline)
> >    "Convert an elisp value to a string."
> > --
> > 2.0.0
> >
> >
> >
> > Regards,
> > Achim.
>
> --
> Eric Schulte
> https://cs.unm.edu/~eschulte
> PGP: 0x614CA05D (see https://u.fsf.org/yw)
>
>


-- 
--paf

[-- Attachment #2: Type: text/html, Size: 7196 bytes --]

Re: [PATCH] ob-shell

[Achim Gratz]

Pascal Fleury writes:
> I was wondering how it would behave if the string that is put into a
> variable contains newlines, backslashes and other things that bash
> (and other shells) treats specially.

Exactly like it does with the earlier method, except it doesn't fork and
doesn't require certain versions of bash.

> The trick with cat and BABEL_TABLE is resistant to this.

I don't think so, some older versions of bash had a parsing bug that
made it croak on single quotes in here documents.  Single quoting the
variable content only reuires special handling of single quotes.  While
that is tedious and error prone to do by hand, we're doing it with elisp
anyway so this doesn't pose a problem.  Also, it works for just about
any shell, not only those that know about $( ) command substitution and
implement here documents within the substitution correctly.

> As in many cases (when I use it at least) the variable come from other
> parts of the org file and are computed through another language
> (python in my case) I have such cases that would break.

If you have an example, please show it.

> Also, I think the test coverage for such things is limited for
> ob-shell.el in the org test suite.
> We should probably add some test cases for this, and then make sure it
> works with your implementation as well.

By all means, please add such tests.  While you're at it, you might want
to double quote the variable expansions, which will show you that the
shell is wholly capable of returning a list and even a table to Org,
with or without associative arrays.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: [PATCH] ob-shell

[Achim Gratz]

[re-sent, sorry if you get this twice]

Eric Schulte writes:
> If this maintains existing functionality, please go ahead and apply it.

Done on master.  The tests are left untouched, I will not have time to
do anything there for the next few days.  Any helping hand is welcome.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

Re: [PATCH] ob-shell

[Alan Schmitt]

[-- Attachment #1: Type: text/plain, Size: 355 bytes --]

Hello,

On 2014-06-22 14:50, Eric Schulte <schulte.eric@gmail.com> writes:

> If this maintains existing functionality, please go ahead and apply it.

Has this been applied? I'm still seeing a failing test for ob-shell.

1 unexpected results:
   FAILED  ob-shell/bash-uses-assoc-arrays

Best,

Alan

-- 
OpenPGP Key ID : 040D0A3B4ED2E5C7

[-- Attachment #2: Type: application/pgp-signature, Size: 494 bytes --]

Re: [PATCH] ob-shell

[Eric Schulte]

Alan Schmitt <alan.schmitt@polytechnique.org> writes:

> Hello,
>
> On 2014-06-22 14:50, Eric Schulte <schulte.eric@gmail.com> writes:
>
>> If this maintains existing functionality, please go ahead and apply it.
>
> Has this been applied? I'm still seeing a failing test for ob-shell.
>
> 1 unexpected results:
>    FAILED  ob-shell/bash-uses-assoc-arrays
>
> Best,
>
> Alan

It looks like Achim is part way through making the required changes.

Achim Gratz <Stromeko@nexgo.de> writes:

> Eric Schulte writes:
> > If this maintains existing functionality, please go ahead and apply it.
> 
> Done on master.  The tests are left untouched, I will not have time to
> do anything there for the next few days.  Any helping hand is welcome.
> 
> 
> Regards,
> Achim.

Best,
Eric

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D (see https://u.fsf.org/yw)

Re: [PATCH] ob-shell

[Achim Gratz]

Alan Schmitt writes:
> Has this been applied? I'm still seeing a failing test for ob-shell.
>
> 1 unexpected results:
>    FAILED  ob-shell/bash-uses-assoc-arrays

That's to be expected if you use a version of bash that doesn't support
assoc arrays.  WHat is the backtrace from the test?  Babel (and the
tests) blindly assume that these things work when they see you using
"bash".


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: [PATCH] ob-shell

[Alan Schmitt]

[-- Attachment #1: Type: text/plain, Size: 3099 bytes --]

On 2014-08-04 21:48, Achim Gratz <Stromeko@nexgo.de> writes:

> Alan Schmitt writes:
>> Has this been applied? I'm still seeing a failing test for ob-shell.
>>
>> 1 unexpected results:
>>    FAILED  ob-shell/bash-uses-assoc-arrays
>
> That's to be expected if you use a version of bash that doesn't support
> assoc arrays.

I don't use bash, I either use zsh or fish. So I guess the version of
bash on my system is the one that ships with OS X 10.9:

GNU bash, version 3.2.51(1)-release (x86_64-apple-darwin13)

> WHat is the backtrace from the test?  Babel (and the
> tests) blindly assume that these things work when they see you using
> "bash".

Test ob-shell/bash-uses-assoc-arrays backtrace:
  signal(ert-test-failed (((should (equal "20 cm" (org-babel-execute-s
  ert-fail(((should (equal "20 cm" (org-babel-execute-src-block))) :fo
  (if (unwind-protect (setq value-1098 (apply fn-1096 args-1097)) (set
  (let (form-description-1100) (if (unwind-protect (setq value-1098 (a
  (let ((value-1098 (quote ert-form-evaluation-aborted-1099))) (let (f
  (let ((fn-1096 (function equal)) (args-1097 (list "20 cm" (org-babel
  (save-restriction (org-babel-next-src-block 2) (let ((fn-1096 (funct
  (progn (org-id-goto "82320a48-3409-49d7-85c9-5de1c6d3ff87") (setq to
  (unwind-protect (progn (org-id-goto "82320a48-3409-49d7-85c9-5de1c6d
  (let ((save-match-data-internal (match-data))) (unwind-protect (prog
  (progn (let ((save-match-data-internal (match-data))) (unwind-protec
  (unwind-protect (progn (let ((save-match-data-internal (match-data))
  (let ((wconfig (current-window-configuration))) (unwind-protect (pro
  (unwind-protect (let ((wconfig (current-window-configuration))) (unw
  (let* ((id-location (org-id-find "82320a48-3409-49d7-85c9-5de1c6d3ff
  (lambda nil (let* ((id-location (org-id-find "82320a48-3409-49d7-85c
  byte-code("\306\307!q\210\310\216\311 \312\216\313\314\315\316\3
  ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
  byte-code("\306\307!\211\211r\310\311!q\210\312 d\313\223)L\210)\3
  ert-run-test([cl-struct-ert-test ob-shell/bash-uses-assoc-arrays "Ba
  ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st
  ert-run-tests("\\(org\\|ob\\)" #[(event-type &rest event-args) "\306
  ert-run-tests-batch("\\(org\\|ob\\)")
  ert-run-tests-batch-and-exit("\\(org\\|ob\\)")
  (let ((org-id-track-globally t) (org-test-selector (if org-test-sele
  org-test-run-batch-tests("\\(org\\|ob\\)")
  eval((org-test-run-batch-tests org-test-select-re))
  command-line-1(("--eval" "(setq vc-handled-backends nil org-startup-
  command-line()
  normal-top-level()
Test ob-shell/bash-uses-assoc-arrays condition:
    (ert-test-failed
     ((should
       (equal "20 cm"
	      (org-babel-execute-src-block)))
      :form
      (equal "20 cm" "50 dl")
      :value nil :explanation
      (array-elt 0
		 (different-atoms
		  (50 "#x32" "?2")
		  (53 "#x35" "?5")))))
   FAILED  104/510  ob-shell/bash-uses-assoc-arrays

Thanks,

Alan

-- 
OpenPGP Key ID : 040D0A3B4ED2E5C7

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 494 bytes --]

Re: [PATCH] ob-shell

[Pascal Fleury]

[-- Attachment #1: Type: text/plain, Size: 3745 bytes --]

Yes, sorry for late reaction (it's holiday season here :-).

This is an issue I introduced with the assoc. arrays, which does not verify
if the installed version of bash handles them (from bash4 on).
It needs a modification in the setup code that checks the version and only
issues assoc. array code when bash supports it. And it so happens that Macs
have bash3 installed. Will try to find such a machine and add the test.

--paf


On Wed, Aug 13, 2014 at 3:12 PM, Alan Schmitt <
alan.schmitt@polytechnique.org> wrote:

> On 2014-08-04 21:48, Achim Gratz <Stromeko@nexgo.de> writes:
>
> > Alan Schmitt writes:
> >> Has this been applied? I'm still seeing a failing test for ob-shell.
> >>
> >> 1 unexpected results:
> >>    FAILED  ob-shell/bash-uses-assoc-arrays
> >
> > That's to be expected if you use a version of bash that doesn't support
> > assoc arrays.
>
> I don't use bash, I either use zsh or fish. So I guess the version of
> bash on my system is the one that ships with OS X 10.9:
>
> GNU bash, version 3.2.51(1)-release (x86_64-apple-darwin13)
>
> > WHat is the backtrace from the test?  Babel (and the
> > tests) blindly assume that these things work when they see you using
> > "bash".
>
> Test ob-shell/bash-uses-assoc-arrays backtrace:
>   signal(ert-test-failed (((should (equal "20 cm" (org-babel-execute-s
>   ert-fail(((should (equal "20 cm" (org-babel-execute-src-block))) :fo
>   (if (unwind-protect (setq value-1098 (apply fn-1096 args-1097)) (set
>   (let (form-description-1100) (if (unwind-protect (setq value-1098 (a
>   (let ((value-1098 (quote ert-form-evaluation-aborted-1099))) (let (f
>   (let ((fn-1096 (function equal)) (args-1097 (list "20 cm" (org-babel
>   (save-restriction (org-babel-next-src-block 2) (let ((fn-1096 (funct
>   (progn (org-id-goto "82320a48-3409-49d7-85c9-5de1c6d3ff87") (setq to
>   (unwind-protect (progn (org-id-goto "82320a48-3409-49d7-85c9-5de1c6d
>   (let ((save-match-data-internal (match-data))) (unwind-protect (prog
>   (progn (let ((save-match-data-internal (match-data))) (unwind-protec
>   (unwind-protect (progn (let ((save-match-data-internal (match-data))
>   (let ((wconfig (current-window-configuration))) (unwind-protect (pro
>   (unwind-protect (let ((wconfig (current-window-configuration))) (unw
>   (let* ((id-location (org-id-find "82320a48-3409-49d7-85c9-5de1c6d3ff
>   (lambda nil (let* ((id-location (org-id-find "82320a48-3409-49d7-85c
>   byte-code("\306\307!q\210\310\216\311 \312\216\313\314\315\316\3
>   ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
>   byte-code("\306\307!\211\211r\310\311!q\210\312 d\313\223)L\210)\3
>   ert-run-test([cl-struct-ert-test ob-shell/bash-uses-assoc-arrays "Ba
>   ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st
>   ert-run-tests("\\(org\\|ob\\)" #[(event-type &rest event-args) "\306
>   ert-run-tests-batch("\\(org\\|ob\\)")
>   ert-run-tests-batch-and-exit("\\(org\\|ob\\)")
>   (let ((org-id-track-globally t) (org-test-selector (if org-test-sele
>   org-test-run-batch-tests("\\(org\\|ob\\)")
>   eval((org-test-run-batch-tests org-test-select-re))
>   command-line-1(("--eval" "(setq vc-handled-backends nil org-startup-
>   command-line()
>   normal-top-level()
> Test ob-shell/bash-uses-assoc-arrays condition:
>     (ert-test-failed
>      ((should
>        (equal "20 cm"
>               (org-babel-execute-src-block)))
>       :form
>       (equal "20 cm" "50 dl")
>       :value nil :explanation
>       (array-elt 0
>                  (different-atoms
>                   (50 "#x32" "?2")
>                   (53 "#x35" "?5")))))
>    FAILED  104/510  ob-shell/bash-uses-assoc-arrays
>
> Thanks,
>
> Alan
>
> --
> OpenPGP Key ID : 040D0A3B4ED2E5C7
>



-- 
--paf

[-- Attachment #2: Type: text/html, Size: 4896 bytes --]

Re: [PATCH] ob-shell

[Alan Schmitt]

[-- Attachment #1: Type: text/plain, Size: 683 bytes --]

On 2014-08-13 15:20, Pascal Fleury <fleury@google.com> writes:

> Yes, sorry for late reaction (it's holiday season here :-).

Funny, it's holiday season here as well (in Bretagne) ;-)

> This is an issue I introduced with the assoc. arrays, which does not
> verify if the installed version of bash handles them (from bash4 on).
> It needs a modification in the setup code that checks the version and
> only issues assoc. array code when bash supports it. And it so happens
> that Macs have bash3 installed. Will try to find such a machine and
> add the test.

Thanks a lot. If I can help, please let me know.

Best,

Alan

-- 
OpenPGP Key ID : 040D0A3B4ED2E5C7

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 494 bytes --]

Re: [PATCH] ob-shell

[Pascal Fleury]

[-- Attachment #1.1: Type: text/plain, Size: 1216 bytes --]

Hello,

I have a little patch (attached) that does check the version of bash, and
uses associative arrays (or doesn't) depending on that version.

The problem now is that the test suite will only work on one platform t a
time, i.e. if there is a bash4 there, it will pass, but if it has bash3, it
will work as intended, but return a different value set. Is there a way to
have platform-dependent tests ?

--Pascal



On Wed, Aug 13, 2014 at 3:29 PM, Alan Schmitt <
alan.schmitt@polytechnique.org> wrote:

> On 2014-08-13 15:20, Pascal Fleury <fleury@google.com> writes:
>
> > Yes, sorry for late reaction (it's holiday season here :-).
>
> Funny, it's holiday season here as well (in Bretagne) ;-)
>
> > This is an issue I introduced with the assoc. arrays, which does not
> > verify if the installed version of bash handles them (from bash4 on).
> > It needs a modification in the setup code that checks the version and
> > only issues assoc. array code when bash supports it. And it so happens
> > that Macs have bash3 installed. Will try to find such a machine and
> > add the test.
>
> Thanks a lot. If I can help, please let me know.
>
> Best,
>
> Alan
>
> --
> OpenPGP Key ID : 040D0A3B4ED2E5C7
>



-- 
--paf

[-- Attachment #1.2: Type: text/html, Size: 1891 bytes --]

[-- Attachment #2: 0001-ob-shell-now-checks-for-the-current-bash-version-bef.patch --]
[-- Type: application/octet-stream, Size: 1555 bytes --]

From d28d82dd88d2e61a2ce08cc69eee02b8213c6252 Mon Sep 17 00:00:00 2001
From: Pascal Fleury <fleury@google.com>
Date: Fri, 22 Aug 2014 10:45:20 +0200
Subject: [PATCH] ob-shell now checks for the current bash version before
 emitting code that uses associative arrays, because bash supports them only
 from version 4 and on, and on some platforms (e.g. MacOSX) the default bash
 is v3.

---
 lisp/ob-shell.el | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el
index 8c58dfd..27d6de7 100644
--- a/lisp/ob-shell.el
+++ b/lisp/ob-shell.el
@@ -121,10 +121,15 @@ This function is called by `org-babel-execute-src-block'."
      values
      "\n")))
 
+(defun org-babel-shell-bash-version
+  "Returns the major revision of the current bash running on the system."
+  (car (parse-integer (org-babel-execute:shell "echo ${BASH_VERSINFO[0]}" nil))))
+
 (defun org-babel-variable-assignments:bash (varname values &optional sep hline)
   "Represents the parameters as useful Bash shell variables."
-  (if (listp values)
-      (if (and (listp (car values)) (= 1 (length (car values))))
+   (if (listp values)
+      (if (and (< 3 (org-babel-shell-bash-version) ; bash supports assoc arrays only from version 4 on.
+	       (listp (car values)) (= 1 (length (car values)))))
 	  (org-babel-variable-assignments:bash_array varname values sep hline)
 	(org-babel-variable-assignments:bash_assoc varname values sep hline))
     (org-babel-variable-assignments:sh-generic varname values sep hline)))
-- 
1.9.3

Re: [PATCH] ob-shell

[Pascal Fleury]

[-- Attachment #1.1: Type: text/plain, Size: 1729 bytes --]

Me again,

Please forget about the previous patch I submitted, I have a new one that
should work now on all platforms.
Let me know if it should be formatted differently.

I will also add some more tests to check that such arrays in bash are
handled properly when the input contains whitespace, quotes, commas and
other chars that one may have to deal with.

--paf



On Fri, Aug 22, 2014 at 10:52 AM, Pascal Fleury <fleury@google.com> wrote:

> Hello,
>
> I have a little patch (attached) that does check the version of bash, and
> uses associative arrays (or doesn't) depending on that version.
>
> The problem now is that the test suite will only work on one platform t a
> time, i.e. if there is a bash4 there, it will pass, but if it has bash3, it
> will work as intended, but return a different value set. Is there a way to
> have platform-dependent tests ?
>
> --Pascal
>
>
>
> On Wed, Aug 13, 2014 at 3:29 PM, Alan Schmitt <
> alan.schmitt@polytechnique.org> wrote:
>
>> On 2014-08-13 15:20, Pascal Fleury <fleury@google.com> writes:
>>
>> > Yes, sorry for late reaction (it's holiday season here :-).
>>
>> Funny, it's holiday season here as well (in Bretagne) ;-)
>>
>> > This is an issue I introduced with the assoc. arrays, which does not
>> > verify if the installed version of bash handles them (from bash4 on).
>> > It needs a modification in the setup code that checks the version and
>> > only issues assoc. array code when bash supports it. And it so happens
>> > that Macs have bash3 installed. Will try to find such a machine and
>> > add the test.
>>
>> Thanks a lot. If I can help, please let me know.
>>
>> Best,
>>
>> Alan
>>
>> --
>> OpenPGP Key ID : 040D0A3B4ED2E5C7
>>
>
>
>
> --
> --paf
>



-- 
--paf

[-- Attachment #1.2: Type: text/html, Size: 2858 bytes --]

[-- Attachment #2: 0001-ob-shell.el-now-emits-associative-array-code-only-wh.patch --]
[-- Type: application/octet-stream, Size: 3093 bytes --]

From 251566b07d75e0254f5044b3807cb9d0ea9ff749 Mon Sep 17 00:00:00 2001
From: Pascal Fleury <fleury@google.com>
Date: Fri, 22 Aug 2014 15:04:00 +0200
Subject: [PATCH] ob-shell.el now emits associative array code only when the
 installed version of bash supports them. This takes care of differences in
 platforms that ship with bash v4+, that can deal with them, and those (e.g.
 MacOSX) that ship with bash v3.x that does not support such arrays.

---
 lisp/ob-shell.el                   | 12 +++++++++---
 testing/examples/ob-shell-test.org |  8 +++++++-
 testing/lisp/test-ob-shell.el      |  2 +-
 3 files changed, 17 insertions(+), 5 deletions(-)

diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el
index 8c58dfd..e6b6429 100644
--- a/lisp/ob-shell.el
+++ b/lisp/ob-shell.el
@@ -121,12 +121,18 @@ This function is called by `org-babel-execute-src-block'."
      values
      "\n")))
 
+(defun bash-supports-assoc-arrays ()
+  "Returns if the current bash running on the system supports associative arrays."
+  (let ((bash_version (org-babel-execute:shell "echo ${BASH_VERSINFO[0]}" nil)))
+       (> 3 bash_version)))
+
 (defun org-babel-variable-assignments:bash (varname values &optional sep hline)
   "Represents the parameters as useful Bash shell variables."
-  (if (listp values)
-      (if (and (listp (car values)) (= 1 (length (car values))))
+   (if (listp values)
+      (if (or (not (bash-supports-assoc-arrays))
+	      (and (listp (car values)) (= 1 (length (car values)))))
 	  (org-babel-variable-assignments:bash_array varname values sep hline)
-	(org-babel-variable-assignments:bash_assoc varname values sep hline))
+	  (org-babel-variable-assignments:bash_assoc varname values sep hline))
     (org-babel-variable-assignments:sh-generic varname values sep hline)))
 
 (defun org-babel-variable-assignments:sh (params)
diff --git a/testing/examples/ob-shell-test.org b/testing/examples/ob-shell-test.org
index a54e5c0..cc3031d 100644
--- a/testing/examples/ob-shell-test.org
+++ b/testing/examples/ob-shell-test.org
@@ -80,7 +80,13 @@ echo ${table}
 Bash will see an associative array that contains each row as a single
 string. Bash cannot handle lists in associative arrays.
 #+begin_src bash :exports results :var table=sample_big_table
-echo ${table[spaghetti]}
+if (( ${BASH_VERSINFO[0]} > 3 )); then
+  # understands associative arrays
+  echo spaghetti ${table[spaghetti]}
+else
+  # Default V3.x behavior, has one array item contains the key and values.
+  echo ${table[1]}
+fi
 #+end_src
 
 #+RESULTS:
diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el
index 58a7859..9aa7f35 100644
--- a/testing/lisp/test-ob-shell.el
+++ b/testing/lisp/test-ob-shell.el
@@ -84,7 +84,7 @@ ob-comint.el, which was not previously tested."
   "Bash associative arrays as strings for the row"
   (org-test-at-id "82320a48-3409-49d7-85c9-5de1c6d3ff87"
     (org-babel-next-src-block 2)
-    (should (equal "20 cm" (org-babel-execute-src-block)))))
+    (should (equal "spaghetti 20 cm" (org-babel-execute-src-block)))))
 
 
 (provide 'test-ob-shell)
-- 
1.9.3

Re: [PATCH] ob-shell

[Achim Gratz]

Pascal Fleury writes:
> Please forget about the previous patch I submitted, I have a new one
> that should work now on all platforms.
> Let me know if it should be formatted differently.

You don't really want to fork into bash each time you're about to run a
shell code block just to find out if it suppports associative arrays.
This needs to be determined at customization time.

Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [PATCH] ob-shell

[Pascal Fleury]

[-- Attachment #1.1: Type: text/plain, Size: 892 bytes --]

Hello,

Here is a patch that will figure out the version of bash in a less fork-y
way.
It keeps the result in a variable after having gotten it the first time by
indeed forking to bash.

--Pascal

On Fri, Aug 22, 2014 at 8:16 PM, Achim Gratz <Stromeko@nexgo.de> wrote:

> Pascal Fleury writes:
> > Please forget about the previous patch I submitted, I have a new one
> > that should work now on all platforms.
> > Let me know if it should be formatted differently.
>
> You don't really want to fork into bash each time you're about to run a
> shell code block just to find out if it suppports associative arrays.
> This needs to be determined at customization time.
>
> Regards,
> Achim.
> --
> +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+
>
> Factory and User Sound Singles for Waldorf Q+, Q and microQ:
> http://Synth.Stromeko.net/Downloads.html#WaldorfSounds
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 1451 bytes --]

[-- Attachment #2: 0001-PATCH-ob-shell.el-now-emits-associative-array-code-o.patch --]
[-- Type: application/octet-stream, Size: 3566 bytes --]

From 993c8e1da79c229a60aa76ade64780484a405df0 Mon Sep 17 00:00:00 2001
From: Pascal Fleury <fleury@google.com>
Date: Wed, 21 Jan 2015 23:55:34 +0100
Subject: [PATCH] [PATCH]ob-shell.el now emits associative array code only when
 the  installed version of bash supports them. This takes care of differences
 in  platforms that ship with bash v4+, that can deal with them, and those
 (e.g.  MacOSX) that ship with bash v3.x that does not support such arrays.

---
 lisp/ob-shell.el                   | 15 +++++++++++++--
 testing/examples/ob-shell-test.org | 10 ++++++++--
 testing/lisp/test-ob-shell.el      |  2 +-
 3 files changed, 22 insertions(+), 5 deletions(-)

diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el
index aa14a69..48b38cc 100644
--- a/lisp/ob-shell.el
+++ b/lisp/ob-shell.el
@@ -38,6 +38,10 @@
 
 (defvar org-babel-default-header-args:sh '())
 
+;; Contains the version of bash that is installed. If nil, will be
+;; found by using bash itself once.
+(defvar org-babel-shell-bash-version nil)
+
 (defcustom org-babel-shell-names
   '("sh" "bash" "csh" "ash" "dash" "ksh" "mksh" "posh")
   "List of names of shell supported by babel shell code blocks."
@@ -121,12 +125,19 @@ This function is called by `org-babel-execute-src-block'."
      values
      "\n")))
 
+(defun bash-supports-assoc-arrays ()
+  "Returns if the current bash running on the system supports associative arrays."
+  (if (= org-babel-shell-bash-version nil)
+      (setq org-babel-shell-bash-version (org-babel-execute:shell "echo ${BASH_VERSINFO[0]}" nil)))
+  (> 3 org-babel-shell-bash-version)))
+
 (defun org-babel-variable-assignments:bash (varname values &optional sep hline)
   "Represents the parameters as useful Bash shell variables."
   (if (listp values)
-      (if (and (listp (car values)) (= 1 (length (car values))))
+      (if (or (not (bash-supports-assoc-arrays))
+	      (and (listp (car values)) (= 1 (length (car values)))))
 	  (org-babel-variable-assignments:bash_array varname values sep hline)
-	(org-babel-variable-assignments:bash_assoc varname values sep hline))
+	  (org-babel-variable-assignments:bash_assoc varname values sep hline))
     (org-babel-variable-assignments:sh-generic varname values sep hline)))
 
 (defun org-babel-variable-assignments:sh (params)
diff --git a/testing/examples/ob-shell-test.org b/testing/examples/ob-shell-test.org
index a54e5c0..ebd7421 100644
--- a/testing/examples/ob-shell-test.org
+++ b/testing/examples/ob-shell-test.org
@@ -80,9 +80,15 @@ echo ${table}
 Bash will see an associative array that contains each row as a single
 string. Bash cannot handle lists in associative arrays.
 #+begin_src bash :exports results :var table=sample_big_table
-echo ${table[spaghetti]}
+if (( ${BASH_VERSINFO[0]} > 3 )); then
+  # understands associative arrays
+  echo spaghetti ${table[spaghetti]}
+else
+  # Default V3.x behavior, has one array item contains the key and values.
+  echo ${table[1]}
+fi
 #+end_src
 
 #+RESULTS:
-: 20 cm
+: spaghetti 20 cm
 
diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el
index 58a7859..9aa7f35 100644
--- a/testing/lisp/test-ob-shell.el
+++ b/testing/lisp/test-ob-shell.el
@@ -84,7 +84,7 @@ ob-comint.el, which was not previously tested."
   "Bash associative arrays as strings for the row"
   (org-test-at-id "82320a48-3409-49d7-85c9-5de1c6d3ff87"
     (org-babel-next-src-block 2)
-    (should (equal "20 cm" (org-babel-execute-src-block)))))
+    (should (equal "spaghetti 20 cm" (org-babel-execute-src-block)))))
 
 
 (provide 'test-ob-shell)
-- 
2.2.1

Re: [PATCH] ob-shell

[Achim Gratz]

Pascal Fleury writes:
> Here is a patch that will figure out the version of bash in a less
> fork-y way.  It keeps the result in a variable after having gotten it
> the first time by indeed forking to bash.

I still think this should be a defcustom instead with a setter function
that checks for the actual availablity of array variables.  The user
should have the opportunity to not use arrays at all even when the shell
supports it.  Using array variable syntax to get the major version
number of the installled Bash might not work in those shells that don't
have arrays.  I can't test an old Bash, but dash doesn't recognize the
syntax at all.  There should be no fallback to checking just the version
of bash since array variables may have been backported or disabled for
the actual variant of Bash that is used.  Also, the code as presented
doesn't deal with the possibility that Tramp might be invoking a
different shell on a remote machine, but that is a problem in more areas.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds