* 2 Org tests failing
@ 2014-05-22 10:03 Sebastien Vauban
2014-05-22 10:48 ` Bastien
0 siblings, 1 reply; 24+ messages in thread
From: Sebastien Vauban @ 2014-05-22 10:03 UTC (permalink / raw)
To: emacs-orgmode-mXXj517/zsQ
Hello,
I dunno if this must be signalled but, when runnning:
--8<---------------cut here---------------start------------->8---
emacs -Q --batch -L lisp/ -L testing/ -l org-test.el
--eval '(setq org-confirm-babel-evaluate nil)' -f org-test-run-batch-tests
--8<---------------cut here---------------end--------------->8---
on the fresh Org-mode version 8.2.6 (release_8.2.6-1010-g1ca86f), I do get:
--8<---------------cut here---------------start------------->8---
Running org_test, please wait (this can take a while)...
FAILED test-org-table/org-table-calc-current-TBLFM
FAILED test-org-table/org-table-calc-current-TBLFM-when-stop-because-of-error
--8<---------------cut here---------------end--------------->8---
Best regards,
Seb
--
Sebastien Vauban
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: 2 Org tests failing 2014-05-22 10:03 2 Org tests failing Sebastien Vauban @ 2014-05-22 10:48 ` Bastien [not found] ` <87tx8iks22.fsf-E3UqQZAQFPqWIDz0JBNUog@public.gmane.org> 0 siblings, 1 reply; 24+ messages in thread From: Bastien @ 2014-05-22 10:48 UTC (permalink / raw) To: Sebastien Vauban; +Cc: public-emacs-orgmode-mXXj517/zsQ Sebastien Vauban <sva-news-D0wtAvR13HarG/iDocfnWg@public.gmane.org> writes: > Running org_test, please wait (this can take a while)... > FAILED test-org-table/org-table-calc-current-TBLFM > FAILED test-org-table/org-table-calc-current-TBLFM-when-stop-because-of-error I can't reproduce this. Can you run the tests manually and report the backtrace? -- Bastien ^ permalink raw reply [flat|nested] 24+ messages in thread
[parent not found: <87tx8iks22.fsf-E3UqQZAQFPqWIDz0JBNUog@public.gmane.org>]
* Re: 2 Org tests failing [not found] ` <87tx8iks22.fsf-E3UqQZAQFPqWIDz0JBNUog@public.gmane.org> @ 2014-06-18 14:55 ` Sebastien Vauban 2014-06-18 21:49 ` Achim Gratz 0 siblings, 1 reply; 24+ messages in thread From: Sebastien Vauban @ 2014-06-18 14:55 UTC (permalink / raw) To: Bastien; +Cc: public-emacs-orgmode-mXXj517/zsQ-wOFGN7rlS/M9smdsby/KFg Bastien wrote: > Sebastien Vauban <sva-news-D0wtAvR13HarG/iDocfnWg-XMD5yJDbdMReXY1tMh2IBg@public.gmane.org> > writes: > >> Running org_test, please wait (this can take a while)... >> FAILED test-org-table/org-table-calc-current-TBLFM >> FAILED test-org-table/org-table-calc-current-TBLFM-when-stop-because-of-error > > I can't reproduce this. Can you run the tests manually and report > the backtrace? --8<---------------cut here---------------start------------->8--- Test test-org-table/org-table-calc-current-TBLFM condition: (file-error "Opening output file" "No such file or directory" "/tmp/org_test/org-test8984aQq") FAILED 354/427 test-org-table/org-table-calc-current-TBLFM Test test-org-table/org-table-calc-current-TBLFM-when-stop-because-of-error backtrace: write-region("" nil "/tmp/org_test/org-test8984naw" nil silent nil e byte-code("\305\30G!\203\307 !\202\310 \"! \203\n P\f\203 make-temp-file("org-test") (let ((file (make-temp-file "org-test")) (kill-buffer-query-function (lambda nil (let ((file (make-temp-file "org-test")) (kill-buffer-qu byte-code("\306\307!q\210\310\216\311 \312\216\313\314\315\316\3 ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc byte-code("\306\307!\211\211r\310\311!q\210\312 d\313\223)L\210)\3 ert-run-test([cl-struct-ert-test test-org-table/org-table-calc-curre ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st ert-run-tests("\\(org\\|ob\\)" #[(event-type &rest event-args) "\306 ert-run-tests-batch("\\(org\\|ob\\)") ert-run-tests-batch-and-exit("\\(org\\|ob\\)") (let ((org-id-track-globally t) (org-test-selector (if org-test-sele org-test-run-batch-tests() call-interactively(org-test-run-batch-tests nil nil) command-execute(org-test-run-batch-tests) command-line-1(("-L" "lisp/" "-L" "testing/" "-l" "org-test.el" "--e command-line() normal-top-level() --8<---------------cut here---------------end--------------->8--- These were run successfully from within Emacs itself (on Windows). I've solved the reported problem by creating the missing TMPDIR (set to /tmp/org_test) for the invocation from my Zsh (Cygwin) terminal... Though, now, I do have another test failing when testing from the shell: --8<---------------cut here---------------start------------->8--- Test ob-shell/bash-uses-assoc-arrays backtrace: signal(ert-test-failed (((should (equal "20 cm" (org-babel-execute-s ert-fail(((should (equal "20 cm" (org-babel-execute-src-block))) :fo (if (unwind-protect (setq value-502 (apply fn-500 args-501)) (setq f (let (form-description-504) (if (unwind-protect (setq value-502 (app (let ((value-502 (quote ert-form-evaluation-aborted-503))) (let (for (let ((fn-500 (function equal)) (args-501 (list "20 cm" (org-babel-e (save-restriction (org-babel-next-src-block 2) (let ((fn-500 (functi (progn (org-id-goto "82320a48-3409-49d7-85c9-5de1c6d3ff87") (setq to (unwind-protect (progn (org-id-goto "82320a48-3409-49d7-85c9-5de1c6d (let ((save-match-data-internal (match-data))) (unwind-protect (prog (progn (let ((save-match-data-internal (match-data))) (unwind-protec (unwind-protect (progn (let ((save-match-data-internal (match-data)) (let ((wconfig (current-window-configuration))) (unwind-protect (pro (unwind-protect (let ((wconfig (current-window-configuration))) (unw (let* ((id-location (org-id-find "82320a48-3409-49d7-85c9-5de1c6d3ff (lambda nil (let* ((id-location (org-id-find "82320a48-3409-49d7-85c byte-code("\306\307!q\210\310\216\311 \312\216\313\314\315\316\3 ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc byte-code("\306\307!\211\211r\310\311!q\210\312 d\313\223)L\210)\3 ert-run-test([cl-struct-ert-test ob-shell/bash-uses-assoc-arrays "Ba ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st ert-run-tests("\\(org\\|ob\\)" #[(event-type &rest event-args) "\306 ert-run-tests-batch("\\(org\\|ob\\)") ert-run-tests-batch-and-exit("\\(org\\|ob\\)") (let ((org-id-track-globally t) (org-test-selector (if org-test-sele org-test-run-batch-tests() call-interactively(org-test-run-batch-tests nil nil) command-execute(org-test-run-batch-tests) command-line-1(("-L" "lisp/" "-L" "testing/" "-l" "org-test.el" "--e command-line() normal-top-level() Test ob-shell/bash-uses-assoc-arrays condition: (ert-test-failed ((should (equal "20 cm" (org-babel-execute-src-block))) :form (equal "20 cm" nil) :value nil :explanation (different-types "20 cm" nil))) FAILED 24/427 ob-shell/bash-uses-assoc-arrays --8<---------------cut here---------------end--------------->8--- Best regards, Seb -- Sebastien Vauban ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: 2 Org tests failing 2014-06-18 14:55 ` Sebastien Vauban @ 2014-06-18 21:49 ` Achim Gratz 2014-06-19 8:36 ` Sebastien Vauban 0 siblings, 1 reply; 24+ messages in thread From: Achim Gratz @ 2014-06-18 21:49 UTC (permalink / raw) To: emacs-orgmode Sebastien Vauban writes: > I've solved the reported problem by creating the missing TMPDIR (set to > /tmp/org_test) for the invocation from my Zsh (Cygwin) terminal... The first thing the build system does before testing is install -m 755 -d /tmp/tmp-orgtest so you either pick up the wrong install command via path or have otherwise misconfigured MKDIR in local.mk. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Waldorf MIDI Implementation & additional documentation: http://Synth.Stromeko.net/Downloads.html#WaldorfDocs ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: 2 Org tests failing 2014-06-18 21:49 ` Achim Gratz @ 2014-06-19 8:36 ` Sebastien Vauban 2014-06-19 9:04 ` Sebastien Vauban 2014-06-19 20:05 ` Achim Gratz 0 siblings, 2 replies; 24+ messages in thread From: Sebastien Vauban @ 2014-06-19 8:36 UTC (permalink / raw) To: emacs-orgmode-mXXj517/zsQ Achim Gratz wrote: > Sebastien Vauban writes: >> I've solved the reported problem by creating the missing TMPDIR (set to >> /tmp/org_test) for the invocation from my Zsh (Cygwin) terminal... > > The first thing the build system does before testing is > > install -m 755 -d /tmp/tmp-orgtest > > so you either pick up the wrong install command via path or have > otherwise misconfigured MKDIR in local.mk. No, I was using my own function: --8<---------------cut here---------------start------------->8--- org_test () { ( cd $HOME/Public/Repositories/org-mode; TMPDIR=/tmp/org_test emacs -Q --batch -L lisp/ -L testing/ -l org-test.el --eval '(setq org-confirm-babel-evaluate nil)' -f org-test-run-batch-tests ) } --8<---------------cut here---------------end--------------->8--- I just ran `make test' and got the same error for `ob-shell/bash-uses-assoc-arrays'. BTW, how can I avoid compiling the source directory before launching the tests? I prefer having only .el files, as I was mistaken multiple times when trying to understand executions of code I modified. Best regards, Seb -- Sebastien Vauban ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: 2 Org tests failing 2014-06-19 8:36 ` Sebastien Vauban @ 2014-06-19 9:04 ` Sebastien Vauban 2014-06-19 9:08 ` Sebastien Vauban 2014-06-19 20:05 ` Achim Gratz 1 sibling, 1 reply; 24+ messages in thread From: Sebastien Vauban @ 2014-06-19 9:04 UTC (permalink / raw) To: emacs-orgmode-mXXj517/zsQ Sebastien Vauban wrote: > I just ran `make test' and got the same error for > `ob-shell/bash-uses-assoc-arrays'. Now that I updated to Org-mode version 8.2.7 (release_8.2.7-1114-g6c2838), I have an extra test failing: --8<---------------cut here---------------start------------->8--- Test test-org-element/headline-properties backtrace: signal(ert-test-failed (((should-not (org-test-with-temp-text "* Hea ert-fail(((should-not (org-test-with-temp-text "* Headline\n#+BEGIN_ (if (not (unwind-protect (setq value-1904 (let ((inside-text (if ... (let (form-description-1905) (if (not (unwind-protect (setq value-19 (let ((value-1904 (ert--gensym "ert-form-evaluation-aborted-"))) (le (lambda nil (let ((value-1900 (ert--gensym "ert-form-evaluation-abor byte-code("\306\307!q\210\310\216\311 \312\216\313\314\315\316\3 ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc byte-code("\306\307!\211\211r\310\311!q\210\312 d\313\223)L\210)\3 ert-run-test([cl-struct-ert-test test-org-element/headline-propertie ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st ert-run-tests("\\(org\\|ob\\)" #[(event-type &rest event-args) "\306 ert-run-tests-batch("\\(org\\|ob\\)") ert-run-tests-batch-and-exit("\\(org\\|ob\\)") (let ((org-id-track-globally t) (org-test-selector (if org-test-sele org-test-run-batch-tests() call-interactively(org-test-run-batch-tests nil nil) command-execute(org-test-run-batch-tests) command-line-1(("-L" "lisp/" "-L" "testing/" "-l" "org-test.el" "--e command-line() normal-top-level() Test test-org-element/headline-properties condition: (ert-test-failed ((should-not (org-test-with-temp-text "* Headline\n#+BEGIN_EXAMPLE\n:PROPERTIES:\n:foo: bar\n:END:\n#+END_EXAMPLE" (org-element-property :FOO ...))) :form (let ((inside-text ...)) (with-temp-buffer (org-mode) (let ... ...) (org-element-property :FOO ...))) :value "bar")) FAILED 169/427 test-org-element/headline-properties --8<---------------cut here---------------end--------------->8--- Best regards, Seb -- Sebastien Vauban ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: 2 Org tests failing 2014-06-19 9:04 ` Sebastien Vauban @ 2014-06-19 9:08 ` Sebastien Vauban 0 siblings, 0 replies; 24+ messages in thread From: Sebastien Vauban @ 2014-06-19 9:08 UTC (permalink / raw) To: emacs-orgmode-mXXj517/zsQ Sebastien Vauban wrote: > Sebastien Vauban wrote: >> I just ran `make test' and got the same error for >> `ob-shell/bash-uses-assoc-arrays'. > > Now that I updated to Org-mode version 8.2.7 > (release_8.2.7-1114-g6c2838), I have an extra test failing: > > Test test-org-element/headline-properties backtrace: > signal(ert-test-failed (((should-not (org-test-with-temp-text "* Hea > ert-fail(((should-not (org-test-with-temp-text "* Headline\n#+BEGIN_ > (if (not (unwind-protect (setq value-1904 (let ((inside-text (if ... > (let (form-description-1905) (if (not (unwind-protect (setq value-19 > (let ((value-1904 (ert--gensym "ert-form-evaluation-aborted-"))) (le > (lambda nil (let ((value-1900 (ert--gensym "ert-form-evaluation-abor > byte-code("\306\307!q\210\310\216\311 \312\216\313\314\315\316\3 > ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc > byte-code("\306\307!\211\211r\310\311!q\210\312 d\313\223)L\210)\3 > ert-run-test([cl-struct-ert-test test-org-element/headline-propertie > ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st > ert-run-tests("\\(org\\|ob\\)" #[(event-type &rest event-args) "\306 > ert-run-tests-batch("\\(org\\|ob\\)") > ert-run-tests-batch-and-exit("\\(org\\|ob\\)") > (let ((org-id-track-globally t) (org-test-selector (if org-test-sele > org-test-run-batch-tests() > call-interactively(org-test-run-batch-tests nil nil) > command-execute(org-test-run-batch-tests) > command-line-1(("-L" "lisp/" "-L" "testing/" "-l" "org-test.el" "--e > command-line() > normal-top-level() > Test test-org-element/headline-properties condition: > (ert-test-failed > ((should-not > (org-test-with-temp-text "* Headline\n#+BEGIN_EXAMPLE\n:PROPERTIES:\n:foo: bar\n:END:\n#+END_EXAMPLE" > (org-element-property :FOO ...))) > :form > (let > ((inside-text ...)) > (with-temp-buffer > (org-mode) > (let ... ...) > (org-element-property :FOO ...))) > :value "bar")) > FAILED 169/427 test-org-element/headline-properties False positive: was given by my test function. After using `make test' (hence recompiling the source), that one disappeared. Only one staying: 1 unexpected results: FAILED ob-shell/bash-uses-assoc-arrays Best regards, Seb -- Sebastien Vauban ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: 2 Org tests failing 2014-06-19 8:36 ` Sebastien Vauban 2014-06-19 9:04 ` Sebastien Vauban @ 2014-06-19 20:05 ` Achim Gratz 2014-06-22 8:35 ` [PATCH] ob-shell (was: 2 Org tests failing) Achim Gratz 1 sibling, 1 reply; 24+ messages in thread From: Achim Gratz @ 2014-06-19 20:05 UTC (permalink / raw) To: emacs-orgmode Sebastien Vauban writes: > I just ran `make test' and got the same error for > `ob-shell/bash-uses-assoc-arrays'. Yes, that's because not all versions of bash that have associative arrays can parse the bizarre quoting style that goes through a sub-process and here-document that is used to fill in the parameters. A better quoting function would be (defun org-babel-sh-var-to-sh (var &optional sep hline) "Convert an elisp value to a shell variable. Convert an elisp var into a string of shell commands specifying a var of the same value." (concat "'" (replace-regexp-in-string "'" "'\"'\"'" (org-babel-sh-var-to-string var sep hline)) "'")) (and remove the double quotes around the places where the value is going to be used). It'd have the advantage of working with more types of shells, but I'm not 100% certain that there aren't any corner cases. > BTW, how can I avoid compiling the source directory before launching the > tests? I prefer having only .el files, as I was mistaken multiple times > when trying to understand executions of code I modified. Take a few minutes and read: http://orgmode.org/worg/dev/org-build-system.html Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for KORG EX-800 and Poly-800MkII V0.9: http://Synth.Stromeko.net/Downloads.html#KorgSDada ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] ob-shell (was: 2 Org tests failing) 2014-06-19 20:05 ` Achim Gratz @ 2014-06-22 8:35 ` Achim Gratz 2014-06-22 12:50 ` [PATCH] ob-shell Eric Schulte 0 siblings, 1 reply; 24+ messages in thread From: Achim Gratz @ 2014-06-22 8:35 UTC (permalink / raw) To: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 446 bytes --] Achim Gratz writes: > Sebastien Vauban writes: >> I just ran `make test' and got the same error for >> `ob-shell/bash-uses-assoc-arrays'. > > Yes, that's because not all versions of bash that have associative > arrays can parse the bizarre quoting style that goes through a > sub-process and here-document that is used to fill in the parameters. Here's a patch that implements the suggestion and tested to work correctly with Cygwin and Linux. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-ob-shell-stratify-shell-variable-quoting.patch --] [-- Type: text/x-patch, Size: 3322 bytes --] From a79aff65d562e59ed4e01e550224eb96a665c1ae Mon Sep 17 00:00:00 2001 From: Achim Gratz <Stromeko@Stromeko.DE> Date: Thu, 19 Jun 2014 21:23:28 +0200 Subject: [PATCH] ob-shell: stratify shell variable quoting * lisp/ob-shell.el: Remove unused defcustom `org-babel-sh-var-quote-fmt'. (org-babel-variable-assignments:bash_array): (org-babel-variable-assignments:bash_assoc): Remove superfluous `mapcar' and double quotes around parameters. (org-babel-sh-var-to-sh): Single-quote the whole string and escape all single quotes in the original string. --- lisp/ob-shell.el | 42 ++++++++++++++++++------------------------ 1 file changed, 18 insertions(+), 24 deletions(-) diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el index 474a8f2..7d87026 100644 --- a/lisp/ob-shell.el +++ b/lisp/ob-shell.el @@ -45,12 +45,6 @@ (defcustom org-babel-sh-command shell-file-name :group 'org-babel :type 'string) -(defcustom org-babel-sh-var-quote-fmt - "$(cat <<'BABEL_TABLE'\n%s\nBABEL_TABLE\n)" - "Format string used to escape variables when passed to shell scripts." - :group 'org-babel - :type 'string) - (defcustom org-babel-shell-names '("sh" "bash" "csh" "ash" "dash" "ksh" "mksh" "posh") "List of names of shell supported by babel shell code blocks." @@ -113,28 +107,26 @@ (defun org-babel-variable-assignments:sh-generic (defun org-babel-variable-assignments:bash_array (varname values &optional sep hline) "Returns a list of statements declaring the values as a bash array." - (format "unset %s\ndeclare -a %s=( \"%s\" )" - varname varname - (mapconcat 'identity - (mapcar - (lambda (value) (org-babel-sh-var-to-sh value sep hline)) - values) - "\" \""))) + (format "unset %s\ndeclare -a %s=( %s )" + varname varname + (mapconcat + (lambda (value) (org-babel-sh-var-to-sh value sep hline)) + values + " "))) (defun org-babel-variable-assignments:bash_assoc (varname values &optional sep hline) "Returns a list of statements declaring the values as bash associative array." (format "unset %s\ndeclare -A %s\n%s" varname varname - (mapconcat 'identity - (mapcar - (lambda (items) - (format "%s[\"%s\"]=%s" - varname - (org-babel-sh-var-to-sh (car items) sep hline) - (org-babel-sh-var-to-sh (cdr items) sep hline))) - values) - "\n"))) + (mapconcat + (lambda (items) + (format "%s[%s]=%s" + varname + (org-babel-sh-var-to-sh (car items) sep hline) + (org-babel-sh-var-to-sh (cdr items) sep hline))) + values + "\n"))) (defun org-babel-variable-assignments:bash (varname values &optional sep hline) "Represents the parameters as useful Bash shell variables." @@ -163,8 +155,10 @@ (defun org-babel-sh-var-to-sh (var &optional sep hline) "Convert an elisp value to a shell variable. Convert an elisp var into a string of shell commands specifying a var of the same value." - (format org-babel-sh-var-quote-fmt - (org-babel-sh-var-to-string var sep hline))) + (concat "'" (replace-regexp-in-string + "'" "'\"'\"'" + (org-babel-sh-var-to-string var sep hline)) + "'")) (defun org-babel-sh-var-to-string (var &optional sep hline) "Convert an elisp value to a string." -- 2.0.0 [-- Attachment #3: Type: text/plain, Size: 201 bytes --] Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Waldorf MIDI Implementation & additional documentation: http://Synth.Stromeko.net/Downloads.html#WaldorfDocs ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH] ob-shell 2014-06-22 8:35 ` [PATCH] ob-shell (was: 2 Org tests failing) Achim Gratz @ 2014-06-22 12:50 ` Eric Schulte 2014-06-23 6:26 ` Pascal Fleury ` (2 more replies) 0 siblings, 3 replies; 24+ messages in thread From: Eric Schulte @ 2014-06-22 12:50 UTC (permalink / raw) To: Achim Gratz; +Cc: emacs-orgmode If this maintains existing functionality, please go ahead and apply it. Thanks, Achim Gratz <Stromeko@nexgo.de> writes: > Achim Gratz writes: >> Sebastien Vauban writes: >>> I just ran `make test' and got the same error for >>> `ob-shell/bash-uses-assoc-arrays'. >> >> Yes, that's because not all versions of bash that have associative >> arrays can parse the bizarre quoting style that goes through a >> sub-process and here-document that is used to fill in the parameters. > > Here's a patch that implements the suggestion and tested to work > correctly with Cygwin and Linux. > > From a79aff65d562e59ed4e01e550224eb96a665c1ae Mon Sep 17 00:00:00 2001 > From: Achim Gratz <Stromeko@Stromeko.DE> > Date: Thu, 19 Jun 2014 21:23:28 +0200 > Subject: [PATCH] ob-shell: stratify shell variable quoting > > * lisp/ob-shell.el: Remove unused defcustom > `org-babel-sh-var-quote-fmt'. > (org-babel-variable-assignments:bash_array): > (org-babel-variable-assignments:bash_assoc): Remove superfluous > `mapcar' and double quotes around parameters. > (org-babel-sh-var-to-sh): Single-quote the whole string and escape > all single quotes in the original string. > --- > lisp/ob-shell.el | 42 ++++++++++++++++++------------------------ > 1 file changed, 18 insertions(+), 24 deletions(-) > > diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el > index 474a8f2..7d87026 100644 > --- a/lisp/ob-shell.el > +++ b/lisp/ob-shell.el > @@ -45,12 +45,6 @@ (defcustom org-babel-sh-command shell-file-name > :group 'org-babel > :type 'string) > > -(defcustom org-babel-sh-var-quote-fmt > - "$(cat <<'BABEL_TABLE'\n%s\nBABEL_TABLE\n)" > - "Format string used to escape variables when passed to shell scripts." > - :group 'org-babel > - :type 'string) > - > (defcustom org-babel-shell-names > '("sh" "bash" "csh" "ash" "dash" "ksh" "mksh" "posh") > "List of names of shell supported by babel shell code blocks." > @@ -113,28 +107,26 @@ (defun org-babel-variable-assignments:sh-generic > (defun org-babel-variable-assignments:bash_array > (varname values &optional sep hline) > "Returns a list of statements declaring the values as a bash array." > - (format "unset %s\ndeclare -a %s=( \"%s\" )" > - varname varname > - (mapconcat 'identity > - (mapcar > - (lambda (value) (org-babel-sh-var-to-sh value sep hline)) > - values) > - "\" \""))) > + (format "unset %s\ndeclare -a %s=( %s )" > + varname varname > + (mapconcat > + (lambda (value) (org-babel-sh-var-to-sh value sep hline)) > + values > + " "))) > > (defun org-babel-variable-assignments:bash_assoc > (varname values &optional sep hline) > "Returns a list of statements declaring the values as bash associative array." > (format "unset %s\ndeclare -A %s\n%s" > varname varname > - (mapconcat 'identity > - (mapcar > - (lambda (items) > - (format "%s[\"%s\"]=%s" > - varname > - (org-babel-sh-var-to-sh (car items) sep hline) > - (org-babel-sh-var-to-sh (cdr items) sep hline))) > - values) > - "\n"))) > + (mapconcat > + (lambda (items) > + (format "%s[%s]=%s" > + varname > + (org-babel-sh-var-to-sh (car items) sep hline) > + (org-babel-sh-var-to-sh (cdr items) sep hline))) > + values > + "\n"))) > > (defun org-babel-variable-assignments:bash (varname values &optional sep hline) > "Represents the parameters as useful Bash shell variables." > @@ -163,8 +155,10 @@ (defun org-babel-sh-var-to-sh (var &optional sep hline) > "Convert an elisp value to a shell variable. > Convert an elisp var into a string of shell commands specifying a > var of the same value." > - (format org-babel-sh-var-quote-fmt > - (org-babel-sh-var-to-string var sep hline))) > + (concat "'" (replace-regexp-in-string > + "'" "'\"'\"'" > + (org-babel-sh-var-to-string var sep hline)) > + "'")) > > (defun org-babel-sh-var-to-string (var &optional sep hline) > "Convert an elisp value to a string." > -- > 2.0.0 > > > > Regards, > Achim. -- Eric Schulte https://cs.unm.edu/~eschulte PGP: 0x614CA05D (see https://u.fsf.org/yw) ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] ob-shell 2014-06-22 12:50 ` [PATCH] ob-shell Eric Schulte @ 2014-06-23 6:26 ` Pascal Fleury 2014-06-23 17:20 ` Achim Gratz 2014-06-23 20:02 ` Achim Gratz 2014-07-24 7:38 ` Alan Schmitt 2 siblings, 1 reply; 24+ messages in thread From: Pascal Fleury @ 2014-06-23 6:26 UTC (permalink / raw) To: Eric Schulte; +Cc: Achim Gratz, emacs-org list [-- Attachment #1: Type: text/plain, Size: 5225 bytes --] Hi Achim, I was wondering how it would behave if the string that is put into a variable contains newlines, backslashes and other things that bash (and other shells) treats specially. The trick with cat and BABEL_TABLE is resistant to this. As in many cases (when I use it at least) the variable come from other parts of the org file and are computed through another language (python in my case) I have such cases that would break. Also, I think the test coverage for such things is limited for ob-shell.el in the org test suite. We should probably add some test cases for this, and then make sure it works with your implementation as well. --paf On Sun, Jun 22, 2014 at 2:50 PM, Eric Schulte <schulte.eric@gmail.com> wrote: > If this maintains existing functionality, please go ahead and apply it. > > Thanks, > > Achim Gratz <Stromeko@nexgo.de> writes: > > > Achim Gratz writes: > >> Sebastien Vauban writes: > >>> I just ran `make test' and got the same error for > >>> `ob-shell/bash-uses-assoc-arrays'. > >> > >> Yes, that's because not all versions of bash that have associative > >> arrays can parse the bizarre quoting style that goes through a > >> sub-process and here-document that is used to fill in the parameters. > > > > Here's a patch that implements the suggestion and tested to work > > correctly with Cygwin and Linux. > > > > From a79aff65d562e59ed4e01e550224eb96a665c1ae Mon Sep 17 00:00:00 2001 > > From: Achim Gratz <Stromeko@Stromeko.DE> > > Date: Thu, 19 Jun 2014 21:23:28 +0200 > > Subject: [PATCH] ob-shell: stratify shell variable quoting > > > > * lisp/ob-shell.el: Remove unused defcustom > > `org-babel-sh-var-quote-fmt'. > > (org-babel-variable-assignments:bash_array): > > (org-babel-variable-assignments:bash_assoc): Remove superfluous > > `mapcar' and double quotes around parameters. > > (org-babel-sh-var-to-sh): Single-quote the whole string and escape > > all single quotes in the original string. > > --- > > lisp/ob-shell.el | 42 ++++++++++++++++++------------------------ > > 1 file changed, 18 insertions(+), 24 deletions(-) > > > > diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el > > index 474a8f2..7d87026 100644 > > --- a/lisp/ob-shell.el > > +++ b/lisp/ob-shell.el > > @@ -45,12 +45,6 @@ (defcustom org-babel-sh-command shell-file-name > > :group 'org-babel > > :type 'string) > > > > -(defcustom org-babel-sh-var-quote-fmt > > - "$(cat <<'BABEL_TABLE'\n%s\nBABEL_TABLE\n)" > > - "Format string used to escape variables when passed to shell scripts." > > - :group 'org-babel > > - :type 'string) > > - > > (defcustom org-babel-shell-names > > '("sh" "bash" "csh" "ash" "dash" "ksh" "mksh" "posh") > > "List of names of shell supported by babel shell code blocks." > > @@ -113,28 +107,26 @@ (defun org-babel-variable-assignments:sh-generic > > (defun org-babel-variable-assignments:bash_array > > (varname values &optional sep hline) > > "Returns a list of statements declaring the values as a bash array." > > - (format "unset %s\ndeclare -a %s=( \"%s\" )" > > - varname varname > > - (mapconcat 'identity > > - (mapcar > > - (lambda (value) (org-babel-sh-var-to-sh value sep hline)) > > - values) > > - "\" \""))) > > + (format "unset %s\ndeclare -a %s=( %s )" > > + varname varname > > + (mapconcat > > + (lambda (value) (org-babel-sh-var-to-sh value sep hline)) > > + values > > + " "))) > > > > (defun org-babel-variable-assignments:bash_assoc > > (varname values &optional sep hline) > > "Returns a list of statements declaring the values as bash > associative array." > > (format "unset %s\ndeclare -A %s\n%s" > > varname varname > > - (mapconcat 'identity > > - (mapcar > > - (lambda (items) > > - (format "%s[\"%s\"]=%s" > > - varname > > - (org-babel-sh-var-to-sh (car items) sep hline) > > - (org-babel-sh-var-to-sh (cdr items) sep hline))) > > - values) > > - "\n"))) > > + (mapconcat > > + (lambda (items) > > + (format "%s[%s]=%s" > > + varname > > + (org-babel-sh-var-to-sh (car items) sep hline) > > + (org-babel-sh-var-to-sh (cdr items) sep hline))) > > + values > > + "\n"))) > > > > (defun org-babel-variable-assignments:bash (varname values &optional > sep hline) > > "Represents the parameters as useful Bash shell variables." > > @@ -163,8 +155,10 @@ (defun org-babel-sh-var-to-sh (var &optional sep > hline) > > "Convert an elisp value to a shell variable. > > Convert an elisp var into a string of shell commands specifying a > > var of the same value." > > - (format org-babel-sh-var-quote-fmt > > - (org-babel-sh-var-to-string var sep hline))) > > + (concat "'" (replace-regexp-in-string > > + "'" "'\"'\"'" > > + (org-babel-sh-var-to-string var sep hline)) > > + "'")) > > > > (defun org-babel-sh-var-to-string (var &optional sep hline) > > "Convert an elisp value to a string." > > -- > > 2.0.0 > > > > > > > > Regards, > > Achim. > > -- > Eric Schulte > https://cs.unm.edu/~eschulte > PGP: 0x614CA05D (see https://u.fsf.org/yw) > > -- --paf [-- Attachment #2: Type: text/html, Size: 7196 bytes --] ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] ob-shell 2014-06-23 6:26 ` Pascal Fleury @ 2014-06-23 17:20 ` Achim Gratz 0 siblings, 0 replies; 24+ messages in thread From: Achim Gratz @ 2014-06-23 17:20 UTC (permalink / raw) To: emacs-orgmode Pascal Fleury writes: > I was wondering how it would behave if the string that is put into a > variable contains newlines, backslashes and other things that bash > (and other shells) treats specially. Exactly like it does with the earlier method, except it doesn't fork and doesn't require certain versions of bash. > The trick with cat and BABEL_TABLE is resistant to this. I don't think so, some older versions of bash had a parsing bug that made it croak on single quotes in here documents. Single quoting the variable content only reuires special handling of single quotes. While that is tedious and error prone to do by hand, we're doing it with elisp anyway so this doesn't pose a problem. Also, it works for just about any shell, not only those that know about $( ) command substitution and implement here documents within the substitution correctly. > As in many cases (when I use it at least) the variable come from other > parts of the org file and are computed through another language > (python in my case) I have such cases that would break. If you have an example, please show it. > Also, I think the test coverage for such things is limited for > ob-shell.el in the org test suite. > We should probably add some test cases for this, and then make sure it > works with your implementation as well. By all means, please add such tests. While you're at it, you might want to double quote the variable expansions, which will show you that the shell is wholly capable of returning a list and even a table to Org, with or without associative arrays. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Samples for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] ob-shell 2014-06-22 12:50 ` [PATCH] ob-shell Eric Schulte 2014-06-23 6:26 ` Pascal Fleury @ 2014-06-23 20:02 ` Achim Gratz 2014-07-24 7:38 ` Alan Schmitt 2 siblings, 0 replies; 24+ messages in thread From: Achim Gratz @ 2014-06-23 20:02 UTC (permalink / raw) To: emacs-orgmode [re-sent, sorry if you get this twice] Eric Schulte writes: > If this maintains existing functionality, please go ahead and apply it. Done on master. The tests are left untouched, I will not have time to do anything there for the next few days. Any helping hand is welcome. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Waldorf MIDI Implementation & additional documentation: http://Synth.Stromeko.net/Downloads.html#WaldorfDocs ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] ob-shell 2014-06-22 12:50 ` [PATCH] ob-shell Eric Schulte 2014-06-23 6:26 ` Pascal Fleury 2014-06-23 20:02 ` Achim Gratz @ 2014-07-24 7:38 ` Alan Schmitt 2014-08-04 11:56 ` Eric Schulte 2014-08-04 19:48 ` Achim Gratz 2 siblings, 2 replies; 24+ messages in thread From: Alan Schmitt @ 2014-07-24 7:38 UTC (permalink / raw) To: Eric Schulte; +Cc: Achim Gratz, emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 355 bytes --] Hello, On 2014-06-22 14:50, Eric Schulte <schulte.eric@gmail.com> writes: > If this maintains existing functionality, please go ahead and apply it. Has this been applied? I'm still seeing a failing test for ob-shell. 1 unexpected results: FAILED ob-shell/bash-uses-assoc-arrays Best, Alan -- OpenPGP Key ID : 040D0A3B4ED2E5C7 [-- Attachment #2: Type: application/pgp-signature, Size: 494 bytes --] ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] ob-shell 2014-07-24 7:38 ` Alan Schmitt @ 2014-08-04 11:56 ` Eric Schulte 2014-08-04 19:48 ` Achim Gratz 1 sibling, 0 replies; 24+ messages in thread From: Eric Schulte @ 2014-08-04 11:56 UTC (permalink / raw) To: Alan Schmitt; +Cc: Achim Gratz, emacs-orgmode Alan Schmitt <alan.schmitt@polytechnique.org> writes: > Hello, > > On 2014-06-22 14:50, Eric Schulte <schulte.eric@gmail.com> writes: > >> If this maintains existing functionality, please go ahead and apply it. > > Has this been applied? I'm still seeing a failing test for ob-shell. > > 1 unexpected results: > FAILED ob-shell/bash-uses-assoc-arrays > > Best, > > Alan It looks like Achim is part way through making the required changes. Achim Gratz <Stromeko@nexgo.de> writes: > Eric Schulte writes: > > If this maintains existing functionality, please go ahead and apply it. > > Done on master. The tests are left untouched, I will not have time to > do anything there for the next few days. Any helping hand is welcome. > > > Regards, > Achim. Best, Eric -- Eric Schulte https://cs.unm.edu/~eschulte PGP: 0x614CA05D (see https://u.fsf.org/yw) ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] ob-shell 2014-07-24 7:38 ` Alan Schmitt 2014-08-04 11:56 ` Eric Schulte @ 2014-08-04 19:48 ` Achim Gratz 2014-08-13 13:12 ` Alan Schmitt 1 sibling, 1 reply; 24+ messages in thread From: Achim Gratz @ 2014-08-04 19:48 UTC (permalink / raw) To: emacs-orgmode Alan Schmitt writes: > Has this been applied? I'm still seeing a failing test for ob-shell. > > 1 unexpected results: > FAILED ob-shell/bash-uses-assoc-arrays That's to be expected if you use a version of bash that doesn't support assoc arrays. WHat is the backtrace from the test? Babel (and the tests) blindly assume that these things work when they see you using "bash". Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Samples for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] ob-shell 2014-08-04 19:48 ` Achim Gratz @ 2014-08-13 13:12 ` Alan Schmitt 2014-08-13 13:20 ` Pascal Fleury 0 siblings, 1 reply; 24+ messages in thread From: Alan Schmitt @ 2014-08-13 13:12 UTC (permalink / raw) To: Achim Gratz; +Cc: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 3099 bytes --] On 2014-08-04 21:48, Achim Gratz <Stromeko@nexgo.de> writes: > Alan Schmitt writes: >> Has this been applied? I'm still seeing a failing test for ob-shell. >> >> 1 unexpected results: >> FAILED ob-shell/bash-uses-assoc-arrays > > That's to be expected if you use a version of bash that doesn't support > assoc arrays. I don't use bash, I either use zsh or fish. So I guess the version of bash on my system is the one that ships with OS X 10.9: GNU bash, version 3.2.51(1)-release (x86_64-apple-darwin13) > WHat is the backtrace from the test? Babel (and the > tests) blindly assume that these things work when they see you using > "bash". Test ob-shell/bash-uses-assoc-arrays backtrace: signal(ert-test-failed (((should (equal "20 cm" (org-babel-execute-s ert-fail(((should (equal "20 cm" (org-babel-execute-src-block))) :fo (if (unwind-protect (setq value-1098 (apply fn-1096 args-1097)) (set (let (form-description-1100) (if (unwind-protect (setq value-1098 (a (let ((value-1098 (quote ert-form-evaluation-aborted-1099))) (let (f (let ((fn-1096 (function equal)) (args-1097 (list "20 cm" (org-babel (save-restriction (org-babel-next-src-block 2) (let ((fn-1096 (funct (progn (org-id-goto "82320a48-3409-49d7-85c9-5de1c6d3ff87") (setq to (unwind-protect (progn (org-id-goto "82320a48-3409-49d7-85c9-5de1c6d (let ((save-match-data-internal (match-data))) (unwind-protect (prog (progn (let ((save-match-data-internal (match-data))) (unwind-protec (unwind-protect (progn (let ((save-match-data-internal (match-data)) (let ((wconfig (current-window-configuration))) (unwind-protect (pro (unwind-protect (let ((wconfig (current-window-configuration))) (unw (let* ((id-location (org-id-find "82320a48-3409-49d7-85c9-5de1c6d3ff (lambda nil (let* ((id-location (org-id-find "82320a48-3409-49d7-85c byte-code("\306\307!q\210\310\216\311 \312\216\313\314\315\316\3 ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc byte-code("\306\307!\211\211r\310\311!q\210\312 d\313\223)L\210)\3 ert-run-test([cl-struct-ert-test ob-shell/bash-uses-assoc-arrays "Ba ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st ert-run-tests("\\(org\\|ob\\)" #[(event-type &rest event-args) "\306 ert-run-tests-batch("\\(org\\|ob\\)") ert-run-tests-batch-and-exit("\\(org\\|ob\\)") (let ((org-id-track-globally t) (org-test-selector (if org-test-sele org-test-run-batch-tests("\\(org\\|ob\\)") eval((org-test-run-batch-tests org-test-select-re)) command-line-1(("--eval" "(setq vc-handled-backends nil org-startup- command-line() normal-top-level() Test ob-shell/bash-uses-assoc-arrays condition: (ert-test-failed ((should (equal "20 cm" (org-babel-execute-src-block))) :form (equal "20 cm" "50 dl") :value nil :explanation (array-elt 0 (different-atoms (50 "#x32" "?2") (53 "#x35" "?5"))))) FAILED 104/510 ob-shell/bash-uses-assoc-arrays Thanks, Alan -- OpenPGP Key ID : 040D0A3B4ED2E5C7 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 494 bytes --] ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] ob-shell 2014-08-13 13:12 ` Alan Schmitt @ 2014-08-13 13:20 ` Pascal Fleury 2014-08-13 13:29 ` Alan Schmitt 0 siblings, 1 reply; 24+ messages in thread From: Pascal Fleury @ 2014-08-13 13:20 UTC (permalink / raw) To: Alan Schmitt; +Cc: Achim Gratz, emacs-org list [-- Attachment #1: Type: text/plain, Size: 3745 bytes --] Yes, sorry for late reaction (it's holiday season here :-). This is an issue I introduced with the assoc. arrays, which does not verify if the installed version of bash handles them (from bash4 on). It needs a modification in the setup code that checks the version and only issues assoc. array code when bash supports it. And it so happens that Macs have bash3 installed. Will try to find such a machine and add the test. --paf On Wed, Aug 13, 2014 at 3:12 PM, Alan Schmitt < alan.schmitt@polytechnique.org> wrote: > On 2014-08-04 21:48, Achim Gratz <Stromeko@nexgo.de> writes: > > > Alan Schmitt writes: > >> Has this been applied? I'm still seeing a failing test for ob-shell. > >> > >> 1 unexpected results: > >> FAILED ob-shell/bash-uses-assoc-arrays > > > > That's to be expected if you use a version of bash that doesn't support > > assoc arrays. > > I don't use bash, I either use zsh or fish. So I guess the version of > bash on my system is the one that ships with OS X 10.9: > > GNU bash, version 3.2.51(1)-release (x86_64-apple-darwin13) > > > WHat is the backtrace from the test? Babel (and the > > tests) blindly assume that these things work when they see you using > > "bash". > > Test ob-shell/bash-uses-assoc-arrays backtrace: > signal(ert-test-failed (((should (equal "20 cm" (org-babel-execute-s > ert-fail(((should (equal "20 cm" (org-babel-execute-src-block))) :fo > (if (unwind-protect (setq value-1098 (apply fn-1096 args-1097)) (set > (let (form-description-1100) (if (unwind-protect (setq value-1098 (a > (let ((value-1098 (quote ert-form-evaluation-aborted-1099))) (let (f > (let ((fn-1096 (function equal)) (args-1097 (list "20 cm" (org-babel > (save-restriction (org-babel-next-src-block 2) (let ((fn-1096 (funct > (progn (org-id-goto "82320a48-3409-49d7-85c9-5de1c6d3ff87") (setq to > (unwind-protect (progn (org-id-goto "82320a48-3409-49d7-85c9-5de1c6d > (let ((save-match-data-internal (match-data))) (unwind-protect (prog > (progn (let ((save-match-data-internal (match-data))) (unwind-protec > (unwind-protect (progn (let ((save-match-data-internal (match-data)) > (let ((wconfig (current-window-configuration))) (unwind-protect (pro > (unwind-protect (let ((wconfig (current-window-configuration))) (unw > (let* ((id-location (org-id-find "82320a48-3409-49d7-85c9-5de1c6d3ff > (lambda nil (let* ((id-location (org-id-find "82320a48-3409-49d7-85c > byte-code("\306\307!q\210\310\216\311 \312\216\313\314\315\316\3 > ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc > byte-code("\306\307!\211\211r\310\311!q\210\312 d\313\223)L\210)\3 > ert-run-test([cl-struct-ert-test ob-shell/bash-uses-assoc-arrays "Ba > ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st > ert-run-tests("\\(org\\|ob\\)" #[(event-type &rest event-args) "\306 > ert-run-tests-batch("\\(org\\|ob\\)") > ert-run-tests-batch-and-exit("\\(org\\|ob\\)") > (let ((org-id-track-globally t) (org-test-selector (if org-test-sele > org-test-run-batch-tests("\\(org\\|ob\\)") > eval((org-test-run-batch-tests org-test-select-re)) > command-line-1(("--eval" "(setq vc-handled-backends nil org-startup- > command-line() > normal-top-level() > Test ob-shell/bash-uses-assoc-arrays condition: > (ert-test-failed > ((should > (equal "20 cm" > (org-babel-execute-src-block))) > :form > (equal "20 cm" "50 dl") > :value nil :explanation > (array-elt 0 > (different-atoms > (50 "#x32" "?2") > (53 "#x35" "?5"))))) > FAILED 104/510 ob-shell/bash-uses-assoc-arrays > > Thanks, > > Alan > > -- > OpenPGP Key ID : 040D0A3B4ED2E5C7 > -- --paf [-- Attachment #2: Type: text/html, Size: 4896 bytes --] ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] ob-shell 2014-08-13 13:20 ` Pascal Fleury @ 2014-08-13 13:29 ` Alan Schmitt 2014-08-22 8:52 ` Pascal Fleury 0 siblings, 1 reply; 24+ messages in thread From: Alan Schmitt @ 2014-08-13 13:29 UTC (permalink / raw) To: Pascal Fleury; +Cc: Achim Gratz, emacs-org list [-- Attachment #1: Type: text/plain, Size: 683 bytes --] On 2014-08-13 15:20, Pascal Fleury <fleury@google.com> writes: > Yes, sorry for late reaction (it's holiday season here :-). Funny, it's holiday season here as well (in Bretagne) ;-) > This is an issue I introduced with the assoc. arrays, which does not > verify if the installed version of bash handles them (from bash4 on). > It needs a modification in the setup code that checks the version and > only issues assoc. array code when bash supports it. And it so happens > that Macs have bash3 installed. Will try to find such a machine and > add the test. Thanks a lot. If I can help, please let me know. Best, Alan -- OpenPGP Key ID : 040D0A3B4ED2E5C7 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 494 bytes --] ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] ob-shell 2014-08-13 13:29 ` Alan Schmitt @ 2014-08-22 8:52 ` Pascal Fleury 2014-08-22 13:08 ` Pascal Fleury 0 siblings, 1 reply; 24+ messages in thread From: Pascal Fleury @ 2014-08-22 8:52 UTC (permalink / raw) To: Alan Schmitt; +Cc: Achim Gratz, emacs-org list [-- Attachment #1.1: Type: text/plain, Size: 1216 bytes --] Hello, I have a little patch (attached) that does check the version of bash, and uses associative arrays (or doesn't) depending on that version. The problem now is that the test suite will only work on one platform t a time, i.e. if there is a bash4 there, it will pass, but if it has bash3, it will work as intended, but return a different value set. Is there a way to have platform-dependent tests ? --Pascal On Wed, Aug 13, 2014 at 3:29 PM, Alan Schmitt < alan.schmitt@polytechnique.org> wrote: > On 2014-08-13 15:20, Pascal Fleury <fleury@google.com> writes: > > > Yes, sorry for late reaction (it's holiday season here :-). > > Funny, it's holiday season here as well (in Bretagne) ;-) > > > This is an issue I introduced with the assoc. arrays, which does not > > verify if the installed version of bash handles them (from bash4 on). > > It needs a modification in the setup code that checks the version and > > only issues assoc. array code when bash supports it. And it so happens > > that Macs have bash3 installed. Will try to find such a machine and > > add the test. > > Thanks a lot. If I can help, please let me know. > > Best, > > Alan > > -- > OpenPGP Key ID : 040D0A3B4ED2E5C7 > -- --paf [-- Attachment #1.2: Type: text/html, Size: 1891 bytes --] [-- Attachment #2: 0001-ob-shell-now-checks-for-the-current-bash-version-bef.patch --] [-- Type: application/octet-stream, Size: 1555 bytes --] From d28d82dd88d2e61a2ce08cc69eee02b8213c6252 Mon Sep 17 00:00:00 2001 From: Pascal Fleury <fleury@google.com> Date: Fri, 22 Aug 2014 10:45:20 +0200 Subject: [PATCH] ob-shell now checks for the current bash version before emitting code that uses associative arrays, because bash supports them only from version 4 and on, and on some platforms (e.g. MacOSX) the default bash is v3. --- lisp/ob-shell.el | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el index 8c58dfd..27d6de7 100644 --- a/lisp/ob-shell.el +++ b/lisp/ob-shell.el @@ -121,10 +121,15 @@ This function is called by `org-babel-execute-src-block'." values "\n"))) +(defun org-babel-shell-bash-version + "Returns the major revision of the current bash running on the system." + (car (parse-integer (org-babel-execute:shell "echo ${BASH_VERSINFO[0]}" nil)))) + (defun org-babel-variable-assignments:bash (varname values &optional sep hline) "Represents the parameters as useful Bash shell variables." - (if (listp values) - (if (and (listp (car values)) (= 1 (length (car values)))) + (if (listp values) + (if (and (< 3 (org-babel-shell-bash-version) ; bash supports assoc arrays only from version 4 on. + (listp (car values)) (= 1 (length (car values))))) (org-babel-variable-assignments:bash_array varname values sep hline) (org-babel-variable-assignments:bash_assoc varname values sep hline)) (org-babel-variable-assignments:sh-generic varname values sep hline))) -- 1.9.3 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH] ob-shell 2014-08-22 8:52 ` Pascal Fleury @ 2014-08-22 13:08 ` Pascal Fleury 2014-08-22 18:16 ` Achim Gratz 0 siblings, 1 reply; 24+ messages in thread From: Pascal Fleury @ 2014-08-22 13:08 UTC (permalink / raw) To: Alan Schmitt; +Cc: Achim Gratz, emacs-org list [-- Attachment #1.1: Type: text/plain, Size: 1729 bytes --] Me again, Please forget about the previous patch I submitted, I have a new one that should work now on all platforms. Let me know if it should be formatted differently. I will also add some more tests to check that such arrays in bash are handled properly when the input contains whitespace, quotes, commas and other chars that one may have to deal with. --paf On Fri, Aug 22, 2014 at 10:52 AM, Pascal Fleury <fleury@google.com> wrote: > Hello, > > I have a little patch (attached) that does check the version of bash, and > uses associative arrays (or doesn't) depending on that version. > > The problem now is that the test suite will only work on one platform t a > time, i.e. if there is a bash4 there, it will pass, but if it has bash3, it > will work as intended, but return a different value set. Is there a way to > have platform-dependent tests ? > > --Pascal > > > > On Wed, Aug 13, 2014 at 3:29 PM, Alan Schmitt < > alan.schmitt@polytechnique.org> wrote: > >> On 2014-08-13 15:20, Pascal Fleury <fleury@google.com> writes: >> >> > Yes, sorry for late reaction (it's holiday season here :-). >> >> Funny, it's holiday season here as well (in Bretagne) ;-) >> >> > This is an issue I introduced with the assoc. arrays, which does not >> > verify if the installed version of bash handles them (from bash4 on). >> > It needs a modification in the setup code that checks the version and >> > only issues assoc. array code when bash supports it. And it so happens >> > that Macs have bash3 installed. Will try to find such a machine and >> > add the test. >> >> Thanks a lot. If I can help, please let me know. >> >> Best, >> >> Alan >> >> -- >> OpenPGP Key ID : 040D0A3B4ED2E5C7 >> > > > > -- > --paf > -- --paf [-- Attachment #1.2: Type: text/html, Size: 2858 bytes --] [-- Attachment #2: 0001-ob-shell.el-now-emits-associative-array-code-only-wh.patch --] [-- Type: application/octet-stream, Size: 3093 bytes --] From 251566b07d75e0254f5044b3807cb9d0ea9ff749 Mon Sep 17 00:00:00 2001 From: Pascal Fleury <fleury@google.com> Date: Fri, 22 Aug 2014 15:04:00 +0200 Subject: [PATCH] ob-shell.el now emits associative array code only when the installed version of bash supports them. This takes care of differences in platforms that ship with bash v4+, that can deal with them, and those (e.g. MacOSX) that ship with bash v3.x that does not support such arrays. --- lisp/ob-shell.el | 12 +++++++++--- testing/examples/ob-shell-test.org | 8 +++++++- testing/lisp/test-ob-shell.el | 2 +- 3 files changed, 17 insertions(+), 5 deletions(-) diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el index 8c58dfd..e6b6429 100644 --- a/lisp/ob-shell.el +++ b/lisp/ob-shell.el @@ -121,12 +121,18 @@ This function is called by `org-babel-execute-src-block'." values "\n"))) +(defun bash-supports-assoc-arrays () + "Returns if the current bash running on the system supports associative arrays." + (let ((bash_version (org-babel-execute:shell "echo ${BASH_VERSINFO[0]}" nil))) + (> 3 bash_version))) + (defun org-babel-variable-assignments:bash (varname values &optional sep hline) "Represents the parameters as useful Bash shell variables." - (if (listp values) - (if (and (listp (car values)) (= 1 (length (car values)))) + (if (listp values) + (if (or (not (bash-supports-assoc-arrays)) + (and (listp (car values)) (= 1 (length (car values))))) (org-babel-variable-assignments:bash_array varname values sep hline) - (org-babel-variable-assignments:bash_assoc varname values sep hline)) + (org-babel-variable-assignments:bash_assoc varname values sep hline)) (org-babel-variable-assignments:sh-generic varname values sep hline))) (defun org-babel-variable-assignments:sh (params) diff --git a/testing/examples/ob-shell-test.org b/testing/examples/ob-shell-test.org index a54e5c0..cc3031d 100644 --- a/testing/examples/ob-shell-test.org +++ b/testing/examples/ob-shell-test.org @@ -80,7 +80,13 @@ echo ${table} Bash will see an associative array that contains each row as a single string. Bash cannot handle lists in associative arrays. #+begin_src bash :exports results :var table=sample_big_table -echo ${table[spaghetti]} +if (( ${BASH_VERSINFO[0]} > 3 )); then + # understands associative arrays + echo spaghetti ${table[spaghetti]} +else + # Default V3.x behavior, has one array item contains the key and values. + echo ${table[1]} +fi #+end_src #+RESULTS: diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el index 58a7859..9aa7f35 100644 --- a/testing/lisp/test-ob-shell.el +++ b/testing/lisp/test-ob-shell.el @@ -84,7 +84,7 @@ ob-comint.el, which was not previously tested." "Bash associative arrays as strings for the row" (org-test-at-id "82320a48-3409-49d7-85c9-5de1c6d3ff87" (org-babel-next-src-block 2) - (should (equal "20 cm" (org-babel-execute-src-block))))) + (should (equal "spaghetti 20 cm" (org-babel-execute-src-block))))) (provide 'test-ob-shell) -- 1.9.3 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH] ob-shell 2014-08-22 13:08 ` Pascal Fleury @ 2014-08-22 18:16 ` Achim Gratz 2015-01-21 22:58 ` Pascal Fleury 0 siblings, 1 reply; 24+ messages in thread From: Achim Gratz @ 2014-08-22 18:16 UTC (permalink / raw) To: emacs-orgmode Pascal Fleury writes: > Please forget about the previous patch I submitted, I have a new one > that should work now on all platforms. > Let me know if it should be formatted differently. You don't really want to fork into bash each time you're about to run a shell code block just to find out if it suppports associative arrays. This needs to be determined at customization time. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Q+, Q and microQ: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] ob-shell 2014-08-22 18:16 ` Achim Gratz @ 2015-01-21 22:58 ` Pascal Fleury 2015-01-25 11:41 ` Achim Gratz 0 siblings, 1 reply; 24+ messages in thread From: Pascal Fleury @ 2015-01-21 22:58 UTC (permalink / raw) To: Achim Gratz; +Cc: emacs orgmode-mailinglist [-- Attachment #1.1: Type: text/plain, Size: 892 bytes --] Hello, Here is a patch that will figure out the version of bash in a less fork-y way. It keeps the result in a variable after having gotten it the first time by indeed forking to bash. --Pascal On Fri, Aug 22, 2014 at 8:16 PM, Achim Gratz <Stromeko@nexgo.de> wrote: > Pascal Fleury writes: > > Please forget about the previous patch I submitted, I have a new one > > that should work now on all platforms. > > Let me know if it should be formatted differently. > > You don't really want to fork into bash each time you're about to run a > shell code block just to find out if it suppports associative arrays. > This needs to be determined at customization time. > > Regards, > Achim. > -- > +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ > > Factory and User Sound Singles for Waldorf Q+, Q and microQ: > http://Synth.Stromeko.net/Downloads.html#WaldorfSounds > > > [-- Attachment #1.2: Type: text/html, Size: 1451 bytes --] [-- Attachment #2: 0001-PATCH-ob-shell.el-now-emits-associative-array-code-o.patch --] [-- Type: application/octet-stream, Size: 3566 bytes --] From 993c8e1da79c229a60aa76ade64780484a405df0 Mon Sep 17 00:00:00 2001 From: Pascal Fleury <fleury@google.com> Date: Wed, 21 Jan 2015 23:55:34 +0100 Subject: [PATCH] [PATCH]ob-shell.el now emits associative array code only when the installed version of bash supports them. This takes care of differences in platforms that ship with bash v4+, that can deal with them, and those (e.g. MacOSX) that ship with bash v3.x that does not support such arrays. --- lisp/ob-shell.el | 15 +++++++++++++-- testing/examples/ob-shell-test.org | 10 ++++++++-- testing/lisp/test-ob-shell.el | 2 +- 3 files changed, 22 insertions(+), 5 deletions(-) diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el index aa14a69..48b38cc 100644 --- a/lisp/ob-shell.el +++ b/lisp/ob-shell.el @@ -38,6 +38,10 @@ (defvar org-babel-default-header-args:sh '()) +;; Contains the version of bash that is installed. If nil, will be +;; found by using bash itself once. +(defvar org-babel-shell-bash-version nil) + (defcustom org-babel-shell-names '("sh" "bash" "csh" "ash" "dash" "ksh" "mksh" "posh") "List of names of shell supported by babel shell code blocks." @@ -121,12 +125,19 @@ This function is called by `org-babel-execute-src-block'." values "\n"))) +(defun bash-supports-assoc-arrays () + "Returns if the current bash running on the system supports associative arrays." + (if (= org-babel-shell-bash-version nil) + (setq org-babel-shell-bash-version (org-babel-execute:shell "echo ${BASH_VERSINFO[0]}" nil))) + (> 3 org-babel-shell-bash-version))) + (defun org-babel-variable-assignments:bash (varname values &optional sep hline) "Represents the parameters as useful Bash shell variables." (if (listp values) - (if (and (listp (car values)) (= 1 (length (car values)))) + (if (or (not (bash-supports-assoc-arrays)) + (and (listp (car values)) (= 1 (length (car values))))) (org-babel-variable-assignments:bash_array varname values sep hline) - (org-babel-variable-assignments:bash_assoc varname values sep hline)) + (org-babel-variable-assignments:bash_assoc varname values sep hline)) (org-babel-variable-assignments:sh-generic varname values sep hline))) (defun org-babel-variable-assignments:sh (params) diff --git a/testing/examples/ob-shell-test.org b/testing/examples/ob-shell-test.org index a54e5c0..ebd7421 100644 --- a/testing/examples/ob-shell-test.org +++ b/testing/examples/ob-shell-test.org @@ -80,9 +80,15 @@ echo ${table} Bash will see an associative array that contains each row as a single string. Bash cannot handle lists in associative arrays. #+begin_src bash :exports results :var table=sample_big_table -echo ${table[spaghetti]} +if (( ${BASH_VERSINFO[0]} > 3 )); then + # understands associative arrays + echo spaghetti ${table[spaghetti]} +else + # Default V3.x behavior, has one array item contains the key and values. + echo ${table[1]} +fi #+end_src #+RESULTS: -: 20 cm +: spaghetti 20 cm diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el index 58a7859..9aa7f35 100644 --- a/testing/lisp/test-ob-shell.el +++ b/testing/lisp/test-ob-shell.el @@ -84,7 +84,7 @@ ob-comint.el, which was not previously tested." "Bash associative arrays as strings for the row" (org-test-at-id "82320a48-3409-49d7-85c9-5de1c6d3ff87" (org-babel-next-src-block 2) - (should (equal "20 cm" (org-babel-execute-src-block))))) + (should (equal "spaghetti 20 cm" (org-babel-execute-src-block))))) (provide 'test-ob-shell) -- 2.2.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH] ob-shell 2015-01-21 22:58 ` Pascal Fleury @ 2015-01-25 11:41 ` Achim Gratz 0 siblings, 0 replies; 24+ messages in thread From: Achim Gratz @ 2015-01-25 11:41 UTC (permalink / raw) To: emacs-orgmode Pascal Fleury writes: > Here is a patch that will figure out the version of bash in a less > fork-y way. It keeps the result in a variable after having gotten it > the first time by indeed forking to bash. I still think this should be a defcustom instead with a setter function that checks for the actual availablity of array variables. The user should have the opportunity to not use arrays at all even when the shell supports it. Using array variable syntax to get the major version number of the installled Bash might not work in those shells that don't have arrays. I can't test an old Bash, but dash doesn't recognize the syntax at all. There should be no fallback to checking just the version of bash since array variables may have been backported or disabled for the actual variant of Bash that is used. Also, the code as presented doesn't deal with the possibility that Tramp might be invoking a different shell on a remote machine, but that is a problem in more areas. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf rackAttack: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2015-01-25 11:42 UTC | newest]
Thread overview: 24+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-05-22 10:03 2 Org tests failing Sebastien Vauban
2014-05-22 10:48 ` Bastien
[not found] ` <87tx8iks22.fsf-E3UqQZAQFPqWIDz0JBNUog@public.gmane.org>
2014-06-18 14:55 ` Sebastien Vauban
2014-06-18 21:49 ` Achim Gratz
2014-06-19 8:36 ` Sebastien Vauban
2014-06-19 9:04 ` Sebastien Vauban
2014-06-19 9:08 ` Sebastien Vauban
2014-06-19 20:05 ` Achim Gratz
2014-06-22 8:35 ` [PATCH] ob-shell (was: 2 Org tests failing) Achim Gratz
2014-06-22 12:50 ` [PATCH] ob-shell Eric Schulte
2014-06-23 6:26 ` Pascal Fleury
2014-06-23 17:20 ` Achim Gratz
2014-06-23 20:02 ` Achim Gratz
2014-07-24 7:38 ` Alan Schmitt
2014-08-04 11:56 ` Eric Schulte
2014-08-04 19:48 ` Achim Gratz
2014-08-13 13:12 ` Alan Schmitt
2014-08-13 13:20 ` Pascal Fleury
2014-08-13 13:29 ` Alan Schmitt
2014-08-22 8:52 ` Pascal Fleury
2014-08-22 13:08 ` Pascal Fleury
2014-08-22 18:16 ` Achim Gratz
2015-01-21 22:58 ` Pascal Fleury
2015-01-25 11:41 ` Achim Gratz
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).