emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
From: Eric Schulte <schulte.eric@gmail.com>
To: Pascal Fleury <pascal@telefleuries.com>
Cc: Pascal Fleury <fleury@google.com>, emacs-orgmode@gnu.org
Subject: Re: Export arrays for 'sh' code blocks when using bash
Date: Thu, 10 Apr 2014 20:38:09 -0600	[thread overview]
Message-ID: <871tx4zj3i.fsf@gmail.com> (raw)
In-Reply-To: <CA+p9wPBiZWEmFC+F-DFTUkDW1dMM-8q5o1NODdTwu2T3RsGo+Q@mail.gmail.com> (Pascal Fleury's message of "Mon, 7 Apr 2014 01:25:19 +0200")

Hi Pascal,

This patch looks great, thanks for making the additions.  Once last
request, would you mind formatting it with "git format-patch"?  With
that done I'll be happy to apply this to the repo.

Also, I think the google-wide copyright stuff is sorted out.

Best,

Pascal Fleury <pascal@telefleuries.com> writes:

> Hi Eric,
>
> Please find attached a new patch that contains the same changes of the
> ob-shell.el exporter as before, but also contains unit tests for checking
> that it works.
>
> Let me know if you need some changes in there, and how the google-wide
> copyright issues comes along. My information came from Chris DiBona (our
> master of open-source licenses), is this helps to find the relevant
> documents.
>
> Thanks,
> Pascal
>
>
>
> On Sat, Mar 29, 2014 at 8:37 PM, Eric Schulte <schulte.eric@gmail.com>wrote:
>
>> Thanks for making these changes.  Once they're in and Bastien figures
>> out how the google-wide copyright attribution works we should be good to
>> go.
>>
>> Thanks!
>>
>> Pascal Fleury <fleury@google.com> writes:
>>
>> > Hi Eric, see comments inline.
>> >
>> >
>> > On Thu, Mar 27, 2014 at 6:43 PM, Eric Schulte <schulte.eric@gmail.com
>> >wrote:
>> >
>> >> Hi Pascal,
>> >>
>> >> This looks like a good patch.  If I could make three changes/requests.
>> >>
>> >>
>> > Thanks, I am happy to see it makes sense.
>> >
>> >
>> >> 1. This patch should also work for "begin_src bash" code blocks.  After
>> >>    a very quick glance it doesn't appear to.
>> >>
>> >> I have actually tried this with the latest org-mode release (8.2.5h) and
>> > it did plainly not accept 'begin_src bash'. Maybe that's new with the
>> > ob-shell.el (I noticed it has been renamed :-)
>> >
>> >
>> >> 2. Could you include one or two tests ensuring that this patch does what
>> >>    it intends with bash code blocks, and does not affect code blocks
>> >>    executed with bin/sh?
>> >>
>> >
>> > Will do, seeing it has some traction.
>> >
>> >
>> >>
>> >> 3. Large contributions like this require some FSF paperwork.  Please see
>> >>    the following page for information on requirements to contribute.
>> >>
>> >>
>> > I checked internally, and apparently, Google has already such a signed
>> > document. It would run on the same document, as I am a google employee.
>> >
>> >
>> >>    http://orgmode.org/worg/org-contribute.html
>> >>
>> >> Thanks, and I look forward to seeing this merged into Org-mode!
>> >> Eric
>> >>
>> >> Pascal Fleury <fleury@google.com> writes:
>> >>
>> >> > Hello,
>> >> >
>> >> > I'dl like to propose a patch for inclusion into org-mode (ob-shell.el
>> in
>> >> > particular).
>> >> >
>> >> > *TL;DR:* use arrays and associative arrays when exporting variables to
>> >> bash
>> >> > in 'sh' code blocks.
>> >> >
>> >> > *Details:*
>> >> > When variables are defined in a 'sh' code block, they are exported as
>> >> > strings. when the variable itself is an array or a table, then we
>> simply
>> >> > get a shell variable that contains the list of all values in a
>> >> > non-structured form. E.g.
>> >> >
>> >> > #+NAME: my_list
>> >> > | one   |
>> >> > | two   |
>> >> > | three |
>> >> >
>> >> > #+NAME: experiment
>> >> > | name | first_attempt    |
>> >> > | date | [2014-03-27 Thu] |
>> >> > | user | fleury           |
>> >> >
>> >> > #+BEGIN_SRC sh :var scalar="some value" :var array=my_list :var
>> >> table=config
>> >> > echo ${scalar}  # -> prints 'some value'
>> >> > echo ${array}   # -> prints 'one two three'
>> >> > echo ${table}   # -> prints 'first attempt [2014-03-27 Thu] fleury'
>> >> > #+END_SRC
>> >> >
>> >> > This will print simple strings. Also, there is no easy way to access
>> the
>> >> > date of the experiment, for example. Now bash has things like arrays
>> and
>> >> > associative arrays, but the current ob-shell.el does not use these.
>> >> > Probably because their syntax is bash-specific, and ob-shell.el is
>> >> > shell-agnostic.
>> >> >
>> >> > My patch (attached) changes this in the case you have
>> >> > (setq org-babel-sh-command "bash")
>> >> > in your emacs config somewhere. If any other value is used, it
>> continues
>> >> to
>> >> > export them as we do today (I don't know enough about other shells).
>> >> >
>> >> > In that case, it will export the list as an array, the table as an
>> >> > associative array, and the scalar as it does already. So the 'sh' code
>> >> > block could then use
>> >> >
>> >> > #+BEGIN_SRC sh :var scalar="some value" :var array=my_list :var
>> >> table=config
>> >> > echo ${scalar}
>> >> > echo ${array[1]} # -> prints "two"
>> >> > echo ${table[user]} # -> prints "fleury"
>> >> > #+END_SRC
>> >> >
>> >> > In the case we have a bigger table, then the first row is used as key,
>> >> the
>> >> > others are represented as a string with all the values (as it is for
>> >> array
>> >> > currently). bash does not have multi-dimensional arrays, so it needs
>> to
>> >> be
>> >> > a string.
>> >> >
>> >> > This makes writing shell snippets much easier in my experience, and
>> there
>> >> > I'd like to propose this fix for the org-mode community at large.
>> >> >
>> >> > --paf
>> >> >
>> >> > diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el
>> >> > index 3ede701..0a691e2 100644
>> >> > --- a/lisp/ob-shell.el
>> >> > +++ b/lisp/ob-shell.el
>> >> > @@ -106,6 +106,30 @@ This function is called by
>> >> `org-babel-execute-src-block'."
>> >> >
>> >> >  ;; helper functions
>> >> >
>> >> > +(defun org-babel-variable-assignments:bash_array (varname values
>> >> &optional sep hline)
>> >> > +  "Returns a list of statements declaring the values as a bash
>> array."
>> >> > +  (format "declare -a %s=( \"%s\" )"
>> >> > +     varname
>> >> > +     (mapconcat 'identity
>> >> > +       (mapcar
>> >> > +         (lambda (value) (org-babel-sh-var-to-sh value sep hline))
>> >> > +         values)
>> >> > +       "\" \"")))
>> >> > +
>> >> > +(defun org-babel-variable-assignments:bash_associative (varname
>> values
>> >> &optional sep hline)
>> >> > +  "Returns a list of statements declaring the values as bash
>> >> associative array."
>> >> > +  (format "declare -A %s\n%s"
>> >> > +    varname
>> >> > +    (mapconcat 'identity
>> >> > +      (mapcar
>> >> > +        (lambda (items)
>> >> > +          (format "%s[\"%s\"]=%s"
>> >> > +            varname
>> >> > +            (org-babel-sh-var-to-sh (car items) sep hline)
>> >> > +            (org-babel-sh-var-to-sh (cdr items) sep hline)))
>> >> > +        values)
>> >> > +      "\n")))
>> >> > +
>> >> >  (defun org-babel-variable-assignments:sh (params)
>> >> >    "Return list of shell statements assigning the block's variables."
>> >> >    (let ((sep (cdr (assoc :separator params)))
>> >> > @@ -114,9 +138,17 @@ This function is called by
>> >> `org-babel-execute-src-block'."
>> >> >                    "hline"))))
>> >> >      (mapcar
>> >> >       (lambda (pair)
>> >> > -       (format "%s=%s"
>> >> > -            (car pair)
>> >> > -            (org-babel-sh-var-to-sh (cdr pair) sep hline)))
>> >> > +       (if (and (string= org-babel-sh-command "bash") (listp (cdr
>> >> pair)))
>> >> > +         (if (listp (car (cdr pair)))
>> >> > +           (org-babel-variable-assignments:bash_associative
>> >> > +           (car pair) (cdr pair) sep hline)
>> >> > +           (org-babel-variable-assignments:bash_array
>> >> > +           (car pair) (cdr pair) sep) hline)
>> >> > +         (format "%s=%s"
>> >> > +         (car pair)
>> >> > +         (org-babel-sh-var-to-sh (cdr pair) sep hline))
>> >> > +       )
>> >> > +     )
>> >> >       (mapcar #'cdr (org-babel-get-header params :var)))))
>> >> >
>> >> >  (defun org-babel-sh-var-to-sh (var &optional sep hline)
>> >>
>> >> --
>> >> Eric Schulte
>> >> https://cs.unm.edu/~eschulte
>> >> PGP: 0x614CA05D
>> >>
>>
>> --
>> Eric Schulte
>> https://cs.unm.edu/~eschulte
>> PGP: 0x614CA05D
>>
>>
>

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

  reply	other threads:[~2014-04-11  2:38 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-03-27 10:26 Export arrays for 'sh' code blocks when using bash Pascal Fleury
2014-03-27 17:43 ` Eric Schulte
2014-03-27 22:52   ` Pascal Fleury
2014-03-29 19:37     ` Eric Schulte
2014-04-06 23:25       ` Pascal Fleury
2014-04-11  2:38         ` Eric Schulte [this message]
2014-04-11  9:33           ` Bastien
2014-04-14  8:48             ` Pascal Fleury
2014-04-15  3:35               ` Eric Schulte
2014-04-15 11:15                 ` Pascal Fleury
2014-04-22 14:47                 ` Skip Collins
2014-04-22 15:37                   ` Bastien
2014-04-22 17:19                     ` Skip Collins
2014-04-22 20:59                       ` Bastien
2014-04-22 21:04                         ` Skip Collins
2014-04-22 21:09                           ` Skip Collins
2014-04-22 21:22                             ` Bastien
2014-04-23 13:51                               ` Skip Collins
2014-04-23 14:13                                 ` Pascal Fleury
2014-04-23 16:31                                   ` Skip Collins
2014-04-24  1:23                                     ` Eric Schulte
2014-04-24  7:44                                       ` Pascal Fleury
2014-04-22 21:15                           ` Bastien
2014-04-15  3:37             ` Eric Schulte
2014-04-15 15:37               ` Nick Dokos
2014-04-17  6:31               ` Bastien

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.orgmode.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=871tx4zj3i.fsf@gmail.com \
    --to=schulte.eric@gmail.com \
    --cc=emacs-orgmode@gnu.org \
    --cc=fleury@google.com \
    --cc=pascal@telefleuries.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).