[babel] Sh problem when echo'ing input data with ' inside

[babel] Sh problem when echo'ing input data with ' inside

[Sébastien Vauban]

Hi,

I tried to work on a CSV2Ledger, accepting the particular (European) CSV
format of my bank:

* Data

#+results: excel-from-bank
#+begin_example
Numéro du compte :;979-9500975-24;Compte Maxi
Date valeur;Référence de l'opération;Description;Montant de l'opération;Devise;Date d'opération;Compte de contrepartie;Nom de la contrepartie :;Communication 1 :;Communication 2 :
04-06-2009;A9F04NT01WK300TG;Virem. internet;420,00;EUR;04-06-2009;799-5900947-23;PAYEE ONE; ; 
24-02-2009;A9B24NT012K4018Z;Virem. internet;-54,93;EUR;24-02-2009;799-9974005-30;ME;Eigen rekening; 
18-05-2008;A8E19NT000S604QI;Virem. internet;-1.000,00;EUR;19-05-2008;799-8068445-18;PAYEE TWO; ; 
#+end_example

* "Europeanize" the CSV file

Here, we have to make several manipulations on the input file, such as:
- remove dots from amounts
- replace commas from amounts by dots
- replace semi-commas by commas
- etc.

My goal is to make that as a suite of bash commands (=echo | cmd 1 | cmd 2=),
tangled in a script file.

However, I can't do any work on the input file, the very basic echo command
already giving an error:

#+begin_src sh :var data=excel-from-bank :results output
echo "$data"
#+end_src

#+results:
Code block produced no output.

#+begin_src stderr
sh: line 3: unexpected EOF while looking for matching `''
sh: line 10: syntax error: unexpected end of file
#+end_src

In this sample, this is due to the apostrophe in the header line ("Montant de
l'opération").

Any idea on how to:
- circumvent such problem?
- or, eventually, do this better?

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: [babel] Sh problem when echo'ing input data with ' inside

[Eric Schulte]

Hi Seb,

I can't reproduce this problem, on my system, I get the following
correct behavior

#+results: excel-from-bank
#+begin_example
Num.ro du compte :;979-9500975-24;Compte Maxi
Date valeur;R.f.rence de l'op.ration;Description;Montant de l'op.ration;Devise;Date d'op.ration;Compte de contrepartie;Nom de la contrepartie :;Communication 1 :;Communication 2 :
04-06-2009;A9F04NT01WK300TG;Virem. internet;420,00;EUR;04-06-2009;799-5900947-23;PAYEE ONE; ; 
24-02-2009;A9B24NT012K4018Z;Virem. internet;-54,93;EUR;24-02-2009;799-9974005-30;ME;Eigen rekening; 
18-05-2008;A8E19NT000S604QI;Virem. internet;-1.000,00;EUR;19-05-2008;799-8068445-18;PAYEE TWO; ; 
#+end_example

#+begin_src sh :var data=excel-from-bank :results output
echo "$data"
#+end_src

#+results:
: Num.ro du compte :;979-9500975-24;Compte Maxi
: Date valeur;R.f.rence de l'op.ration;Description;Montant de l'op.ration;Devise;Date d'op.ration;Compte de contrepartie;Nom de la contrepartie :;Communication 1 :;Communication 2 :
: 04-06-2009;A9F04NT01WK300TG;Virem. internet;420,00;EUR;04-06-2009;799-5900947-23;PAYEE ONE; ; 
: 24-02-2009;A9B24NT012K4018Z;Virem. internet;-54,93;EUR;24-02-2009;799-9974005-30;ME;Eigen rekening; 
: 18-05-2008;A8E19NT000S604QI;Virem. internet;-1.000,00;EUR;19-05-2008;799-8068445-18;PAYEE TWO; ;

and the sh code block expands to the following (with C-c C-v v) which is
not affected by commas...

#+begin_src sh
  data=$(cat <<BABEL_STRING
  Num.ro du compte :;979-9500975-24;Compte Maxi
  Date valeur;R.f.rence de l'op.ration;Description;Montant de l'op.ration;Devise;Date d'op.ration;Compte de contrepartie;Nom de la contrepartie :;Communication 1 :;Communication 2 :
  04-06-2009;A9F04NT01WK300TG;Virem. internet;420,00;EUR;04-06-2009;799-5900947-23;PAYEE ONE; ; 
  24-02-2009;A9B24NT012K4018Z;Virem. internet;-54,93;EUR;24-02-2009;799-9974005-30;ME;Eigen rekening; 
  18-05-2008;A8E19NT000S604QI;Virem. internet;-1.000,00;EUR;19-05-2008;799-8068445-18;PAYEE TWO; ;
  BABEL_STRING
  )
  echo "$data"
#+end_src

Are you on a windows machine?  I'm not sure what else could be different
about the shell environment on our machines.

Cheers -- Eric

note: some of the characters in your email weren't printable when I
tried to send this with gnus, so they've been replaced with "."s

Sébastien Vauban <wxhgmqzgwmuf@spammotel.com> writes:

> Hi,
>
> I tried to work on a CSV2Ledger, accepting the particular (European) CSV
> format of my bank:
>
> * Data
>
> #+results: excel-from-bank
> #+begin_example
> Num.ro du compte :;979-9500975-24;Compte Maxi
> Date valeur;R.f.rence de l'op.ration;Description;Montant de l'op.ration;Devise;Date d'op.ration;Compte de contrepartie;Nom de la contrepartie :;Communication 1 :;Communication 2 :
> 04-06-2009;A9F04NT01WK300TG;Virem. internet;420,00;EUR;04-06-2009;799-5900947-23;PAYEE ONE; ; 
> 24-02-2009;A9B24NT012K4018Z;Virem. internet;-54,93;EUR;24-02-2009;799-9974005-30;ME;Eigen rekening; 
> 18-05-2008;A8E19NT000S604QI;Virem. internet;-1.000,00;EUR;19-05-2008;799-8068445-18;PAYEE TWO; ; 
> #+end_example
>
> * "Europeanize" the CSV file
>
> Here, we have to make several manipulations on the input file, such as:
> - remove dots from amounts
> - replace commas from amounts by dots
> - replace semi-commas by commas
> - etc.
>
> My goal is to make that as a suite of bash commands (=echo | cmd 1 | cmd 2=),
> tangled in a script file.
>
> However, I can't do any work on the input file, the very basic echo command
> already giving an error:
>
> #+begin_src sh :var data=excel-from-bank :results output
> echo "$data"
> #+end_src
>
> #+results:
> Code block produced no output.
>
> #+begin_src stderr
> sh: line 3: unexpected EOF while looking for matching `''
> sh: line 10: syntax error: unexpected end of file
> #+end_src
>
> In this sample, this is due to the apostrophe in the header line ("Montant de
> l'opération").
>
> Any idea on how to:
> - circumvent such problem?
> - or, eventually, do this better?
>
> Best regards,
>   Seb

Re: [babel] Sh problem when echo'ing input data with ' inside

[Sébastien Vauban]

Hi Eric,

"Eric Schulte" wrote:
>> I tried to work on a CSV2Ledger, accepting the particular (European) CSV
>> format of my bank:
>>
>> * "Europeanize" the CSV file
>>
>> Here, we have to make several manipulations on the input file [...]. My
>> goal is to make that as a suite of bash commands (=echo | cmd 1 | cmd 2=),
>> tangled in a script file.
>>
>> However, I can't do any work on the input file, the very basic echo command
>> already giving an error:
>>
>> #+begin_src stderr
>> sh: line 3: unexpected EOF while looking for matching `''
>> sh: line 10: syntax error: unexpected end of file
>> #+end_src
>>
>> In this sample, this is due to the apostrophe in the header line ("Montant de
>> l'opération").
>
> I can't reproduce this problem, on my system, I get the following
> correct behavior
>
> #+results: excel-from-bank
> #+begin_example
> Num.ro du compte :;979-9500975-24;Compte Maxi
> Date valeur;R.f.rence de l'op.ration;Description;Montant de l'op.ration;Devise;Date d'op.ration;Compte de contrepartie;Nom de la contrepartie :;Communication 1 :;Communication 2 :
> 04-06-2009;A9F04NT01WK300TG;Virem. internet;420,00;EUR;04-06-2009;799-5900947-23;PAYEE ONE; ; 
> 24-02-2009;A9B24NT012K4018Z;Virem. internet;-54,93;EUR;24-02-2009;799-9974005-30;ME;Eigen rekening; 
> 18-05-2008;A8E19NT000S604QI;Virem. internet;-1.000,00;EUR;19-05-2008;799-8068445-18;PAYEE TWO; ; 
> #+end_example
>
> #+begin_src sh :var data=excel-from-bank :results output
> echo "$data"
> #+end_src
>
> #+results:
> : Num.ro du compte :;979-9500975-24;Compte Maxi
> : Date valeur;R.f.rence de l'op.ration;Description;Montant de l'op.ration;Devise;Date d'op.ration;Compte de contrepartie;Nom de la contrepartie :;Communication 1 :;Communication 2 :
> : 04-06-2009;A9F04NT01WK300TG;Virem. internet;420,00;EUR;04-06-2009;799-5900947-23;PAYEE ONE; ; 
> : 24-02-2009;A9B24NT012K4018Z;Virem. internet;-54,93;EUR;24-02-2009;799-9974005-30;ME;Eigen rekening; 
> : 18-05-2008;A8E19NT000S604QI;Virem. internet;-1.000,00;EUR;19-05-2008;799-8068445-18;PAYEE TWO; ;
>
> and the sh code block expands to the following (with C-c C-v v) which is
> not affected by commas...

I've no trouble with commas, except that it's not an American formatting for
amounts. Though, I does well have problems, as showed, with the apostrophe...

Previewing does work for me as well:

> #+begin_src sh
>   data=$(cat <<BABEL_STRING
>   Num.ro du compte :;979-9500975-24;Compte Maxi
>   Date valeur;R.f.rence de l'op.ration;Description;Montant de l'op.ration;Devise;Date d'op.ration;Compte de contrepartie;Nom de la contrepartie :;Communication 1 :;Communication 2 :
>   04-06-2009;A9F04NT01WK300TG;Virem. internet;420,00;EUR;04-06-2009;799-5900947-23;PAYEE ONE; ; 
>   24-02-2009;A9B24NT012K4018Z;Virem. internet;-54,93;EUR;24-02-2009;799-9974005-30;ME;Eigen rekening; 
>   18-05-2008;A8E19NT000S604QI;Virem. internet;-1.000,00;EUR;19-05-2008;799-8068445-18;PAYEE TWO; ;
>   BABEL_STRING
>   )
>   echo "$data"
> #+end_src

Evaluation does not.

> Are you on a windows machine?  I'm not sure what else could be different
> about the shell environment on our machines.

Yes, it is on Windows ;-)  Though, with Cygwin bash support... which should
behave as on any Linux...

> note: some of the characters in your email weren't printable when I
> tried to send this with gnus, so they've been replaced with "."s

These are French accentuated letters.

>> Any idea on how to:
>> - circumvent such problem?
>> - or, eventually, do this better?

No idea on any workaround, then?  ;-(

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: [babel] Sh problem when echo'ing input data with ' inside

[Eric Schulte]

>>
>> and the sh code block expands to the following (with C-c C-v v) which is
>> not affected by commas...
>
> I've no trouble with commas, except that it's not an American formatting for
> amounts. Though, I does well have problems, as showed, with the apostrophe...
>

Sorry, I mis-typed, as far as I can tell the apostrophe works on my
machine without causing any issues.

>
> Previewing does work for me as well:
>
>> #+begin_src sh
>>   data=$(cat <<BABEL_STRING
>>   Num.ro du compte :;979-9500975-24;Compte Maxi
>>   Date valeur;R.f.rence de l'op.ration;Description;Montant de
>> l'op.ration;Devise;Date d'op.ration;Compte de contrepartie;Nom de la
>> contrepartie :;Communication 1 :;Communication 2 :
>>   04-06-2009;A9F04NT01WK300TG;Virem. internet;420,00;EUR;04-06-2009;799-5900947-23;PAYEE ONE; ; 
>>   24-02-2009;A9B24NT012K4018Z;Virem. internet;-54,93;EUR;24-02-2009;799-9974005-30;ME;Eigen rekening; 
>>   18-05-2008;A8E19NT000S604QI;Virem. internet;-1.000,00;EUR;19-05-2008;799-8068445-18;PAYEE TWO; ;
>>   BABEL_STRING
>>   )
>>   echo "$data"
>> #+end_src
>

if you execute the previewed code block as an independent shell script
on your machine does it throw errors or does it echo correctly?

>
> Evaluation does not.
>
>> Are you on a windows machine?  I'm not sure what else could be different
>> about the shell environment on our machines.
>
> Yes, it is on Windows ;-)  Though, with Cygwin bash support... which should
> behave as on any Linux...
>
>> note: some of the characters in your email weren't printable when I
>> tried to send this with gnus, so they've been replaced with "."s
>
> These are French accentuated letters.
>
>>> Any idea on how to:
>>> - circumvent such problem?
>>> - or, eventually, do this better?
>
> No idea on any workaround, then?  ;-(
>

I can't reproduce the problem locally, and I believe that this is due to
differences in our shell environments rather than Emacs-specific issues,
so it is hard for me to diagnose or propose a solution.

Sorry I can't be of more help -- Eric

>
> Best regards,
>   Seb

Re: [babel] Sh problem when echo'ing input data with ' inside

[Sébastien Vauban]

Eric,

"Eric Schulte" wrote:
> if you execute the previewed code block as an independent shell script
> on your machine does it throw errors or does it echo correctly?

On this one: the behavior seen from Org Babel is the same as the one seen when
saving the previewed script as an executable file and launching it from Bash.
No diff.

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: [babel] Sh problem when echo'ing input data with ' inside

[Achim Gratz]

Sébastien Vauban <wxhgmqzgwmuf@spammotel.com>
writes:
> However, I can't do any work on the input file, the very basic echo command
> already giving an error:

This is because quoting the arguments to echo does not work when your
input contains unescaped quote characters.  I've no idea how exactly
babel tries to get it's arguments to the command, but it looks like it
simply copies text and executes the resulting script?  If it could open
a pipe and send the data through, no quoting would be necessary since
the shell never sees the data.  I'm not sure if that's possible because
I can't find specific documentation on language "sh".

[Note: orgmode online manual links to
 http://orgmode.org/worg/org-contrib/babel/languages/ which is broken.]

> #+begin_src sh :var data=excel-from-bank :results output
> echo "$data"
> #+end_src

So, you're using a un*xoid?  This is your lucky day, since in your case you
should be able to use a here script:

#+begin_src sh :var data=excel-from-bank :results output
echo <<EOF | cmd1 | cmd2
$data
EOF
#+end_src


HTH,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: Re: [babel] Sh problem when echo'ing input data with ' inside

[Eric Schulte]

Achim Gratz <Stromeko@nexgo.de> writes:

> Sébastien Vauban <wxhgmqzgwmuf@spammotel.com>
> writes:
>> However, I can't do any work on the input file, the very basic echo command
>> already giving an error:
>
> This is because quoting the arguments to echo does not work when your
> input contains unescaped quote characters.  I've no idea how exactly
> babel tries to get it's arguments to the command, but it looks like it
> simply copies text and executes the resulting script?  If it could open
> a pipe and send the data through, no quoting would be necessary since
> the shell never sees the data.  I'm not sure if that's possible because
> I can't find specific documentation on language "sh".
>
> [Note: orgmode online manual links to
>  http://orgmode.org/worg/org-contrib/babel/languages/ which is broken.]
>

yes, the Worg documentation is in dire need of maintenance, see below
for discussion of how arguments are passed in the sh blocks

>
>> #+begin_src sh :var data=excel-from-bank :results output
>> echo "$data"
>> #+end_src
>
> So, you're using a un*xoid?  This is your lucky day, since in your case you
> should be able to use a here script:
>
> #+begin_src sh :var data=excel-from-bank :results output
> echo <<EOF | cmd1 | cmd2
> $data
> EOF
> #+end_src
>

here scripts are how Babel sends data through to a shell script, as far
as I know they should have no issues with quotes (or any other
character)

specifically, the following code block

    #+results: something
    #+begin_example
      something
      and something else
      with apostrophes ' and quotes "
      this should all work...
    #+end_example

    #+begin_src sh :var data=something
      echo $data
    #+end_src

Expands to

    #+begin_src sh
      data=$(cat <<BABEL_STRING
      something
        and something else
        with apostrophes ' and quotes "
        this should all work...
      BABEL_STRING
      )
      echo $data
    #+end_src

Cheers -- Eric

Re: [babel] Sh problem when echo'ing input data with ' inside

[Achim Gratz]

Achim Gratz <Stromeko@nexgo.de> writes:
> #+begin_src sh :var data=excel-from-bank :results output
> echo <<EOF | cmd1 | cmd2
> $data
> EOF
> #+end_src

Sent too soon: use cat, not echo.  Actually that is redundant, too:

#+begin_src sh :var data=excel-from-bank :results output
cmd1 <<EOF | cmd2
$data
EOF
#+end_src

Or even simpler (but that might be a bashism...):

#+begin_src sh :var data=excel-from-bank :results output
cmd1 <<<"$data" | cmd2
#+end_src

And I missed the preview in Erics post, which gave me the clue I was
looking for; the script produced by babel should actually look like
this:

#+begin_src sh
data=$(cat <<'BABEL_STRING'
Num.ro du compte :;979-9500975-24;Compte Maxi
Date valeur;R.f.rence de l'op.ration;Description;Montant de l'op.ration;Devise;Date d'op.ration;Compte de contrepartie;Nom de la contrepartie :;Communication 1 :;Communication 2 :
04-06-2009;A9F04NT01WK300TG;Virem. internet;420,00;EUR;04-06-2009;799-5900947-23;PAYEE ONE; ; 
24-02-2009;A9B24NT012K4018Z;Virem. internet;-54,93;EUR;24-02-2009;799-9974005-30;ME;Eigen rekening; 
18-05-2008;A8E19NT000S604QI;Virem. internet;-1.000,00;EUR;19-05-2008;799-8068445-18;PAYEE TWO; ;
BABEL_STRING
)
echo "$data"
#+end_src

That is no indentation and the here-tag should be in quotes (otherwise the
here-script is subject to shell expansion).  It's probably possible to
do a contrived redirection and not use a variable to store the data, but
I'm afraid I won't grok this today.


Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: [babel] Sh problem when echo'ing input data with ' inside

[Sébastien Vauban]

Hi Achim and Eric,

Achim Gratz wrote:
> The script produced by babel should actually look like this:
>
> #+begin_src sh
> data=$(cat <<'BABEL_STRING'
> Num.ro du compte :;979-9500975-24;Compte Maxi
> Date valeur;R.f.rence de l'op.ration;Description;Montant de l'op.ration;Devise;Date d'op.ration;Compte de contrepartie;Nom de la contrepartie :;Communication 1 :;Communication 2 :
> 04-06-2009;A9F04NT01WK300TG;Virem. internet;420,00;EUR;04-06-2009;799-5900947-23;PAYEE ONE; ; 
> 24-02-2009;A9B24NT012K4018Z;Virem. internet;-54,93;EUR;24-02-2009;799-9974005-30;ME;Eigen rekening; 
> 18-05-2008;A8E19NT000S604QI;Virem. internet;-1.000,00;EUR;19-05-2008;799-8068445-18;PAYEE TWO; ;
> BABEL_STRING
> )
> echo "$data"
> #+end_src
>
> That is no indentation and the here-tag should be in quotes (otherwise the
> here-script is subject to shell expansion).

The above, from which I even removed the accentuated characters (from French),
still does not work...

Now, I can imagine it is due to Cygwin, not to Emacs/Org/Babel. Though, that's
a problem in general for the use of sh blocks from Org... under Windows, then.

- Has anyone any experience with this?
- Can some Bash user confirm what I see?
- Can some Windows user confirm this with other Unix-emulation shells (MinGW
  and the like)?

> It's probably possible to do a contrived redirection and not use a variable
> to store the data, but I'm afraid I won't grok this today.

Or maybe using a real file as temporary storage?  Overkill, or simply needed?

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: [babel] Sh problem when echo'ing input data with ' inside

[Achim Gratz]

Sébastien Vauban <wxhgmqzgwmuf@spammotel.com>
writes:
> The above, from which I even removed the accentuated characters (from French),
> still does not work...

That's strange.

> Now, I can imagine it is due to Cygwin, not to Emacs/Org/Babel. Though, that's
> a problem in general for the use of sh blocks from Org... under Windows, then.

I can try tomorrow, I won't start the Windows box today...

> Or maybe using a real file as temporary storage?  Overkill, or simply needed?

Temporary files should work from emacs, but on Windows it is somewhat
troublesome to reliably get rid of them when they're no longer needed.


Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [babel] Sh problem when echo'ing input data with ' inside

[Sébastien Vauban]

Hi Achim,

Achim Gratz wrote:
> Sébastien Vauban <wxhgmqzgwmuf-geNee64TY+gS+FvcfC7Uqw@public.gmane.org> writes:
>> The above, from which I even removed the accentuated characters (from French),
>> still does not work...
>
> That's strange.
>
>> Now, I can imagine it is due to Cygwin, not to Emacs/Org/Babel. Though, that's
>> a problem in general for the use of sh blocks from Org... under Windows, then.
>
> I can try tomorrow, I won't start the Windows box today...

Thanks for trying.

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: [babel] Sh problem when echo'ing input data with ' inside

[Sébastien Vauban]

Hi Eric and Achim,

Sébastien Vauban wrote:
> Achim Gratz wrote:
>> The script produced by babel should actually look like this:
>>
>> #+begin_src sh
>> data=$(cat <<'BABEL_STRING'
>> Num.ro du compte :;979-9500975-24;Compte Maxi
>> Date valeur;R.f.rence de l'op.ration;Description;Montant de l'op.ration;Devise;Date d'op.ration;Compte de contrepartie;Nom de la contrepartie :;Communication 1 :;Communication 2 :
>> 04-06-2009;A9F04NT01WK300TG;Virem. internet;420,00;EUR;04-06-2009;799-5900947-23;PAYEE ONE; ; 
>> 24-02-2009;A9B24NT012K4018Z;Virem. internet;-54,93;EUR;24-02-2009;799-9974005-30;ME;Eigen rekening; 
>> 18-05-2008;A8E19NT000S604QI;Virem. internet;-1.000,00;EUR;19-05-2008;799-8068445-18;PAYEE TWO; ;
>> BABEL_STRING
>> )
>> echo "$data"
>> #+end_src
>>
>> That is no indentation and the here-tag should be in quotes (otherwise the
>> here-script is subject to shell expansion).
>
> The above, from which I even removed the accentuated characters (from French),
> still does not work...
>
> Now, I can imagine it is due to Cygwin, not to Emacs/Org/Babel. Though, that's
> a problem in general for the use of sh blocks from Org... under Windows, then.
>
> - Has anyone any experience with this?
> - Can some Bash user confirm what I see?
> - Can some Windows user confirm this with other Unix-emulation shells (MinGW
>   and the like)?

I asked the question on the Cygwin mailing list, and got an answer from Tobias
Schlottke, telling me to use the 'plain old' syntax (=`...`= instead of
=$(...)=):

#+begin_src sh
data=`cat <<EOF
...
EOF
`
echo "$data"
#+end_src

and... it indeed works!

Can we move to that syntax?

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: [babel] Sh problem when echo'ing input data with ' inside

[Eric Schulte]

>
> I asked the question on the Cygwin mailing list, and got an answer from Tobias
> Schlottke, telling me to use the 'plain old' syntax (=`...`= instead of
> =$(...)=):
>
> #+begin_src sh
> data=`cat <<EOF
> ...
> EOF
> `
> echo "$data"
> #+end_src
>
> and... it indeed works!
>
> Can we move to that syntax?
>

I just made this customizable, after a git pull the following elisp code
will use that syntax on your system.

#+begin_src emacs-lisp
  (setq org-babel-sh-var-quote-fmt "`cat <<'BABEL_TABLE'\n%s\nBABEL_TABLE\n`")
#+end_src

Best -- Eric

Re: [babel] Sh problem when echo'ing input data with ' inside

[Sébastien Vauban]

Hi Eric,

"Eric Schulte" wrote:
>> I asked the question on the Cygwin mailing list, and got an answer from Tobias
>> Schlottke, telling me to use the 'plain old' syntax (=`...`= instead of
>> =$(...)=):
>>
>> #+begin_src sh
>> data=`cat <<EOF
>> ...
>> EOF
>> `
>> echo "$data"
>> #+end_src
>>
>> and... it indeed works!
>>
>> Can we move to that syntax?
>
> I just made this customizable, after a git pull the following elisp code
> will use that syntax on your system.
>
> #+begin_src emacs-lisp
>   (setq org-babel-sh-var-quote-fmt "`cat <<'BABEL_TABLE'\n%s\nBABEL_TABLE\n`")
> #+end_src

That's, indeed, better than what I had in mind, as it now is customizable...
... and, of course, that does the trick!

Thanks a lot for your prompt fix.

BTW, I would, though, advice to make this the default, as it should be fully
functional under any Linux, and then would be directly usable by any user on
Windows/Cygwin, without having to wonder why sh code blocks don't work the way
they should.

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: [babel] Sh problem when echo'ing input data with ' inside

[Achim Gratz]

Sébastien Vauban <wxhgmqzgwmuf@spammotel.com>
writes:
> I asked the question on the Cygwin mailing list, and got an answer from Tobias
> Schlottke, telling me to use the 'plain old' syntax (=`...`= instead of
> =$(...)=):

I can confirm.  This is really bad, since now you'd have to escape
backticks in the data coming from emacs.  Do you have a link to the
discussion, provided there is a web-mirror of the mailing list
someplace?  I'm a bit out of my waters here since mostly I'm using tcsh,
but it appears that dash on cygwin (not ash, even though dash is just a
symlink to ash) does the right thing.  It is also vastly smaller than
bash, so you might just use it for babel scripting.

> Can we move to that syntax?

For the reason above this would not be safe without more changes to
babel and backticks are actively discouraged to use in new scripts by
just about any shell introduction/ FAQ for years now.  Eric has
implemented a workaround and the problem deserves to be documented,
though.


Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [babel] Sh problem when echo'ing input data with ' inside

[Achim Gratz]

Sébastien Vauban <wxhgmqzgwmuf@spammotel.com>
writes:
> I asked the question on the Cygwin mailing list, and got an answer from Tobias
> Schlottke, telling me to use the 'plain old' syntax (=`...`= instead of
> =$(...)=):

On further investigation, this is a bug in bash3, fixed in bash4
(hence why I didn't see it on Linux).  Bash4 is now available as
"experimental" in the latest version of Cygwin.  It's a bit tricky to
install since you will also need the experimental version of
libreadline7 (package version 6.1) and the setup kind of works against
you: I've not managed to install both of them in one go and you will
need to be careful with updates since I believe setup will always try to
reverse these to the non-experimental versions unless you allow them
globally (which I don't want).  In any case, the problem with the '$( )'
construct is indeed fixed and the next Cygwin release will hopefully
have this new bash as standard.


Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [babel] Sh problem when echo'ing input data with ' inside

[Sébastien Vauban]

Hi Achim,

Achim Gratz wrote:
> Sébastien Vauban <wxhgmqzgwmuf-geNee64TY+gS+FvcfC7Uqw@public.gmane.org> writes:
>> I asked the question on the Cygwin mailing list, and got an answer from Tobias
>> Schlottke, telling me to use the 'plain old' syntax (=`...`= instead of
>> =$(...)=):
>
> On further investigation, this is a bug in bash3, fixed in bash4 (hence why
> I didn't see it on Linux). Bash4 is now available as "experimental" in the
> latest version of Cygwin. It's a bit tricky to install since you will also
> need the experimental version of libreadline7 (package version 6.1) and the
> setup kind of works against you: I've not managed to install both of them in
> one go and you will need to be careful with updates since I believe setup
> will always try to reverse these to the non-experimental versions unless you
> allow them globally (which I don't want). In any case, the problem with the
> '$( )' construct is indeed fixed and the next Cygwin release will hopefully
> have this new bash as standard.

Thanks a lot for the follow-up, and for the above details about the problem!

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode