Re: [BABEL] "unset" :var definitions for subtree

[Rainer M Krug <r.m.krug@gmail.com>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 02/11/2011 12:55 PM, Dan Davison wrote:
> Rainer M Krug <r.m.krug@gmail.com> writes:
> 
>> On 02/11/2011 10:32 AM, Dan Davison wrote:
>>> Rainer M Krug <r.m.krug@gmail.com> writes:
>>>
>>>> On 02/10/2011 05:48 PM, Eric Schulte wrote:
>>>>> Rainer M Krug <r.m.krug@gmail.com> writes:
>>>>>
>>>>>> On 02/10/2011 02:27 AM, Eric Schulte wrote:
>>>>>>> Rainer M Krug <r.m.krug@gmail.com> writes:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> For one project, I am usinr org to write submit scripte to a cluster
>>>>>>>> runing torqu. The important bit in this is, that between the shebang and
>>>>>>>> the code, no other executable line must occur. As I am using variables
>>>>>>>> in org (:var) they will occur just after the shebang, which causes a
>>>>>>>> problem for torque. So, my question is, is there a way to "unset"
>>>>>>>> variables defined by using :var for a subtree?
>>>>>>>>
>>>>>>>
>>>>>>> Hi Rainer,
>>>>>>>
>>>>>>> Interesting question... unfortunately I don't think that removing
>>>>>>> variables from header arguments is possible under the current setup.
>>>>>>>
>>>>>>> Perhaps in your case you could add a function to the post-tangle hook,
>>>>>>> which recognizes when it is being called in a just-tangled torqu script
>>>>>>> (maybe by searching for a series of #PBS lines), and then removes any
>>>>>>> lines between the shebang and the first #PBS line?
>>>>>>
>>>>>> That is also an option - what I am using at the moment is to use
>>>>>> :no-expand as a code block specific header argument. But this raises the
>>>>>> other question:
>>>>>>
>>>>>> Can I set the :no-expand in a properties block? As far as I understand,
>>>>>> in the properties block I have the argument and the value - but what do
>>>>>> I do with :noexpand?
>>>>>>
>>>>>> :PROPERTIES:
>>>>>> :var: A=13
>>>>>> :no-expand
>>>>>> :END:
>>>>>>
>>>>>
>>>>> You can just set it to "yes" or really any value you like (the value
>>>>> will be ignored).  I did however have to add "no-expand" to the list of
>>>>> header argument names searched for in property blocks -- I just pushed
>>>>> up a patch to this effect.
>>>>
>>>> Thanks - I'll try it today and come back if it does not work.
>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> More generally, I wonder what a natural method would be to allow
>>>>>>> unsetting of pre-set header arguments for local blocks or subtrees?
>>>>>>> This may only apply to the :var header argument, as most others have a
>>>>>>> default setting which can be actively set.  If you have any ideas for a
>>>>>>> natural syntax for such an operation I'd be happy to hear it.
>>>>>>
>>>>>> First solution (probably the easiest to implement) would be along the
>>>>>> lines of the :no-expand header argument -
>>>>>>
>>>>>> :expand-var yes
>>>>>> and
>>>>>> :expand-var no
>>>>>>
>>>>>> This could possibly be expanded to
>>>>>> :expand-var A B C
>>>>>>
>>>>>> which would expand only the variables A B and C
>>>>>>
>>>>>> One step further: one could define groups of variables, like
>>>>>> :var-group X=A,B,C
>>>>>> or a similar syntax
>>>>>>
>>>>>> and then
>>>>>> :expand-var X
>>>>>> would expand A B and C
>>>>>>
>>>>>> This all would not be real unset - but a possibility for unsetting would be
>>>>>>
>>>>>> :var B=
>>>>>>
>>>>>> or
>>>>>>
>>>>>> :var-unset B
>>>>>>
>>>>>> i.e. if no value is specified in :var, the variable will be removed
>>>>>> (i.e. unset) - one could also use a null value (if it exists in elisp):
>>>>>>
>>>>>> :var B=(null)
>>>>>>
>>>>>
>>>>> Thanks for the ideas,
>>>>>
>>>>> I think you're right that something along the lines of the above should
>>>>> be the easiest to implement, however after reading these suggestions,
>>>>> I'm thinking that more generally there are a couple of other header
>>>>> arguments which could need to be unset, namely
>>>>> - file
>>>>> - dir
>>>>> - session
>>>>> - shebang
>>>>> some of these (like session) accept a "none" value which has the effect
>>>>> of un-setting the header argument.
>>>>
>>>> True - haven't thought about those (and did not know about :dir useful
>>>> one!). And the :session might definitely come in handy - I have cases,
>>>> where I reset it manually before evaluating certain sections of the block.
>>>>
>>>>>
>>>>> It would be nice to generalize whatever solution we apply across all
>>>>> types of header argument (both for implementation and for user
>>>>> simplicity).
>>>>
>>>> Absolutely - coherent solutions are definitely the best.
>>>>
>>>>> The simplest option would probably be to ensure that
>>>>> setting any header argument to :none would remove all instances of that
>>>>> header argument.
>>>>
>>>> Agreed - makes perfect sense. But probably for readibility use something
>>>> like:
>>>>
>>>> : header :remove()
>>>>
>>>> or
>>>>
>>>> :header :remove
>>>>
>>>>> The only problem there is cases like var, where you
>>>>> might not want to remove all :var's.  Maybe this could be expanded
>>>>> s.t. :none could take arguments, e.g.
>>>>>
>>>>> :header :none(A, B)
>>>>>
>>>>> which would remove all instances of the "header" header argument whose
>>>>> value is or is named "A" or "B"?
>>>>
>>>> I would stick to the name of the variable - that is more consistent.
>>>>
>>>> But instead of :none() I would suggest :remove :
>>>>
>>>> :header :remove(A, B)
>>>>
>>>> and if one wants to remove all variables with *value "A"*, one could use
>>>>
>>>> :header :remove("A")
>>>>
>>>> Or does that look too funky?
>>>>
>>>> No - I like it.
>>>
>>> I'm concerned that all this is looking rather complex. And I'm a bit
>>> dubious about the :xxx syntax -- those should correspond to keys in an
>>> association list. Could we step back a moment -- would someone mind
>>> giving me a concrete example of a problem whose solution requires these
>>> new features?
>>
> 
> Hi Rainer,
> 
>> Here is my situation:
>>
>> I am using org-babel to write analysis for a cluster environment, which
>> uses torque. To make my scripts flexible, I am using variables defined
>> in the beginning of the org document.
>> Everything works fine for after tangling in most of the scripts, but in
>> the submission script for the cluster (a nash scripd with definitions
>> for processing of the cluster job, following *directly/* after the
>> shebang, this causes problems, as the options are expected directly
>> after the shebang, and when variables are defined in the org file, these
>> will be included directly after the shebang. This results in torque not
>> finding the parameter it needs to successfully process the job.
>>
>> So at the moment, I am using :no-expand to avoid this,
> 
> But don't you need the variables?

I need them, but not in the submit script.

> 
> OK, so let's have a concrete example to play with. Here are two ideas --
> are either of these any good for you?
> 
> 1. Use noweb to include the variables
> 
> -------------------------------------------
> #+srcname: set-variables
> #+begin_src sh
> A=13
> B=14
> #+end_src
> 
> #+begin_src sh :tangle script.sh :noweb yes :shebang #!/bin/bash
> #$ -cwd
> <<set-variables>>
> #+end_src

I have scripts in R and in bash and I wold like to be able to use the
same variables in both script types - so no and yes - at the moment this
would work, but definitely would not be my favourite solution.

In addition, I like that the variable definition are in org and not in a
source block somewhere.

> -------------------------------------------
> 
> 2. It should be possible somehow to include the options together with
>    the shebang, or to replace :shebang with a more general "preamble"
>    concept. Or to make :shebang accept a code block as its value.
> 
> --------------------------------------------
> #+BABEL: :var A=13
> #+BABEL: :var B=14
> 
> #+header: :shebang #!/bin/bash #$ -cwd
> #+begin_src sh :tangle script.sh
> # empty
> #+end_src
> --------------------------------------------

Similar to above - should work, but now the options for torque are
hidden in the shebang - which is not to nice, but should work - although
changing the options is not nearly as nice as if the options are in the
source block.

> 
> [...]
> 
>> I must admit, I am not clear what you men with "those should correspond
>> to keys in an association list" - I am definitely not a lisp / elisp
>> person and only a regular user of emacs / org mode / ESS.
> 
> In the header lines, words that start with a colon are the *names* of
> options, and the bits inbetween are the *values* of those options. So
> I'm just whinging about using a word starting with a colon for something
> which is essentially a value.

OK - point taken.

> 
> I'm going to put some separate suggestions that involve code changes in
> a separate reply.

> 
> Dan
> 
> 
> 
>>
>> Hope this clarifies my suggestion a bit,
>>
>> Cheers,
>>
>> Rainer
>>
>>
>>>
>>>>
>>>> For consistency, one could also have a function :set() which could be
>>>> used as follow:
>>>>
>>>> :header :set(A=12, B=13)
>>>>
>>>> to set the "header" header argument A to 12 and B to 13.
>>>>
>>>> And then probably use :unset instead of :remove?
>>>>
>>>> Just thinking along while I am typing...
>>>>
>>>>
>>>>>
>>>>>>
>>>>>> But this raises another question of mine:
>>>>>>
>>>>>> I tried to use two :var headers in a properties block, but it only used
>>>>>> the first - did I miss something here?
>>>>>>
>>>>>
>>>>> Nope, it appears that property blocks (like a hash) assume that there is
>>>>> only one instance of each key->value pair, so once it is satisfied it
>>>>> will quit looking for more instances.
>>>>
>>>> But not always - there are exceptions, e.g. #+LATEX_HEADER: of which I
>>>> can specify more then one.
>>>> I am asking, because I have a horribly long line of #+BABEL: :var ....
>>>> and would really like to split it in two.
>>>
>>> I believe that I fixed that at 05ef2ae7c (2011/01/20) -- i.e., you can
>>> split it in two.
>>
>> Org-mode version 7.4 (release_7.4.338.gda8dc) (updated this morning)
>>
>> Thanks - that makes life easier. When I use
>>
>> #+BABEL: :var A=13
>> #+BABEL: :var B=14
>>
>> Both are defined - but it still does not work in a properties block (as
>> in the Eris's example).
>>
>>
>>
>>>
>>> Dan
>>>
>>>>
>>>>> Maybe the following alternative
>>>>> will suffice
>>>>>
>>>>> Best -- Eric
>>>>>
>>>>> ** two vars in a properties block -- not possible
>>>>>    :PROPERTIES:
>>>>>    :var:      test1=7
>>>>>    :var:      test2=8
>>>>>    :END:
>>>>>
>>>>> #+begin_src emacs-lisp
>>>>>   (message "test1=%S test2=%S" test1 test2)
>>>>> #+end_src
>>>>>
>>>>> results in Error
>>>>> : let: Symbol's value as variable is void: test2
>>>>>
>>>>> *** an alternative
>>>>>     :PROPERTIES:
>>>>>     :var:      tests=all-tests
>>>>>     :END:
>>>>>
>>>>> #+tblname: all-tests
>>>>> - 7
>>>>> - 8
>>>>>
>>>>> #+begin_src emacs-lisp :var eric=89
>>>>>   (message "test1=%S test2=%S" (first tests) (second tests))
>>>>> #+end_src
>>>>>
>>>>> #+results:
>>>>> : test1=7 test2=8
>>>>
>>>> That's sneaky - I like that solution. But I would have to rename my
>>>> variables in my code blocks - but I must say I like it. I'll probably
>>>> use this - it keeps the variables nicely together.
>>>>
>>>> Just an idea:
>>>>
>>>> :var: :in_table(all-variables)
>>>>
>>>> #+tblname: all-tests
>>>> - 7
>>>> - 8
>>>>
>>>> #+tblname: all-variables
>>>> | A    | 20           |
>>>> | B    | "next value" |
>>>> | C    | 13           |
>>>> | test | all-tests    |
>>>>
>>>> which would create the variables A with value 20, B with value "next
>>>> value" and C with value 13, and the variable tests would be based on the
>>>> table all-tests (i.e. test1 = 7, test2 = 8).
>>>>
>>>> That would be *really* neat and useful to use. as one would have a
>>>> nicely formatted table of all variables and their values.
>>>>
>>>> One could even go one step further and say that when no value is
>>>> specified in the second column, the variable is removed and if the
>>>> variable already exists it is overwritten.
>>>>
>>>> In this case, the limitation of one line :var only would not be a
>>>> problem any more.
>>>>
>>>> While we are at variables, it could be useful to rename or copy
>>>> variables as well:
>>>>
>>>> rename A to B:
>>>>
>>>> :var :rename(A, B)
>>>>
>>>> or copy A to B
>>>>
>>>> :var :copy(A, B)
>>>>
>>>> But probably not that useful.
>>>>
>>>> Cheers and thanks,
>>>>
>>>> Rainer


- -- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Natural Sciences Building
Office Suite 2039
Stellenbosch University
Main Campus, Merriman Avenue
Stellenbosch
South Africa

Tel:        +33 - (0)9 53 10 27 44
Cell:       +27 - (0)8 39 47 90 42
Fax (SA):   +27 - (0)8 65 16 27 82
Fax (D) :   +49 - (0)3 21 21 25 22 44
Fax (FR):   +33 - (0)9 58 10 27 44
email:      Rainer@krugs.de

Skype:      RMkrug
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1VK8YACgkQoYgNqgF2egroxgCfRiu5UwdWE+6Xk/7x3/yPS5AK
FaIAnAgX5xYe1UbdO65RHxMC7ta2SHE9
=Wqzm
-----END PGP SIGNATURE-----