Re: [ANN] Org-babel integrated into Org-mode

[Carsten Dominik <carsten.dominik@gmail.com>]

On Jun 30, 2010, at 7:01 PM, Dan Davison wrote:

> "Eric Schulte" <schulte.eric@gmail.com> writes:
>
>> Hi Carsten, Matt, Scott,
>>
>> Carsten Dominik <carsten.dominik@gmail.com> writes:
>>
>>> Hi Matt, hi Eric,
>>>
>>> Matt, thanks a lot for bringing this up.  This is indeed a very
>>> important and serious issue.  We need to address it.  We need to
>>> step back and reconsider this carefully.
>>>
>>> Don't get me wrong, I absolutely think that Org Babel should give
>>> you enough rope to hang yourself.  But we have to make sure that
>>> this will not happen to a happy and unsuspecting Org mode, or even
>>> an unsuspecting Emacs user who by chance opens a file with extension
>>> .org.
>>>
>>> I remember very well when  first realized that shell links could
>>> really affect you badly.  It scared me.
>>>
>>> You main proposal was to make Org Babel an optional module.
>>> This will not solve the problem fully, I think, because we also
>>> don't want that people who turn it on automatically commit
>>> to potentially dangerous operations.  There is a lot of good stuff
>>> in Babel which has nothing to do with code evaluation.
>>>
>>> Here is what I propose (several items are similar to what Eric  
>>> proposes)
>>>
>>> 1. A new variable org-turn-on-babel.  We can discuss the default.
>>>   If it is nil, org-babel should not be loaded.
>>>   A default of t would be fine with me if we implement other
>>>   measures listed below.
>>>
>>
>> This sounds like a good idea to me, and it should address Matt's  
>> desire
>> for enabling minimal Org-mode installs.  I would like this to  
>> default to
>> t, so that new users can try out Org-babel without overmuch effort.
>
> I'm not clear yet what the point of this is. Unless it is the load  
> time
> which is the issue, what else is gained by this variable? In principle
> I'm also all for minimalism and modularity, but what does it actually
> mean here?
>
> If the effect of this variable is to not load org-babel code at all,
> then this needs to be thought about carefully, as it is tantamount  
> to a
> statement that all org-babel code is orthogonal to the rest of
> org-mode. I.e. core org-mode will not be able to make use of any
> org-babel code, because there will always be the risk that the user  
> has
> set this variable to nil. Are we sure that we might not want some
> org-babel code (e.g. block export or tangling or something) to be used
> in core Org functionality?

Like Eric, I agree.

>
> As an example of an area of overlap of core org-mode and babel, I'd  
> been
> considering extending the [[shell:]] and [[elisp:]] links that are
> present in core Org-mode to support other languages. If that happens  
> it
> might make sense to let babel code handle the shell and elisp
> evaluation, rather than having that functionality duplicated in the  
> code
> base.

:-)  Actually, in this specific area I had been thinking to
removing or at least deprecating shell and elisp links,
because the Org-babel way is much better and clearer to have
that code in a block, rather than hiding it in the invisible
part of of a link.

> Essentially I had been envisioning the incorporation of org-babel into
> org-mode as the beginning of a phase in which the code bases can
> interact and evolve together, rather than as a promise of eternal
> orthogonality.

Yes, I agree.  This is definitely the point of this integration.

- Carsten

>
>>
>>>
>>> 2. As Eric proposes, a variable similar to org-confirm-shell-link-
>>>   function
>>>   This should by default query for confirmation on any org-babel
>>>   code execution, and can be configured to shut up by people who  
>>> know
>>>   what they are doing.
>>>
>>
>> Sounds good, I think this is a reasonable safety measure.
>
> Yes, I think this is the key. Other than the doubts expressed above I
> agree with what Eric wrote.
>
> Dan
>
>>
>>>
>>> 3. Not loading emacs lisp evaluation by default.
>>>
>>
>> I would push back on this point.  Largely because we have now crossed
>> the like to where it is impossible to play with a code block w/o  
>> first
>> dropping down to your configuration files, and evaluating require
>> statements.
>>
>>>
>>> 4. A new key in the babel keymap for org-babel-execute-code-block,
>>>   for example `C-c C-v e'.  This should be documented as the default
>>>   key for this operation.
>>>
>>
>> Hmm, I'm less enthusiastic about this point and point 5.  I really  
>> like
>> how 'C-c C-c' naturally does whatever-I-want given the context in  
>> which
>> it's called, and I wouldn't want to lose that intuitiveness.   
>> Similarly
>> 'C-c C-o' currently opens the results of a code block, I also find  
>> this
>> very appealing as it allows for a uniform top-level interface  
>> across an
>> Org-mode document, be it a code block or a link.
>>
>> Here are my reasons why I think leaving this keybinding is safe.
>>
>> 1) Unlike with shell/elisp links, the contents of code blocks is  
>> almost
>>   always visible right under the user's point.  So it is less  
>> likely to
>>   evaluate something w/o having any idea what you are evaluating.
>>
>> 2) Adding a protection variable (e.g. org-confirm-babel-eval) means  
>> that
>>   the only users who could potentially evaluate a code block with a
>>   slip of the fingers would be users who have explicitly said that  
>> they
>>   want to be able to easily run code blocks without confirmation.
>>
>> 3) Emacs exposes a number of entry points into code evaluation.  M-!
>>   allows users to run shell commands, C-M-x evaluate the elisp at
>>   point, and these have not caused problems in the past.
>>
>>>
>>> 5. Removing org-babel-execute-code-block from `C-c C-c'.  Inclusion
>>>   should be optional.
>>>
>>> 6. A section in the manual on code execution and associated security
>>>   risks in Org mode.  This is not only about babel, but also about
>>>   org-eval, org-eval-light, shell links and elisp links.  I have  
>>> meant
>>>   to write this section for a long time and would be willing to
>>>   draft it. We could then refer to this section from a couple of
>>>   places in the docs, without cluttering the docs with disclaimers.
>>>
>>
>> This sounds like a very good idea.  I'd be happy to help write such a
>> section.
>>
>>>
>>> The reason for 4 and 5 is that I believe Org-mode users are trained
>>> to blindly press `C-c C-c' whenever they want to update something at
>>> point.  Matt's example of a blog post about `rm -rf' is a very
>>> realistic example for bad code being evaluated by mistake, not even
>>> due to malicious cations.  I belive that a special key for this
>>> action would gove a good measure of protection.
>>>
>>
>> As I mentioned, I personally feel that an org-confirm-babel-eval
>> variable is sufficient protection.  I think it's safe to assume  
>> that if
>> a user has explicitly customized that variable, then they know what
>> they're doing and trust themselves to execute code responsibly.  I  
>> think
>> it's likely that the casual Org-babel user would never customize this
>> variable, which seems to me entirely appropriate.
>>
>>>
>>> This is what I think - please let me know if you think I am  
>>> overdoing
>>> it.
>>>
>>
>> So to summarize, I think that the combination of (1), (2) and (6),
>> should be sufficient to protect users from accidental code  
>> evaluation.
>> Please let me know what you think, I am of course looking to  
>> compromise
>> and I fully understand that the general consensus may be that we need
>> more layers of protection.
>>
>> Best -- Eric
>>
>>>
>>> - Carsten
>>>
>>>
>>> On Jun 29, 2010, at 8:23 PM, Matt Lundin wrote:
>>>
>>>> Hi Eric,
>>>>
>>>> Thanks again for all the work that you, Dan, and Tom have put into
>>>> org-babel. I'm glad to see it become part of org-mode!
>>>>
>>>> "Eric Schulte" <schulte.eric@gmail.com> writes:
>>>>
>>>>> 2) Babel will now be loaded by default along with the rest of Org-
>>>>> mode.
>>>>>  This means that *everyone* currently using babel will need to
>>>>> change
>>>>>  their Emacs config and remove the (require 'org-babel-int) and/or
>>>>>  (require 'org-babel) lines.
>>>>
>>>> I would like to request that org-babel be made an optional  
>>>> module. I
>>>> ask
>>>> this as someone who uses org-babel regularly. Here are my reasons:
>>>>
>>>> - Org-babel adds rather specific and complex functionality to org-
>>>> mode
>>>>   that those who use it as a simple outliner and todo manager do  
>>>> not
>>>>   require. (In other words, an option to turn it off might be nice
>>>> for
>>>>   those who are worried about "feature creep.")
>>>>
>>>> - Org-babel increases the risk of accidentally executing  
>>>> malicious or
>>>>   dangerous code when typing C-c C-c on a src block or exporting a
>>>>   file. Perhaps users should activate it only after they understand
>>>>   the risks.
>>>>
>>>>   + For instance, I might write a blog post warning about the  
>>>> dangers
>>>>     of typing "rm -rf ~/". If I put this between #+begin_src sh
>>>>     and #+end_src and unthinkingly hit C-c C-c, I would be in
>>>> trouble.
>>>>     I believe this is the reason for the variables
>>>>     org-confirm-shell-link-function and
>>>>     org-confirm-elisp-link-function.
>>>>
>>>>   + This is admitted a bit far-fetched as an example, as it would
>>>>     require one to have loaded ob-sh.el. But since elisp  
>>>> execution is
>>>>     activated by default, there remain opportunities for  
>>>> unwittingly
>>>>     executing code that is meant for other purposes (e.g.,  
>>>> warnings,
>>>>     examples, etc.).
>>>>
>>>>>  Support for evaluating emacs-lisp code blocks is loaded by  
>>>>> default.
>>>>>  All other languages will need to be required explicitly.  To
>>>>> conform
>>>>>  to Emacs filename specifications all language require lines have
>>>>> been
>>>>>  shortened from e.g.
>>>>>
>>>>>  (require 'org-babel-sh)
>>>>>
>>>>>  to
>>>>>
>>>>>  (require 'ob-sh)
>>>>
>>>> When I run make clean && make && make install I find that the  
>>>> language
>>>> directory is not installed. Does the langs directory require a  
>>>> manual
>>>> installation?
>>>>
>>>> Also, with make install, the ob-* files are installed on the same
>>>> level
>>>> as the org-files, yet lines 108-114 in org.el indicate that they
>>>> should
>>>> be installed in a babel subdirectory.
>>>>
>>>> Thanks!
>>>> Matt
>>>>
>>>> _______________________________________________
>>>> Emacs-orgmode mailing list
>>>> Please use `Reply All' to send replies to the list.
>>>> Emacs-orgmode@gnu.org
>>>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>>>
>>> - Carsten
>>
>> _______________________________________________
>> Emacs-orgmode mailing list
>> Please use `Reply All' to send replies to the list.
>> Emacs-orgmode@gnu.org
>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

- Carsten