From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Eric Schulte" Subject: Re: Re: [ANN] Org-babel integrated into Org-mode Date: Fri, 02 Jul 2010 11:52:06 -0700 Message-ID: <87mxu9ra21.fsf@gmail.com> References: <87wrtp78rg.fsf@gmail.com> <874oglofzs.fsf@fastmail.fm> <87sk4432t8.fsf@gmail.com> <874ogjrzg4.fsf@gmail.com> <871vbnj5rj.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Received: from [140.186.70.92] (port=39375 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OUlLT-0001DL-Ce for emacs-orgmode@gnu.org; Fri, 02 Jul 2010 14:52:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OUlLR-000469-0i for emacs-orgmode@gnu.org; Fri, 02 Jul 2010 14:52:15 -0400 Received: from mail-px0-f169.google.com ([209.85.212.169]:46465) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OUlLQ-000460-Kg for emacs-orgmode@gnu.org; Fri, 02 Jul 2010 14:52:12 -0400 Received: by pxi17 with SMTP id 17so3387930pxi.0 for ; Fri, 02 Jul 2010 11:52:11 -0700 (PDT) In-Reply-To: (Carsten Dominik's message of "Fri, 2 Jul 2010 06:22:57 +0200") List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Carsten Dominik Cc: Matt Lundin , Org Mode Hi Carsten, Carsten Dominik writes: > Hi Eric, thanks for this. > > I would actually like to have a variable that will exclude evaluation > from being added to the C-c C-c hook. I think users will understand > better how to use this, and customizing it will work for sure. > Explicitly removing it from the hook will only work after load time. > I've made this change and updated it in the safety-babel branch, which is also rebase'd against the current master head, so it will need to be pulled with a "git pull -f" (and will overwrite any local changes you have in that branch so be careful). > > We should also add > > (put 'org-confirm-babel-evaluate > 'safe-local-variable > (lambda (x) (eq x t))) > This is also added > > After the definition of org-confirm-babel-evaluate to avoid that > malicious code can change this setting through file variables. > > I have made a first draft of the security section in the manual, > please take a look, add to it, and add a link to it from a good location > in chapter 14. > I'm not able to find this section. Which branch is it in? Thanks -- Eric > > Cheers > > - Carsten > > On Jul 1, 2010, at 10:39 PM, Eric Schulte wrote: > >> Hi, >> >> Pursuant to the below, I've created a new "babel-safety" branch of the >> repository. It includes two new commits, the first of which >> implements >> confirmation before *any* code block evaluation, adds the keybinds for >> code block evaluation, and adds a documentation for disassociating >> code >> block evaluation from C-c C-c. >> >> ,----[1d0b2f48c7f4b12b07dcfff0b5c1d29cc2c863bb] >> | babel: evaluation of code blocks now requires confirmation >> | >> | * lisp/babel/ob.el (org-confirm-babel-evaluate): variable used to >> | control evaluation of code blocks, default value it t, meaning all >> | code block evaluation requires confirmation >> | >> | (org-babel-confirm-evaluate): function used to request >> confirmation >> | of code block evaluation from the user >> | >> | (org-babel-execute-src-block): this function is the single point >> of >> | entry for evaluation of code blocks (whether initiated through lob >> | call, through direct code block evaluation, or as part of file >> | exportation). Every time this function is called it will now >> | request confirmation from the user. The newly added >> | `org-confirm-babel-evaluate' variable can be used to configure >> this >> | behavior. >> | >> | * lisp/org.el (org-ctrl-c-ctrl-c): added documentation of code block >> | evaluation behavior >> | >> | * lisp/babel/ob-keys.el (org-babel-key-bindings): adding keybindings >> | for executing code blocks and for opening their results >> `---- >> >> the second commit creates org-babel-load-languages, which can be >> used to >> enable or disable any babel language. >> >> ,----[5f5f41a206ccef10b8ff49af8c689995d05da37c] >> | babel: `org-babel-load-languages' activates code blocks by language >> | >> | * lisp/org.el (org-babel-load-languages): this variable controls >> which >> | languages will be loaded by org-babel. It is customizable through >> | the customize interface. >> | >> | (org-babel-do-load-languages): load those languages in >> | org-babel-load-languages and disable those with nil cdr's >> `---- >> >> While this variable works, and has a very nice customization widget >> interface, it is awkward to customize from a user's configuration >> file, >> this is because it relies on the defcustom :set function with which is >> it associated, and as far as I can tell, the only way to ensure that >> the >> set function is called, is to set this variable with something along >> the >> lines of the following in your configuration. >> >> --8<---------------cut here---------------start------------->8--- >> (org-babel-do-load-languages >> 'org-babel-load-languages >> '((emacs-lisp . nil) >> (ruby . t) >> (python . nil) >> (R . t))) >> --8<---------------cut here---------------end--------------->8--- >> >> As of right now I can't think of a more natural way to implement >> such a >> variable -- suggestions welcom :). >> >> As I mentioned I'm keeping this is the babel-safety branch for now, as >> it *will* disrupt the configuration and experience of Babel users, and >> I've been hard on them already these past few weeks. Maybe this is >> best >> folded into main along with the version 7.0 release so that it will be >> accompanied with web-accessible documentation how to handle the >> incomparable changes. >> >> Thanks -- Eric >> >> "Eric Schulte" writes: >> >>> Hi, >>> >>> Thanks for finding such a good compromises solution. This new plan >>> looks great to me, specifics below >>> >>> Carsten Dominik writes: >>> >>>> Hi everyone, >>>> >>>> first of all, I think it is clear that I may have overreacted >>>> with the "6 point plan". But it is good that we are having >>>> this discussion. >>>> >>>> On Jun 30, 2010, at 6:25 PM, Eric Schulte wrote: >>>> >>>>> Hi Carsten, Matt, Scott, >>>>> >>>>> Carsten Dominik writes: >>>>> >>>>>> Hi Matt, hi Eric, >>>>>> >>>>>> Matt, thanks a lot for bringing this up. This is indeed a very >>>>>> important and serious issue. We need to address it. We need to >>>>>> step back and reconsider this carefully. >>>>>> >>>>>> Don't get me wrong, I absolutely think that Org Babel should give >>>>>> you enough rope to hang yourself. But we have to make sure that >>>>>> this will not happen to a happy and unsuspecting Org mode, or even >>>>>> an unsuspecting Emacs user who by chance opens a file with >>>>>> extension >>>>>> .org. >>>>>> >>>>>> I remember very well when first realized that shell links could >>>>>> really affect you badly. It scared me. >>>>>> >>>>>> You main proposal was to make Org Babel an optional module. >>>>>> This will not solve the problem fully, I think, because we also >>>>>> don't want that people who turn it on automatically commit >>>>>> to potentially dangerous operations. There is a lot of good stuff >>>>>> in Babel which has nothing to do with code evaluation. >>>>>> >>>>>> Here is what I propose (several items are similar to what Eric >>>>>> proposes) >>>>>> >>>>>> 1. A new variable org-turn-on-babel. We can discuss the default. >>>>>> If it is nil, org-babel should not be loaded. >>>>>> A default of t would be fine with me if we implement other >>>>>> measures listed below. >>>>>> >>>>> >>>>> This sounds like a good idea to me, and it should address Matt's >>>>> desire >>>>> for enabling minimal Org-mode installs. I would like this to >>>>> default to >>>>> t, so that new users can try out Org-babel without overmuch effort. >>>> >>>> Actually, following Dan's argument, I am paddling back on this one. >>>> Lets just keep it on. >>>> >>>> Instead of having a function to unload emacs-lisp, maybe a good way >>>> would be a customize option org-babel-load-languages with a checkbox >>>> for each language, emacs-lisp on by default. This would make it >>>> easy fo >>>> people to select the languages they want, without having to add >>>> several require statements to .emacs. >>>> >>> >>> This sounds like a good idea, such a variable would also play the >>> important role of advertising what languages currently support >>> execution >>> in Babel. One issue with this setup is that I think languages >>> which do >>> not have FSF attribution (currently only oz) would still require an >>> explicit `require' statement, however that shouldn't be too >>> surprising >>> as those statements live in the contrib directory, and users should >>> be >>> used to having to explicitly require functionality located in >>> contrib. >>> >>> One nice thing about this setup is that it shouldn't break user's >>> current config (existing `require' statements will still work). >>> >>>> >>>>>> >>>>>> 2. As Eric proposes, a variable similar to org-confirm-shell-link- >>>>>> function >>>>>> This should by default query for confirmation on any org-babel >>>>>> code execution, and can be configured to shut up by people who >>>>>> know >>>>>> what they are doing. >>>>>> >>>>> >>>>> Sounds good, I think this is a reasonable safety measure. >>>>> >>>>>> >>>>>> 3. Not loading emacs lisp evaluation by default. >>>>>> >>>>> >>>>> I would push back on this point. Largely because we have now >>>>> crossed >>>>> the like to where it is impossible to play with a code block w/o >>>>> first >>>>> dropping down to your configuration files, and evaluating require >>>>> statements. >>>>> >>>>>> >>>>>> 4. A new key in the babel keymap for org-babel-execute-code-block, >>>>>> for example `C-c C-v e'. This should be documented as the default >>>>>> key for this operation. >>>>>> >>>>> >>>>> Hmm, I'm less enthusiastic about this point and point 5. I really >>>>> like >>>>> how 'C-c C-c' naturally does whatever-I-want given the context in >>>>> which >>>>> it's called, and I wouldn't want to lose that intuitiveness. >>>>> Similarly >>>>> 'C-c C-o' currently opens the results of a code block, I also find >>>>> this >>>>> very appealing as it allows for a uniform top-level interface >>>>> across >>>>> an >>>>> Org-mode document, be it a code block or a link. >>>>> >>>>> Here are my reasons why I think leaving this keybinding is safe. >>>>> >>>>> 1) Unlike with shell/elisp links, the contents of code blocks is >>>>> almost >>>>> always visible right under the user's point. So it is less >>>>> likely to >>>>> evaluate something w/o having any idea what you are evaluating. >>>>> >>>>> 2) Adding a protection variable (e.g. org-confirm-babel-eval) means >>>>> that >>>>> the only users who could potentially evaluate a code block with a >>>>> slip of the fingers would be users who have explicitly said that >>>>> they >>>>> want to be able to easily run code blocks without confirmation. >>>>> >>>>> 3) Emacs exposes a number of entry points into code evaluation. >>>>> M-! >>>>> allows users to run shell commands, C-M-x evaluate the elisp at >>>>> point, and these have not caused problems in the past. >>>> >>>> These are all very well taken points. And I agree that a somewhat >>>> regular Org-mode user should be protected by this well enough. >>>> >>>> There are actually two kinds of users and two levels where >>>> we need to think about this. >>>> >>>> 1. I am worried about is this: Org mode (including Babel) >>>> will soon be part of Emacs an be shipped to a very large number of >>>> people who have nothing to do with Org mode and might pick a file >>>> of the web to try playing with it. I want to protect these users >>>> and >>>> also us, as the Org mode community, from a stupid accident happening >>>> like that. But, in fact, a yes-or-no-p confirmation would probably >>>> cover this well enough. OK for this part. BTW, Eric, >>>> I think this confirmation variable should also be allowed to take >>>> a function with a two arguments, the language of the snippet >>>> and the snippet. Users could then write a function which would >>>> get confirmation for some snippets, but not for others. >>>> >>> >>> This sounds like a good idea. I'll update the confirmation >>> function as >>> you've described. One possible issue here is that with complicated >>> setups, a single action could require multiple confirmations -- as >>> executing one code block can call on other code blocks as >>> arguments, but >>> I think that behavior is required to ensure that the user is fully >>> aware >>> of the full extent of the processes taking place. >>> >>>> >>>> 2. The other thing is that I am afraid of myself in this context. >>>> I envision myself turning off the check eval confirmation check >>>> sooner >>>> rather than later because I don't like to press the confirmation key >>>> all the time. Repetitive things like this annoy me and I turn >>>> them off. >>>> So I am happily working with code in a document fine. >>>> >>>> Later, I see myself accidentally pressing C-c C-c in a place where >>>> I did >>>> not mean to press it. Like in Matt's example, this could be a >>>> blog post >>>> or any other document where I have some source code examples. >>>> I press key combinations with C-c *so* many times >>>> a day that a couple of `C-c C-c' come up by accident every day. >>>> In fact, in this context I am more worried about `C-c C-c' than `C-c >>>> C- >>>> o' >>>> This is why I was proposing to not have this in C-c C-c (and, now >>>> you mention it, in C-c C-o) by default. I definitely think >>>> that it would be good to give users a variable to not include >>>> these into `C-c C-c' and `C-c C-o'. Having additional bindings >>>> for these two commands in the `C-c C-v' map would not hurt in >>>> any case. >>>> >>>> On the other hand, I totally see how C-c C-c is a great and >>>> natural binding if you wan to work with source code, of cause, >>>> and I do understand why you defend it and want to have it in by >>>> default. >>>> >>> >>> Thanks for your understanding on this point. >>> >>>> >>>> So in summary, I think I could be fine with a situation >>>> where the variable I just described exists and is set >>>> so that C-c C-c and C-c C-o do the evaluation, and where >>>> the issues are clearly documented. >>>> >>> >>> I see your point, and I think your proposed solution is a good >>> compromise. I'll add C-c C-v keybindings for both evaluation and >>> opening of code blocks. Also, I'll add a variable which can be >>> used to >>> disable the binding of code block execution to C-c C-c. We can also >>> mention this variable in the documentation for the confirmation >>> variable, so that as users disable block confirmation they will >>> (hopefully) read the documentation of the variable they are >>> disabling, >>> and will stop and think "maybe in a world w/o confirmation, I'd feel >>> safer using a higher friction keybinding". >>> >>> BTW: any suggestions for C-c C-v key bindings, I'm thinking of the >>> following, but there may be options with better ergonomics. >>> | C-c C-v e | for code block execution, mnemonic "execute" | >>> | C-c C-v o | for opening of results, mnemonic "open" | >>> >>> Thanks -- Eric >>> >>>> >>>> - Carsten >>>> >>>> >>>> >>>>> >>>>>> >>>>>> 5. Removing org-babel-execute-code-block from `C-c C-c'. >>>>>> Inclusion >>>>>> should be optional. >>>>>> >>>>>> 6. A section in the manual on code execution and associated >>>>>> security >>>>>> risks in Org mode. This is not only about babel, but also about >>>>>> org-eval, org-eval-light, shell links and elisp links. I have >>>>>> meant >>>>>> to write this section for a long time and would be willing to >>>>>> draft it. We could then refer to this section from a couple of >>>>>> places in the docs, without cluttering the docs with disclaimers. >>>>>> >>>>> >>>>> This sounds like a very good idea. I'd be happy to help write >>>>> such a >>>>> section. >>>>> >>>>>> >>>>>> The reason for 4 and 5 is that I believe Org-mode users are >>>>>> trained >>>>>> to blindly press `C-c C-c' whenever they want to update >>>>>> something at >>>>>> point. Matt's example of a blog post about `rm -rf' is a very >>>>>> realistic example for bad code being evaluated by mistake, not >>>>>> even >>>>>> due to malicious cations. I belive that a special key for this >>>>>> action would gove a good measure of protection. >>>>>> >>>>> >>>>> As I mentioned, I personally feel that an org-confirm-babel-eval >>>>> variable is sufficient protection. I think it's safe to assume >>>>> that >>>>> if >>>>> a user has explicitly customized that variable, then they know what >>>>> they're doing and trust themselves to execute code responsibly. I >>>>> think >>>>> it's likely that the casual Org-babel user would never customize >>>>> this >>>>> variable, which seems to me entirely appropriate. >>>>> >>>>>> >>>>>> This is what I think - please let me know if you think I am >>>>>> overdoing >>>>>> it. >>>>>> >>>>> >>>>> So to summarize, I think that the combination of (1), (2) and (6), >>>>> should be sufficient to protect users from accidental code >>>>> evaluation. >>>>> Please let me know what you think, I am of course looking to >>>>> compromise >>>>> and I fully understand that the general consensus may be that we >>>>> need >>>>> more layers of protection. >>>>> >>>>> Best -- Eric >>>>> >>>>>> >>>>>> - Carsten >>>>>> >>>>>> >>>>>> On Jun 29, 2010, at 8:23 PM, Matt Lundin wrote: >>>>>> >>>>>>> Hi Eric, >>>>>>> >>>>>>> Thanks again for all the work that you, Dan, and Tom have put >>>>>>> into >>>>>>> org-babel. I'm glad to see it become part of org-mode! >>>>>>> >>>>>>> "Eric Schulte" writes: >>>>>>> >>>>>>>> 2) Babel will now be loaded by default along with the rest of >>>>>>>> Org- >>>>>>>> mode. >>>>>>>> This means that *everyone* currently using babel will need to >>>>>>>> change >>>>>>>> their Emacs config and remove the (require 'org-babel-int) >>>>>>>> and/ >>>>>>>> or >>>>>>>> (require 'org-babel) lines. >>>>>>> >>>>>>> I would like to request that org-babel be made an optional >>>>>>> module. I >>>>>>> ask >>>>>>> this as someone who uses org-babel regularly. Here are my >>>>>>> reasons: >>>>>>> >>>>>>> - Org-babel adds rather specific and complex functionality to >>>>>>> org- >>>>>>> mode >>>>>>> that those who use it as a simple outliner and todo manager do >>>>>>> not >>>>>>> require. (In other words, an option to turn it off might be nice >>>>>>> for >>>>>>> those who are worried about "feature creep.") >>>>>>> >>>>>>> - Org-babel increases the risk of accidentally executing >>>>>>> malicious >>>>>>> or >>>>>>> dangerous code when typing C-c C-c on a src block or exporting a >>>>>>> file. Perhaps users should activate it only after they understand >>>>>>> the risks. >>>>>>> >>>>>>> + For instance, I might write a blog post warning about the >>>>>>> dangers >>>>>>> of typing "rm -rf ~/". If I put this between #+begin_src sh >>>>>>> and #+end_src and unthinkingly hit C-c C-c, I would be in >>>>>>> trouble. >>>>>>> I believe this is the reason for the variables >>>>>>> org-confirm-shell-link-function and >>>>>>> org-confirm-elisp-link-function. >>>>>>> >>>>>>> + This is admitted a bit far-fetched as an example, as it would >>>>>>> require one to have loaded ob-sh.el. But since elisp >>>>>>> execution is >>>>>>> activated by default, there remain opportunities for >>>>>>> unwittingly >>>>>>> executing code that is meant for other purposes (e.g., >>>>>>> warnings, >>>>>>> examples, etc.). >>>>>>> >>>>>>>> Support for evaluating emacs-lisp code blocks is loaded by >>>>>>>> default. >>>>>>>> All other languages will need to be required explicitly. To >>>>>>>> conform >>>>>>>> to Emacs filename specifications all language require lines have >>>>>>>> been >>>>>>>> shortened from e.g. >>>>>>>> >>>>>>>> (require 'org-babel-sh) >>>>>>>> >>>>>>>> to >>>>>>>> >>>>>>>> (require 'ob-sh) >>>>>>> >>>>>>> When I run make clean && make && make install I find that the >>>>>>> language >>>>>>> directory is not installed. Does the langs directory require a >>>>>>> manual >>>>>>> installation? >>>>>>> >>>>>>> Also, with make install, the ob-* files are installed on the same >>>>>>> level >>>>>>> as the org-files, yet lines 108-114 in org.el indicate that they >>>>>>> should >>>>>>> be installed in a babel subdirectory. >>>>>>> >>>>>>> Thanks! >>>>>>> Matt >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Emacs-orgmode mailing list >>>>>>> Please use `Reply All' to send replies to the list. >>>>>>> Emacs-orgmode@gnu.org >>>>>>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode >>>>>> >>>>>> - Carsten >>>>> >>>>> _______________________________________________ >>>>> Emacs-orgmode mailing list >>>>> Please use `Reply All' to send replies to the list. >>>>> Emacs-orgmode@gnu.org >>>>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode >>>> >>>> - Carsten > > - Carsten