From mboxrd@z Thu Jan 1 00:00:00 1970 From: Carsten Dominik Subject: Re: [ANN] Org-babel integrated into Org-mode Date: Thu, 1 Jul 2010 08:27:31 +0200 Message-ID: References: <87wrtp78rg.fsf@gmail.com> <874oglofzs.fsf@fastmail.fm> <87sk4432t8.fsf@gmail.com> <87zkyc4fql.fsf@gmail.com> Mime-Version: 1.0 (Apple Message framework v936) Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Return-path: Received: from [140.186.70.92] (port=41415 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OUE2g-0004Az-41 for emacs-orgmode@gnu.org; Thu, 01 Jul 2010 03:18:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OUE2b-00059X-EL for emacs-orgmode@gnu.org; Thu, 01 Jul 2010 03:18:38 -0400 Received: from mail-fx0-f41.google.com ([209.85.161.41]:49517) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OUE2b-00059M-4g for emacs-orgmode@gnu.org; Thu, 01 Jul 2010 03:18:33 -0400 Received: by fxm17 with SMTP id 17so1104737fxm.0 for ; Thu, 01 Jul 2010 00:18:31 -0700 (PDT) In-Reply-To: <87zkyc4fql.fsf@gmail.com> List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Dan Davison Cc: Matt Lundin , Org Mode On Jun 30, 2010, at 7:01 PM, Dan Davison wrote: > "Eric Schulte" writes: > >> Hi Carsten, Matt, Scott, >> >> Carsten Dominik writes: >> >>> Hi Matt, hi Eric, >>> >>> Matt, thanks a lot for bringing this up. This is indeed a very >>> important and serious issue. We need to address it. We need to >>> step back and reconsider this carefully. >>> >>> Don't get me wrong, I absolutely think that Org Babel should give >>> you enough rope to hang yourself. But we have to make sure that >>> this will not happen to a happy and unsuspecting Org mode, or even >>> an unsuspecting Emacs user who by chance opens a file with extension >>> .org. >>> >>> I remember very well when first realized that shell links could >>> really affect you badly. It scared me. >>> >>> You main proposal was to make Org Babel an optional module. >>> This will not solve the problem fully, I think, because we also >>> don't want that people who turn it on automatically commit >>> to potentially dangerous operations. There is a lot of good stuff >>> in Babel which has nothing to do with code evaluation. >>> >>> Here is what I propose (several items are similar to what Eric >>> proposes) >>> >>> 1. A new variable org-turn-on-babel. We can discuss the default. >>> If it is nil, org-babel should not be loaded. >>> A default of t would be fine with me if we implement other >>> measures listed below. >>> >> >> This sounds like a good idea to me, and it should address Matt's >> desire >> for enabling minimal Org-mode installs. I would like this to >> default to >> t, so that new users can try out Org-babel without overmuch effort. > > I'm not clear yet what the point of this is. Unless it is the load > time > which is the issue, what else is gained by this variable? In principle > I'm also all for minimalism and modularity, but what does it actually > mean here? > > If the effect of this variable is to not load org-babel code at all, > then this needs to be thought about carefully, as it is tantamount > to a > statement that all org-babel code is orthogonal to the rest of > org-mode. I.e. core org-mode will not be able to make use of any > org-babel code, because there will always be the risk that the user > has > set this variable to nil. Are we sure that we might not want some > org-babel code (e.g. block export or tangling or something) to be used > in core Org functionality? Like Eric, I agree. > > As an example of an area of overlap of core org-mode and babel, I'd > been > considering extending the [[shell:]] and [[elisp:]] links that are > present in core Org-mode to support other languages. If that happens > it > might make sense to let babel code handle the shell and elisp > evaluation, rather than having that functionality duplicated in the > code > base. :-) Actually, in this specific area I had been thinking to removing or at least deprecating shell and elisp links, because the Org-babel way is much better and clearer to have that code in a block, rather than hiding it in the invisible part of of a link. > Essentially I had been envisioning the incorporation of org-babel into > org-mode as the beginning of a phase in which the code bases can > interact and evolve together, rather than as a promise of eternal > orthogonality. Yes, I agree. This is definitely the point of this integration. - Carsten > >> >>> >>> 2. As Eric proposes, a variable similar to org-confirm-shell-link- >>> function >>> This should by default query for confirmation on any org-babel >>> code execution, and can be configured to shut up by people who >>> know >>> what they are doing. >>> >> >> Sounds good, I think this is a reasonable safety measure. > > Yes, I think this is the key. Other than the doubts expressed above I > agree with what Eric wrote. > > Dan > >> >>> >>> 3. Not loading emacs lisp evaluation by default. >>> >> >> I would push back on this point. Largely because we have now crossed >> the like to where it is impossible to play with a code block w/o >> first >> dropping down to your configuration files, and evaluating require >> statements. >> >>> >>> 4. A new key in the babel keymap for org-babel-execute-code-block, >>> for example `C-c C-v e'. This should be documented as the default >>> key for this operation. >>> >> >> Hmm, I'm less enthusiastic about this point and point 5. I really >> like >> how 'C-c C-c' naturally does whatever-I-want given the context in >> which >> it's called, and I wouldn't want to lose that intuitiveness. >> Similarly >> 'C-c C-o' currently opens the results of a code block, I also find >> this >> very appealing as it allows for a uniform top-level interface >> across an >> Org-mode document, be it a code block or a link. >> >> Here are my reasons why I think leaving this keybinding is safe. >> >> 1) Unlike with shell/elisp links, the contents of code blocks is >> almost >> always visible right under the user's point. So it is less >> likely to >> evaluate something w/o having any idea what you are evaluating. >> >> 2) Adding a protection variable (e.g. org-confirm-babel-eval) means >> that >> the only users who could potentially evaluate a code block with a >> slip of the fingers would be users who have explicitly said that >> they >> want to be able to easily run code blocks without confirmation. >> >> 3) Emacs exposes a number of entry points into code evaluation. M-! >> allows users to run shell commands, C-M-x evaluate the elisp at >> point, and these have not caused problems in the past. >> >>> >>> 5. Removing org-babel-execute-code-block from `C-c C-c'. Inclusion >>> should be optional. >>> >>> 6. A section in the manual on code execution and associated security >>> risks in Org mode. This is not only about babel, but also about >>> org-eval, org-eval-light, shell links and elisp links. I have >>> meant >>> to write this section for a long time and would be willing to >>> draft it. We could then refer to this section from a couple of >>> places in the docs, without cluttering the docs with disclaimers. >>> >> >> This sounds like a very good idea. I'd be happy to help write such a >> section. >> >>> >>> The reason for 4 and 5 is that I believe Org-mode users are trained >>> to blindly press `C-c C-c' whenever they want to update something at >>> point. Matt's example of a blog post about `rm -rf' is a very >>> realistic example for bad code being evaluated by mistake, not even >>> due to malicious cations. I belive that a special key for this >>> action would gove a good measure of protection. >>> >> >> As I mentioned, I personally feel that an org-confirm-babel-eval >> variable is sufficient protection. I think it's safe to assume >> that if >> a user has explicitly customized that variable, then they know what >> they're doing and trust themselves to execute code responsibly. I >> think >> it's likely that the casual Org-babel user would never customize this >> variable, which seems to me entirely appropriate. >> >>> >>> This is what I think - please let me know if you think I am >>> overdoing >>> it. >>> >> >> So to summarize, I think that the combination of (1), (2) and (6), >> should be sufficient to protect users from accidental code >> evaluation. >> Please let me know what you think, I am of course looking to >> compromise >> and I fully understand that the general consensus may be that we need >> more layers of protection. >> >> Best -- Eric >> >>> >>> - Carsten >>> >>> >>> On Jun 29, 2010, at 8:23 PM, Matt Lundin wrote: >>> >>>> Hi Eric, >>>> >>>> Thanks again for all the work that you, Dan, and Tom have put into >>>> org-babel. I'm glad to see it become part of org-mode! >>>> >>>> "Eric Schulte" writes: >>>> >>>>> 2) Babel will now be loaded by default along with the rest of Org- >>>>> mode. >>>>> This means that *everyone* currently using babel will need to >>>>> change >>>>> their Emacs config and remove the (require 'org-babel-int) and/or >>>>> (require 'org-babel) lines. >>>> >>>> I would like to request that org-babel be made an optional >>>> module. I >>>> ask >>>> this as someone who uses org-babel regularly. Here are my reasons: >>>> >>>> - Org-babel adds rather specific and complex functionality to org- >>>> mode >>>> that those who use it as a simple outliner and todo manager do >>>> not >>>> require. (In other words, an option to turn it off might be nice >>>> for >>>> those who are worried about "feature creep.") >>>> >>>> - Org-babel increases the risk of accidentally executing >>>> malicious or >>>> dangerous code when typing C-c C-c on a src block or exporting a >>>> file. Perhaps users should activate it only after they understand >>>> the risks. >>>> >>>> + For instance, I might write a blog post warning about the >>>> dangers >>>> of typing "rm -rf ~/". If I put this between #+begin_src sh >>>> and #+end_src and unthinkingly hit C-c C-c, I would be in >>>> trouble. >>>> I believe this is the reason for the variables >>>> org-confirm-shell-link-function and >>>> org-confirm-elisp-link-function. >>>> >>>> + This is admitted a bit far-fetched as an example, as it would >>>> require one to have loaded ob-sh.el. But since elisp >>>> execution is >>>> activated by default, there remain opportunities for >>>> unwittingly >>>> executing code that is meant for other purposes (e.g., >>>> warnings, >>>> examples, etc.). >>>> >>>>> Support for evaluating emacs-lisp code blocks is loaded by >>>>> default. >>>>> All other languages will need to be required explicitly. To >>>>> conform >>>>> to Emacs filename specifications all language require lines have >>>>> been >>>>> shortened from e.g. >>>>> >>>>> (require 'org-babel-sh) >>>>> >>>>> to >>>>> >>>>> (require 'ob-sh) >>>> >>>> When I run make clean && make && make install I find that the >>>> language >>>> directory is not installed. Does the langs directory require a >>>> manual >>>> installation? >>>> >>>> Also, with make install, the ob-* files are installed on the same >>>> level >>>> as the org-files, yet lines 108-114 in org.el indicate that they >>>> should >>>> be installed in a babel subdirectory. >>>> >>>> Thanks! >>>> Matt >>>> >>>> _______________________________________________ >>>> Emacs-orgmode mailing list >>>> Please use `Reply All' to send replies to the list. >>>> Emacs-orgmode@gnu.org >>>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode >>> >>> - Carsten >> >> _______________________________________________ >> Emacs-orgmode mailing list >> Please use `Reply All' to send replies to the list. >> Emacs-orgmode@gnu.org >> http://lists.gnu.org/mailman/listinfo/emacs-orgmode - Carsten