From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id qBT8KLVD7mA1owAAgWs5BA (envelope-from ) for ; Wed, 14 Jul 2021 03:53:57 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id uOa5JLVD7mDYEAAA1q6Kng (envelope-from ) for ; Wed, 14 Jul 2021 01:53:57 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A59139830 for ; Wed, 14 Jul 2021 03:53:56 +0200 (CEST) Received: from localhost ([::1]:53634 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m3U5z-000115-Od for larch@yhetil.org; Tue, 13 Jul 2021 21:53:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35788) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3U5a-00010q-TL for emacs-orgmode@gnu.org; Tue, 13 Jul 2021 21:53:30 -0400 Received: from mail-lf1-x12e.google.com ([2a00:1450:4864:20::12e]:41848) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m3U5Y-0007Zh-Me for emacs-orgmode@gnu.org; Tue, 13 Jul 2021 21:53:30 -0400 Received: by mail-lf1-x12e.google.com with SMTP id n14so700106lfu.8 for ; Tue, 13 Jul 2021 18:53:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=DVBEUXRj0oQFYg+ob2t7nVd6LMI1q8Lv/tegzPdPMkQ=; b=KPzn04Y+ztkC+qUF/cARp+pB3y3fZXY2SJJRBu4FA2RoVAHVY+BZY5YjYeM3OPUyZc o6zaqOlv6ZFNxuxjhmU6LqU+ZiBEPASxoInjigWmqF+fAgAeHmL6IKoF4TeJF39K0yVX Em0Qy+Mip9Fg1vUkwC9IwoTx0Sdj6phqDAUcG3BhK0pyThbarlgTI19+VLYe7jH5EOi0 fm02VYr5L/ShAsVsbSWI+LtNFQV5qdkHKq1e4eVyE4w4iwhkKBun1KD05fTnG2xK1Z87 iICHw3sy1e4t211YMLvA2jpx0xgTGOzheQPRc1VOqzyw2ae/5e8k9cOzDxXlYwD+jD4y Fx4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=DVBEUXRj0oQFYg+ob2t7nVd6LMI1q8Lv/tegzPdPMkQ=; b=ioHGFPofwuLKTkir9aVAtR3LOhMDRLxN5+iIIXkUcPT+BKi8zI8SR/DoFBTbgUUkJH RU1CcrLouhKnij256z+HO8usyiwAeBZl/0qByKdIEiD2wd8Np2/8gQJUb3+VNdWBmMTs qkj1SYDgwilM2mhCDuw208z9j8H+wNIrD3XdlRZ7PmlcYzkgg6jnPisC45bOB0lbS7of y9+MylTmwZoz98bbJnetp/Ie8Xzn8B6uyCJBxdnTGjKez++AmSKP4nGNP49wIORnJEn/ f7QbI7LYqkJWNDOVkrMlwIF5up74SomYOCJ9At+SmB0INa9cxciIiRTylzkYgosk2zwL yWsQ== X-Gm-Message-State: AOAM53339RLy6i4wT2DpUNt4HOW0p09wW7fewl6WB3T1nBK2qdJf8eXl tb/W0r8IrZ+dcgaElqrOAXohjKOmXRN4PoxclzI= X-Google-Smtp-Source: ABdhPJxhO4fFxUiD61PX7TIP6sJcvW5JtGHCLU8KEF8UoEgAv8Pp0lnZWYt0jgUxqZqE4dcuZ8srnqHbVcHKUFXyyvo= X-Received: by 2002:ac2:425a:: with SMTP id m26mr5932956lfl.458.1626227605934; Tue, 13 Jul 2021 18:53:25 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a05:6504:13fc:0:0:0:0 with HTTP; Tue, 13 Jul 2021 18:53:25 -0700 (PDT) In-Reply-To: <87h7gxr8ql.fsf@gmail.com> References: <95_WQAdCOQRo2u4VTanEqQ9QrbmOhFNk4aTCGlvgNU8PkVz3j7SQfwIZuwoxtQj0v5bGnRnP6NPf-Z6ImfWKpg_pHF6sTFHMgw0KVfir06U=@protonmail.ch> <9F8F3589-6E58-45AE-A0E6-F359ACA2A4D9@condition-alpha.com> <87h7gxr8ql.fsf@gmail.com> From: Samuel Wales Date: Tue, 13 Jul 2021 18:53:25 -0700 Message-ID: Subject: Re: A requires/provides approach to linking source code blocks To: Tim Cross Content-Type: text/plain; charset="UTF-8" Received-SPF: pass client-ip=2a00:1450:4864:20::12e; envelope-from=samologist@gmail.com; helo=mail-lf1-x12e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Tom Gillespie , emacs-orgmode Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1626227636; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=DVBEUXRj0oQFYg+ob2t7nVd6LMI1q8Lv/tegzPdPMkQ=; b=OQY7rl9TLH2Wq9/bD4WZWUgQz7j2ir0dbmQOVivY4V7OEuC1925TU3nwvmwA7ecWxCL2+A 6xTXsE01uGVOQQYyUiQSsHiCe/IcG78n6URT5uxGEzEMEZvOWn47l9N6Wt6Byp+MThfA0M j5UV8QbWP3/NI4bQxFaKAR4C79mmzw29E1+ikiiBZdbZMXf7gWiGCFMQS4wrFHNc0LSY/G VdWzwJ5obf4s0b4j76YImdxCupuBBZ9QmIG2Moz/dgoGoYzvwdFDrmc/qyCHQVVQVWArb4 whMfG2q94CFMsd4ApizCaHiTtg+cW2cJpleeVfXjOfXS71M9NtItheoDZZkl4Q== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1626227636; a=rsa-sha256; cv=none; b=RYCgPQiYJfXHdOL47gIPPlu8GeB+zU1FslrtOxrRG3eNShhFKIkO2toeeQeF+obzmGKV0Q 1QQDzAuY6WsGXKGDfnjbZRGpYWe9tEuq34msubhb7EbPAD6EE8czoigEugjHhQ4TatBFeC yAoAqQ585IEMnVEvUifNkCHZDZdQIHK8hwn5kuKqHazjuLihXlByiw2QsPHzxqby3dhwCe sK35EDGnYgjt2N1nC1fEtbgPhG3gnW0FnhxlbjjNldprk7gO2SPNA9ap06LN166bizB+EH ZhcR+tHZLhJbeVvDEb0hoJAhpq+qHg/qxMOCjMH+qoppsCY9nhUgs2r7Y3ABQQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=KPzn04Y+; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Spam-Score: -3.10 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=KPzn04Y+; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Queue-Id: A59139830 X-Spam-Score: -3.10 X-Migadu-Scanner: scn0.migadu.com X-TUID: GpNTH5EyJb1Q one thing about org that i think has been making it complex, in addition to number of features, is non-orthogonality. On 7/13/21, Tim Cross wrote: > > Tom Gillespie writes: > > [snip] > >> >> Once I wrap up the formal grammar for org, one of the next things I >> plan to work on is a clear specification for org babel. This is >> critical because so many of the suggestions that come in deal with >> individuals' specific problems and thus fail to account for how such >> features interact with existing features and how the newly proposed >> feature would block some other features in the future, confuse users, >> etc. Such suggestions also often fail to account for increased >> complexity, nor have they been exposed to a sufficient number of >> examples to reveal fundamental ambiguities in how they could be >> interpreted. The issues with variable behavior between ob langs for >> :pre :post :prologue :epilogue etc. are already enough to keep us busy >> for quite some time. >> > > Yes, that clearly summs up my concerns as well. Often, a 'fix' for some > problem can seem straight-forward, possibly even trivial, when > considered only within one person's use case/workflow. Part of what > makes org so powerful is the level of flexibility it supports. However, > this flexibility makes it extremely difficult to consider, or even know > about, all the existing use cases. > > I suspect that once we have some formal specifications for babel, the > next step will be to develop some good unit tests to verify these > specifications. This would at least highlight/alert developers to > unforeseen impact from changes and alert them to things they may > not have considered. > >> With regard to this thread in particular, it is of some interest, but >> there are fundamental issues, including the fact that certain >> languages (e.g. racket) expect module code to exist somewhere on the >> file system. There are ways around many of these issues, in fact there >> are likely many ways around any individual issue, so org babel needs >> to systematically consider the issues and provide a clear >> specification, or at least a guide for how such cases should be >> handled. >> >> To give an example from one of my continual pain points: I start >> writing python or racket in an org src block and then I want it to be >> a library so that it can be reused by other code both inside and >> outside the org file without having to resort to noweb. >> > > This is an interesting point. I think a number of languages have > challenges here. I run into very similar issues with Clojure. > > To some extent, I think this is a grey area within the literate > programming paradigm. The original literate programming model was > developed at a time when most languages were compiled rather then > interpreted. You generated source code, compiled it and then ran it. > > These days, many 'modern' languages are based around an interpreter and > concepts like 'just in time' compilation. In org mode, things become > even more complex because in addition to generation (tangling) of code, > we also want to have evaluation of code blocks, plus we have added the > concept of 'sessions'. > >> What is the best way to handle this? I don't know. Right now I tangle >> things and resort to awful hacks for the reuse-in-this-org-file case, but >> I'm guessing there is a better generic solution which would allow _any_ >> org block to be exported as a library instead of nowebbed in. >> >> Before jumping for any particular suggestion for how to handle this >> we need to explore the diversity of cases that various ob langs >> present, so that we can find a solution that will work for all of >> them. After all, packaging code to a library for reuse is an >> extremely common pattern that org babel should be able to >> abstract, but it is a major undertaking, not just the addition of a >> keyword here and there. >> > > Agree. I'm not convinced we really understand the requirements here and > more exploration and specification is required. The more we add 'simple' > extensions, options, keywords etc the more likely it is we will make > this a much harder process and will likely result in even larger > 'breakage' once we do define a clearer specification. > >> In short I suggest that we issue a general moratorium on new org babel >> feature suggestions until we can stabilize what we already have and >> provide a clear specification for correct behavior. Until we have that >> spec >> we could encourage users to create extensions that implement those >> features. >> > > Yes. While it might sound harsh or overly limiting, I do think such a > moratorium may be required. We may be able to lift it once we have some > core specifications in place. We would still accept bug fixes (though we > may need to provide some clarity on what is a bug fix and what is a > feature enhancement/change - I regularly see posts flagged as bug fixes > which are actually feature enhancements or extensions). >> >> PS The other next thing that I am working on might be another way out >> for this particular feature request. Namely, it is simplifying and >> extending org keyword syntax so that new keywords (with options) and >> associated keywords can be specified using keyword syntax within a >> single org file. This would make it possible to get useful high level >> keyword behavior in a single file without burdening the core >> implementation with more special cases for associated keywords, and it >> would allow users to write small elisp functions that could do some of >> what is suggested here, all without need to add anything to the core >> org implementation. > > Sounds interesting. To some extent, recognising there may be at least > two types of code blocks (library code, executable/driver code) and at > least two types of languages (compiled/interpreted) may help identify > base code block permutation and required options, keywords etc. However, > we are unlikely to find a definitive set which supports all use > cases/workflows and the ability to easily extend/customise behaviours > would be very useful. > -- > Tim Cross > > -- The Kafka Pandemic Please learn what misopathy is. https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html