From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 4BN9ICc87mDklgAAgWs5BA (envelope-from ) for ; Wed, 14 Jul 2021 03:21:43 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id mr0zHCc87mAxFAAAB5/wlQ (envelope-from ) for ; Wed, 14 Jul 2021 01:21:43 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 9B7EF9332 for ; Wed, 14 Jul 2021 03:21:42 +0200 (CEST) Received: from localhost ([::1]:50356 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m3Tan-0004X4-2U for larch@yhetil.org; Tue, 13 Jul 2021 21:21:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58398) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3TaO-0004Wf-E3 for emacs-orgmode@gnu.org; Tue, 13 Jul 2021 21:21:16 -0400 Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]:42888) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m3TaM-0002Yn-Hu for emacs-orgmode@gnu.org; Tue, 13 Jul 2021 21:21:16 -0400 Received: by mail-pl1-x630.google.com with SMTP id v14so480597plg.9 for ; Tue, 13 Jul 2021 18:21:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=references:user-agent:from:to:cc:subject:date:in-reply-to :message-id:mime-version; bh=KuXfxIv9Xo43LL/IOwR6FT+OoY8xzg0nMPQYBE7gtlc=; b=NEi+X6+1ts5dhbEIi59pK5h78gRQ9Gyg7/hExIZmvOEv9VwgieGmnfULt3qdlSTARf 0wPZ8slj07r6LA1Dv8jJ7/sGrhEXtIY7znP1qpXEt92M2ZePIXBhi2InGZPmM8DadksV 4VW9uFpwdc7fzusoTDt1Qca8FhOANLf4ELJe70JvPBTZToZzPOOLG7Mo1bnRJwNEO5UF PohA5baQ1jyiROjWB8bCN+almlpNMO6AJNCnJAQS2y5MBjfvpk/SQKg2rvAEQrfJGezp 2qRO6fJWy26qE+WBtdNXLeEzSUoep13qb60L7ohx9+tOfNZMVCRjBBwcFEFr5fhOtPIj 9v/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:references:user-agent:from:to:cc:subject:date :in-reply-to:message-id:mime-version; bh=KuXfxIv9Xo43LL/IOwR6FT+OoY8xzg0nMPQYBE7gtlc=; b=qbKDrnwBTHyd4OpRmXY+E6QSzy0lXaKiCSwU0PYSpS6P+8KfXfXyWO28qYOXRL3Zz6 EO/SGBxyFR5Y20Km+wMt3WZjtaFDak1nP36Q3QKGeLexWQO+RLS0IzvY38WQWNVtMjzo kxdNBhFBfUr/t/xGC3DYTkSP42mIa7xJRicNdP0TlLNxJMKuAabRJ3FSq4U5QkdpsAI/ kRoCozHt5fMzeIUOJ7AcY03eIcns3hfyYpqtp7YCEaVI5vWyq+NSTk4Zj/9efqYLsEyw Tsisa80PnXpN1hRW9EQBPclwIRihD3JCEpEj4fMUiOjf8z7Z7+l8tg3JJ8pZvMkXm07e dVvA== X-Gm-Message-State: AOAM530jeZGIpIi41Qd+X0Vn9jUTXgE5+SUCY4V2scOsWX+p/X9oEFCJ ZzJSugNISwT4F0Wwm00RwmCPGjAJ8gI= X-Google-Smtp-Source: ABdhPJzNZMJXbDrDk9NBwkqIGPjgHLRT3dxIlTenOwCTwZbV3RYxhwH95KCCndgADGOrTgXIHnA88A== X-Received: by 2002:a17:902:8ec7:b029:119:a15f:3a1c with SMTP id x7-20020a1709028ec7b0290119a15f3a1cmr5897962plo.48.1626225671766; Tue, 13 Jul 2021 18:21:11 -0700 (PDT) Received: from tim-desktop (220-235-4-230.dyn.iinet.net.au. [220.235.4.230]) by smtp.gmail.com with ESMTPSA id 1sm387384pfm.123.2021.07.13.18.21.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Jul 2021 18:21:11 -0700 (PDT) References: <95_WQAdCOQRo2u4VTanEqQ9QrbmOhFNk4aTCGlvgNU8PkVz3j7SQfwIZuwoxtQj0v5bGnRnP6NPf-Z6ImfWKpg_pHF6sTFHMgw0KVfir06U=@protonmail.ch> <9F8F3589-6E58-45AE-A0E6-F359ACA2A4D9@condition-alpha.com> User-agent: mu4e 1.5.13; emacs 28.0.50 From: Tim Cross To: Tom Gillespie Subject: Re: A requires/provides approach to linking source code blocks Date: Wed, 14 Jul 2021 10:44:10 +1000 In-reply-to: Message-ID: <87h7gxr8ql.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::630; envelope-from=theophilusx@gmail.com; helo=mail-pl1-x630.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: emacs-orgmode Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1626225703; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=KuXfxIv9Xo43LL/IOwR6FT+OoY8xzg0nMPQYBE7gtlc=; b=NikLtydTtye8kJRe1XYicF6hx1gJnRutWApSErDJR0sQm1WVucfNH8Cffz0t51VkLbgE+3 2lTlmXMCQtQe8jPDo4CwKoRtxD/4MQKm8Tl5FJqo4G2Zz6irgdkZpxhKDlpYsWhJn00fBH siwS9F4LxkUHD5+hv5OtufjU9XzyJx3+1j0etkXOYBEO13Vb2osRgAk2Z7iFtNAKR/UJzh BjbdHe5N01Z53SJuiGm7y2aBU7rkF57sAlm4F7zaHT+AZUQjvHSk+xJmKdfA/fu9aoKw6V Dp8dNeMa9SpkOAFI1IN3J2kLfjXUQmtBWR/ezTxzp0iqOLN6itimq6EBM6yPnA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1626225703; a=rsa-sha256; cv=none; b=Fx7VpnvRKVOK/J+Bk0/1mS5YRegEqcizsPt8RmVXBa0f5ARHLmX2ZzMgaOqZS7hnZe6xoT vyMZyxK3qu0zogaR4mPUhs4av/SvpJdLHY/PbBmvD7g6ULT5sJ7e8YUg9Y2kDIlNyyJg0H 19n51SfeXyr3xIl/cn4PXILD14QcZjF5P2P0sf75zhfGSnlY5vjok3wQCZwZa9pNaqmQLk /SO5wLEoQhOaFa4pUYUOvmyUiwm9Mskg+BLdjUWomicCP/Jg0GMi4jR+5a0g1XCWkUlrsi wYBHbDBJp8wooulUtW03S2ffUdeJ80xaXjYr8K7T7Ilj7B95zA09IPuFZtfWnQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=NEi+X6+1; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Spam-Score: -3.10 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=NEi+X6+1; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Queue-Id: 9B7EF9332 X-Spam-Score: -3.10 X-Migadu-Scanner: scn0.migadu.com X-TUID: j3t/HNKf0WeQ Tom Gillespie writes: [snip] > > Once I wrap up the formal grammar for org, one of the next things I > plan to work on is a clear specification for org babel. This is > critical because so many of the suggestions that come in deal with > individuals' specific problems and thus fail to account for how such > features interact with existing features and how the newly proposed > feature would block some other features in the future, confuse users, > etc. Such suggestions also often fail to account for increased > complexity, nor have they been exposed to a sufficient number of > examples to reveal fundamental ambiguities in how they could be > interpreted. The issues with variable behavior between ob langs for > :pre :post :prologue :epilogue etc. are already enough to keep us busy > for quite some time. > Yes, that clearly summs up my concerns as well. Often, a 'fix' for some problem can seem straight-forward, possibly even trivial, when considered only within one person's use case/workflow. Part of what makes org so powerful is the level of flexibility it supports. However, this flexibility makes it extremely difficult to consider, or even know about, all the existing use cases. I suspect that once we have some formal specifications for babel, the next step will be to develop some good unit tests to verify these specifications. This would at least highlight/alert developers to unforeseen impact from changes and alert them to things they may not have considered. > With regard to this thread in particular, it is of some interest, but > there are fundamental issues, including the fact that certain > languages (e.g. racket) expect module code to exist somewhere on the > file system. There are ways around many of these issues, in fact there > are likely many ways around any individual issue, so org babel needs > to systematically consider the issues and provide a clear > specification, or at least a guide for how such cases should be > handled. > > To give an example from one of my continual pain points: I start > writing python or racket in an org src block and then I want it to be > a library so that it can be reused by other code both inside and > outside the org file without having to resort to noweb. > This is an interesting point. I think a number of languages have challenges here. I run into very similar issues with Clojure. To some extent, I think this is a grey area within the literate programming paradigm. The original literate programming model was developed at a time when most languages were compiled rather then interpreted. You generated source code, compiled it and then ran it. These days, many 'modern' languages are based around an interpreter and concepts like 'just in time' compilation. In org mode, things become even more complex because in addition to generation (tangling) of code, we also want to have evaluation of code blocks, plus we have added the concept of 'sessions'. > What is the best way to handle this? I don't know. Right now I tangle > things and resort to awful hacks for the reuse-in-this-org-file case, but > I'm guessing there is a better generic solution which would allow _any_ > org block to be exported as a library instead of nowebbed in. > > Before jumping for any particular suggestion for how to handle this > we need to explore the diversity of cases that various ob langs > present, so that we can find a solution that will work for all of > them. After all, packaging code to a library for reuse is an > extremely common pattern that org babel should be able to > abstract, but it is a major undertaking, not just the addition of a > keyword here and there. > Agree. I'm not convinced we really understand the requirements here and more exploration and specification is required. The more we add 'simple' extensions, options, keywords etc the more likely it is we will make this a much harder process and will likely result in even larger 'breakage' once we do define a clearer specification. > In short I suggest that we issue a general moratorium on new org babel > feature suggestions until we can stabilize what we already have and > provide a clear specification for correct behavior. Until we have that spec > we could encourage users to create extensions that implement those > features. > Yes. While it might sound harsh or overly limiting, I do think such a moratorium may be required. We may be able to lift it once we have some core specifications in place. We would still accept bug fixes (though we may need to provide some clarity on what is a bug fix and what is a feature enhancement/change - I regularly see posts flagged as bug fixes which are actually feature enhancements or extensions). > > PS The other next thing that I am working on might be another way out > for this particular feature request. Namely, it is simplifying and > extending org keyword syntax so that new keywords (with options) and > associated keywords can be specified using keyword syntax within a > single org file. This would make it possible to get useful high level > keyword behavior in a single file without burdening the core > implementation with more special cases for associated keywords, and it > would allow users to write small elisp functions that could do some of > what is suggested here, all without need to add anything to the core > org implementation. Sounds interesting. To some extent, recognising there may be at least two types of code blocks (library code, executable/driver code) and at least two types of languages (compiled/interpreted) may help identify base code block permutation and required options, keywords etc. However, we are unlikely to find a definitive set which supports all use cases/workflows and the ability to easily extend/customise behaviours would be very useful. -- Tim Cross