From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:306:2d92::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id wFA4OTZvvmQjAAAASxT56A (envelope-from ) for ; Mon, 24 Jul 2023 14:31:51 +0200 Received: from aspmx1.migadu.com ([2001:41d0:306:2d92::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id CCOXODZvvmTrkAAAauVa8A (envelope-from ) for ; Mon, 24 Jul 2023 14:31:50 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 9A1FF39AF6 for ; Mon, 24 Jul 2023 14:31:50 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=lipklim.org header.s=key1 header.b=4J0ILutJ; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=quarantine) header.from=lipklim.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1690201910; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=bn0jTLbgaDpISB7+UDMRfVBGqkXznJr6BFFNY4yVvR8=; b=GihdY1Tm255onL8w+sHUDvmKvHn1Ha67AkoO4lU10H3Nhs2/GqaezjrRr2a6oxSj/kbjio KqQd23/d9j+8EJ8uQnk+LbHLzEmNU1WfLSpkxn8GHF59EAEz50ZFUndts3scupZlTZ6Ub8 ZwWSH7tzKTtK0e3xr2OfxunfE5udKTJWnJnavbn29izE9upTCSjuVR/48NE20doPP9UycM A7+63YbKzsOHP215Hsifla1J7S6Trrau0nzylCY7/Z5HTgFv/stpcjTb3b1c5YKsPHX0GR Bte/TPEzA5mxt0c0/WiFID+cfEq2zf7wrG7vRxpg4ryvhQKC/xxA+CF3PukdgQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1690201910; a=rsa-sha256; cv=none; b=XD6xRDu3xminpFIukWgPZIfTE/u1+lqtYK8b+xolMGxh1RJQi4PM7zlHWeMJlRQvcxQyyG AaILof/BTy54EqnfPLZczgnGEemITWfm+C/ht0wuuGrXAFHkkEcsc3ATb7x0SHOKZvZ9aP 7Bl+ozWjQQ88moctMZEwQmbXRfSRIPHNJ6QSUj4zAo1LFd1Mlvwtx80FgwnlstPIhi1dt6 t0TBl2KMxJ40TU3LyqlvGucf1EmsQxR3iHQGa0SNB3VfXyAkG0B+5qoF+qAConuDl+v7kv 7jjNNfU8ARX8UDp3t+dLwFFPeIAHtkk8+0Vm4QlJumnvcfJWdT7EAmtM1t1uWg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=lipklim.org header.s=key1 header.b=4J0ILutJ; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=quarantine) header.from=lipklim.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qNuiI-0001Hk-MA; Mon, 24 Jul 2023 08:30:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qNuiA-0001Fz-BT for emacs-orgmode@gnu.org; Mon, 24 Jul 2023 08:30:53 -0400 Received: from out-7.mta1.migadu.com ([2001:41d0:203:375::7]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qNui0-0000vW-1g for emacs-orgmode@gnu.org; Mon, 24 Jul 2023 08:30:48 -0400 References: <87lefkd9tb.fsf@lipklim.org> <87h6q8d9lz.fsf@lipklim.org> <87cz0wcbtt.fsf@lipklim.org> <87ttu6j2zr.fsf@localhost> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lipklim.org; s=key1; t=1690201833; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=bn0jTLbgaDpISB7+UDMRfVBGqkXznJr6BFFNY4yVvR8=; b=4J0ILutJA+ix+Gs7hweBiLUSgVxkr23f1XT7+gogUejzxmfeoLHTzHqvLufBlHyCNTJYVe KmNiHIrirtveUq5J951eladNDq7f32sX5gygU4QEEJArkUW07Pd644ROqK4LcXxlr9SgMa w14j3pLEIqoMbXbTIezP1fFjBZlfFL8= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Evgenii Klimov To: Ihor Radchenko Cc: emacs-orgmode@gnu.org Subject: [PATCH v3] ob-tangle.el: Blocks overwrite each other when grouping before tangling Date: Mon, 24 Jul 2023 13:28:11 +0100 In-reply-to: <87ttu6j2zr.fsf@localhost> Message-ID: <87bkg1h4q3.fsf@lipklim.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=2001:41d0:203:375::7; envelope-from=eugene.dev@lipklim.org; helo=out-7.mta1.migadu.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Scanner: mx2.migadu.com X-Migadu-Spam-Score: -5.00 X-Spam-Score: -5.00 X-Migadu-Queue-Id: 9A1FF39AF6 X-TUID: Fstn/NcsgJE2 --=-=-= Content-Type: text/plain Hi Here are the new tests that demonstrate the bug in block grouping during block collection, along with the patch to address the issue, taking your previous remarks into account. I split it into two patches so you can apply the tests first to see the bug. And probably tests should be rewritten as they look too complex and mostly duplicate each other. I'd appreciate your suggestions on how to enhance them. Ihor Radchenko writes: > Evgenii Klimov writes: > >> In this version I just updated the docstrings for the relevant >> functions, because prior to that it wasn't clear: does this "default >> export file for *all* source blocks" influence blocks with :tangle >> "yes"/FILENAME? > > Thanks for the patch, but we need to be careful changing things in > ob-tangle. Not everything is well-documented there. > >> Optional argument TARGET-FILE can be used to specify a default >> -export file for all source blocks. >> +export file for all source blocks without :tangle header >> +argument. > > This is confusing. > Is :tangle yes "without"? > What about inheritance? > What about default header args? I just find current lack of details confusing as well and want to express the place of TARGET-FILE in the lineage of :tangle in ~org-babel-get-src-block-info~: 1. org-babel-default-header-args 1. TANGLE-FILE of ~org-babel-tangle~ 2. org-babel-default-header-args: 3. org-babel-params-from-properties 4. org-element-property :parameters datum 5. org-element-property :header datum It wasn't clear for me: will ":tangle yes" or explicit ":tangle no" be affected by TARGET-FILE. Maybe if we rephrase as follows it will be clear for both of us: Optional argument TARGET-FILE can be used to overwrite a default export file in `org-babel-default-header-args' for all source blocks. > What if we have :tangle "/path/to/foo" and TARGET-FILE = "/path/to/foo"? > What if they are :tangle "./foo" and TARGET-FILE = "/full/path/to/foo"? See the new tests in the patch, I tried to take it into account. >> (defun org-babel-effective-tangled-filename (buffer-fn src-lang src-tfile) >> - "Return effective tangled filename of a source-code block. >> + "Return effective tangled absolute filename of a source-code block. > > This will likely cause breakage. > There are two callers of `org-babel-effective-tangled-filename: > 1. `org-babel-tangle-collect-blocks' > 2. `org-babel-tangle-single-block' > > `org-babel-tangle-single-block' passes (nth 1 result) as BUFFER-FN. > Its value is > > (if org-babel-tangle-use-relative-file-links > (file-relative-name file) > file) > > So, > >> + (let* ((fnd (file-name-directory (buffer-file-name >> + (get-buffer buffer-fn)))) > > will fail when FILE contains file path. > And it does: (file (buffer-file-name (buffer-base-buffer))) Thanks, fixed: both `org-babel-tangle-single-block' and `org-babel-tangle-collect-blocks' now pass absolute value to `org-babel-effective-tangled-filename'. --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v3-0001-testing-lisp-test-ob-tangle.el-Test-block-collect.patch >From 7235bf0306d12f6644838ad8542ac8822bcde258 Mon Sep 17 00:00:00 2001 From: Evgenii Klimov Date: Fri, 21 Jul 2023 22:40:06 +0100 Subject: [PATCH v3 1/2] testing/lisp/test-ob-tangle.el: Test block collection into groups for tangling * testing/lisp/test-ob-tangle.el (ob-tangle/collect-blocks): Test block collection into groups for tangling. (ob-tangle/collect-blocks-with-target-file): The same but with TARGET-FILE. --- testing/examples/babel.org | 75 ++++++++++++++++++++++++++++++++++ testing/lisp/test-ob-tangle.el | 60 +++++++++++++++++++++++++++ 2 files changed, 135 insertions(+) diff --git a/testing/examples/babel.org b/testing/examples/babel.org index d46afeb5e..2d7b39d4e 100644 --- a/testing/examples/babel.org +++ b/testing/examples/babel.org @@ -490,3 +490,78 @@ The =[[= causes a false positive which ~org-babel-detangle~ should handle proper :END: #+begin_src emacs-lisp :tangle yes :comments link #+end_src +* tangle collect blocks +:PROPERTIES: +:ID: fae6bb5b-555a-4d68-9658-a30ac5d1b2ba +:END: +** with :tangle in properties +:PROPERTIES: +:ID: b2021d51-253c-4b26-9988-dac9193eb00b +:header-args: :tangle relative.el +:END: +#+begin_src emacs-lisp +"H1: no :tangle, but :tangle relative.el in properties" +#+end_src + +#+begin_src emacs-lisp :tangle yes +"H1: :tangle yes (to babel.el)" +#+end_src + +#+begin_src emacs-lisp :tangle no +"H1: should be ignored" +#+end_src + +#+begin_src emacs-lisp :tangle ./babel.el +"H1: :tangle ./babel.el" +#+end_src + +#+begin_src emacs-lisp :tangle relative.el +"H1: :tangle relative.el" +#+end_src + +#+begin_src emacs-lisp :tangle ./relative.el +"H1: :tangle ./relative.el" +#+end_src + +#+begin_src emacs-lisp :tangle /tmp/absolute.el +"H1: :tangle /tmp/absolute.el" +#+end_src + +#+begin_src emacs-lisp :tangle ~/../../tmp/absolute.el +"H1: :tangle ~/../../tmp/absolute.el" +#+end_src +** without :tangle in properties +:PROPERTIES: +:ID: 9f9afb0e-ba6d-4f63-9735-71af48ecd4e6 +:END: +#+begin_src emacs-lisp +"H2: no :tangle" +#+end_src + +#+begin_src emacs-lisp :tangle yes +"H2: :tangle yes (to babel.el)" +#+end_src + +#+begin_src emacs-lisp :tangle no +"H2: should be ignored" +#+end_src + +#+begin_src emacs-lisp :tangle babel.el +"H2: :tangle babel.el" +#+end_src + +#+begin_src emacs-lisp :tangle relative.el +"H2: :tangle relative.el" +#+end_src + +#+begin_src emacs-lisp :tangle ./relative.el +"H2: :tangle ./relative.el" +#+end_src + +#+begin_src emacs-lisp :tangle /tmp/absolute.el +"H2: :tangle /tmp/absolute.el" +#+end_src + +#+begin_src emacs-lisp :tangle ~/../../tmp/absolute.el +"H2: :tangle ~/../../tmp/absolute.el" +#+end_src diff --git a/testing/lisp/test-ob-tangle.el b/testing/lisp/test-ob-tangle.el index 07e75f4d3..09eeffed7 100644 --- a/testing/lisp/test-ob-tangle.el +++ b/testing/lisp/test-ob-tangle.el @@ -569,6 +569,66 @@ another block (set-buffer-modified-p nil)) (kill-buffer buffer)))) +(ert-deftest ob-tangle/collect-blocks () + "Test block collection into groups for tangling." + (org-test-at-id "fae6bb5b-555a-4d68-9658-a30ac5d1b2ba" + (org-narrow-to-subtree) + (let ((expected-targets (cons '("/tmp/absolute.el" . 4) + (map-apply + (lambda (file numblocks) + (cons (expand-file-name file + org-test-example-dir) + numblocks)) + '(("relative.el" . 5) ("babel.el" . 4))))) + (collected-blocks (org-babel-tangle-collect-blocks))) + (should (= (length expected-targets) + (length (map-keys collected-blocks)))) + (let ((collected-targets (map-apply (lambda (file blocks) ; full blocks itself + (cons (expand-file-name file + org-test-example-dir) + (length blocks))) + collected-blocks))) + (should (equal (length expected-targets) + (length (map-filter + (lambda (file numblocks) + (= numblocks + (cdr (assoc-string file collected-targets)))) + expected-targets)))))))) + +(ert-deftest ob-tangle/collect-blocks-with-target-file () + "Test block collection into groups for tangling with TARGET-FILE +as `org-babel-tangle' would do." + (org-test-at-id "fae6bb5b-555a-4d68-9658-a30ac5d1b2ba" + (org-narrow-to-subtree) + (let* ((expected-targets (cons '("/tmp/absolute.el" . 4) + (map-apply + (lambda (file numblocks) + (cons (expand-file-name file + org-test-example-dir) + numblocks)) + '(("relative.el" . 5) ("babel.el" . 5))))) + ;; simulate `org-babel-tangle' and `org-babel-load-file' + ;; TARGET-FILE + (org-babel-default-header-args + (org-babel-merge-params + org-babel-default-header-args + (list (cons :tangle + (expand-file-name "babel.el" org-test-example-dir))))) + (collected-blocks (org-babel-tangle-collect-blocks))) + (should (= (length expected-targets) + (length (map-keys collected-blocks)))) + (let ((collected-targets (map-apply (lambda (file blocks) ; full blocks itself + (cons (expand-file-name file + org-test-example-dir) + (length blocks))) + collected-blocks))) + (should (equal (length expected-targets) + (length (map-filter + (lambda (file numblocks) + (= numblocks + (cdr (assoc-string file collected-targets)))) + expected-targets)))))))) + (provide 'test-ob-tangle) ;;; test-ob-tangle.el ends here -- 2.34.1 --=-=-= Content-Type: message/external-body; name="/home/eugene/git/org-mode/patches/v3-0002-ob-tangle.el-Avoid-relative-file-names-when-group.patch"; access-type=local-file Content-Type: text/x-diff Content-ID: <87a5vlh4q3.fsf@lipklim.org> Content-Transfer-Encoding: binary --=-=-=--