Re: [FR] :noweb-wrap header arg

[Amy Grinn <grinn.amy@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2803 bytes --]

Ihor Radchenko <yantar92@posteo.net> writes:

> Amy Grinn <grinn.amy@gmail.com> writes:
>
>>> +1
>>> You may even use obsolete alias (add it to lisp/org-compat.el)
>>
>> Here's a patch to rename org-babel-noweb-wrap to
>> org-babel-noweb-make-regexp.
>
> Thanks!
> News entry is not necessary here - we are just renaming a function.
> Otherwise, the patch looks good.
> I am not yet merging it though - let's have all the patches you plan for
> the new noweb-wrap argument first and then apply them together.

Got it, not a problem!  I've attached both patches here.

Regarding the other conversation,

>>>> +        (while (< (point) (point-max))
>>>> +          (unless (looking-at " *\"\\([^\"]+\\)\" *")
>>>> +            (looking-at " *\\([^ ]+\\)"))
>>>
>>> May you please explain the rationale behind this regexp? AFAIU, it
>>> implies that you want to allow whitespace characters inside :noweb-wrap
>>> boundaries. But I do not think that we need to complicate things so much.
>>
>> That is exactly what I was going for.  I thought about the ways this
>> could be used and the most general-purpose, non-syntax-breaking,
>> easily-recognizable way I could think of was to use the language's
>> line-comment construct followed by the standard << >> characters.
>>
>> # <<foo>>
>> ;; <<bar>>
>> // <<baz>>
>>
>> I can see how it might be harder to maintain to allow whitespace in the
>> noweb-wrap header arg.  I could create a separate org-parse-arg-list
>> function to ease that burden somewhat.  My opinion is that having the
>> option to use the examples above is preferable to using non-standard
>> wrappers, from a third-person point-of-view.
>
> Makes sense. Also, see a similar idea being discussed in
> https://list.orgmode.org/orgmode/87o7jlzxgn.fsf@localhost/
>
> I recommend the following:
>
> If the value starts from ", use Elisp's `read':
>    |"# <<" ">>"
>    (read (current-buffer)) ; => "# <<"
>    otherwise, consider read until the first whitespace.
>    |#<<; >>;
>    (re-search-forward (rx (1+ (not whitespace))))
>    #<<;|
>
> However, there may be edge cases like
>
> "<< >>"
> "<< >>
> << << >>
> << "asd" >>

I didn't implement this recommendation yet; I still think the regex is a
more clear way of putting it, even without using a temporary buffer:

    ;; If a pair of " is found separated by one or more
    ;; characters, capture those characters as a group
    (unless (eq i (string-match (rx (* space) ?\"
                                    (group (+ (not ?\")))
                                    ?\" (* space))
                                raw i))
      ;; Otherwise, capture the next non-whitespace group of
      ;; characters
      (string-match (rx (* space) (group (* (not space))) (* space))
                    raw i))

Let me know what you think!

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: rename to org-babel-noweb-make-regexp --]
[-- Type: text/x-patch, Size: 4910 bytes --]

From 9442c029a7b2f1ec061e42a047b3d1bff88441d8 Mon Sep 17 00:00:00 2001
From: Amy Grinn <grinn.amy@gmail.com>
Date: Wed, 17 Apr 2024 16:01:40 -0400
Subject: [PATCH 1/2] lisp/ob-core.el: (org-babel-noweb-wrap): renamed to
 org-babel-noweb-make-regexp

* lisp/org-compat.el: Declare org-babel-noweb-wrap to be an obselete
function alias for org-babel-noweb-make-regexp.
* lisp/ob-core.el (org-babel-noweb-make-regexp): Rename the function.
(org-babel-goto-named-src-block):
(org-babel-expand-noweb-references):
* lisp/ob-exp.el (org-babel-exp-code):
* lisp/ob-tangle.el (org-babel-tangle-clean):
(org-babel-tangle-single-block): Use the new function name.
---
 lisp/ob-core.el    | 8 ++++----
 lisp/ob-exp.el     | 2 +-
 lisp/ob-tangle.el  | 5 +++--
 lisp/org-compat.el | 2 +-
 4 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index c5dd20b0e..1518d7726 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -194,7 +194,7 @@ This string must include a \"%s\" which will be replaced by the results."
   :package-version '(Org . "9.1")
   :safe #'booleanp)
 
-(defun org-babel-noweb-wrap (&optional regexp)
+(defun org-babel-noweb-make-regexp (&optional regexp)
   "Return regexp matching a Noweb reference.
 
 Match any reference, or only those matching REGEXP, if non-nil.
@@ -1976,7 +1976,7 @@ src block, then return nil."
 		   (type (org-element-type context))
 		   (noweb-ref
 		    (and (memq type '(inline-src-block src-block))
-			 (org-in-regexp (org-babel-noweb-wrap)))))
+			 (org-in-regexp (org-babel-noweb-make-regexp)))))
 	      (cond
 	       (noweb-ref
 		(buffer-substring
@@ -3125,7 +3125,7 @@ block but are passed literally to the \"example-block\"."
                                   (not (equal (cdr v) "no"))))))
 	 (noweb-re (format "\\(.*?\\)\\(%s\\)"
 			   (with-current-buffer parent-buffer
-			     (org-babel-noweb-wrap)))))
+			     (org-babel-noweb-make-regexp)))))
     (unless (equal (cons parent-buffer
                          (with-current-buffer parent-buffer
                            (buffer-chars-modified-tick)))
@@ -3175,7 +3175,7 @@ block but are passed literally to the \"example-block\"."
 	               ((guard (or org-babel-noweb-error-all-langs
 			           (member lang org-babel-noweb-error-langs)))
 	                (error "Cannot resolve %s (see `org-babel-noweb-error-langs')"
-		               (org-babel-noweb-wrap ,ref)))
+		               (org-babel-noweb-make-regexp ,ref)))
 	               (_ ""))))
       (replace-regexp-in-string
        noweb-re
diff --git a/lisp/ob-exp.el b/lisp/ob-exp.el
index 34f12fdcc..33de7a4aa 100644
--- a/lisp/ob-exp.el
+++ b/lisp/ob-exp.el
@@ -418,7 +418,7 @@ replaced with its value."
   (setf (nth 1 info)
 	(if (string= "strip-export" (cdr (assq :noweb (nth 2 info))))
 	    (replace-regexp-in-string
-	     (org-babel-noweb-wrap) "" (nth 1 info))
+	     (org-babel-noweb-make-regexp) "" (nth 1 info))
 	  (if (org-babel-noweb-p (nth 2 info) :export)
 	      (org-babel-expand-noweb-references
 	       info org-babel-exp-reference-buffer)
diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el
index 79fe6448b..4427250ae 100644
--- a/lisp/ob-tangle.el
+++ b/lisp/ob-tangle.el
@@ -412,7 +412,7 @@ references."
   (interactive)
   (goto-char (point-min))
   (while (or (re-search-forward "\\[\\[file:.*\\]\\[.*\\]\\]" nil t)
-             (re-search-forward (org-babel-noweb-wrap) nil t))
+             (re-search-forward (org-babel-noweb-make-regexp) nil t))
     (delete-region (save-excursion (forward-line) (point))
                    (save-excursion (end-of-line 1) (forward-char 1) (point)))))
 
@@ -580,7 +580,8 @@ non-nil, return the full association list to be used by
 	  ;; Run the tangle-body-hook.
           (let ((body (if (org-babel-noweb-p params :tangle)
                           (if (string= "strip-tangle" (cdr (assq :noweb (nth 2 info))))
-                            (replace-regexp-in-string (org-babel-noweb-wrap) "" (nth 1 info))
+                              (replace-regexp-in-string (org-babel-noweb-make-regexp)
+                                                        "" (nth 1 info))
 			    (org-babel-expand-noweb-references info))
 			(nth 1 info))))
 	    (with-temp-buffer
diff --git a/lisp/org-compat.el b/lisp/org-compat.el
index 92cad3d6e..ce46fd399 100644
--- a/lisp/org-compat.el
+++ b/lisp/org-compat.el
@@ -433,7 +433,7 @@ This is a floating point number if the size is too large for an integer."
 (define-obsolete-function-alias 'org-show-context 'org-fold-show-context "9.6")
 (define-obsolete-function-alias 'org-show-entry 'org-fold-show-entry "9.6")
 (define-obsolete-function-alias 'org-show-children 'org-fold-show-children "9.6")
-
+(define-obsolete-function-alias 'org-babel-noweb-wrap 'org-babel-noweb-make-regexp "9.7")
 
 (defmacro org-re (s)
   "Replace posix classes in regular expression S."
-- 
2.39.2


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: new header argument :noweb-wrap --]
[-- Type: text/x-patch, Size: 13399 bytes --]

From aad9b3926c711dfed411e2f439ae8b0fe3db7d10 Mon Sep 17 00:00:00 2001
From: Amy Grinn <grinn.amy@gmail.com>
Date: Wed, 17 Apr 2024 16:56:37 -0400
Subject: [PATCH 2/2] org-babel: New header argument :noweb-wrap

* lisp/ob-core.el (org-babel-get-noweb-wrap): New API function that
parses the :noweb-wrap parameter of a babel src block.
(org-babel-noweb-make-regexp): Add a new optional parameter 'wrap' to
control how noweb references are wrapped.
(org-babel-goto-named-src-block):
(org-babel-expand-noweb-references):
* lisp/ob-exp.el (org-babel-exp-code):
* lisp/ob-tangle.el (org-babel-tangle-single-block): Use the new
org-babel-get-noweb-wrap function as the new optional parameter to
org-babel-noweb-make-regexp.
* lisp/ob-tangle.el (org-babel-tangle-clean): Add an additional
warning about not being able to resolve noweb references if :noweb
wrap was specified.
* etc/ORG-NEWS (New =:noweb-wrap= babel header argument): Describe new
argument.
* testing/examples/babel.org:
* testing/lisp/test-ob-exp.el:
* testing/lisp/test-ob.el: Add tests which use alternate wrapping
syntax for noweb references.
---
 etc/ORG-NEWS                | 14 +++++++++
 lisp/ob-core.el             | 57 +++++++++++++++++++++++++++++++------
 lisp/ob-exp.el              |  3 +-
 lisp/ob-tangle.el           | 12 +++++---
 testing/examples/babel.org  | 17 +++++++++++
 testing/lisp/test-ob-exp.el | 55 +++++++++++++++++++++++++++++++++++
 testing/lisp/test-ob.el     | 32 +++++++++++++++++++++
 7 files changed, 176 insertions(+), 14 deletions(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 585b2b262..e1fb95744 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -589,6 +589,20 @@ Org mode provides 3 possible values for ~org-sort-function~:
    and on MacOS.
 3. Custom function, if the above does not fit the needs.
 
+*** New =:noweb-wrap= babel header argument
+
+This argument changes the default noweb reference syntax by masking
+the options ~org-babel-noweb-wrap-start~ and
+~org-babel-noweb-wrap-end~.
+
+=:noweb-wrap= takes two parameters, start and end, corresponding to
+each option.
+
+For example:
+: #+begin_src sh :noweb-wrap <<< >>>
+:   echo <<<message>>>
+: #+end_src
+
 *** =ob-latex= now uses a new option ~org-babel-latex-process-alist~ to generate png output
 
 Previously, =ob-latex= used ~org-preview-latex-default-process~ from
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 1518d7726..4f286c64c 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -194,15 +194,21 @@ This string must include a \"%s\" which will be replaced by the results."
   :package-version '(Org . "9.1")
   :safe #'booleanp)
 
-(defun org-babel-noweb-make-regexp (&optional regexp)
+(defun org-babel-noweb-make-regexp (&optional regexp wrap)
   "Return regexp matching a Noweb reference.
 
 Match any reference, or only those matching REGEXP, if non-nil.
 
+If WRAP is provided, it should be a list of 2 strings describing
+the start and end of a noweb reference, such as that returned by
+`org-babel-get-noweb-wrap'.  Otherwise
+`org-babel-noweb-wrap-start' and `org-babel-noweb-wrap-end' will
+be used.
+
 When matching, reference is stored in match group 1."
-  (concat (regexp-quote org-babel-noweb-wrap-start)
-	  (or regexp "\\([^ \t\n]\\(?:.*?[^ \t\n]\\)?\\)")
-	  (regexp-quote org-babel-noweb-wrap-end)))
+  (concat (regexp-quote (or (car wrap) org-babel-noweb-wrap-start))
+	    (or regexp "\\([^ \t\n]\\(?:.*?[^ \t\n]\\)?\\)")
+	    (regexp-quote (or (cadr wrap) org-babel-noweb-wrap-end))))
 
 (defvar org-babel-src-name-regexp
   "^[ \t]*#\\+name:[ \t]*"
@@ -1963,6 +1969,33 @@ src block, then return nil."
   (let ((head (org-babel-where-is-src-block-head)))
     (if head (goto-char head) (error "Not currently in a code block"))))
 
+(defun org-babel-get-noweb-wrap (&optional info)
+  "Retrieve a description the :noweb-wrap header arg from INFO.
+
+The description will be in the form of a list of two of strings
+for the start and end of a reference.  INFO can be the result of
+`org-babel-get-src-block-info' otherwise this function will parse
+info at point."
+  (unless info
+    (setq info (org-babel-get-src-block-info 'no-eval)))
+  (when-let* ((raw (cdr (assq :noweb-wrap (nth 2 info))))
+              (len (length raw)))
+    (let ((i 0) result)
+      (while (< i len)
+        ;; If a pair of " is found separated by one or more
+        ;; characters, capture those characters as a group
+        (unless (eq i (string-match (rx (* space) ?\"
+                                        (group (+ (not ?\")))
+                                        ?\" (* space))
+                                    raw i))
+          ;; Otherwise, capture the next non-whitespace group of
+          ;; characters
+          (string-match (rx (* space) (group (* (not space))) (* space))
+                        raw i))
+        (setq i (match-end 0))
+        (push (match-string 1 raw) result))
+      (reverse result))))
+
 ;;;###autoload
 (defun org-babel-goto-named-src-block (name)
   "Go to a source-code block with NAME."
@@ -1974,14 +2007,18 @@ src block, then return nil."
 	    "source-block name: " all-block-names nil t
 	    (let* ((context (org-element-context))
 		   (type (org-element-type context))
+                   (noweb-wrap (org-babel-get-noweb-wrap))
 		   (noweb-ref
 		    (and (memq type '(inline-src-block src-block))
-			 (org-in-regexp (org-babel-noweb-make-regexp)))))
+			 (org-in-regexp (org-babel-noweb-make-regexp
+                                         nil noweb-wrap)))))
 	      (cond
 	       (noweb-ref
 		(buffer-substring
-		 (+ (car noweb-ref) (length org-babel-noweb-wrap-start))
-		 (- (cdr noweb-ref) (length org-babel-noweb-wrap-end))))
+		 (+ (car noweb-ref) (length (or (car noweb-wrap)
+                                                org-babel-noweb-wrap-start)))
+		 (- (cdr noweb-ref) (length (or (cadr noweb-wrap)
+                                                org-babel-noweb-wrap-end)))))
 	       ((memq type '(babel-call inline-babel-call)) ;#+CALL:
 		(org-element-property :call context))
 	       ((car (org-element-property :results context))) ;#+RESULTS:
@@ -3125,7 +3162,8 @@ block but are passed literally to the \"example-block\"."
                                   (not (equal (cdr v) "no"))))))
 	 (noweb-re (format "\\(.*?\\)\\(%s\\)"
 			   (with-current-buffer parent-buffer
-			     (org-babel-noweb-make-regexp)))))
+			     (org-babel-noweb-make-regexp
+                              nil (org-babel-get-noweb-wrap info))))))
     (unless (equal (cons parent-buffer
                          (with-current-buffer parent-buffer
                            (buffer-chars-modified-tick)))
@@ -3175,7 +3213,8 @@ block but are passed literally to the \"example-block\"."
 	               ((guard (or org-babel-noweb-error-all-langs
 			           (member lang org-babel-noweb-error-langs)))
 	                (error "Cannot resolve %s (see `org-babel-noweb-error-langs')"
-		               (org-babel-noweb-make-regexp ,ref)))
+		               (org-babel-noweb-make-regexp
+                                ,ref (org-babel-get-noweb-wrap))))
 	               (_ ""))))
       (replace-regexp-in-string
        noweb-re
diff --git a/lisp/ob-exp.el b/lisp/ob-exp.el
index 33de7a4aa..14a32a8e6 100644
--- a/lisp/ob-exp.el
+++ b/lisp/ob-exp.el
@@ -418,7 +418,8 @@ replaced with its value."
   (setf (nth 1 info)
 	(if (string= "strip-export" (cdr (assq :noweb (nth 2 info))))
 	    (replace-regexp-in-string
-	     (org-babel-noweb-make-regexp) "" (nth 1 info))
+	     (org-babel-noweb-make-regexp nil (org-babel-get-noweb-wrap info))
+             "" (nth 1 info))
 	  (if (org-babel-noweb-p (nth 2 info) :export)
 	      (org-babel-expand-noweb-references
 	       info org-babel-exp-reference-buffer)
diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el
index 4427250ae..5fcb40443 100644
--- a/lisp/ob-tangle.el
+++ b/lisp/ob-tangle.el
@@ -406,9 +406,11 @@ Did you give the decimal value %1$d by mistake?" mode)))
   "Remove comments inserted by `org-babel-tangle'.
 Call this function inside of a source-code file generated by
 `org-babel-tangle' to remove all comments inserted automatically
-by `org-babel-tangle'.  Warning, this comment removes any lines
+by `org-babel-tangle'.  Warning, this command removes any lines
 containing constructs which resemble Org file links or noweb
-references."
+references.  It also cannot determine which noweb syntax is being
+used for any given source file, if :noweb-wrap was specified in
+the original Org file."
   (interactive)
   (goto-char (point-min))
   (while (or (re-search-forward "\\[\\[file:.*\\]\\[.*\\]\\]" nil t)
@@ -580,8 +582,10 @@ non-nil, return the full association list to be used by
 	  ;; Run the tangle-body-hook.
           (let ((body (if (org-babel-noweb-p params :tangle)
                           (if (string= "strip-tangle" (cdr (assq :noweb (nth 2 info))))
-                              (replace-regexp-in-string (org-babel-noweb-make-regexp)
-                                                        "" (nth 1 info))
+                              (replace-regexp-in-string
+			       (org-babel-noweb-make-regexp
+				nil (org-babel-get-noweb-wrap info))
+			       "" (nth 1 info))
 			    (org-babel-expand-noweb-references info))
 			(nth 1 info))))
 	    (with-temp-buffer
diff --git a/testing/examples/babel.org b/testing/examples/babel.org
index d46afeb5e..680d4bf3e 100644
--- a/testing/examples/babel.org
+++ b/testing/examples/babel.org
@@ -346,6 +346,23 @@ Here is a call line with more than just the results exported.
   echo "1$i"
 #+END_SRC
 
+* strip noweb references with alternative wrap
+  :PROPERTIES:
+  :ID:       da9bcfdd-c1bd-47b4-b520-67974b9f9856
+  :END:
+
+#+name: strip-export-2
+#+BEGIN_SRC sh :exports none
+  i="10"
+#+END_SRC
+
+#+RESULTS: strip-export-2
+
+#+BEGIN_SRC sh :noweb strip-export :noweb-wrap #[[ ]] :exports code :results silent
+  #[[strip-export-2]]
+  echo "1$i"
+#+END_SRC
+
 * use case of reading entry properties
   :PROPERTIES:
   :ID:       cc5fbc20-bca5-437a-a7b8-2b4d7a03f820
diff --git a/testing/lisp/test-ob-exp.el b/testing/lisp/test-ob-exp.el
index d029dadfb..abfda4368 100644
--- a/testing/lisp/test-ob-exp.el
+++ b/testing/lisp/test-ob-exp.el
@@ -394,6 +394,61 @@ be evaluated."
 		 (regexp-quote " :foo  :bar \n")
 		 ascii))))))
 
+(ert-deftest ob-exp/noweb-wrap-header-arg ()
+  (let ((org-export-use-babel t))
+    (org-test-with-temp-text
+	"
+#+Title: exporting from a temporary buffer
+
+#+name: foo
+#+BEGIN_SRC emacs-lisp
+  :foo
+#+END_SRC
+
+#+BEGIN_SRC emacs-lisp :noweb yes :noweb-wrap {{ }} :exports results
+  (list {{foo}})
+#+END_SRC
+"
+      (let* ((ascii (org-export-as 'ascii)))
+	(should (string-match
+		 (regexp-quote " :foo \n")
+		 ascii))))))
+
+(ert-deftest ob-exp/noweb-strip-export-with-wrap ()
+  (org-test-at-id "da9bcfdd-c1bd-47b4-b520-67974b9f9856"
+    (org-narrow-to-subtree)
+    (org-babel-next-src-block 1)
+    (org-babel-execute-src-block)
+    (let ((result (org-test-with-expanded-babel-code (buffer-string))))
+      (should-not (string-match (regexp-quote "#[[strip-export-2]]") result))
+      (should-not (string-match (regexp-quote "i=\"10\"") result)))))
+
+(ert-deftest ob-exp/noweb-wrap-strip-export ()
+  (let ((org-export-use-babel t))
+    (org-test-with-temp-text
+	"
+#+Title: exporting from a temporary buffer
+
+#+name: foo
+#+BEGIN_SRC emacs-lisp
+  :foo
+#+END_SRC
+
+#+name: bar
+#+BEGIN_SRC emacs-lisp
+  :bar
+#+END_SRC
+
+#+BEGIN_SRC emacs-lisp :noweb yes :noweb-wrap {{ }} :exports results
+  (list {{foo}} {{bar}})
+#+END_SRC
+"
+      (let* ((ascii (org-export-as 'ascii)))
+        
+	(should (string-match
+		 (regexp-quote ":foo  :bar")
+		 ascii))))))
+
 (ert-deftest ob-export/export-with-results-before-block ()
   "Test export when results are inserted before source block."
   (let ((org-export-use-babel t))
diff --git a/testing/lisp/test-ob.el b/testing/lisp/test-ob.el
index 544e68267..b0051d2a2 100644
--- a/testing/lisp/test-ob.el
+++ b/testing/lisp/test-ob.el
@@ -988,6 +988,38 @@ x
             (search-forward "begin_src")
             (org-babel-expand-noweb-references)))))
 
+(ert-deftest test-ob/noweb-wrap ()
+  ;; Standard test.
+  (should
+   (string=
+    "bar"
+    (org-test-with-temp-text "#+begin_src sh :results output :tangle yes :noweb-wrap <<< >>>
+  <<<foo>>>
+#+end_src
+
+#+name: foo
+#+begin_src sh
+  bar
+#+end_src"
+        (org-babel-expand-noweb-references)))))
+
+(ert-deftest test-ob/get-noweb-wrap ()
+  (should (equal '("<< >>")
+                 (org-babel-get-noweb-wrap
+                  '(nil nil ((:noweb-wrap . "\"<< >>\""))))))
+  (should (equal '("\"<<" ">>")
+                 (org-babel-get-noweb-wrap
+                  '(nil nil ((:noweb-wrap . "\"<< >>"))))))
+  (should (equal '("<<" "<<" ">>")
+                 (org-babel-get-noweb-wrap
+                  '(nil nil ((:noweb-wrap . "<< << >>"))))))
+  (should (equal '("<<" "asd" ">>")
+                 (org-babel-get-noweb-wrap
+                  '(nil nil ((:noweb-wrap . "<< \"asd\" >>"))))))
+  (should (equal '("\"\"\"" ">>>")
+                 (org-babel-get-noweb-wrap
+                  '(nil nil ((:noweb-wrap . "\"\"\" >>>")))))))
+
 (ert-deftest test-ob/splitting-variable-lists-in-references ()
   (org-test-with-temp-text ""
     (should (= 1 (length (org-babel-ref-split-args
-- 
2.39.2


[-- Attachment #4: Type: text/plain, Size: 14 bytes --]

--
Best,

Amy