Re: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments

["Sébastien Miquel" <sebastien.miquel@posteo.eu>]

[-- Attachment #1.1: Type: text/plain, Size: 1788 bytes --]

Hi,

Ihor Radchenko writes:
> prin1-to-string is too specific and only solves a single use-case.
prin1-to-string is actually universal in a way, since any other
manipulation can then be achieved with

: (setq var (do-something <<nw>>))

at least as long as you're tangling to a programming language, that
can read lisp strings.
> Consider the following example:
>
> #+BEGIN_SRC emacs-lisp :noweb yes :tangle yes :noweb-prefix no :noweb-trans prin1-to-string
> <<preamble>>
> (setq latex-header <<nw>>)
> #+END_SRC
>
> There are two noweb references here. Setting source block-wide
> :noweb-trans is not helpful because the first reference will be
> incorrectly filtered through prin1-to-string.
Indeed. Originally I had thought of adding a new syntax <<"nw">> to
insert a string representation. I've attached a new patch, that does
this instead of introducing :noweb-trans. Now that I think of the
universality of prin1-to-string, I actually like it slightly better
than :noweb-trans. It would break existing "nw"-like noweb references.

Of course, one can work around this easily enough by using two blocks.
> I'd rather introduce a new syntax to transform the noweb reference
> inline. Something like
>
> #+BEGIN_SRC emacs-lisp :noweb yes :tangle yes :noweb-prefix no
> <<preamble>>
> (setq latex-header <<(prin1-to-string nw)>>)
> #+END_SRC
You'd need to only allow a single function call with only one
argument, or use something like <<(prin1-to-string <<nw>>)>>. The
change would be much more complex than what I propose, for maybe
little benefit.
>> [...]
> This sounds a bit confusing. I would also add an example where it is
> useful to set :noweb-prefix to no.

I've added such an example in the revised patch attached.

Thanks for the feedback.

Regards,

-- 
Sébastien Miquel

[-- Attachment #1.2: Type: text/html, Size: 3394 bytes --]

[-- Attachment #2: 0001-ob-core.el-Add-noweb-prefix-noweb-trans-babel-header.patch --]
[-- Type: text/x-patch, Size: 6090 bytes --]

From 99d043b9d837a2658e60fb4b4913454d9566519b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Mon, 6 Sep 2021 18:45:42 +0200
Subject: [PATCH] ob-core.el: Add `:noweb-prefix`, `:noweb-trans` babel header
 arguments

* lisp/ob-core.el (org-babel-expand-noweb-references): Add support for
`noweb-prefix' header argument, to not repeat the prefix characters
when expanding a noweb reference. Add support for `noweb-trans' header
argument, to apply a function to the noweb content upon
expansion.
(org-babel-common-header-args-w-values):
(org-babel-safe-header-args): Add `noweb-prefix' and `noweb-trans' values.
* doc/org-manual.org: Document `noweb-prefix' and `noweb-trans' babel header
arguments.
* etc/ORG-NEWS: Document `:noweb-prefix' and `:noweb-trans'.
---
 doc/org-manual.org | 42 ++++++++++++++++++++++++++++++++++++++++++
 etc/ORG-NEWS       | 10 +++++++++-
 lisp/ob-core.el    | 26 ++++++++++++++++++++------
 3 files changed, 71 insertions(+), 7 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 6768ca98d..5ef8e2f8b 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -18760,6 +18760,48 @@ else:
     print('do things when false')
 #+end_example
 
+This prefix behavior can be turned off in a block by setting the
+=noweb-prefix= header argument to =no=, as in:
+
+#+begin_example
+,#+BEGIN_SRC elisp :noweb-prefix no
+  (setq example-data "<<example>>")
+,#+END_SRC
+#+end_example
+
+#+texinfo: @noindent
+which expands to:
+
+#+begin_example
+(setq example-data "this is the
+multi-line body of example")
+#+end_example
+
+The header argument =noweb-trans= can be set to =prin1-to-string= to
+insert a lisp string representing the content of the referenced src
+block. With:
+
+#+begin_example
+,#+NAME: latex-header
+,#+BEGIN_SRC latex
+  \usepackage{amsmath}
+,#+END_SRC
+#+end_example
+
+#+texinfo: @noindent
+this code block:
+
+#+begin_example
+,#+BEGIN_SRC elisp :noweb yes :noweb-trans prin1-to-string
+  (setq header <<latex-header>>)
+,#+END_SRC
+#+end_example
+
+#+texinfo: @noindent
+expands to:
+
+: (setq header "\\usepackage{amsmath}")
+
 When in doubt about the outcome of a source code block expansion, you
 can preview the results with the following command:
 
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 2b539d305..70f7606db 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -150,7 +150,7 @@ The entry points are ~org-persist-register~, ~org-persist-unregister~,
 ~org-persist-read~, and ~org-persist-read-all~.  Storing circular
 structures is supported.  Storing references between different
 variables is also supported (see =:inherit= key in
-~org-persist-register~).  
+~org-persist-register~).
 
 The library permits storing buffer-local variables.  Such variables
 are linked to the buffer text, file =inode=, and file path.
@@ -175,6 +175,14 @@ the =compact-itemx= export option, or globally using the
 Items in a description list that begin with =Function:=, =Variable:=
 or certain related prefixes are converted using Texinfo definition
 commands.
+*** New =:noweb-prefix= and =:noweb-trans= babel header arguments
+
+=:noweb-prefix= can be set to =no= to prevent the prefix characters
+from being repeated when expanding a multiline noweb reference.
+
+=:noweb-trans= can be set to =prin1-to-string=. Noweb reference
+therein will be expanded to an elisp string representation of their
+content.
 
 ** New functions and changes in function arguments
 
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 6590eeee7..b5fb68661 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -413,6 +413,8 @@ then run `org-babel-switch-to-session'."
     (noweb	. ((yes no tangle no-export strip-export)))
     (noweb-ref	. :any)
     (noweb-sep  . :any)
+    (noweb-prefix . ((no yes)))
+    (noweb-trans  . ((prin1-to-string)))
     (output-dir . :any)
     (padline	. ((yes no)))
     (post       . :any)
@@ -438,9 +440,10 @@ specific header arguments as well.")
 
 (defconst org-babel-safe-header-args
   '(:cache :colnames :comments :exports :epilogue :hlines :noeval
-	   :noweb :noweb-ref :noweb-sep :padline :prologue :rownames
-	   :sep :session :tangle :wrap
+	   :noweb :noweb-ref :noweb-sep :noweb-prefix :padline
+           :prologue :rownames :sep :session :tangle :wrap
 	   (:eval . ("never" "query"))
+           (:noweb-trans . ("prin1-to-string"))
 	   (:results . (lambda (str) (not (string-match "file" str)))))
   "A list of safe header arguments for babel source blocks.
 
@@ -2827,6 +2830,12 @@ block but are passed literally to the \"example-block\"."
          (lang (nth 0 info))
          (body (nth 1 info))
 	 (comment (string= "noweb" (cdr (assq :comments (nth 2 info)))))
+	 (noweb-trans (when (cdr (assq :noweb-trans (nth 2 info)))
+                        (intern (cdr (assq :noweb-trans (nth 2 info))))))
+         (noweb-prefix (let ((v (assq :noweb-prefix (nth 2 info))))
+                         (or (not v)
+                             (and (org-not-nil (cdr v))
+                                  (not (equal (cdr v) "no"))))))
 	 (noweb-re (format "\\(.*?\\)\\(%s\\)"
 			   (with-current-buffer parent-buffer
 			     (org-babel-noweb-wrap))))
@@ -2921,11 +2930,16 @@ block but are passed literally to the \"example-block\"."
 			  (let* ((info (org-babel-get-src-block-info t))
 				 (ref (cdr (assq :noweb-ref (nth 2 info)))))
 			    (push info (gethash ref cache))))))
-		     (funcall expand-references id cache)))))
+		     (funcall expand-references id cache))))
+                  (expansion (if (functionp noweb-trans)
+                                 (funcall noweb-trans expansion)
+                               expansion)))
 	     ;; Interpose PREFIX between every line.
-	     (mapconcat #'identity
-			(split-string expansion "[\n\r]")
-			(concat "\n" prefix))))))
+	     (if noweb-prefix
+                 (mapconcat #'identity
+			    (split-string expansion "[\n\r]")
+			    (concat "\n" prefix))
+               expansion)))))
      body t t 2)))
 
 (defun org-babel--script-escape-inner (str)
-- 
2.36.0


[-- Attachment #3: 0001-ob-core.el-noweb-prefix-header-argument-and-nw-synta.patch --]
[-- Type: text/x-patch, Size: 7143 bytes --]

From 8fdd163b3f5c8049c67a6f3eab4587cac4a6d04a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Mon, 6 Sep 2021 18:45:42 +0200
Subject: [PATCH] ob-core.el: `:noweb-prefix` header argument and <<"nw">>
 syntax

* lisp/ob-core.el (org-babel-expand-noweb-references): Add support for
`noweb-prefix' header argument, to not repeat the prefix characters
when expanding a noweb reference. Add support for <<"nw">> noweb
syntax, to insert a lisp string representation of the content.
(org-babel-common-header-args-w-values):
(org-babel-safe-header-args): Add `noweb-prefix' values.
(org-babel-noweb-wrap): Add support for <<"nw">> syntax.
* doc/org-manual.org: Document `noweb-prefix' and <<"nw">> noweb
syntax.
* etc/ORG-NEWS: Document `:noweb-prefix' and <<"nw">> noweb syntax.
---
 doc/org-manual.org | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 etc/ORG-NEWS       | 25 ++++++++++++++++++++++++-
 lisp/ob-core.el    | 34 +++++++++++++++++++++++-----------
 3 files changed, 93 insertions(+), 12 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 6768ca98d..2e1435279 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -18760,6 +18760,52 @@ else:
     print('do things when false')
 #+end_example
 
+This prefix behavior can be turned off in a block by setting the
+=noweb-prefix= header argument to =no=, as in:
+
+#+begin_example
+,#+BEGIN_SRC elisp :noweb-prefix no
+  (setq example-data "<<example>>")
+,#+END_SRC
+#+end_example
+
+#+texinfo: @noindent
+which expands to:
+
+#+begin_example
+(setq example-data "this is the
+multi-line body of example")
+#+end_example
+
+Instead of inserting the content of a block, one may insert a lisp
+string representation of the content using the syntax
+
+: <<"CODE-BLOCK-ID">>
+
+#+texinfo: @noindent
+With:
+
+#+begin_example
+,#+NAME: latex-header
+,#+BEGIN_SRC latex
+  \usepackage{amsmath}
+,#+END_SRC
+#+end_example
+
+#+texinfo: @noindent
+this code block:
+
+#+begin_example
+,#+BEGIN_SRC elisp :noweb yes
+  (setq header <<"latex-header">>)
+,#+END_SRC
+#+end_example
+
+#+texinfo: @noindent
+expands to:
+
+: (setq header "\\usepackage{amsmath}")
+
 When in doubt about the outcome of a source code block expansion, you
 can preview the results with the following command:
 
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 2b539d305..3a8229d73 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -150,7 +150,7 @@ The entry points are ~org-persist-register~, ~org-persist-unregister~,
 ~org-persist-read~, and ~org-persist-read-all~.  Storing circular
 structures is supported.  Storing references between different
 variables is also supported (see =:inherit= key in
-~org-persist-register~).  
+~org-persist-register~).
 
 The library permits storing buffer-local variables.  Such variables
 are linked to the buffer text, file =inode=, and file path.
@@ -176,6 +176,29 @@ Items in a description list that begin with =Function:=, =Variable:=
 or certain related prefixes are converted using Texinfo definition
 commands.
 
+*** New =:noweb-prefix= babel header argument
+
+=:noweb-prefix= can be set to =no= to prevent the prefix characters
+from being repeated when expanding a multiline noweb reference.
+
+*** New syntax to insert a string representation of a noweb reference
+
+Use =<<"foo">>= to insert a lisp string representing the content of
+the =foo= block. For example:
+
+#+BEGIN_SRC org
+  ,#+BEGIN_SRC LaTeX :noweb-ref foo
+    \usepackage{bar}
+  ,#+END_SRC
+
+  ,#+BEGIN_SRC emacs-lisp :noweb yes
+    (setq latex-header <<"foo">>)
+  ,#+END_SRC
+#+END_SRC
+
+will expand to
+: (setq latex-header "\\usepackage{bar}")
+
 ** New functions and changes in function arguments
 
 *** New function ~org-element-cache-map~ for quick mapping across Org elements
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 6590eeee7..cac1a4162 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -187,7 +187,9 @@ Match any reference, or only those matching REGEXP, if non-nil.
 
 When matching, reference is stored in match group 1."
   (concat (regexp-quote org-babel-noweb-wrap-start)
-	  (or regexp "\\([^ \t\n]\\(?:.*?[^ \t\n]\\)?\\)")
+          "\\(?2:\"?\\)"
+	  (or regexp "\\(?1:[^ \t\n]\\(?:.*?[^ \t\n]\\)?\\)")
+          "\\2"
 	  (regexp-quote org-babel-noweb-wrap-end)))
 
 (defvar org-babel-src-name-regexp
@@ -413,6 +415,7 @@ then run `org-babel-switch-to-session'."
     (noweb	. ((yes no tangle no-export strip-export)))
     (noweb-ref	. :any)
     (noweb-sep  . :any)
+    (noweb-prefix . ((no yes)))
     (output-dir . :any)
     (padline	. ((yes no)))
     (post       . :any)
@@ -438,8 +441,8 @@ specific header arguments as well.")
 
 (defconst org-babel-safe-header-args
   '(:cache :colnames :comments :exports :epilogue :hlines :noeval
-	   :noweb :noweb-ref :noweb-sep :padline :prologue :rownames
-	   :sep :session :tangle :wrap
+	   :noweb :noweb-ref :noweb-sep :noweb-prefix :padline
+           :prologue :rownames :sep :session :tangle :wrap
 	   (:eval . ("never" "query"))
 	   (:results . (lambda (str) (not (string-match "file" str)))))
   "A list of safe header arguments for babel source blocks.
@@ -2827,7 +2830,11 @@ block but are passed literally to the \"example-block\"."
          (lang (nth 0 info))
          (body (nth 1 info))
 	 (comment (string= "noweb" (cdr (assq :comments (nth 2 info)))))
-	 (noweb-re (format "\\(.*?\\)\\(%s\\)"
+         (noweb-prefix (let ((v (assq :noweb-prefix (nth 2 info))))
+                         (or (not v)
+                             (and (org-not-nil (cdr v))
+                                  (not (equal (cdr v) "no"))))))
+	 (noweb-re (format "\\(?3:.*?\\)\\(?4:%s\\)"
 			   (with-current-buffer parent-buffer
 			     (org-babel-noweb-wrap))))
 	 (cache nil)
@@ -2878,9 +2885,10 @@ block but are passed literally to the \"example-block\"."
      (lambda (m)
        (with-current-buffer parent-buffer
 	 (save-match-data
-	   (let* ((prefix (match-string 1 m))
-		  (id (match-string 3 m))
+	   (let* ((prefix (match-string 3 m))
+		  (id (match-string 1 m))
 		  (evaluate (string-match-p "(.*)" id))
+                  (print-str (> (length (match-string 2 m)) 0))
 		  (expansion
 		   (cond
 		    (evaluate
@@ -2921,12 +2929,16 @@ block but are passed literally to the \"example-block\"."
 			  (let* ((info (org-babel-get-src-block-info t))
 				 (ref (cdr (assq :noweb-ref (nth 2 info)))))
 			    (push info (gethash ref cache))))))
-		     (funcall expand-references id cache)))))
+		     (funcall expand-references id cache))))
+                  (expansion (if print-str (prin1-to-string expansion)
+                               expansion)))
 	     ;; Interpose PREFIX between every line.
-	     (mapconcat #'identity
-			(split-string expansion "[\n\r]")
-			(concat "\n" prefix))))))
-     body t t 2)))
+	     (if noweb-prefix
+                 (mapconcat #'identity
+			    (split-string expansion "[\n\r]")
+			    (concat "\n" prefix))
+               expansion)))))
+     body t t 4)))
 
 (defun org-babel--script-escape-inner (str)
   (let (in-single in-double backslash out)
-- 
2.36.0