[PATCH] Allow tangling to a list of files

[PATCH] Allow tangling to a list of files

[Jacopo De Simoi]

[-- Attachment #1.1: Type: text/plain, Size: 675 bytes --]

Dear All,

Please find attached a patch (against master) that adds a feature to the tangle framework. Essentially, the following block would now tangle to two files

#+begin_src sh '("filename1" "filename2")
#my script
#+end_src

Usecases

- literate config (e.g. .zshrc) of several machines at once (e.g. tangling via tramp).
- literate similar versions of the same script which differ only in small chunks (I use it for a slightly different latexmkrc for my projects on Dropbox)

The patch also streamlines the tangling routines.

Tests have been checked.

I am of course open to discussions and comments. Looking forward to hear from you.
Thanks for your time

Best
Jacopo

[-- Attachment #1.2: Type: text/html, Size: 758 bytes --]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-tangle-Accept-lists-of-files-as-tangling-target.patch --]
[-- Type: text/x-diff; name=0001-ob-tangle-Accept-lists-of-files-as-tangling-target.patch, Size: 8030 bytes --]

From 3a04df5e636c11bfcd8e2183e4a3e336daeb46a9 Mon Sep 17 00:00:00 2001
From: Jacopo De Simoi <wilderjds@protonmail.com>
Date: Fri, 2 Jul 2021 18:31:41 -0400
Subject: [PATCH 1/2] ob-tangle: Accept lists of files as tangling target

* lisp/ob-tangle.el (org-babel-tangle): Drop the now
superfluous parameter `only-this-block'.

(org-babel-effective-tangled-filenames): Return a list
of filenames to use as targets for tangling.

(org-babel-tangle-collect-blocks): Adapt to changes to
`org-babel-tangle-single-block', which now returns an
alist.

(org-babel-tangle-single-block): Return an alist to be
used from `org-babel-tangle'.  A single block can now
be tangled to several files at once.

This commit correctly parses list of filenames as
arguments of the :tangle parameter.  In doing so it
streamlines both `org-babel-tangle-single-block' and
`org-babel-tangle-collect-blocks'.  This was inspired
by the solution to the question
[https://emacs.stackexchange.com/questions/39032/tangle-the-same-src-block-to-different-files]
which I asked few years ago.

The suggested solution does not work with recent org-mode,
so I came up with a working rewrite.
---
 lisp/ob-tangle.el | 88 ++++++++++++++++++++++++-----------------------
 1 file changed, 45 insertions(+), 43 deletions(-)

diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el
index 2f60ef9a4..3799da03f 100644
--- a/lisp/ob-tangle.el
+++ b/lisp/ob-tangle.el
@@ -275,7 +275,7 @@ matching a regular expression."
 		   (mapc (lambda (mode) (set-file-modes file-name mode)) modes)
                    (push file-name path-collector))))))
 	 (if (equal arg '(4))
-	     (org-babel-tangle-single-block 1 t)
+	     (org-babel-tangle-single-block 1)
 	   (org-babel-tangle-collect-blocks lang-re tangle-file)))
 	(message "Tangled %d code block%s from %s" block-counter
 		 (if (= block-counter 1) "" "s")
@@ -350,22 +350,24 @@ that the appropriate major-mode is set.  SPEC has the form:
 	       (org-fill-template
 		org-babel-tangle-comment-format-end link-data)))))
 
-(defun org-babel-effective-tangled-filename (buffer-fn src-lang src-tfile)
-  "Return effective tangled filename of a source-code block.
+(defun org-babel-effective-tangled-filenames (buffer-fn src-lang src-tfile)
+  "Return a list of effective tangled filename of a source-code block.
 BUFFER-FN is the name of the buffer, SRC-LANG the language of the
 block and SRC-TFILE is the value of the :tangle header argument,
 as computed by `org-babel-tangle-single-block'."
-  (let ((base-name (cond
+  (if (consp src-tfile)
+      src-tfile
+    (list (let ((base-name (cond
                     ((string= "yes" src-tfile)
                      ;; Use the buffer name
                      (file-name-sans-extension buffer-fn))
                     ((string= "no" src-tfile) nil)
                     ((> (length src-tfile) 0) src-tfile)))
         (ext (or (cdr (assoc src-lang org-babel-tangle-lang-exts)) src-lang)))
-    (when base-name
+        (when base-name
       ;; decide if we want to add ext to base-name
       (if (and ext (string= "yes" src-tfile))
-          (concat base-name "." ext) base-name))))
+          (concat base-name "." ext) base-name))))))
 
 (defun org-babel-tangle-collect-blocks (&optional lang-re tangle-file)
   "Collect source blocks in the current Org file.
@@ -388,27 +390,26 @@ be used to limit the collected code blocks by target file."
 	(let* ((info (org-babel-get-src-block-info 'light))
 	       (src-lang (nth 0 info))
 	       (src-tfile (cdr (assq :tangle (nth 2 info)))))
-	  (unless (or (string= src-tfile "no")
+	  (unless (or (and (not (consp src-tfile)) (string= src-tfile "no"))
 		      (and tangle-file (not (equal tangle-file src-tfile)))
 		      (and lang-re (not (string-match-p lang-re src-lang))))
 	    ;; Add the spec for this block to blocks under its tangled
 	    ;; file name.
-	    (let* ((block (org-babel-tangle-single-block counter))
-                   (src-tfile (cdr (assq :tangle (nth 4 block))))
-		   (file-name (org-babel-effective-tangled-filename
-                               (nth 1 block) src-lang src-tfile))
-		   (by-fn (assoc file-name blocks)))
-	      (if by-fn (setcdr by-fn (cons (cons src-lang block) (cdr by-fn)))
-		(push (cons file-name (list (cons src-lang block))) blocks)))))))
+	    (let ((new-blocks (org-babel-tangle-single-block counter)))
+              (dolist (bl new-blocks)
+		(let* ((block (cdr bl))
+                       (file-name (car bl))
+                       (by-fn (assoc file-name blocks)))
+	          (if by-fn (setcdr by-fn (cons (car block) (cdr by-fn)))
+		    (push (cons file-name block) blocks)))))))))
     ;; Ensure blocks are in the correct order.
     (mapcar (lambda (b) (cons (car b) (nreverse (cdr b))))
 	    (nreverse blocks))))
 
-(defun org-babel-tangle-single-block (block-counter &optional only-this-block)
+(defun org-babel-tangle-single-block (block-counter)
   "Collect the tangled source for current block.
-Return the list of block attributes needed by
-`org-babel-tangle-collect-blocks'.  When ONLY-THIS-BLOCK is
-non-nil, return the full association list to be used by
+Return the association list of blocks needed by
+`org-babel-tangle-collect-blocks' or
 `org-babel-tangle' directly."
   (let* ((info (org-babel-get-src-block-info))
 	 (start-line
@@ -470,31 +471,32 @@ non-nil, return the full association list to be used by
 			 (match-end 0)
 		       (point-min))))
 	      (point)))))
-	 (result
-	  (list start-line
-		(if org-babel-tangle-use-relative-file-links
-		    (file-relative-name file)
-		  file)
-		(if (and org-babel-tangle-use-relative-file-links
-			 (string-match org-link-types-re link)
-			 (string= (match-string 1 link) "file"))
-		    (concat "file:"
-			    (file-relative-name (substring link (match-end 0))
-						(file-name-directory
-						 (cdr (assq :tangle params)))))
-		  link)
-		source-name
-		params
-		(if org-src-preserve-indentation
-		    (org-trim body t)
-		  (org-trim (org-remove-indentation body)))
-		comment)))
-    (if only-this-block
-        (let* ((src-tfile (cdr (assq :tangle (nth 4 result))))
-               (file-name (org-babel-effective-tangled-filename
-                           (nth 1 result) src-lang src-tfile)))
-          (list (cons file-name (list (cons src-lang result)))))
-      result)))
+         (src-tfile (cdr (assq :tangle params)))
+         (file-names (org-babel-effective-tangled-filenames
+                      (if org-babel-tangle-use-relative-file-links
+		          (file-relative-name file)
+		        file) src-lang src-tfile)))
+    (mapcar (lambda (file-name)
+              (cons file-name (list (cons src-lang
+                                          (list start-line
+		                                (if org-babel-tangle-use-relative-file-links
+		                                    (file-relative-name file)
+		                                  file)
+		                                (if (and org-babel-tangle-use-relative-file-links
+			                                 (string-match org-link-types-re link)
+			                                 (string= (match-string 1 link) "file"))
+		                                    (concat "file:"
+			                                    (file-relative-name (substring link (match-end 0))
+						                                (file-name-directory
+                                                                                 file-name)))
+		                                            link)
+		                                source-name
+		                                params
+		                                (if org-src-preserve-indentation
+		                                    (org-trim body t)
+		                                  (org-trim (org-remove-indentation body)))
+		                                comment)))))
+            file-names)))
 
 (defun org-babel-tangle-comment-links (&optional info)
   "Return a list of begin and end link comments for the code block at point.
-- 
2.31.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-org-manual-Update-docs.patch --]
[-- Type: text/x-diff; name=0002-org-manual-Update-docs.patch, Size: 1186 bytes --]

From b7d1e15f5f79bdbec72a36125c700e06d1b33b39 Mon Sep 17 00:00:00 2001
From: Jacopo De Simoi <wilderjds@protonmail.com>
Date: Fri, 2 Jul 2021 18:53:53 -0400
Subject: [PATCH 2/2] org-manual: Update docs

* doc/org-manual.org (Header arguments): document that
lists can be passed as arguments to :tangle
---
 doc/org-manual.org | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 977ef80b9..eaee9b248 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -17961,6 +17961,14 @@ to source file(s).
   file name as being relative to the directory of the Org file's
   location.  Example: =:tangle FILENAME=.
 
+- {{{var(FILENAME-LIST)}}} ::
+
+  Export the code block to possibly several source files whose file name is derived from
+  a list of strings passed to the =tangle= header argument.  Org derives the
+  file names as being relative to the directory of the Org file's
+  location.  Example: =:tangle ’("FILENAME1" "FILENAME2")=.
+
+
 #+cindex: @samp{mkdirp}, header argument
 The =mkdirp= header argument creates parent directories for tangled
 files if the directory does not exist.  A =yes= value enables
-- 
2.31.1

Re: [PATCH] Allow tangling to a list of files

[Vladimir Lomov]

[-- Attachment #1: Type: text/plain, Size: 2057 bytes --]

Hello
** Jacopo De Simoi <wilderjds@protonmail.com> [2021-07-05 18:54:14 +0000]:

> Dear All,

I do use 'tangle' feature and I have several Org documents that tangle shell
scripts as well as shell configurations so I could comment your proposal a
bit.

> Please find attached a patch (against master) that adds a feature to the
> tangle framework. Essentially, the following block would now tangle to two
> files

> #+begin_src sh '("filename1" "filename2")
> #my script
> #+end_src

May be you mean

#+begin_src sh :tangle '("filename1" "filemane2")
# some shell script
#+end_src

> Usecases

> - literate config (e.g. .zshrc) of several machines at once (e.g. tangling
> via tramp).

IMO this is very specific use case but still it could be solved using existing
technique. I would suggest storing all configuration files (as well as Org
documents) in git repository, tangle them on a host, push to a repository and
pull them on destination host (I do this myself, it helps me to avoid errors).

> - literate similar versions of the same script which differ only in small
> chunks (I use it for a slightly different latexmkrc for my projects on
> Dropbox)

Again, IMO, this is very specific use case. You could use standard way to get
desire result. Besides if you want tangle to several files they content will
be the same, won't it be? So you couldn't get different results from one code
block. I do a bit similar thing: I have Org document describing shell
configuration for several hosts. I tangle to different files using standard
technique and use 'noweb' feature to adapt to different hosts. 

May be it would worth if you describe what you want to achieve and how you are
doing it now?

[...]

---
WBR, Vladimir Lomov

-- 
Software suppliers are trying to make their software packages more
"user-friendly".  ...  Their best approach, so far, has been to take all
the old brochures, and stamp the words, "user-friendly" on the cover.
		-- Bill Gates, Microsoft, Inc.
	[Pot. Kettle. Black.]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH] Allow tangling to a list of files

[Greg Minshall]

hi, Jacopo,

i'm not convinced this is needed over and above your old "solution" of
using <<noweb>> witn N-different source blocks, each :tangle'ing to a
different file.

but, i'm curious -- in the example you sent, did you miss a ":tangle" on
the "#+begin_src" line?

> #+begin_src sh '("filename1" "filename2")
> #my script
> #+end_src

cheers, Greg

Re: [PATCH] Allow tangling to a list of files

[Jacopo De Simoi via General discussions about Org-mode.]

Hi Greg, 

thanks for your comments!

On Tuesday, July 6, 2021 12:43:54 AM EDT Greg Minshall wrote:
> hi, Jacopo,
> 
> i'm not convinced this is needed over and above your old "solution" of
> using <<noweb>> witn N-different source blocks, each :tangle'ing to a
> different file.

To be honest I never quite managed to get it work... =) 

My point here is to be able to have one org file tangle'ing to several, 
slightly different outputs.  Ideally I want to use one readable literate config 
for all my machines; the config can then be published (or exported) to html

Say I want to create an org file to tangle .tmux.conf (or .zshrc) for different 
machines; then most of the conf file would be the same (and each such block 
would be tangled to all files) whereas some specifics could be tangled to 
corresponding files only (e.g. ALIASes or EDITORs) 

Even if a solution using noweb could work, I find being able to tangle to a 
list of files more readable and elegant. Especially when exporting the org in 
an external format, I think the noweb solution would look like a hack, whereas 
a solution with tangle-to-list would be much easier to parse. 

> 
> but, i'm curious -- in the example you sent, did you miss a ":tangle" on
> the "#+begin_src" line?

Yikes! of course I did! Good catch.
> 
> > #+begin_src sh '("filename1" "filename2")
> > #my script
> > #+end_src
> 

Best,
 Jacopo

Re: [PATCH] Allow tangling to a list of files

[Vladimir Lomov]

[-- Attachment #1: Type: text/plain, Size: 2270 bytes --]

Hello!

** Jacopo De Simoi via General discussions about Org-mode.
<emacs-orgmode@gnu.org> [2021-07-06 01:09:30 -0400]:

> Hi Greg, 

> thanks for your comments!

> On Tuesday, July 6, 2021 12:43:54 AM EDT Greg Minshall wrote:
>> hi, Jacopo,
>> 
>> i'm not convinced this is needed over and above your old "solution" of
>> using <<noweb>> witn N-different source blocks, each :tangle'ing to a
>> different file.

> To be honest I never quite managed to get it work... =) 

> My point here is to be able to have one org file tangle'ing to several, 
> slightly different outputs.  Ideally I want to use one readable literate config 
> for all my machines; the config can then be published (or exported) to html

I do the same with 'noweb' feature.

> Say I want to create an org file to tangle .tmux.conf (or .zshrc) for different 
> machines; then most of the conf file would be the same (and each such block 
> would be tangled to all files) whereas some specifics could be tangled to 
> corresponding files only (e.g. ALIASes or EDITORs) 

How this could be solved by your approach by tangling to several files? They
will be identical!

> Even if a solution using noweb could work, I find being able to tangle to a 
> list of files more readable and elegant.

Could you explain this a bit more? How you will see that (assuming you have
exported Org to HTML) this goes into different files? Again, if you tangle to
several files they content will be the same! Then why you need different
files?

> Especially when exporting the org in an external format,

> I think the noweb solution would look like a hack,

IMHO, yes, this is a hack and the "standard" way. I would suggest to read
again Org manual concerning "Literate programming".

> whereas a solution with tangle-to-list would be much easier to parse. 

How that? You will tangle source block to several files, they content will be
identical! Or you suggesting to add some syntax for source block indicating to
which file to tangle that code snippet?

I would recommend you to describe your needs and how you do things right now.

[...]

> Best,
>  Jacopo

---
WBR, Vladimir Lomov

-- 
When a camel flies, no one laughs if it doesn't get very far!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH] Allow tangling to a list of files

[Tim Cross]

Jacopo De Simoi via "General discussions about Org-mode." <emacs-orgmode@gnu.org> writes:

> Hi Greg, 
>
> thanks for your comments!
>
> On Tuesday, July 6, 2021 12:43:54 AM EDT Greg Minshall wrote:
>> hi, Jacopo,
>> 
>> i'm not convinced this is needed over and above your old "solution" of
>> using <<noweb>> witn N-different source blocks, each :tangle'ing to a
>> different file.
>
> To be honest I never quite managed to get it work... =) 
>
> My point here is to be able to have one org file tangle'ing to several, 
> slightly different outputs.  Ideally I want to use one readable literate config 
> for all my machines; the config can then be published (or exported) to html
>
> Say I want to create an org file to tangle .tmux.conf (or .zshrc) for different 
> machines; then most of the conf file would be the same (and each such block 
> would be tangled to all files) whereas some specifics could be tangled to 
> corresponding files only (e.g. ALIASes or EDITORs) 
>
> Even if a solution using noweb could work, I find being able to tangle to a 
> list of files more readable and elegant. Especially when exporting the org in 
> an external format, I think the noweb solution would look like a hack, whereas 
> a solution with tangle-to-list would be much easier to parse. 
>
>> 
>> but, i'm curious -- in the example you sent, did you miss a ":tangle" on
>> the "#+begin_src" line?
>
> Yikes! of course I did! Good catch.
>> 
>> > #+begin_src sh '("filename1" "filename2")
>> > #my script
>> > #+end_src
>> 

I'm not sure I fully understand the rationale behind this patch. It
seems to be very niche oriented and not a terribly useful general
feature. It feels like it is just a partial solution to a problem (i.e.
generate multiple different files from the same org file). If this is
the case, then you probably need some additional control structures to
determine which bits/blocks go into which files. From what I can see,
all the patch is doing is creating multiple files, which I imagine would
then need to be modified anyway?

If I understand it correctly, all the files will end up with the same
content. This seems odd to me. Am I missing something (like some ability
to have different contents tangled to different files)?

If it is just generating multiple copies of the same file, I don't
really see the value. It would be less processing/overhead to just
create one file and then copy it (possibly copying it to different
locations, such as a tramp filespec). Using the 'devops' style of org
files, this could even be coded into a script block which could be
executed after tangling. 

Personally, I've used a different approach to a similar problem. For
example, my .zshrc and init.el files have conditional tests which check
either the hostname or platform type and set things accordingly. This
way, I only ever have one .zshrc and one init.el file for all systems,
but they behave differently based on the system they are running on.
When the config does not support conditional tests, then I'll have
different source blocks included in the tangle. Currently, I just turn
off blocks I don't want (:tangle no), but I guess I could do something
more sophisticated using noweb or tags, but the manual setting has
worked fine so far as I don't have many files requiring this. 

-- 
Tim Cross

Re: [PATCH] Allow tangling to a list of files

[Jacopo De Simoi via General discussions about Org-mode.]

Dear Vladimir, 

 thanks for your reply.

Let me show you two examples; they are both WIP and incomplete, but you'll get 
the gist. 

- Example 1

https://gist.github.com/85c9b9c9ab151c5fd35bd761ef6ef569

This is my literate config for .zshrc; as you can see different blocks are 
tangled to different set of files; hence the output files differ (as some blocks 
are present and other are not) 

- Example 2

https://gist.github.com/2bce5d6df3f04c934fb4beda0c6560ad

This is my literate config for xkb maps; I have several keyboards, and some of 
them require slightly different maps.

Once again, likely all this can be accomplished by some noweb magic, but I find 
my approach much less intrusive (particularly the initial bit in the .zshrc, 
which I am still refactoring, where the list of tangle targets is inferred 
from the headline tags)

Thanks again for your comments. I'd really like to see how to do the same with 
noweb and compare my approach with the "standard" one. Once more, I hope these 
examples clarify my point of view. 

Best 
 Jacopo

On Tuesday, July 6, 2021 2:11:09 AM EDT Vladimir Lomov wrote:
> Hello!
> 
> ** Jacopo De Simoi via General discussions about Org-mode.
> 
> <emacs-orgmode@gnu.org> [2021-07-06 01:09:30 -0400]:
> > Hi Greg,
> > 
> > thanks for your comments!
> > 
> > On Tuesday, July 6, 2021 12:43:54 AM EDT Greg Minshall wrote:
> >> hi, Jacopo,
> >> 
> >> i'm not convinced this is needed over and above your old "solution" of
> >> using <<noweb>> witn N-different source blocks, each :tangle'ing to a
> >> different file.
> > 
> > To be honest I never quite managed to get it work... =)
> > 
> > My point here is to be able to have one org file tangle'ing to several,
> > slightly different outputs.  Ideally I want to use one readable literate
> > config for all my machines; the config can then be published (or
> > exported) to html
> I do the same with 'noweb' feature.
> 
> > Say I want to create an org file to tangle .tmux.conf (or .zshrc) for
> > different machines; then most of the conf file would be the same (and
> > each such block would be tangled to all files) whereas some specifics
> > could be tangled to corresponding files only (e.g. ALIASes or EDITORs)
> 
> How this could be solved by your approach by tangling to several files? They
> will be identical!
> 
> > Even if a solution using noweb could work, I find being able to tangle to
> > a
> > list of files more readable and elegant.
> 
> Could you explain this a bit more? How you will see that (assuming you have
> exported Org to HTML) this goes into different files? Again, if you tangle
> to several files they content will be the same! Then why you need different
> files?
> 
> > Especially when exporting the org in an external format,
> > 
> > I think the noweb solution would look like a hack,
> 
> IMHO, yes, this is a hack and the "standard" way. I would suggest to read
> again Org manual concerning "Literate programming".
> 
> > whereas a solution with tangle-to-list would be much easier to parse.
> 
> How that? You will tangle source block to several files, they content will
> be identical! Or you suggesting to add some syntax for source block
> indicating to which file to tangle that code snippet?
> 
> I would recommend you to describe your needs and how you do things right
> now.
> 
> [...]
> 
> > Best,
> > 
> >  Jacopo
> 
> ---
> WBR, Vladimir Lomov

Re: [PATCH] Allow tangling to a list of files

[Vladimir Lomov]

[-- Attachment #1: Type: text/plain, Size: 3738 bytes --]

Hello Jacopo.
** Jacopo De Simoi <wilderjds@protonmail.com> [2021-07-06 11:24:40 -0400]:

> Dear Vladimir, 

>  thanks for your reply.

> Let me show you two examples; they are both WIP and incomplete, but you'll get 
> the gist. 

> - Example 1

> https://gist.github.com/85c9b9c9ab151c5fd35bd761ef6ef569

> This is my literate config for .zshrc; as you can see different blocks are
> tangled to different set of files; hence the output files differ (as some
> blocks are present and other are not) 

I couldn't find in Org manual how tangling should work if there are several
source code blocks with the same file name for ':tangle'. The Org manual
section "15.8 Extracting Source Code" is a bit obscure. There are these two
sentences

  When Org tangles code blocks, it expands, merges, and transforms
  them.  Then Org recomposes them into one or more separate files, as
  configured through the options.

The second sentence assumes that it might be possible to tangle to more than
one file but possible options for ':tangle' don't give certain answer:

  yes’

       Export the code block to source file.  The file name for the source
       file is derived from the name of the Org file, and the file extension
       is derived from the source code language identifier.
       Example: ‘:tangle yes’.

  FILENAME

       Export the code block to source file whose file name is derived from
       any string passed to the ‘tangle’ header argument.  Org derives the
       file name as being relative to the directory of the Org file’s
       location.  Example: ‘:tangle FILENAME’.

If I understand first choice correctly then the only source block ("the code
block") will be tangled (exported) to a source file. What Org will do if there
are several source blocks? Concatenate them or write to a file only last one?
(I didn't test that).

The same applies to 'FILENAME' case: what Org will do if there are several
source code blocks with the same 'FILENAME' for ':tangle'.

I would say that you use here undocumented feature and you workflow could be
broken if in some future Org version developers decide to implement only
documented features.

> - Example 2

> https://gist.github.com/2bce5d6df3f04c934fb4beda0c6560ad

> This is my literate config for xkb maps; I have several keyboards, and some of 
> them require slightly different maps.

> Once again, likely all this can be accomplished by some noweb magic, but I find 
> my approach much less intrusive (particularly the initial bit in the .zshrc, 
> which I am still refactoring, where the list of tangle targets is inferred 
> from the headline tags)

I found you way of doing things a bit strange. As I already told I use 'noweb'
feature a lot. I won't advise you to use it or not to use it but I find it
very powerful.

> Thanks again for your comments. I'd really like to see how to do the same
> with noweb and compare my approach with the "standard" one. Once more, I
> hope these examples clarify my point of view. 

If you don't distract by foreign language (Russian) you could look how I use
'noweb' feature for scripts (one example):

https://git.sr.ht/~vp1981/scripts/tree/master/item/docs/scripts/misc.org

for shell configurations:

https://git.sr.ht/~vp1981/scripts/tree/master/item/docs/shell/bash.org

and for Emacs configuration:

https://git.sr.ht/~vp1981/scripts/tree/master/item/docs/emacs/emacs.org

> Best 
>  Jacopo

[...]

---
WBR, Vladimir Lomov

-- 
str->str_pok |= SP_FBM;                     /* deep magic */
s = (unsigned char*)(str->str_ptr);         /* deeper magic */
		-- Larry Wall in util.c from the perl source code

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH] Allow tangling to a list of files

[Tim Cross]

Vladimir Lomov <lomov.vl@yandex.ru> writes:

> [[PGP Signed Part:Undecided]]
> Hello Jacopo.
> ** Jacopo De Simoi <wilderjds@protonmail.com> [2021-07-06 11:24:40 -0400]:
>
>> Dear Vladimir, 
>
>>  thanks for your reply.
>
>> Let me show you two examples; they are both WIP and incomplete, but you'll get 
>> the gist. 
>
>> - Example 1
>
>> https://gist.github.com/85c9b9c9ab151c5fd35bd761ef6ef569
>
>> This is my literate config for .zshrc; as you can see different blocks are
>> tangled to different set of files; hence the output files differ (as some
>> blocks are present and other are not) 
>
> I couldn't find in Org manual how tangling should work if there are several
> source code blocks with the same file name for ':tangle'. The Org manual
> section "15.8 Extracting Source Code" is a bit obscure. There are these two
> sentences
>
>   When Org tangles code blocks, it expands, merges, and transforms
>   them.  Then Org recomposes them into one or more separate files, as
>   configured through the options.
>
> The second sentence assumes that it might be possible to tangle to more than
> one file but possible options for ':tangle' don't give certain answer:
>
>   yes’
>
>        Export the code block to source file.  The file name for the source
>        file is derived from the name of the Org file, and the file extension
>        is derived from the source code language identifier.
>        Example: ‘:tangle yes’.
>
>   FILENAME
>
>        Export the code block to source file whose file name is derived from
>        any string passed to the ‘tangle’ header argument.  Org derives the
>        file name as being relative to the directory of the Org file’s
>        location.  Example: ‘:tangle FILENAME’.
>
> If I understand first choice correctly then the only source block ("the code
> block") will be tangled (exported) to a source file. What Org will do if there
> are several source blocks? Concatenate them or write to a file only last one?
> (I didn't test that).
>
> The same applies to 'FILENAME' case: what Org will do if there are several
> source code blocks with the same 'FILENAME' for ':tangle'.
>
> I would say that you use here undocumented feature and you workflow could be
> broken if in some future Org version developers decide to implement only
> documented features.
>


As it stands at the moment, if the :tangle keyword is followed by a
filename, that block will be tangled into that file. If there is more
than one block with the same filename, the blocks will all be
concatenated into the same file. If you have multiple source blocks, but
with different filenames for the blocks, multiple files will be created
when you tangle the file - one file for each unique filename given to
the :tangle argument in each block.

I use this to generate my emacs config. I have both an early-init.el and
an init.el. These files are created from multiple source blocks where
some blocks have :tangle early-init.el and some have :tangle init.el.
(actually, I have changed that now. I tangle to tangle-early-init.el and
tangle-init.el. I then have a simple 'install.sh' script which first
moves the old early-init.el and init.el to new files with a timestamp
added to the name i.e. init.el-20210722 and then it installs the new
init files, renaming them to early-init.el and init.el).

I'm not sure this is an undocumented feature. The manual just says that
the code will be tangled into the file specified with the :tangle
keyword. Nothing states the filename can only appear once or you cannot
have different filenames for different blocks.

If you don't specify a filename i.e. :tangle yes, the block will be
tangled into a file with the same name as the org file and the extension
appropriate to the language in the source block. For example, if my org
file is called emacs-config.org and I just have :tangle yes, I will get
the source tangled into emacs-config.el. Same will happen if I just have
the keyword :tangle.

When I don't want a source block added to a file, I just add :tangle no
to the block header. This is a handy way of removing something from a
config file. I find it useful when trying to track down problems with my
emacs init as I can quickly remove lots of stuff, generate a new config
and see if the issue persists. I find this easier than commenting and
uncommenting multiple lines in my init file. I also find it useful when
I run into a problem with something (possibly due to a package update)
as my source blocks are all broken up into specific bits of
configuration which are as far as possible completely independent of
each other. For example, a while ago, I had problems with one of the
magit add on packages I use after it was updated. I changed the header
to :tangle no, generated a new init file and waited until a new version
of the package was released. This happened a few days later and I just
changed the header back to :tangle init.el and generated the new config
and all was back to normal.

-- 
Tim Cross

Re: [PATCH] Allow tangling to a list of files

[Jacopo De Simoi]

On Tuesday, July 6, 2021 11:27:18 PM EDT Vladimir Lomov wrote:
> Hello Jacopo.
>
> ** Jacopo De Simoi <wilderjds@protonmail.com> [2021-07-06 11:24:40 -0400]:
> > Dear Vladimir,
> >
> >  thanks for your reply.
> >
> > Let me show you two examples; they are both WIP and incomplete, but you'll
> > get the gist.
> >
> > - Example 1
> >
> > https://gist.github.com/85c9b9c9ab151c5fd35bd761ef6ef569
> >
> > This is my literate config for .zshrc; as you can see different blocks are
> > tangled to different set of files; hence the output files differ (as some
> > blocks are present and other are not)
>
> I couldn't find in Org manual how tangling should work if there are several
> source code blocks with the same file name for ':tangle'. The Org manual
> section "15.8 Extracting Source Code" is a bit obscure. There are these two
> sentences
>
>   When Org tangles code blocks, it expands, merges, and transforms
>   them.  Then Org recomposes them into one or more separate files, as
>   configured through the options.
>
> The second sentence assumes that it might be possible to tangle to more than
> one file but possible options for ':tangle' don't give certain answer:
>
>   yes’
>
>        Export the code block to source file.  The file name for the source
>        file is derived from the name of the Org file, and the file extension
> is derived from the source code language identifier.
>        Example: ‘:tangle yes’.
>
>   FILENAME
>
>        Export the code block to source file whose file name is derived from
>        any string passed to the ‘tangle’ header argument.  Org derives the
>        file name as being relative to the directory of the Org file’s
>        location.  Example: ‘:tangle FILENAME’.
>
> If I understand first choice correctly then the only source block ("the code
> block") will be tangled (exported) to a source file. What Org will do if
> there are several source blocks? Concatenate them or write to a file only
> last one? (I didn't test that).
>
> The same applies to 'FILENAME' case: what Org will do if there are several
> source code blocks with the same 'FILENAME' for ':tangle'.
>
> I would say that you use here undocumented feature and you workflow could be
> broken if in some future Org version developers decide to implement only
> documented features.

I am quite sure that if more than one blocks are tangled to the same file, then
the blocks must be concatenated.  There is at least one test in the test suite
which is designed to check precisely this behavior.

>
> > - Example 2
> >
> > https://gist.github.com/2bce5d6df3f04c934fb4beda0c6560ad
> >
> > This is my literate config for xkb maps; I have several keyboards, and
> > some of them require slightly different maps.
> >
> > Once again, likely all this can be accomplished by some noweb magic, but I
> > find my approach much less intrusive (particularly the initial bit in the
> > .zshrc, which I am still refactoring, where the list of tangle targets is
> > inferred from the headline tags)
>
> I found you way of doing things a bit strange. As I already told I use
> 'noweb' feature a lot. I won't advise you to use it or not to use it but I
> find it very powerful.
>
> > Thanks again for your comments. I'd really like to see how to do the same
> > with noweb and compare my approach with the "standard" one. Once more, I
> > hope these examples clarify my point of view.
>

Thanks, I will certainly have a good look.

> If you don't distract by foreign language (Russian) you could look how I use
> 'noweb' feature for scripts (one example):
>
> https://git.sr.ht/~vp1981/scripts/tree/master/item/docs/scripts/misc.org
>
> for shell configurations:
>
> https://git.sr.ht/~vp1981/scripts/tree/master/item/docs/shell/bash.org
>
> and for Emacs configuration:
>
> https://git.sr.ht/~vp1981/scripts/tree/master/item/docs/emacs/emacs.org
>
> > Best
> >
> >  Jacopo
>
> [...]
>
> ---
> WBR, Vladimir Lomov

Re: [PATCH] Allow tangling to a list of files

[Greg Minshall]

Vladimir,

> I couldn't find in Org manual how tangling should work if there are
> several source code blocks with the same file name for ':tangle'. The
> Org manual section "15.8 Extracting Source Code" is a bit
> obscure. There are these two sentences

i think what Tim answered is correct.  but, i believe the "desired"
approach is to put all those blocks to be tangled to the same file under
a headline with a property drawer containing something like:
----
        :header-args+:    :tangle "submsim.jl"
----

i believe this is for performance of tangling, but possibly the
"multiple source blocks to same :tangle'd file" feature may disappear?

cheers, Greg

Re: [PATCH] Allow tangling to a list of files

[Jacopo De Simoi]

[-- Attachment #1: Type: text/plain, Size: 1342 bytes --]

Dear Greg,

Thanks for bringing this up

-------- Original Message --------
On Jul. 7, 2021, 02:56, Greg Minshall < minshall@umich.edu> wrote:
Vladimir,
> I couldn't find in Org manual how tangling should work if there are
> several source code blocks with the same file name for ':tangle'. The
> Org manual section "15.8 Extracting Source Code" is a bit
> obscure. There are these two sentences
i think what Tim answered is correct. but, i believe the "desired"
approach is to put all those blocks to be tangled to the same file under
a headline with a property drawer containing something like:
----
:header-args+: :tangle "submsim.jl"
----
i believe this is for performance of tangling, but possibly the
"multiple source blocks to same :tangle'd file" feature may disappear?
cheers, Greg

Once again, there is a unit test designed to check that different blocks tangling to the same file are concatenated in the right order, this without a common property drawer. Hence I believe this behavior is not accidental, but a wanted feature.

I always understood that this was the expected behavior, to allow descriptions to interleave code in a literate config situation.

However, if the documentation is ambiguous, or lacking, then we should probably make the docs clear, so that this question does not rise again in the future.

Cheers
Jacopo

[-- Attachment #2: Type: text/html, Size: 1460 bytes --]

Re: [PATCH] Allow tangling to a list of files

[Jacopo De Simoi via General discussions about Org-mode.]

[-- Attachment #1: Type: text/plain, Size: 4439 bytes --]

Dear Tim, 

On Tuesday, July 6, 2021 3:30:40 AM EDT Tim Cross wrote:
> Jacopo De Simoi via "General discussions about Org-mode." <emacs-
orgmode@gnu.org> writes:
> > Hi Greg,
> > 
> > thanks for your comments!
> > 
> > On Tuesday, July 6, 2021 12:43:54 AM EDT Greg Minshall wrote:
> >> hi, Jacopo,
> >> 
> >> i'm not convinced this is needed over and above your old "solution" of
> >> using <<noweb>> witn N-different source blocks, each :tangle'ing to a
> >> different file.
> > 
> > To be honest I never quite managed to get it work... =)
> > 
> > My point here is to be able to have one org file tangle'ing to several,
> > slightly different outputs.  Ideally I want to use one readable literate
> > config for all my machines; the config can then be published (or
> > exported) to html
> > 
> > Say I want to create an org file to tangle .tmux.conf (or .zshrc) for
> > different machines; then most of the conf file would be the same (and
> > each such block would be tangled to all files) whereas some specifics
> > could be tangled to corresponding files only (e.g. ALIASes or EDITORs)
> > 
> > Even if a solution using noweb could work, I find being able to tangle to
> > a
> > list of files more readable and elegant. Especially when exporting the org
> > in an external format, I think the noweb solution would look like a hack,
> > whereas a solution with tangle-to-list would be much easier to parse.
> > 
> >> but, i'm curious -- in the example you sent, did you miss a ":tangle" on
> >> the "#+begin_src" line?
> > 
> > Yikes! of course I did! Good catch.
> > 
> >> > #+begin_src sh '("filename1" "filename2")
> >> > #my script
> >> > #+end_src
> 
> I'm not sure I fully understand the rationale behind this patch. It
> seems to be very niche oriented and not a terribly useful general
> feature. It feels like it is just a partial solution to a problem (i.e.
> generate multiple different files from the same org file). If this is
> the case, then you probably need some additional control structures to
> determine which bits/blocks go into which files. From what I can see,
> all the patch is doing is creating multiple files, which I imagine would
> then need to be modified anyway?
> 
> If I understand it correctly, all the files will end up with the same
> content. This seems odd to me. Am I missing something (like some ability
> to have different contents tangled to different files)?
> 
> If it is just generating multiple copies of the same file, I don't
> really see the value. It would be less processing/overhead to just
> create one file and then copy it (possibly copying it to different
> locations, such as a tramp filespec). Using the 'devops' style of org
> files, this could even be coded into a script block which could be
> executed after tangling.

In fact the files are different, since each source block is tangled to a 
possibly different subset of files. 

The logic for which files to tangle according to which parameter or tags can be 
implemented by some lisp magic such as 

#+begin_src sh :tangle (filenames-according-to-situation)
#+end_src

So my patch provides the framework to do this, but the implementation is left 
to the author.

I agree that if you tangle to shell or to any programming language, you can 
set up the checks for each usecase in the programming language itself.  

I developed this solution for plaintext config files (e.g. xkb maps, or KDE 
shortcut config files)  which are static and parsed as a single chunk.

I admit that the use case is quite specific, but it seemed to me a quite 
natural generalization of the current behavior, and worth adding.

Thanks for the discussion! It's really helpful to put things in perspective. 


> 
> Personally, I've used a different approach to a similar problem. For
> example, my .zshrc and init.el files have conditional tests which check
> either the hostname or platform type and set things accordingly. This
> way, I only ever have one .zshrc and one init.el file for all systems,
> but they behave differently based on the system they are running on.
> When the config does not support conditional tests, then I'll have
> different source blocks included in the tangle. Currently, I just turn
> off blocks I don't want (:tangle no), but I guess I could do something
> more sophisticated using noweb or tags, but the manual setting has
> worked fine so far as I don't have many files requiring this.
> 
> --
> Tim Cross


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH] Allow tangling to a list of files

[Tim Cross]

Jacopo De Simoi <jacopods@protonmail.com> writes:

>
> In fact the files are different, since each source block is tangled to a 
> possibly different subset of files. 
>
> The logic for which files to tangle according to which parameter or tags can be 
> implemented by some lisp magic such as 
>
> #+begin_src sh :tangle (filenames-according-to-situation)
> #+end_src
>
> So my patch provides the framework to do this, but the implementation is left 
> to the author.
>

This possibly makes your intention a little clearer.

It seems to me that what you are asking for is not support for
specifying a list of files, but support for specifying a function which
will return the filename of the file to use for the tangled output? That
is something I could see as being more useful than the ability to set a
list of output filename. Things could be defined so that if the function
returns nil, the block is not tangled, if it returns 't the block is
tangled to the default output file and if the function returns a string,
that string is interpreted as a filename which is to be used as the
output target for the tangle.

I think I might have seen another request for this type of functionality
on the list recently? Such functionality seems like a useful addition. 


-- 
Tim Cross

Re: [PATCH] Allow tangling to a list of files

[Jacopo De Simoi]

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Wednesday, July 7th, 2021 at 7:28 PM, Tim Cross <theophilusx@gmail.com> wrote:

> Jacopo De Simoi jacopods@protonmail.com writes:
>
> > In fact the files are different, since each source block is tangled to a
> >
> > possibly different subset of files.
> >
> > The logic for which files to tangle according to which parameter or tags can be
> >
> > implemented by some lisp magic such as
> >
> > #+begin_src sh :tangle (filenames-according-to-situation)
> >
> > #+end_src
> >
> > So my patch provides the framework to do this, but the implementation is left
> >
> > to the author.
>
> This possibly makes your intention a little clearer.
>
> It seems to me that what you are asking for is not support for
>
> specifying a list of files, but support for specifying a function which
>
> will return the filename of the file to use for the tangled output? That
>
> is something I could see as being more useful than the ability to set a
>
> list of output filename. Things could be defined so that if the function
>
> returns nil, the block is not tangled, if it returns 't the block is
>
> tangled to the default output file and if the function returns a string,
>
> that string is interpreted as a filename which is to be used as the
>
> output target for the tangle.

This is (mostly) already in place, lisp is indeed evaluated as an argument of the : tangle argument (although the docs do not mention this fact).

 What my patch is providing is support for the case in which the function returns a list of filenames rather than just one filename (or yes or no)

The "(mostly)" above refer to the fact that 't is not processed correctly (but that would be an easy fix)

nil used to work before being broken by a recent commit.  I posted a patch on July 1 to address this issue, but it has not been considered yet.

>
> I think I might have seen another request for this type of functionality
>
> on the list recently? Such functionality seems like a useful addition.
>
>
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Tim Cross

Re: [PATCH] Allow tangling to a list of files

[Tom Gillespie]

Reading over this with the new information about the use case, it
seems that using noweb to manage the many-to-many nature of a mapping
between blocks and files is a much better way to achieve the desired
result. In addition it is already supported and does not add more
complexity to an already complex part of org.

The one area that a noweb approach does not support is dynamic
construction of files at runtime on the basis of some information that
is only available at runtime, however that does not seem to be
important for this use case.

Therefore I suggest that the tangling behavior be left 1:1 block:file,
and if there is some desire to tangle to multiple files then noweb
should be used with multiple blocks. Obviously there is some
performance penalty here. Also this doesn't help with cases where we
want to tangle to hundreds of servers using tramp, but if that is the
use case then I would suggest that that operation not be hidden behind
:tangle. Instead tangle once and then use another elisp block write
all the files to their final destinations using tramp, ssh, or some
other means.

I personally have use cases for things like this, but even so I don't
think we wan't the :tangle parameter to be the way to do it. I would
suggest instead that if we want to enable a tangled file to be
automatically distributed to a number of different locations that we
provide a separate header argument so that the functionality is not
conflated with the tangle functionality. I don't have a good name for
it, but the objective seems to be something like :tangle-copy-to that
accepts a function returning zero or more paths, or a list of multiple
paths (I don't recall how/whether any of the babel args deal with
accepting multiple values).

Best,
Tom

Re: [PATCH] Allow tangling to a list of files

[Trust me I am a Doctor]

[-- Attachment #1: Type: text/plain, Size: 895 bytes --]


Hi,

I have no particular opinion for the patch proposed but wanted to share
with you some reflections I had on the subject to use one org file to
tangle to multiple setup.

I use abundantly virtual machines and my emacs configurations have many
bits that I don't want everywhere, be it a mail setup, a rss setup, or
extra languages setup ... etc.

It is not exactly the same usage, I do not tangle different versions in
one pass, I tangle a version at a time, and take care of installing it
in the right place later.

Naively I started to use the header's tags to tangle blocks. I thought
because there was already in place a query syntax for the tags it
may been interesting to use that.

You can see attached a proto of a tangle function that accept tags
specification and that will only tangle the blocks of the headings
matching this specification; and some utilities that I use with it.


[-- Attachment #2: tag-tangler.el --]
[-- Type: application/emacs-lisp, Size: 8953 bytes --]

[-- Attachment #3: Type: text/plain, Size: 376 bytes --]


However I must say that solution is quite simplistic. It effectively
allow me to filter dynamically what I tangle from my files, but I cannot
rely on it to specify elegantly, eg, complex dependencies between
packages.

At some point I wished to use org-properties to abstract some logics
from Elisp to org, and then forget about I always have something else to
tweak ...

--

Re: [PATCH] Allow tangling to a list of files

[Jacopo De Simoi]

Hi Doctor,

 It's good to have such constructive discussion here.

I like your approach to tangle based on tags.  I accomplish more or less the same in this (unfinished) literate config for zshrc

https://gist.github.com/85c9b9c9ab151c5fd35bd761ef6ef569

Here the function `org-tags-to-filenames' simply fetches the list of tags and returns a list of filenames corresponding to the tags.  Each block is then tangled to each file in the returned list (using the patch under discussion)

Best
 J
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Thursday, July 8th, 2021 at 12:41 PM, Trust me I am a Doctor <pillule@riseup.net> wrote:

> Hi,
>
> I have no particular opinion for the patch proposed but wanted to share
>
> with you some reflections I had on the subject to use one org file to
>
> tangle to multiple setup.
>
> I use abundantly virtual machines and my emacs configurations have many
>
> bits that I don't want everywhere, be it a mail setup, a rss setup, or
>
> extra languages setup ... etc.
>
> It is not exactly the same usage, I do not tangle different versions in
>
> one pass, I tangle a version at a time, and take care of installing it
>
> in the right place later.
>
> Naively I started to use the header's tags to tangle blocks. I thought
>
> because there was already in place a query syntax for the tags it
>
> may been interesting to use that.
>
> You can see attached a proto of a tangle function that accept tags
>
> specification and that will only tangle the blocks of the headings
>
> matching this specification; and some utilities that I use with it.
>
> However I must say that solution is quite simplistic. It effectively
>
> allow me to filter dynamically what I tangle from my files, but I cannot
>
> rely on it to specify elegantly, eg, complex dependencies between
>
> packages.
>
> At some point I wished to use org-properties to abstract some logics
>
> from Elisp to org, and then forget about I always have something else to
>
> tweak ...
>
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Re: [PATCH] Allow tangling to a list of files

[Vladimir Lomov]

[-- Attachment #1: Type: text/plain, Size: 1427 bytes --]

Hello,
** Greg Minshall <minshall@umich.edu> [2021-07-07 09:56:06 +0300]:

> Vladimir,

>> I couldn't find in Org manual how tangling should work if there are
>> several source code blocks with the same file name for ':tangle'. The
>> Org manual section "15.8 Extracting Source Code" is a bit
>> obscure. There are these two sentences

> i think what Tim answered is correct.  but, i believe the "desired"
> approach is to put all those blocks to be tangled to the same file under
> a headline with a property drawer containing something like:
> ----
>         :header-args+:    :tangle "submsim.jl"
> ----

Hmm, the more I read the manual and your answers the less I understand. As I
said I didn't find in the manual any mention of feature you and Tim referring
to. Besides I didn't find definition of [source] code block. If Org document
has several #+BEGIN/END_SRC constructions is this the one "code block" or not?
May be they are different if they use different "language" identifier? Again,
I didn't find any definition or explanation in the manual. This is either lack
of documentation or feature of how Org deals with source blocks. In my
opinion, this is undocumented feature.

> i believe this is for performance of tangling, but possibly the
> "multiple source blocks to same :tangle'd file" feature may disappear?

> cheers, Greg

---
WBR, Vladimir Lomov

-- 
How do I get HOME?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH] Allow tangling to a list of files

[Jacopo De Simoi]

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Friday, July 9th, 2021 at 8:26 AM, Vladimir Lomov <lomov.vl@yandex.ru> wrote:

> Hello,
>
> ** Greg Minshall minshall@umich.edu [2021-07-07 09:56:06 +0300]:
>
> > Vladimir,
>
> > > I couldn't find in Org manual how tangling should work if there are
> > >
> > > several source code blocks with the same file name for ':tangle'. The
> > >
> > > Org manual section "15.8 Extracting Source Code" is a bit
> > >
> > > obscure. There are these two sentences
>
> > i think what Tim answered is correct. but, i believe the "desired"
> >
> > approach is to put all those blocks to be tangled to the same file under
> >
> > a headline with a property drawer containing something like:
> > ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >
> >         :header-args+:    :tangle "submsim.jl"
> >
>
> Hmm, the more I read the manual and your answers the less I understand. As I
>
> said I didn't find in the manual any mention of feature you and Tim referring
>
> to. Besides I didn't find definition of [source] code block. If Org document
>
> has several #+BEGIN/END_SRC constructions is this the one "code block" or not?
>
> May be they are different if they use different "language" identifier? Again,
>
> I didn't find any definition or explanation in the manual. This is either lack
>
> of documentation or feature of how Org deals with source blocks. In my
>
> opinion, this is undocumented feature.

+1 for clarifying the docs. My point is that there is even a unit test designed to check in which order different source blocks are tangled to the same file.  Hence it is a desired feature.

 If the documentation lacks the description of this feature, then the documentation needs to be updated.


>
> > i believe this is for performance of tangling, but possibly the
> >
> > "multiple source blocks to same :tangle'd file" feature may disappear?
>
> > cheers, Greg
>
> WBR, Vladimir Lomov
>
> --------------------
>
> How do I get HOME?

Re: [PATCH] Allow tangling to a list of files

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 5153 bytes --]

On Fri, 9 Jul 2021 at 22:28, Vladimir Lomov <lomov.vl@yandex.ru> wrote:

> Hello,
> ** Greg Minshall <minshall@umich.edu> [2021-07-07 09:56:06 +0300]:
>
> > Vladimir,
>
> >> I couldn't find in Org manual how tangling should work if there are
> >> several source code blocks with the same file name for ':tangle'. The
> >> Org manual section "15.8 Extracting Source Code" is a bit
> >> obscure. There are these two sentences
>
> > i think what Tim answered is correct.  but, i believe the "desired"
> > approach is to put all those blocks to be tangled to the same file under
> > a headline with a property drawer containing something like:
> > ----
> >         :header-args+:    :tangle "submsim.jl"
> > ----
>
> Hmm, the more I read the manual and your answers the less I understand. As
> I
> said I didn't find in the manual any mention of feature you and Tim
> referring
> to. Besides I didn't find definition of [source] code block. If Org
> document
> has several #+BEGIN/END_SRC constructions is this the one "code block" or
> not?
> May be they are different if they use different "language" identifier?
> Again,
> I didn't find any definition or explanation in the manual. This is either
> lack
> of documentation or feature of how Org deals with source blocks. In my
> opinion, this is undocumented feature.
>
> > i believe this is for performance of tangling, but possibly the
> > "multiple source blocks to same :tangle'd file" feature may disappear?
>
> Personally, I don't find the documentation lacking in this area. I think
it is quite clear and the use of multiple source blocks and multiple
destination files is within the scope of documented features. Therefore,
I'm not in a good position to suggest what modifications to the
documentation are necessary. However, I did use literate programming before
org mode, so perhaps my understanding or expectations have been influenced
by thiat.

Perhaps if I outline my understanding and the sections from the manual
which guide my interpretation, that will help.

If we start from a high level literate programming perspective, we can
assume a file can contain multiple source blocks. This is largely the whole
idea. You have a file which has source code and 'pros' mixed together. To
distinguish them, source code is wrapped in a source block. In org, this
means a block of the form

+begin_src
...
+end_src

To provide some additional functionality, the source block definition above
can be extended with a number of useful options, including specifying the
language of the source code and the :tangle option. e.g.

+begin_src emacs-lisp :tangle yes
...
+end_src

The above block will be tangled as emacs lisp code and written to the
default output file, which is the org filename with a language appropriate
extension. e.g. if the org file is my-code.org, the tangled output file
will be my-code.el

The org manual states in the section on extracting source code

"When Org tangles code blocks, it expands, merges, and transforms
them.  Then Org recomposes them into one or more separate files, as
configured through the options.  During this tangling process, Org
expands variables in the source code, and resolves any noweb style
references (see *note Noweb Reference Syntax::)."

I think the above paragraph largely describes the feature of supporting
both multiple code blocks (which are expanded, merged and transformed) and
multiple destination files ("...recomposes them into one or more separate
files...").

The manual then goes on to describe the :tangle option

"The ‘tangle’ header argument specifies if the code block is exported to
source file(s)."

Note the use of 'file(s)" to indicate one or more files. The documentation
goes on to explain that the tangle argument can have the vaules "yes", "no"
or a FILENAME. If the argument is a file name

"Export the code block to source file whose file name is derived
from any string passed to the ‘tangle’ header argument.  Org
derives the file name as being relative to the directory of the Org
file’s location.  Example: ‘:tangle FILENAME’."

So, at this point we know

- An org file can contain multiple source blocks
- Org will expand, merge and transform source blocks as required
- By default, a tangled block will be written to a source file with the
same base name as the org file it comes from with a language appropriate
extension
- If the :tangle option specifies a filename, that filename will be used
for *that* block

I think the above covers the behaviour where you have multiple source
blocks in an org file and you use the :tangle FILENAME option to send the
tangled output to different files. The fact there is a test case for this
behaviour further confirms this is expected behaviour.

The only undocumented behaviour is the use of a function instead of a
filename to specify the destination for tangled output. I was not aware
that this functionality is uspported and have not yet tried it. If this
does indeed work, it does need to be added to the manual.


--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 6545 bytes --]