* [ANN] lisp/ob-tangle-sync.el
@ 2023-04-26 14:48 Mehmet Tekman
2023-04-26 16:43 ` John Wiegley
` (2 more replies)
0 siblings, 3 replies; 30+ messages in thread
From: Mehmet Tekman @ 2023-04-26 14:48 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 2423 bytes --]
Dear fellow org-users,
I would like to contribute some a new library into org-mode, which
performs automatic synchronization between tangled files and their
org-mode source blocks via a global minor mode
`org-babel-tangle-sync-mode' which uses the after-save-hook.
Full disclosure:
I created a MELPA package with similar functionality 3 years ago, but
it did not use the existing org libraries much and needlessly
reinvented many wheels:
https://gitlab.com/mtekman/org-tanglesync.el
This is a complete (and very concise) rewrite that uses the org
framework and that I wish to incorporate into org-mode itself. My
changes to the org-mode main branch can be found here:
https://gitlab.com/mtekman/org-mode (ob-tangle-sync branch)
A toy example of what this does would be as follows:
- Source =emacs_conf= org-mode file
#+begin_src bash :comments yes :tangle ~/.bashrc
export PATH=$PATH:/opt/bin
#+end_src
- Tangled =~/.bashrc= file
#+begin_src bash
# [[file:repos/_mtekman/emacs_conf.org::*bashrc][bashrc:1]]
export PATH=$PATH:/opt/bin
#+end_src
By activating =org-babel-tangle-sync-mode=, I can edit either of those
buffers and every time that I save it would automatically update the
other. If I add the header argument =:tangle-sync <action>= then I
can specify an action of:
- skip :: do nothing, just save the buffer, even if the sync mode is active
- pull :: only pull changes from =~/.bashrc= into the Emacs_conf org-mode file
- export :: only export changes from Emacs_conf to =~/.bashrc=, even if called
from =~/.bashrc=
- both :: (default) synchronize freely between tangled file and source block
(the is also the nil value)
By default, the mode acts on any org-mode source block with =:tangle=
headers and any tangled file with file comments. This can be changed
by setting a list of "sync files" via =org-babel-tangle-sync-files=
which only acts if the source block exists in a file given in that
custom variable.
I'm currently trying to write tests for the library, but I would also
greatly welcome any feedback and comments on the currently written
code (attached as a diff, and also available in the above org-mode
repo)
As this is also my first time submitting directly to GNU Emacs, I, um,
might also need some hand-holding. I think I've followed the main
org-contribute guidelines, but I've likely missed or glossed over a
step or two.
Cheerful regards,
Mehmet
[-- Attachment #2: lisp_ob-tangle-sync.el --]
[-- Type: text/x-emacs-lisp, Size: 7952 bytes --]
diff --git a/lisp/ob-tangle-sync.el b/lisp/ob-tangle-sync.el
new file mode 100644
index 000000000..61c23f647
--- /dev/null
+++ b/lisp/ob-tangle-sync.el
@@ -0,0 +1,172 @@
+;;; ob-tangle-sync.el --- Synchronize Source Code and Org Files -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2009-2023 Free Software Foundation, Inc.
+
+;; Author: Mehmet Tekman
+;; Keywords: literate programming, reproducible research
+;; URL: https://orgmode.org
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs. If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; Synchronize the code between source blocks and raw source-code files.
+
+;;; Code:
+
+(require 'org-macs)
+(org-assert-version)
+
+(require 'ol)
+(require 'org)
+(require 'org-element)
+(require 'ob-core)
+
+(defgroup org-babel-tangle-sync nil
+ "Options for synchronizing source code and code blocks."
+ :tag "Org Babel Tangle sync"
+ :group 'org-babel-tangle)
+
+;;;###autoload
+(define-minor-mode org-babel-tangle-sync-mode
+ "Global minor mode that synchronizes tangled files after every save."
+ :global t
+ :interactive t
+ :lighter " o-ts"
+ (if org-babel-tangle-sync-mode
+ (add-hook 'after-save-hook 'org-babel-tangle-sync-synchronize nil t)
+ (remove-hook 'after-save-hook 'org-babel-tangle-sync-synchronize t)))
+
+(defcustom org-babel-tangle-sync-files nil
+ "A list of `org-mode' files.
+When `org-babel-tangle-sync-mode' is enabled only files listed
+here are subject to the org-babel-tangle-sync treatment. If nil,
+then all org files with tangle headers are considered."
+ :group 'org-babel-tangle-sync
+ :type 'list
+ :package-version '(Org . "9.6.5")
+ :set (lambda (_var val) (mapcar #'(lambda (x) (expand-file-name x)) val)))
+
+
+(defun org-babel-tangle-sync--babel-tangle-jump (link block-name)
+ "Jump from a tangled file to the Org file without returning anything.
+The location of the code block in the Org file is given by a
+combination of the LINK filename and header, followed by the
+BLOCK-NAME Org mode source block number. The code is borrowed
+heavily from `org-babel-tangle-jump-to-org'"
+ ;; Go to the beginning of the relative block in Org file.
+ ;; Explicitly allow fuzzy search even if user customized
+ ;; otherwise.
+ (let (org-link-search-must-match-exact-headline)
+ (org-link-open-from-string link))
+ ;;(setq target-buffer (current-buffer))
+ (if (string-match "[^ \t\n\r]:\\([[:digit:]]+\\)" block-name)
+ (let ((n (string-to-number (match-string 1 block-name))))
+ (if (org-before-first-heading-p) (goto-char (point-min))
+ (org-back-to-heading t))
+ ;; Do not skip the first block if it begins at point min.
+ (cond ((or (org-at-heading-p)
+ (not (eq (org-element-type (org-element-at-point))
+ 'src-block)))
+ (org-babel-next-src-block n))
+ ((= n 1))
+ (t (org-babel-next-src-block (1- n)))))
+ (org-babel-goto-named-src-block block-name))
+ (goto-char (org-babel-where-is-src-block-head))
+ (forward-line 1))
+
+;;;###autoload
+(defun org-babel-tangle-sync-synchronize ()
+ "Synchronize a tangled code block to its source-specific file, or vice versa.
+If the cursor is either within the source file or in destination
+tangled file, perform a desired tangling action. The tangling
+action by default is to detangle the tangled files' changes back
+to its source block, or to tangle the source block to its tangled
+file. Actions are one of `skip' (no action), `pull' (detangle
+only), `export' (tangle only), and `both' (default, synchronize
+in both directions). All `org-mode' source blocks and all tangled
+files with comments are considered valid targets, unless
+specified otherwise by `org-babel-tangle-sync-files'."
+ (interactive)
+ (let* ((link (save-excursion
+ (progn (re-search-backward org-link-bracket-re nil t)
+ (match-string-no-properties 0))))
+ (block-name (match-string 2))
+ (orgfile-p (string= major-mode "org-mode"))
+ (tangled-file-p (and link (not orgfile-p))))
+
+ ;; Tangled File → Source Block
+ (if tangled-file-p
+ ;; Examine the block: Get the source file and the desired tangle-sync action
+ (let* ((parsed-link (with-temp-buffer
+ (let ((org-inhibit-startup nil))
+ (insert link)
+ (org-mode)
+ (goto-char (point-min))
+ (org-element-link-parser))))
+ (source-file (expand-file-name
+ (org-element-property :path parsed-link)))
+ (sync-action (save-window-excursion
+ (progn
+ (org-babel-tangle-sync--babel-tangle-jump link block-name)
+ (alist-get :tangle-sync
+ (nth 2 (org-babel-get-src-block-info
+ 'no-eval)))))))
+ ;; De-tangle file back to source block if:
+ ;; - member of sync file list (or list is empty)
+ ;; - source file tangle-sync action isn't "skip" or "export",
+ (if (or (null org-babel-tangle-sync-files)
+ (member source-file org-babel-tangle-sync-files))
+ (cond ((string= sync-action "skip") nil)
+ ((string= sync-action "export")
+ (save-window-excursion
+ (progn (org-babel-tangle-sync--babel-tangle-jump link block-name)
+ (let ((current-prefix-arg '(16)))
+ (call-interactively 'org-babel-tangle))
+ (message "Exported from %s" source-file))))
+ (t
+ (save-window-excursion
+ (org-babel-detangle)
+ (message "Synced to %s" source-file))))))
+
+ ;; Source Block → Tangled File (or Source Block ← Tangled File (via "pull"))
+ (when orgfile-p
+ ;; Tangle action of Source file on Block if:
+ ;; - member of sync file list (or list is empty)
+ ;; Actions
+ ;; - pull (Source Block ← File)
+ ;; - skip (nothing)
+ ;; - export, both, nil (Source Block → File)
+ (if (or (null org-babel-tangle-sync-files)
+ (member buffer-file-name org-babel-tangle-sync-files))
+
+ (let* ((src-headers (nth 2 (org-babel-get-src-block-info 'no-eval)))
+ (tangle-file (cdr (assq :tangle src-headers)))
+ (tangle-action (alist-get :tangle-sync src-headers)))
+ (when tangle-file
+ (cond ((string= tangle-action "pull") (save-excursion
+ (org-babel-detangle tangle-file)))
+ ((string= tangle-action "skip") nil)
+ (t (let ((current-prefix-arg '(16)))
+ (call-interactively 'org-babel-tangle)
+ ;; Revert to see changes, then re-enable the mode
+ (with-current-buffer (get-file-buffer tangle-file)
+ (revert-buffer)
+ (org-babel-tangle-sync-mode t))))))))))))
+
+(provide 'ob-tangle-sync)
+
+;;; ob-tangle-sync.el ends here
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-04-26 14:48 [ANN] lisp/ob-tangle-sync.el Mehmet Tekman
@ 2023-04-26 16:43 ` John Wiegley
2023-04-26 18:43 ` Mehmet Tekman
2023-04-27 2:55 ` Ruijie Yu via General discussions about Org-mode.
2023-04-27 12:02 ` Ihor Radchenko
2 siblings, 1 reply; 30+ messages in thread
From: John Wiegley @ 2023-04-26 16:43 UTC (permalink / raw)
To: Mehmet Tekman; +Cc: emacs-orgmode
>>>>> "MT" == Mehmet Tekman <mtekman89@gmail.com> writes:
MT> - skip :: do nothing, just save the buffer, even if the sync mode is
MT> active
MT> - pull :: only pull changes from =~/.bashrc= into the Emacs_conf org-mode file
MT> - export :: only export changes from Emacs_conf to =~/.bashrc=, even if called
MT> from =~/.bashrc=
For the sake of consistency, perhaps import and export, rather than pull and
export? And, great idea!
--
John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-04-26 16:43 ` John Wiegley
@ 2023-04-26 18:43 ` Mehmet Tekman
0 siblings, 0 replies; 30+ messages in thread
From: Mehmet Tekman @ 2023-04-26 18:43 UTC (permalink / raw)
To: emacs-orgmode
> For the sake of consistency, perhaps import and export, rather than pull and
> export? And, great idea!
Absolutely - unless the "pull" / "push" crowd wants to chime in!
I'm also not 100% sold on the "both" action keyword. There,
writing "sync" would make more sense, but I'm hesitant to write
a =:tangle-sync sync= in the header.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-04-26 14:48 [ANN] lisp/ob-tangle-sync.el Mehmet Tekman
2023-04-26 16:43 ` John Wiegley
@ 2023-04-27 2:55 ` Ruijie Yu via General discussions about Org-mode.
2023-04-27 6:27 ` Mehmet Tekman
2023-04-27 12:02 ` Ihor Radchenko
2 siblings, 1 reply; 30+ messages in thread
From: Ruijie Yu via General discussions about Org-mode. @ 2023-04-27 2:55 UTC (permalink / raw)
To: Mehmet Tekman; +Cc: emacs-orgmode
Mehmet Tekman <mtekman89@gmail.com> writes:
> Dear fellow org-users,
>
> I would like to contribute some a new library into org-mode, which
> performs automatic synchronization between tangled files and their
> org-mode source blocks via a global minor mode
> `org-babel-tangle-sync-mode' which uses the after-save-hook.
Great idea! Some inline comments below.
> diff --git a/lisp/ob-tangle-sync.el b/lisp/ob-tangle-sync.el
> new file mode 100644
> index 000000000..61c23f647
> --- /dev/null
> +++ b/lisp/ob-tangle-sync.el
> @@ -0,0 +1,172 @@
> +;;; ob-tangle-sync.el --- Synchronize Source Code and Org Files -*- lexical-binding: t; -*-
> +
> +;; Copyright (C) 2009-2023 Free Software Foundation, Inc.
> +
> +;; Author: Mehmet Tekman
> +;; Keywords: literate programming, reproducible research
> +;; URL: https://orgmode.org
> +
> +;; This file is part of GNU Emacs.
> +
> +;; GNU Emacs is free software: you can redistribute it and/or modify
> +;; it under the terms of the GNU General Public License as published by
> +;; the Free Software Foundation, either version 3 of the License, or
> +;; (at your option) any later version.
> +
> +;; GNU Emacs is distributed in the hope that it will be useful,
> +;; but WITHOUT ANY WARRANTY; without even the implied warranty of
> +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +;; GNU General Public License for more details.
> +
> +;; You should have received a copy of the GNU General Public License
> +;; along with GNU Emacs. If not, see <https://www.gnu.org/licenses/>.
> +
> +;;; Commentary:
> +
> +;; Synchronize the code between source blocks and raw source-code files.
> +
> +;;; Code:
> +
> +(require 'org-macs)
> +(org-assert-version)
> +
> +(require 'ol)
> +(require 'org)
> +(require 'org-element)
> +(require 'ob-core)
> +
> +(defgroup org-babel-tangle-sync nil
> + "Options for synchronizing source code and code blocks."
> + :tag "Org Babel Tangle sync"
> + :group 'org-babel-tangle)
> +
> +;;;###autoload
> +(define-minor-mode org-babel-tangle-sync-mode
> + "Global minor mode that synchronizes tangled files after every save."
> + :global t
Is there possibility to have a local minor mode (without introducing too
much code changes)?
> + :interactive t
I believe interactive is the default?
> + :lighter " o-ts"
> + (if org-babel-tangle-sync-mode
> + (add-hook 'after-save-hook 'org-babel-tangle-sync-synchronize nil t)
> + (remove-hook 'after-save-hook 'org-babel-tangle-sync-synchronize t)))
> +
> +(defcustom org-babel-tangle-sync-files nil
> + "A list of `org-mode' files.
> +When `org-babel-tangle-sync-mode' is enabled only files listed
> +here are subject to the org-babel-tangle-sync treatment. If nil,
> +then all org files with tangle headers are considered."
> + :group 'org-babel-tangle-sync
> + :type 'list
> + :package-version '(Org . "9.6.5")
> + :set (lambda (_var val) (mapcar #'(lambda (x) (expand-file-name x)) val)))
You only need to say #'expand-file-name instead of the quoted lambda.
Also, you need to set the variable, otherwise the variable
`org-babel-tangle-sync-files' is undefined.
What I have in mind is this:
:set (lambda (var val) (set var (mapcar #'expand-file-name val)))
> +
> +
> +(defun org-babel-tangle-sync--babel-tangle-jump (link block-name)
> + "Jump from a tangled file to the Org file without returning anything.
> +The location of the code block in the Org file is given by a
> +combination of the LINK filename and header, followed by the
> +BLOCK-NAME Org mode source block number. The code is borrowed
> +heavily from `org-babel-tangle-jump-to-org'"
> + ;; Go to the beginning of the relative block in Org file.
> + ;; Explicitly allow fuzzy search even if user customized
> + ;; otherwise.
> + (let (org-link-search-must-match-exact-headline)
> + (org-link-open-from-string link))
> + ;;(setq target-buffer (current-buffer))
> + (if (string-match "[^ \t\n\r]:\\([[:digit:]]+\\)" block-name)
> + (let ((n (string-to-number (match-string 1 block-name))))
> + (if (org-before-first-heading-p) (goto-char (point-min))
> + (org-back-to-heading t))
> + ;; Do not skip the first block if it begins at point min.
> + (cond ((or (org-at-heading-p)
> + (not (eq (org-element-type (org-element-at-point))
> + 'src-block)))
> + (org-babel-next-src-block n))
> + ((= n 1))
> + (t (org-babel-next-src-block (1- n)))))
> + (org-babel-goto-named-src-block block-name))
> + (goto-char (org-babel-where-is-src-block-head))
> + (forward-line 1))
> +
> +;;;###autoload
> +(defun org-babel-tangle-sync-synchronize ()
> + "Synchronize a tangled code block to its source-specific file, or vice versa.
> +If the cursor is either within the source file or in destination
> +tangled file, perform a desired tangling action. The tangling
> +action by default is to detangle the tangled files' changes back
> +to its source block, or to tangle the source block to its tangled
> +file. Actions are one of `skip' (no action), `pull' (detangle
> +only), `export' (tangle only), and `both' (default, synchronize
> +in both directions). All `org-mode' source blocks and all tangled
> +files with comments are considered valid targets, unless
> +specified otherwise by `org-babel-tangle-sync-files'."
> + (interactive)
> + (let* ((link (save-excursion
> + (progn (re-search-backward org-link-bracket-re nil t)
> + (match-string-no-properties 0))))
Here you don't have to use `progn' because it is implied from `save-excursion'.
> + (block-name (match-string 2))
> + (orgfile-p (string= major-mode "org-mode"))
> + (tangled-file-p (and link (not orgfile-p))))
> +
> + ;; Tangled File → Source Block
> + (if tangled-file-p
> + ;; Examine the block: Get the source file and the desired tangle-sync action
> + (let* ((parsed-link (with-temp-buffer
> + (let ((org-inhibit-startup nil))
> + (insert link)
> + (org-mode)
> + (goto-char (point-min))
> + (org-element-link-parser))))
> + (source-file (expand-file-name
> + (org-element-property :path parsed-link)))
> + (sync-action (save-window-excursion
> + (progn
> + (org-babel-tangle-sync--babel-tangle-jump link block-name)
> + (alist-get :tangle-sync
> + (nth 2 (org-babel-get-src-block-info
> + 'no-eval)))))))
> + ;; De-tangle file back to source block if:
> + ;; - member of sync file list (or list is empty)
> + ;; - source file tangle-sync action isn't "skip" or "export",
> + (if (or (null org-babel-tangle-sync-files)
> + (member source-file org-babel-tangle-sync-files))
> + (cond ((string= sync-action "skip") nil)
> + ((string= sync-action "export")
> + (save-window-excursion
> + (progn (org-babel-tangle-sync--babel-tangle-jump link block-name)
> + (let ((current-prefix-arg '(16)))
> + (call-interactively 'org-babel-tangle))
> + (message "Exported from %s" source-file))))
> + (t
> + (save-window-excursion
> + (org-babel-detangle)
> + (message "Synced to %s" source-file))))))
> +
> + ;; Source Block → Tangled File (or Source Block ← Tangled File (via "pull"))
> + (when orgfile-p
> + ;; Tangle action of Source file on Block if:
> + ;; - member of sync file list (or list is empty)
> + ;; Actions
> + ;; - pull (Source Block ← File)
> + ;; - skip (nothing)
> + ;; - export, both, nil (Source Block → File)
> + (if (or (null org-babel-tangle-sync-files)
> + (member buffer-file-name org-babel-tangle-sync-files))
> +
> + (let* ((src-headers (nth 2 (org-babel-get-src-block-info 'no-eval)))
> + (tangle-file (cdr (assq :tangle src-headers)))
> + (tangle-action (alist-get :tangle-sync src-headers)))
> + (when tangle-file
> + (cond ((string= tangle-action "pull") (save-excursion
> + (org-babel-detangle tangle-file)))
> + ((string= tangle-action "skip") nil)
> + (t (let ((current-prefix-arg '(16)))
> + (call-interactively 'org-babel-tangle)
> + ;; Revert to see changes, then re-enable the mode
> + (with-current-buffer (get-file-buffer tangle-file)
> + (revert-buffer)
> + (org-babel-tangle-sync-mode t))))))))))))
> +
> +(provide 'ob-tangle-sync)
> +
> +;;; ob-tangle-sync.el ends here
--
Best,
RY
[Please note that this mail might go to spam due to some
misconfiguration in my mail server -- still investigating.]
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-04-27 2:55 ` Ruijie Yu via General discussions about Org-mode.
@ 2023-04-27 6:27 ` Mehmet Tekman
2023-04-28 10:57 ` Ruijie Yu via General discussions about Org-mode.
0 siblings, 1 reply; 30+ messages in thread
From: Mehmet Tekman @ 2023-04-27 6:27 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 2255 bytes --]
Ruijie Yu <ruijie@netyu.xyz> writes:
> Great idea! Some inline comments below.
>
> You only need to say #'expand-file-name instead of the quoted lambda.
> Also, you need to set the variable, otherwise the variable
> `org-babel-tangle-sync-files' is undefined.
>
> What I have in mind is this:
> :set (lambda (var val) (set var (mapcar #'expand-file-name val)))
>
> Here you don't have to use `progn' because it is implied from `save-excursion'.
>
Thanks! I've made your changes, and I've also incorporated John
Wiegley's comments about using "import" instead of "pull" as a
tangle-sync action word (small diff attached).
I've also written up my changes in the ~etc/ORG-NEWS~ and
targeted my custom variable for the 9.7 release (diff attached).
> Is there possibility to have a local minor mode (without introducing too
> much code changes)?
I initially tried it this way, but the problem is that an org
source block buffer might be in sync-mode, but it's
corresponding tangle file might not be, making any changes
asymmetric.
Another issue is in order to see the changes in the tangled file,
the tangle buffer needs to be reverted (with user prompt) which
then switches off the sync-mode for that buffer on reload.
One way around this (and it's something I implemented 3 years ago in
my messy org-tanglesync[0] MELPA code) is to set an explicit list of
"sync files", and then for Emacs to parse every =:tangle= header in a
given file when loaded (via =org-src-mode-hook=) to create an alist of
config files and their associated tangled files[1], such as
=((file1.conf . (tanglefile1.txt tanglefile2.txt etc)))=. Then, for
example, when ~tanglefile1.txt~ is loaded, Emacs knows that it should
load the sync-mode too.
This approach works reasonably well when the "sync files" list is
mandatory, but it's also prone to errors if a sync file is edited and
the alist of config files isn't updated, and the user would also lose
the flexibility of having ~ob-tangle-sync~ function everywhere.
I think a global minor mode is really elegant in this regard and I
wish I knew about it 3 years ago!
Best,
Mehmet
0: https://gitlab.com/mtekman/org-tanglesync.el
1: https://gitlab.com/mtekman/org-tanglesync.el/-/blob/master/org-tanglesync.el#L400-L410
[-- Attachment #2: etc_ORG-NEWS.diff --]
[-- Type: text/x-patch, Size: 1543 bytes --]
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 03894f128..29f60c755 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -207,12 +207,35 @@ selection.
TODO state, priority, tags, statistics cookies, and COMMENT keywords
are allowed in the tree structure.
-*** Asynchronous code evaluatation in ~ob-shell~
+*** Asynchronous code evaluation in ~ob-shell~
Running shell blocks with the ~:session~ header freezes Emacs until
execution completes. The new ~:async~ header allows users to continue
editing with Emacs while a ~:session~ block executes.
+*** Automatic sync of source blocks and tangled blocks in ~ob-tangle-sync~
+
+Invoking minor mode =org-babel-tangle-sync-mode= synchronizes contents
+between a currently visited tangled file its org-mode source block
+(and vice versa) via the =after-save-hook=.
+
+Desired tangling actions can be assymetric depending on whether the
+org-mode source block header argument =:tangle-sync <action>= has an
+action of:
+
+- =skip= :: do nothing, just save the buffer, even if the sync mode is
+ active
+
+- =import= :: only pull changes from the tangled block into the
+ org-mode source block (even when visited from either)
+
+- =export= :: only pull changes from the org-mode source block into
+ the tangled block (even when visited from either)
+
+- =both= (or nil) :: freely sync changes of current buffer to
+ associated source or target
+
+
** Miscellaneous
*** Blank lines after removed objects are not retained during export
[-- Attachment #3: lisp_ob-tangle-sync.el.diff --]
[-- Type: text/x-patch, Size: 3369 bytes --]
diff --git a/lisp/ob-tangle-sync.el b/lisp/ob-tangle-sync.el
index 61c23f647..cfa6abdd2 100644
--- a/lisp/ob-tangle-sync.el
+++ b/lisp/ob-tangle-sync.el
@@ -44,7 +44,6 @@
(define-minor-mode org-babel-tangle-sync-mode
"Global minor mode that synchronizes tangled files after every save."
:global t
- :interactive t
:lighter " o-ts"
(if org-babel-tangle-sync-mode
(add-hook 'after-save-hook 'org-babel-tangle-sync-synchronize nil t)
@@ -57,8 +56,8 @@ here are subject to the org-babel-tangle-sync treatment. If nil,
then all org files with tangle headers are considered."
:group 'org-babel-tangle-sync
:type 'list
- :package-version '(Org . "9.6.5")
- :set (lambda (_var val) (mapcar #'(lambda (x) (expand-file-name x)) val)))
+ :package-version '(Org . "9.7")
+ :set (lambda (var val) (set var (mapcar #'expand-file-name val))))
(defun org-babel-tangle-sync--babel-tangle-jump (link block-name)
@@ -95,15 +94,15 @@ If the cursor is either within the source file or in destination
tangled file, perform a desired tangling action. The tangling
action by default is to detangle the tangled files' changes back
to its source block, or to tangle the source block to its tangled
-file. Actions are one of `skip' (no action), `pull' (detangle
+file. Actions are one of `skip' (no action), `import' (detangle
only), `export' (tangle only), and `both' (default, synchronize
in both directions). All `org-mode' source blocks and all tangled
files with comments are considered valid targets, unless
specified otherwise by `org-babel-tangle-sync-files'."
(interactive)
(let* ((link (save-excursion
- (progn (re-search-backward org-link-bracket-re nil t)
- (match-string-no-properties 0))))
+ (re-search-backward org-link-bracket-re nil t)
+ (match-string-no-properties 0)))
(block-name (match-string 2))
(orgfile-p (string= major-mode "org-mode"))
(tangled-file-p (and link (not orgfile-p))))
@@ -142,12 +141,12 @@ specified otherwise by `org-babel-tangle-sync-files'."
(org-babel-detangle)
(message "Synced to %s" source-file))))))
- ;; Source Block → Tangled File (or Source Block ← Tangled File (via "pull"))
+ ;; Source Block → Tangled File (or Source Block ← Tangled File (via "import"))
(when orgfile-p
;; Tangle action of Source file on Block if:
;; - member of sync file list (or list is empty)
;; Actions
- ;; - pull (Source Block ← File)
+ ;; - import (Source Block ← File)
;; - skip (nothing)
;; - export, both, nil (Source Block → File)
(if (or (null org-babel-tangle-sync-files)
@@ -157,7 +156,7 @@ specified otherwise by `org-babel-tangle-sync-files'."
(tangle-file (cdr (assq :tangle src-headers)))
(tangle-action (alist-get :tangle-sync src-headers)))
(when tangle-file
- (cond ((string= tangle-action "pull") (save-excursion
+ (cond ((string= tangle-action "import") (save-excursion
(org-babel-detangle tangle-file)))
((string= tangle-action "skip") nil)
(t (let ((current-prefix-arg '(16)))
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-04-26 14:48 [ANN] lisp/ob-tangle-sync.el Mehmet Tekman
2023-04-26 16:43 ` John Wiegley
2023-04-27 2:55 ` Ruijie Yu via General discussions about Org-mode.
@ 2023-04-27 12:02 ` Ihor Radchenko
2023-04-27 13:01 ` Mehmet Tekman
2 siblings, 1 reply; 30+ messages in thread
From: Ihor Radchenko @ 2023-04-27 12:02 UTC (permalink / raw)
To: Mehmet Tekman; +Cc: emacs-orgmode
Mehmet Tekman <mtekman89@gmail.com> writes:
> I would like to contribute some a new library into org-mode, which
> performs automatic synchronization between tangled files and their
> org-mode source blocks via a global minor mode
> `org-babel-tangle-sync-mode' which uses the after-save-hook.
Thanks for your interest in contributing to Org mode!
> By activating =org-babel-tangle-sync-mode=, I can edit either of those
> buffers and every time that I save it would automatically update the
> other. If I add the header argument =:tangle-sync <action>= then I
> can specify an action of:
> - skip :: do nothing, just save the buffer, even if the sync mode is active
> - pull :: only pull changes from =~/.bashrc= into the Emacs_conf org-mode file
> - export :: only export changes from Emacs_conf to =~/.bashrc=, even if called
> from =~/.bashrc=
> - both :: (default) synchronize freely between tangled file and source block
> (the is also the nil value)
Do I understand correctly that your package is adding the following new
features:
1. Automatically manages balanced
`org-babel-tangle'/`org-babel-detangle' in Org sources and target
buffers.
2. Attempts to make more fine-grained tangling/detanging functionality
by controlling tangling behaviour on per-src-block basis (via
:tangle-sync skip/pull/export/both)
If so, I think that (2) probably belongs to the main babel code
(`org-babel-tangle' and `org-babel-detangle'). Since you are
contributing to Org directly, you are free to modify these functions as
you need.
Note that your current approach with `org-babel-tangle-sync-synchronize'
will give unexpected results when one edits multiple src blocks or the
corresponding tangled source in multiple places - the :tangle-sync value
is only checked at point and `org-babel-tangle'/`org-babel-detange' is
then called for the whole buffer, not just for the current code block.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-04-27 12:02 ` Ihor Radchenko
@ 2023-04-27 13:01 ` Mehmet Tekman
0 siblings, 0 replies; 30+ messages in thread
From: Mehmet Tekman @ 2023-04-27 13:01 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 1637 bytes --]
Ihor Radchenko <yantar92@posteo.net> writes:
> Do I understand correctly that your package is adding the following new
> features:
>
> 1. Automatically manages balanced
> `org-babel-tangle'/`org-babel-detangle' in Org sources and target
> buffers.
> 2. Attempts to make more fine-grained tangling/detanging functionality
> by controlling tangling behaviour on per-src-block basis (via
> :tangle-sync skip/pull/export/both)
>
That is correct
> If so, I think that (2) probably belongs to the main babel code
> (`org-babel-tangle' and `org-babel-detangle'). Since you are
> contributing to Org directly, you are free to modify these functions as
> you need.
>
> Note that your current approach with `org-babel-tangle-sync-synchronize'
> will give unexpected results when one edits multiple src blocks or the
> corresponding tangled source in multiple places - the :tangle-sync value
> is only checked at point and `org-babel-tangle'/`org-babel-detange' is
> then called for the whole buffer, not just for the current code block.
That's... a really good point. I didn't consider that I'm calling a
tangle or detangle function on an entire file based on the
=:tangle-sync action= of a single block... which might have different
actions in another block for the same file!
I've attached a toy.org file which should explain what the desired
action would be in such a situation, and I fully agree that I will
likely need to modify the current tangle and detangle code so that the
sync direction is on a per-block basis, and not per-file.
I will work on this a bit more and submit a full patch with this in
mind.
Best,
Mehmet
[-- Attachment #2: toy.org --]
[-- Type: text/org, Size: 1561 bytes --]
#+TITLE: Sync Many Blocks
#+PROPERTY: header-args :tangle /tmp/default_tangle.txt :comments yes
* Default Tangle
The =:tangle= target here is given by the ~PROPERTY~ header. It is
made up of three parts which synchronize with the tangled destination
file in 3 different ways as given below:
** File Contents
*** Free Head
#+begin_src conf
This is some text that can be synced both ways
#+end_src
*** Pull Middle
#+begin_src conf :tangle-sync import
The middle text only retrieves from the default tangle file
#+end_src
*** Push tail
#+begin_src conf :tangle-sync export
This last piece of text only exports from the source org mode file.
#+end_src
* Desired Function
When =org-babel-tangle= is called, only the [[Free Head]] and [[Push tail]]
sections are tangled. The middle section in the tangled file is
_skipped_.
When =org-babel-detangle= is called, only the [[Pull Middle]] section is
updated here. The other two sections are _skipped_.
* Why skipped?
The functions suffixes explicitly state a direction (=tangle= /
=detangle=), and it's probably important that we retain this
directionality.
If we want to synchronize all blocks (i.e. import the middle section
from the tangled file during =org-babel-tangle=, and export the head
and tail sections during =org-babel-detangle=), then perhaps we set a
specific custom variable =(setq org-babel-tangle-alwayssync t)=?
Or, we create a specific function =org-babel-sync= (for lack of better
name) which tangles/detangles in whatever desired direction the source
block specifies.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-04-27 6:27 ` Mehmet Tekman
@ 2023-04-28 10:57 ` Ruijie Yu via General discussions about Org-mode.
2023-04-28 11:28 ` Mehmet Tekman
0 siblings, 1 reply; 30+ messages in thread
From: Ruijie Yu via General discussions about Org-mode. @ 2023-04-28 10:57 UTC (permalink / raw)
To: Mehmet Tekman; +Cc: emacs-orgmode
Mehmet Tekman <mtekman89@gmail.com> writes:
> Ruijie Yu <ruijie@netyu.xyz> writes:
>
>> Great idea! Some inline comments below.
>>
>> You only need to say #'expand-file-name instead of the quoted lambda.
>> Also, you need to set the variable, otherwise the variable
>> `org-babel-tangle-sync-files' is undefined.
>>
>> What I have in mind is this:
>> :set (lambda (var val) (set var (mapcar #'expand-file-name val)))
>>
>> Here you don't have to use `progn' because it is implied from `save-excursion'.
>>
>
> Thanks! I've made your changes, and I've also incorporated John
> Wiegley's comments about using "import" instead of "pull" as a
> tangle-sync action word (small diff attached).
>
> I've also written up my changes in the ~etc/ORG-NEWS~ and
> targeted my custom variable for the 9.7 release (diff attached).
Thanks. Can you make a full patch from the current main branch to your
changes, with a commit message and so on? This would help reviewers to
look at the full picture of what are modified. Take a look at the
manpage git-format-patch(1) if you aren't sure how to do it.
For the commit message, take a look at
https://orgmode.org/worg/org-contribute.html#commit-messages for
inspirations.
>> Is there possibility to have a local minor mode (without introducing too
>> much code changes)?
>
> I initially tried it this way, but the problem is that an org
> source block buffer might be in sync-mode, but it's
> corresponding tangle file might not be, making any changes
> asymmetric.
>
> Another issue is in order to see the changes in the tangled file,
> the tangle buffer needs to be reverted (with user prompt) which
> then switches off the sync-mode for that buffer on reload.
>
> One way around this (and it's something I implemented 3 years ago in
> my messy org-tanglesync[0] MELPA code) is to set an explicit list of
> "sync files", and then for Emacs to parse every =:tangle= header in a
> given file when loaded (via =org-src-mode-hook=) to create an alist of
> config files and their associated tangled files[1], such as
> =((file1.conf . (tanglefile1.txt tanglefile2.txt etc)))=. Then, for
> example, when ~tanglefile1.txt~ is loaded, Emacs knows that it should
> load the sync-mode too.
>
> This approach works reasonably well when the "sync files" list is
> mandatory, but it's also prone to errors if a sync file is edited and
> the alist of config files isn't updated, and the user would also lose
> the flexibility of having ~ob-tangle-sync~ function everywhere.
>
> I think a global minor mode is really elegant in this regard and I
> wish I knew about it 3 years ago!
Thanks for explaining this. Yes, this sounds like a lot of work with
probably insufficient audience, so I'd wait for more use cases to come
up before thinking about local minor modes.
--
Best,
RY
[Please note that this mail might go to spam due to some
misconfiguration in my mail server -- still investigating.]
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-04-28 10:57 ` Ruijie Yu via General discussions about Org-mode.
@ 2023-04-28 11:28 ` Mehmet Tekman
2023-05-02 20:43 ` Mehmet Tekman
0 siblings, 1 reply; 30+ messages in thread
From: Mehmet Tekman @ 2023-04-28 11:28 UTC (permalink / raw)
To: emacs-orgmode
Ruijie Yu <ruijie@netyu.xyz> writes:
> Thanks. Can you make a full patch from the current main branch to your
> changes, with a commit message and so on? This would help reviewers to
> look at the full picture of what are modified. Take a look at the
> manpage git-format-patch(1) if you aren't sure how to do it.
I'm at the moment patching the =org-babel-tangle= function, to act
appropriately to the =:tangle-sync <action>= requests.
It's harder than I thought, because the way the function currently
works is to populate a temporary buffer with source block contents and
then to overwrite the destination tangle file, i.e. it is unaware of
the tangled files contents during processing.
This is efficient, but works anathema to syncing changes in the
opposite direction, i.e. when a block requests =:tangle-sync import=.
I'm currently doing a small rewrite of the function that populates a
list of comments in the tangled file (restricted to stemming from the
source org file), and then either populates this temporary buffer with
the org mode source block contents or with the already tangled
contents (when "import" is given for that block).
I will submit a full patch with these changes (with properly detailed
commit messages) from the main branch in the next few days.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-04-28 11:28 ` Mehmet Tekman
@ 2023-05-02 20:43 ` Mehmet Tekman
2023-05-03 2:31 ` Ruijie Yu via General discussions about Org-mode.
2023-05-03 11:43 ` Ihor Radchenko
0 siblings, 2 replies; 30+ messages in thread
From: Mehmet Tekman @ 2023-05-02 20:43 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1.1: Type: text/plain, Size: 1319 bytes --]
Hello again, sorry for the delay - I had some holiday time off
that couldn't wait ;-)
I've modified the ob-tangle.el file for the main tangling and
detangling functions. Most importantly, both functions can now
exchange information from the source Org mode file to the target
remote tangle file in either direction, depending on whether the
source Org file has `:tangle-sync <action>' in the header.
The action is one of:
- "export" = always transmit information from the source Org mode
block to the target remote file.
- "import" = always transmit information from the target remote
file to the source Org mode block.
- "skip" = skip the block.
- "both" = transmit information from source block to target block
or target block to source, depending on whether the
tangle or detangle is called from the source buffer or
the target buffer respectively.
These functions work at the whole buffer and at the per-block
level. The `org-babel-tangle-sync' functions automate this
process by hooking into the `after-save-hook' and tangling /
detangling the current block.
I feel that I should write what the main motivation for this is:
Dotfiles that are always in sync with the org-mode files they
stem from.
Hope this turns into something big!
Best,
Mehmet
[-- Attachment #1.2: Type: text/html, Size: 1544 bytes --]
[-- Attachment #2: 0005-lisp-ob-tangle-sync.el-Automatic-synchronization-of-.patch --]
[-- Type: application/x-patch, Size: 6766 bytes --]
[-- Attachment #3: 0004-lisp-ob-tangle.el-Sync-aware-tangle-function-with-be.patch --]
[-- Type: application/x-patch, Size: 5843 bytes --]
[-- Attachment #4: 0002-lisp-ob-tangle.el-Sync-action-aware-detangle-functio.patch --]
[-- Type: application/x-patch, Size: 4628 bytes --]
[-- Attachment #5: 0003-lisp-ob-tangle.el-Tangle-function-made-aware-of-remo.patch --]
[-- Type: application/x-patch, Size: 4121 bytes --]
[-- Attachment #6: 0001-lisp-ob-tangle.el-Detangle-a-single-block.patch --]
[-- Type: application/x-patch, Size: 5474 bytes --]
[-- Attachment #7: 0006-etc-ORG-NEWS-lisp-ob-tangle.el-Added-news-and-name.patch --]
[-- Type: application/x-patch, Size: 2779 bytes --]
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-02 20:43 ` Mehmet Tekman
@ 2023-05-03 2:31 ` Ruijie Yu via General discussions about Org-mode.
2023-05-03 7:53 ` Mehmet Tekman
2023-05-03 11:43 ` Ihor Radchenko
1 sibling, 1 reply; 30+ messages in thread
From: Ruijie Yu via General discussions about Org-mode. @ 2023-05-03 2:31 UTC (permalink / raw)
To: Mehmet Tekman; +Cc: emacs-orgmode
Mehmet Tekman <mtekman89@gmail.com> writes:
> Hello again, sorry for the delay - I had some holiday time off
> that couldn't wait ;-)
>
> I've modified the ob-tangle.el file for the main tangling and
> detangling functions. Most importantly, both functions can now
> exchange information from the source Org mode file to the target
> remote tangle file in either direction, depending on whether the
> source Org file has `:tangle-sync <action>' in the header.
>
> The action is one of:
>
> - "export" = always transmit information from the source Org mode
> block to the target remote file.
> - "import" = always transmit information from the target remote
> file to the source Org mode block.
> - "skip" = skip the block.
> - "both" = transmit information from source block to target block
> or target block to source, depending on whether the
> tangle or detangle is called from the source buffer or
> the target buffer respectively.
>
> These functions work at the whole buffer and at the per-block
> level. The `org-babel-tangle-sync' functions automate this
> process by hooking into the `after-save-hook' and tangling /
> detangling the current block.
>
> I feel that I should write what the main motivation for this is:
> Dotfiles that are always in sync with the org-mode files they
> stem from.
>
> Hope this turns into something big!
> Best,
>
> Mehmet
>
> [4. application/x-patch; 0005-lisp-ob-tangle-sync.el-Automatic-synchronization-of-.patch]...
>
> [5. application/x-patch; 0004-lisp-ob-tangle.el-Sync-aware-tangle-function-with-be.patch]...
>
> [6. application/x-patch; 0002-lisp-ob-tangle.el-Sync-action-aware-detangle-functio.patch]...
>
> [7. application/x-patch; 0003-lisp-ob-tangle.el-Tangle-function-made-aware-of-remo.patch]...
>
> [8. application/x-patch; 0001-lisp-ob-tangle.el-Detangle-a-single-block.patch]...
>
> [9. application/x-patch; 0006-etc-ORG-NEWS-lisp-ob-tangle.el-Added-news-and-name.patch]...
I noticed that you modified argument order of a public function
`org-babel-detangle' -- is it possible that you retain the order of
arguments?
The accumulated change you proposed is this:
-(defun org-babel-detangle (&optional source-code-file)
+(defun org-babel-detangle (&optional arg source-code-file ignore-header)
What I think it should be:
(defun org-babel-detangle (&optional source-code-file arg ignore-header) ...)
This way, existing (external) users of this function can rest assured
that their code is not broken by your patchset. Thoughts?
--
Best,
RY
[Please note that this mail might go to spam due to some
misconfiguration in my mail server -- still investigating.]
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-03 2:31 ` Ruijie Yu via General discussions about Org-mode.
@ 2023-05-03 7:53 ` Mehmet Tekman
2023-05-03 8:34 ` Mehmet Tekman
0 siblings, 1 reply; 30+ messages in thread
From: Mehmet Tekman @ 2023-05-03 7:53 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1.1: Type: text/plain, Size: 3422 bytes --]
> I noticed that you modified argument order of a public function
> `org-babel-detangle' -- is it possible that you retain the order of
> arguments?
Good catch - I think I changed the order so that it would match
the interactivity of its sister function `org-babel-tangle', and
I think in a previous rendition of ob-tangle-sync.el, I called
`org-babel-detangle' interactively. But I see now that I no
longer directly call it anywhere in my edited code, so I can
easily change it back.
Attached is a patch
Best
On Wed, 3 May 2023 at 04:35, Ruijie Yu <ruijie@netyu.xyz> wrote:
>
> Mehmet Tekman <mtekman89@gmail.com> writes:
>
> > Hello again, sorry for the delay - I had some holiday time off
> > that couldn't wait ;-)
> >
> > I've modified the ob-tangle.el file for the main tangling and
> > detangling functions. Most importantly, both functions can now
> > exchange information from the source Org mode file to the target
> > remote tangle file in either direction, depending on whether the
> > source Org file has `:tangle-sync <action>' in the header.
> >
> > The action is one of:
> >
> > - "export" = always transmit information from the source Org mode
> > block to the target remote file.
> > - "import" = always transmit information from the target remote
> > file to the source Org mode block.
> > - "skip" = skip the block.
> > - "both" = transmit information from source block to target block
> > or target block to source, depending on whether the
> > tangle or detangle is called from the source buffer or
> > the target buffer respectively.
> >
> > These functions work at the whole buffer and at the per-block
> > level. The `org-babel-tangle-sync' functions automate this
> > process by hooking into the `after-save-hook' and tangling /
> > detangling the current block.
> >
> > I feel that I should write what the main motivation for this is:
> > Dotfiles that are always in sync with the org-mode files they
> > stem from.
> >
> > Hope this turns into something big!
> > Best,
> >
> > Mehmet
> >
> > [4. application/x-patch;
> 0005-lisp-ob-tangle-sync.el-Automatic-synchronization-of-.patch]...
> >
> > [5. application/x-patch;
> 0004-lisp-ob-tangle.el-Sync-aware-tangle-function-with-be.patch]...
> >
> > [6. application/x-patch;
> 0002-lisp-ob-tangle.el-Sync-action-aware-detangle-functio.patch]...
> >
> > [7. application/x-patch;
> 0003-lisp-ob-tangle.el-Tangle-function-made-aware-of-remo.patch]...
> >
> > [8. application/x-patch;
> 0001-lisp-ob-tangle.el-Detangle-a-single-block.patch]...
> >
> > [9. application/x-patch;
> 0006-etc-ORG-NEWS-lisp-ob-tangle.el-Added-news-and-name.patch]...
>
> I noticed that you modified argument order of a public function
> `org-babel-detangle' -- is it possible that you retain the order of
> arguments?
>
> The accumulated change you proposed is this:
>
> -(defun org-babel-detangle (&optional source-code-file)
> +(defun org-babel-detangle (&optional arg source-code-file ignore-header)
>
> What I think it should be:
>
> (defun org-babel-detangle (&optional source-code-file arg
> ignore-header) ...)
>
> This way, existing (external) users of this function can rest assured
> that their code is not broken by your patchset. Thoughts?
>
> --
> Best,
>
>
> RY
>
> [Please note that this mail might go to spam due to some
> misconfiguration in my mail server -- still investigating.]
>
[-- Attachment #1.2: Type: text/html, Size: 4328 bytes --]
[-- Attachment #2: 0007-lisp-ob-tangle.el-Restore-argument-order-in-org-babe.patch --]
[-- Type: text/x-patch, Size: 2193 bytes --]
From 3399279e2c5c2a56076fa85adffffe7aa5c2ca74 Mon Sep 17 00:00:00 2001
From: MT <mtekman89@gmail.com>
Date: Wed, 3 May 2023 09:47:34 +0200
Subject: [PATCH 7/7] lisp/ob-tangle.el: Restore argument order in
org-babel-detangle
* ob-tangle.el (org-babel-detangle): Change `source-code-file' to be the
first argument again for external usage. This changes the prefix
argument functionality, which is replaced by a much easier `single-p'
argument to denote whether a single block is being detangled.
---
lisp/ob-tangle.el | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el
index f199c77bc..fde6683a6 100644
--- a/lisp/ob-tangle.el
+++ b/lisp/ob-tangle.el
@@ -660,13 +660,13 @@ by `org-babel-get-src-block-info'."
(org-fill-template org-babel-tangle-comment-format-end link-data))))
;; de-tangling functions
-(defun org-babel-detangle (&optional arg source-code-file ignore-header)
+(defun org-babel-detangle (&optional source-code-file single-p ignore-header)
"Propagate changes in source file back original to Org file.
This requires that code blocks were tangled with link comments
-which enable the original code blocks to be found. With one
-universal prefix argument, only detangle the block at point. If
-IGNORE-HEADER then detangle regardless of `:tangle-sync' status."
- (interactive "P")
+which enable the original code blocks to be found. If SINGLE-P
+is t then only detangle a single block. If IGNORE-HEADER then
+detangle regardless of `:tangle-sync' status."
+ (interactive)
(save-excursion
(when source-code-file (find-file source-code-file))
(let ((counter 0) (skip-counter 0) (tang-counter 0) end)
@@ -677,7 +677,7 @@ IGNORE-HEADER then detangle regardless of `:tangle-sync' status."
((string= action "export")
(setq tang-counter (1+ tang-counter))))
(setq counter (1+ counter))))))
- (if (equal arg '(4))
+ (if single-p
(funcall single-block-metrics)
(goto-char (point-min))
(while (re-search-forward org-link-bracket-re nil t)
--
2.40.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-03 7:53 ` Mehmet Tekman
@ 2023-05-03 8:34 ` Mehmet Tekman
2023-05-03 8:44 ` Ihor Radchenko
0 siblings, 1 reply; 30+ messages in thread
From: Mehmet Tekman @ 2023-05-03 8:34 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 405 bytes --]
(Sorry for the previous top posting.)
One issue that I cannot seem to solve by myself is getting the
`org-babel-tangle-sync-mode' to persist on the `after-save-hook'
after it's been activated. My understanding of this hook is
that it is global and persists across buffers, but I'm seeing
some inconsistent behaviour that requires me to toggle the mode
on and off again.
Is there something I'm missing?
[-- Attachment #2: Type: text/html, Size: 519 bytes --]
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-03 8:34 ` Mehmet Tekman
@ 2023-05-03 8:44 ` Ihor Radchenko
0 siblings, 0 replies; 30+ messages in thread
From: Ihor Radchenko @ 2023-05-03 8:44 UTC (permalink / raw)
To: Mehmet Tekman; +Cc: emacs-orgmode
Mehmet Tekman <mtekman89@gmail.com> writes:
> (Sorry for the previous top posting.)
>
> One issue that I cannot seem to solve by myself is getting the
> `org-babel-tangle-sync-mode' to persist on the `after-save-hook'
> after it's been activated. My understanding of this hook is
> that it is global and persists across buffers, but I'm seeing
> some inconsistent behaviour that requires me to toggle the mode
> on and off again.
Any hook can be local.
See LOCAL argument in help:add-hook
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-02 20:43 ` Mehmet Tekman
2023-05-03 2:31 ` Ruijie Yu via General discussions about Org-mode.
@ 2023-05-03 11:43 ` Ihor Radchenko
2023-05-03 13:54 ` Mehmet Tekman
2023-05-03 15:05 ` Mehmet Tekman
1 sibling, 2 replies; 30+ messages in thread
From: Ihor Radchenko @ 2023-05-03 11:43 UTC (permalink / raw)
To: Mehmet Tekman; +Cc: emacs-orgmode
Mehmet Tekman <mtekman89@gmail.com> writes:
> I've modified the ob-tangle.el file for the main tangling and
> detangling functions. Most importantly, both functions can now
> exchange information from the source Org mode file to the target
> remote tangle file in either direction, depending on whether the
> source Org file has `:tangle-sync <action>' in the header.
Thanks!
> The action is one of:
>
> - "export" = always transmit information from the source Org mode
> block to the target remote file.
> - "import" = always transmit information from the target remote
> file to the source Org mode block.
> - "skip" = skip the block.
> - "both" = transmit information from source block to target block
> or target block to source, depending on whether the
> tangle or detangle is called from the source buffer or
> the target buffer respectively.
May it be better to make :tangle header argument compound?
Like ":tangle "file" export". Similar to :results header argument. See
"16.6 Results of Evaluation" section of Org manual.
Also, some general comments on the patches:
1. Please make sure that patches do not leave Org in broken state in the
middle of the patchset. Your patch #1 adds two `defun's in the middle
of `org-babel-detangle', which is not right.
2. When naming private functions, "--" should be after library prefix:
org-babel--...
3. Please do not use private functions from third-party libraries. I am
talking about `cl--set-buffer-substring' in particular.
4. Please make sure that docstrings clearly describe what the function
does, each of its arguments, and the expected return value.
In the patch, `org-babel-detangle--block-contents', NEAREST is
ambiguous. The actual meaning is "block at point or previous block".
5. Pay attention to buffer boundaries. In particular, remember that
buffer may be narrowed when you call a command and that expressions
like (+ 2 (line-beginning-position)) may resolve beyond
`point-min'/`point-max'.
6. Avoid using cryptic list functions like `cdadar' when you can use
something more readable.
7. When naming functions or macros, make sure that the name is roughly
describing the purpose of the function. In `org-babel-detangle', you
added `single-block-metrics' local function that does not only do the
metrics, but also (unexpectedly!) calls
`org-babel-detangle-single-block'. This is especially important for
local functions that lack docstring.
8. In `org-babel-tangle', (setq block-counter (+ 1 block-counter))
appears to be misplaced into outer sexp level after your patch.
9. In commit messages, please mark newly added functions clearly.
Like "(org-babel-foo): New function. It does this and that."
Same for commit summaries - "lisp/ob-tangle.el: Detangle a single
block" is awkward. You should clearly indicate that you added
something new to the library.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-03 11:43 ` Ihor Radchenko
@ 2023-05-03 13:54 ` Mehmet Tekman
2023-05-03 18:06 ` Ihor Radchenko
2023-05-03 15:05 ` Mehmet Tekman
1 sibling, 1 reply; 30+ messages in thread
From: Mehmet Tekman @ 2023-05-03 13:54 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: emacs-orgmode
Ihor Radchenko <yantar92@posteo.net> writes:
> Mehmet Tekman <mtekman89@gmail.com> writes:
>>
>> One issue that I cannot seem to solve by myself is getting the
>> `org-babel-tangle-sync-mode' to persist on the `after-save-hook'
>> after it's been activated. My understanding of this hook is
>> that it is global and persists across buffers, but I'm seeing
>> some inconsistent behaviour that requires me to toggle the mode
>> on and off again.
>
> Any hook can be local.
> See LOCAL argument in help:add-hook
Ah, that solves the problem immediately thank you!
> Mehmet Tekman <mtekman89@gmail.com> writes:
>
>> I've modified the ob-tangle.el file for the main tangling and
>> detangling functions. Most importantly, both functions can now
>> exchange information from the source Org mode file to the target
>> remote tangle file in either direction, depending on whether the
>> source Org file has `:tangle-sync <action>' in the header.
>
> Thanks!
>
>> The action is one of:
>>
>> - "export" = always transmit information from the source Org mode
>> block to the target remote file.
>> - "import" = always transmit information from the target remote
>> file to the source Org mode block.
>> - "skip" = skip the block.
>> - "both" = transmit information from source block to target block
>> or target block to source, depending on whether the
>> tangle or detangle is called from the source buffer or
>> the target buffer respectively.
>
> May it be better to make :tangle header argument compound?
> Like ":tangle "file" export". Similar to :results header argument. See
> "16.6 Results of Evaluation" section of Org manual.
>
That's a great idea I had not considered, and would definitely reduce
the header bloat, especially since `:tangle-sync' *requires* the
`:tangle' header to be there.
> Also, some general comments on the patches:
>
Great!
> 2. When naming private functions, "--" should be after library prefix:
> org-babel--...
>
Thanks, I will rename `org-babel-detangle-single-block--from-source'.
> 3. Please do not use private functions from third-party libraries. I am
> talking about `cl--set-buffer-substring' in particular.
>
So initially I used `(setf (buffer-substring X Y) new-content)` but I
recieved a warning from Emacs that it was an obsolete generalized
variable.
After some searching I found this entry in an emacs fork used the cl
library:
https://github.com/emacs-citar/citar/commit/809953a2191d0e3217ffbed9270be9b3cd6abfd2
Since `(require 'cl-lib)' is already imported in ~ob-tangle.el~, I did
not think it was too taboo to use. How does one then set the buffer
substring?
> 4. Please make sure that docstrings clearly describe what the function
> does, each of its arguments, and the expected return value.
> In the patch, `org-babel-detangle--block-contents', NEAREST is
> ambiguous. The actual meaning is "block at point or previous block".
>
> 5. Pay attention to buffer boundaries. In particular, remember that
> buffer may be narrowed when you call a command and that expressions
> like (+ 2 (line-beginning-position)) may resolve beyond
> `point-min'/`point-max'.
>
> 6. Avoid using cryptic list functions like `cdadar' when you can use
> something more readable.
>
> 7. When naming functions or macros, make sure that the name is roughly
> describing the purpose of the function. In `org-babel-detangle', you
> added `single-block-metrics' local function that does not only do the
> metrics, but also (unexpectedly!) calls
> `org-babel-detangle-single-block'. This is especially important for
> local functions that lack docstring.
>
> 8. In `org-babel-tangle', (setq block-counter (+ 1 block-counter))
> appears to be misplaced into outer sexp level after your patch.
Thanks for these, I will clean up and better document my code.
> 1. Please make sure that patches do not leave Org in broken state in the
> middle of the patchset. Your patch #1 adds two `defun's in the middle
> of `org-babel-detangle', which is not right.
>
> 9. In commit messages, please mark newly added functions clearly.
> Like "(org-babel-foo): New function. It does this and that."
> Same for commit summaries - "lisp/ob-tangle.el: Detangle a single
> block" is awkward. You should clearly indicate that you added
> something new to the library.
Apologies. I rebased and squashed all my commits into one, and then
selectively staged hunks into seperate commits for the git format-patc
process. For some reason the diff function decided that the new
functions should exist right in the middle of an existing function and I
was not sure how to resolve it at the time (though I have a better idea
now).
I will take better care with the messages. I tried to look for previous
"[ANN]" postings in the mailing list that I could emulate, but didn't
pay enough attention it seems.
I'm finally using `gnus' as my mail client so I'm slowly getting into a
more streamlined mindset that should be better at submitting and
formatting patches. (To reply to a mailing list, I do a wide reply to
the author and hope that the `Mail-Followup-To' header is used?)
Apropos patches:
Given how broken my current patches are, my next set of changes will be
not contingent on the previous ones. I will start a new set of patches.
I hope that's okay.
Best,
Mehmet
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-03 11:43 ` Ihor Radchenko
2023-05-03 13:54 ` Mehmet Tekman
@ 2023-05-03 15:05 ` Mehmet Tekman
2023-05-03 15:21 ` Ihor Radchenko
1 sibling, 1 reply; 30+ messages in thread
From: Mehmet Tekman @ 2023-05-03 15:05 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: emacs-orgmode
>> Ihor Radchenko <yantar92@posteo.net> writes:
>>
>> May it be better to make :tangle header argument compound?
>> Like ":tangle "file" export". Similar to :results header argument. See
>> "16.6 Results of Evaluation" section of Org manual.
>>
>
> Mehmet Tekman <mtekman89@gmail.com> writes:
>
> That's a great idea I had not considered, and would definitely reduce
> the header bloat, especially since `:tangle-sync' *requires* the
>`:tangle' header to be there.
On closer inspection it might actually be unpractical to have the sync
action within the `:tangle' syntax, the main reason being that people
might define the tangle target in the document header for ease of use.
e.g.
#+begin_example
#+TITLE: Sync Many Blocks
#+PROPERTY: header-args :tangle /tmp/default_tangle.txt :comments yes
** File Contents
*** Free Head
#+begin_src conf
This is some text that can be synced both ways by default
#+end_src
*** Import
#+begin_src conf :tangle-sync import
This is text that can only be imported from the remote, but it still
refers to the default tangle argument
#+end_src
#+end_example
If we instead encode the sync action in the `:tangle' syntax then users
will have to explicitly type out the tangle file in each block.
Also, tangle-sync wouldn't be the only extra tangle-related property in
a block, since tangle-mode is also a thing.
Is it okay to leave `:tangle-sync' as it is?
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-03 15:05 ` Mehmet Tekman
@ 2023-05-03 15:21 ` Ihor Radchenko
[not found] ` <87lei577g4.fsf@gmail.com>
0 siblings, 1 reply; 30+ messages in thread
From: Ihor Radchenko @ 2023-05-03 15:21 UTC (permalink / raw)
To: Mehmet Tekman; +Cc: emacs-orgmode
Mehmet Tekman <mtekman89@gmail.com> writes:
> On closer inspection it might actually be unpractical to have the sync
> action within the `:tangle' syntax, the main reason being that people
> might define the tangle target in the document header for ease of use.
This is not a problem.
Try
#+PROPERTY: header-args :results drawer output
#+begin_src emacs-lisp :results value
(list 1 2)
#+end_src
#+begin_src bash
echo 1 2
#+end_src
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-03 13:54 ` Mehmet Tekman
@ 2023-05-03 18:06 ` Ihor Radchenko
0 siblings, 0 replies; 30+ messages in thread
From: Ihor Radchenko @ 2023-05-03 18:06 UTC (permalink / raw)
To: Mehmet Tekman; +Cc: emacs-orgmode
Mehmet Tekman <mtekman89@gmail.com> writes:
>> 3. Please do not use private functions from third-party libraries. I am
>> talking about `cl--set-buffer-substring' in particular.
>>
>
> So initially I used `(setf (buffer-substring X Y) new-content)` but I
> recieved a warning from Emacs that it was an obsolete generalized
> variable.
Yup. See https://yhetil.org/emacs-devel/87tu5fzu2r.fsf@localhost/
We need to do things manually.
> After some searching I found this entry in an emacs fork used the cl
> library:
> https://github.com/emacs-citar/citar/commit/809953a2191d0e3217ffbed9270be9b3cd6abfd2
>
> Since `(require 'cl-lib)' is already imported in ~ob-tangle.el~, I did
> not think it was too taboo to use.
Private functions are a subject of change without notice. That's why we
do not use them, unless they are _our_ private functions we have control
about.
> How does one then set the buffer substring?
`replace-region-contents'.
> Apologies. I rebased and squashed all my commits into one, and then
> selectively staged hunks into seperate commits for the git format-patc
> process. For some reason the diff function decided that the new
> functions should exist right in the middle of an existing function and I
> was not sure how to resolve it at the time (though I have a better idea
> now).
Interactive rebase is helpful.
You can also edit, reset files, or apply patches in the middle of
rebasing to do extra adjustments.
> I will take better care with the messages. I tried to look for previous
> "[ANN]" postings in the mailing list that I could emulate, but didn't
> pay enough attention it seems.
Just try to follow what we usually do in commit messages. See
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/log/
Also, see https://www.gnu.org/prep/standards/html_node/Change-Logs.html
and https://git.savannah.gnu.org/cgit/emacs.git/plain/CONTRIBUTE
> I'm finally using `gnus' as my mail client so I'm slowly getting into a
> more streamlined mindset that should be better at submitting and
> formatting patches. (To reply to a mailing list, I do a wide reply to
> the author and hope that the `Mail-Followup-To' header is used?)
Reply all, or wide reply should be the right way. We do not want to
exclude participants who are not subscribed to the mailing list.
> Apropos patches:
> Given how broken my current patches are, my next set of changes will be
> not contingent on the previous ones. I will start a new set of patches.
> I hope that's okay.
A common approach is changing subject to [PATCH v2] ...
You can also use so-called reroll count when generating patchset from
git (or magit).
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
[not found] ` <87fs8duyae.fsf@localhost>
@ 2023-05-09 14:03 ` Mehmet Tekman
2023-05-10 9:46 ` Ihor Radchenko
0 siblings, 1 reply; 30+ messages in thread
From: Mehmet Tekman @ 2023-05-09 14:03 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 1591 bytes --]
Ihor Radchenko <yantar92@posteo.net> writes:
> Yes. See `org-babel-common-header-args-w-values'. In particular, take a
> look at (results ...).
Thanks, it took me some time to get my head around how to use this.
> I now ported a bit of documentation from my refactor branch.
> See https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=f268819d1
I'm having some problems getting:
`:tangle <tangle/yes/no/<filename>> [import/export/both/skip]'
to work nicely with the existing framework. The main issue lies in the
`:any` keyword in `org-babel-common-header-args-w-values' for the tangle
entry making it difficult to determine where one exclusionary group
begins and where the other ends.
e.g.
(defconst org-babel-common-header-args-w-values
...
(tangle . ((tangle yes no :any)
(import export skip sync)))
...
)
If I remove the :any keyword, these two groups work with the existing
framework in `org-babel-merge-params', but this would then mean that the
first tangle argument can't just be a filename string. I can get around
it by changing `:any' to `file' and then letting users describe their
tangle headers via e.g. `:tangle file import file: /some/file' but this
would be a breaking change.
In the meantime I put together a hacky solution that parses the tangle
header independently (`org-babel--handle-tangle-args') with the aim that
the first argument should ideally define a tangle filename action and
the second argument should ideally define a tangle sync action.
Please see the attached minor patch (diff) and a toy org example file.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Exclusive values for TANGLE header arguments added to the `org-babel-common-header-args-w-values' constant, along with a dedicated function `org-babel--handle-tangle-args' which parses it to conform to a `:tangle <filename action> <sync action>' format. --]
[-- Type: text/x-patch, Size: 4013 bytes --]
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 65fa47ab5..026788c00 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -431,7 +431,8 @@ then run `org-babel-switch-to-session'."
(sep . :any)
(session . :any)
(shebang . :any)
- (tangle . ((tangle yes no :any)))
+ (tangle . ((tangle yes no :any)
+ (import export skip sync)))
(tangle-mode . ((#o755 #o555 #o444 :any)))
(var . :any)
(wrap . :any)))
@@ -2802,6 +2803,9 @@ parameters when merging lists."
(exports-exclusive-groups
(mapcar (lambda (group) (mapcar #'symbol-name group))
(cdr (assq 'exports org-babel-common-header-args-w-values))))
+ (tangle-exclusive-groups
+ (mapcar (lambda (group) (mapcar #'symbol-name group))
+ (cdr (assq 'tangle org-babel-common-header-args-w-values))))
(merge
(lambda (exclusive-groups &rest result-params)
;; Maintain exclusivity of mutually exclusive parameters,
@@ -2821,7 +2825,7 @@ parameters when merging lists."
params ;Final parameters list.
;; Some keywords accept multiple values. We need to treat
;; them specially.
- vars results exports)
+ vars results exports tangle)
(dolist (plist plists)
(dolist (pair plist)
(pcase pair
@@ -2872,6 +2876,12 @@ parameters when merging lists."
(cond ((and value (functionp value)) (funcall value))
(value value)
(t ""))))))
+ (`(:tangle . ,value)
+ (setq tangle (funcall merge
+ tangle-exclusive-groups
+ tangle
+ (split-string
+ (or value "")))))
((or '(:dir . attach) '(:dir . "'attach"))
(unless (org-attach-dir nil t)
(error "No attachment directory for element (add :ID: or :DIR: property)"))
@@ -2897,11 +2907,38 @@ parameters when merging lists."
params)))))
;; Handle other special keywords, which accept multiple values.
(setq params (nconc (list (cons :results (mapconcat #'identity results " "))
- (cons :exports (mapconcat #'identity exports " ")))
+ (cons :exports (mapconcat #'identity exports " "))
+ (cons :tangle (org-babel--handle-tangle-args tangle)))
params))
;; Return merged params.
(org-babel-eval-headers params)))
+(defun org-babel--handle-tangle-args (tangle)
+ "Sanitize tangle arguments.
+From the list of TANGLE parameters, assert that that there are at
+maximum only two elements in the following preferential order:
+the first element relates to a filename descriptor (such as a
+path, `tangle', `yes', or `no'); the second element relates to a
+valid sync direction."
+ (let* ((num-args (length tangle))
+ ;; Extract valid mutex groups
+ (valid-tangle-headers (cdr (assoc 'tangle
+ org-babel-common-header-args-w-values)))
+ (valid-fname-args (seq-remove (lambda (x) (equal :any x)) (car valid-tangle-headers)))
+ (valid-sync-args (cadr valid-tangle-headers))
+ ;; Attempt to split TANGLE by these mutex groups
+ (sync-arg (seq-filter (lambda (x) (member (intern x) valid-sync-args)) tangle))
+ (fname-arg (seq-remove (lambda (x) (member x sync-arg)) tangle))
+ ;; Search for a potential filename
+ (filename (seq-remove (lambda (x) (member (intern x) valid-fname-args)) fname-arg)))
+ (setq sync-arg (car sync-arg)
+ ;; Assumption: the last added tangle argument is more
+ ;; important than the one preceding it.
+ fname-arg (or (car filename)
+ (car fname-arg)))
+ (concat fname-arg (if sync-arg " " "" ) sync-arg)))
+
+
(defun org-babel-noweb-p (params context)
"Check if PARAMS require expansion in CONTEXT.
CONTEXT may be one of :tangle, :export or :eval."
[-- Attachment #3: Toy example file --]
[-- Type: text/plain, Size: 721 bytes --]
#+TITLE: Sync test
#+PROPERTY: header-args :tangle /tmp/default_tangle.txt
Running =(assoc :tangle (nth 2 (org-babel-get-src-block-info)))= on
each of these should yield:
#+begin_src conf
(:tangle . /tmp/default_tangle.txt)
#+end_src
#+begin_src conf :tangle skip
(:tangle . /tmp/default_tangle.txt skip)
#+end_src
#+begin_src conf :tangle randomfile sync
(:tangle . randomfile sync)
#+end_src
#+begin_src conf :tangle randomfile
(:tangle . randomfile)
#+end_src
#+begin_src conf :tangle import export
## Ignores import
(:tangle . /tmp/default_tangle.txt export)
#+end_src
#+begin_src conf :tangle fname1 fname2 sync export
## Ignores fname1 and sync
(:tangle . fname2 export)
#+end_src
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-09 14:03 ` Mehmet Tekman
@ 2023-05-10 9:46 ` Ihor Radchenko
2023-05-10 11:06 ` mtekman89
0 siblings, 1 reply; 30+ messages in thread
From: Ihor Radchenko @ 2023-05-10 9:46 UTC (permalink / raw)
To: Mehmet Tekman; +Cc: emacs-orgmode
Mehmet Tekman <mtekman89@gmail.com> writes:
> I'm having some problems getting:
>
> `:tangle <tangle/yes/no/<filename>> [import/export/both/skip]'
>
> to work nicely with the existing framework. The main issue lies in the
> `:any` keyword in `org-babel-common-header-args-w-values' for the tangle
> entry making it difficult to determine where one exclusionary group
> begins and where the other ends.
>
> e.g.
>
> (defconst org-babel-common-header-args-w-values
> ...
> (tangle . ((tangle yes no :any)
> (import export skip sync)))
> ...
> )
And the reason is that tangle function inside `org-babel-merge-params'
does not know how to handle :any in exclusive groups.
We should modify it. For example like the following:
1. We will assume that :any can only occur one time in the exclusive
groups. (Otherwise, there is no single definite way to parse header
arguments)
2. Merge function will treat :any specially - when parameter does not
match any of the argument values from all the groups combined, it is
considered as :any and replace the previous corresponding values in
its exclusive group, if any;
In other words, we will need a special match for :any - "anything not
equal to other values in all the groups combined".
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-10 9:46 ` Ihor Radchenko
@ 2023-05-10 11:06 ` mtekman89
2023-05-10 11:32 ` Ihor Radchenko
0 siblings, 1 reply; 30+ messages in thread
From: mtekman89 @ 2023-05-10 11:06 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: emacs-orgmode
Ihor Radchenko <yantar92@posteo.net> writes:
> 1. We will assume that :any can only occur one time in the exclusive
> groups. (Otherwise, there is no single definite way to parse header
> arguments)
Makes sense (or we could revamp all header parsing into some kind of
finite state machine… (I joke… for now…))
> 2. Merge function will treat :any specially - when parameter does not
> match any of the argument values from all the groups combined, it is
> considered as :any and replace the previous corresponding values in
> its exclusive group, if any;
This is something that I do in my `org-babel--handle-tangle-args', where
the "filename" argument is the result of eliminating all other entries
in the group.
> In other words, we will need a special match for :any - "anything not
> equal to other values in all the groups combined".
Agreed, and I can try putting together something working in a day or
two. Or, if this is something you can easily implement yourself, please
do let me know.
Best,
Mehmet
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-10 11:06 ` mtekman89
@ 2023-05-10 11:32 ` Ihor Radchenko
2023-05-10 16:20 ` Mehmet Tekman
0 siblings, 1 reply; 30+ messages in thread
From: Ihor Radchenko @ 2023-05-10 11:32 UTC (permalink / raw)
To: mtekman89; +Cc: emacs-orgmode
mtekman89@gmail.com writes:
>> In other words, we will need a special match for :any - "anything not
>> equal to other values in all the groups combined".
>
> Agreed, and I can try putting together something working in a day or
> two. Or, if this is something you can easily implement yourself, please
> do let me know.
It will be great if you could do it.
I have other things to work on.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-10 11:32 ` Ihor Radchenko
@ 2023-05-10 16:20 ` Mehmet Tekman
2023-05-12 12:33 ` Ihor Radchenko
0 siblings, 1 reply; 30+ messages in thread
From: Mehmet Tekman @ 2023-05-10 16:20 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 1401 bytes --]
Ihor Radchenko <yantar92@posteo.net> writes:
> It will be great if you could do it.
> I have other things to work on.
Of course! I'm just a little unfamiliar on how one coordinates active
collaboration via mailing list :-)
Anyways - I did it, and it took less time than I thought
> We should modify it. For example like the following:
>
> 1. We will assume that :any can only occur one time in the exclusive
> groups. (Otherwise, there is no single definite way to parse header
> arguments)
> 2. Merge function will treat :any specially - when parameter does not
> match any of the argument values from all the groups combined, it is
> considered as :any and replace the previous corresponding values in
> its exclusive group, if any;
> In other words, we will need a special match for :any - "anything not
> equal to other values in all the groups combined".
I've modified the `merge' function within `org-babel-merge-params' so
that the main logic now accumulates a list of potential candidates that
could be the :any keyword, and selects the last added candidate as the
match.
The first two patches are very minor, simply adding
tangle-exclusive-groups using the existing code templates. The last
patch is the merge function rewrite.
It all seems to be passing tests, though I would like to add my toy.org
file to the org testing framework at some point.
Best,
Mehmet
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Added new tangle exclusion group for tangle sync action --]
[-- Type: text/x-patch, Size: 916 bytes --]
From eeb3f165498fcc420b862f67fb616b474a14b684 Mon Sep 17 00:00:00 2001
From: MT <mtekman89@gmail.com>
Date: Wed, 10 May 2023 17:38:22 +0200
Subject: [PATCH 1/3] * lisp/ob-core.el
(org-babel-common-header-args-w-values): Added mutually exclusive tangle
groups relating to desired tangle sync actions (e.g. :tangle
(yes|no|<filename>) [(import|export|sync)])
---
lisp/ob-core.el | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 65fa47ab5..013a37ce5 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -431,7 +431,8 @@ then run `org-babel-switch-to-session'."
(sep . :any)
(session . :any)
(shebang . :any)
- (tangle . ((tangle yes no :any)))
+ (tangle . ((tangle yes no :any)
+ (import export skip sync)))
(tangle-mode . ((#o755 #o555 #o444 :any)))
(var . :any)
(wrap . :any)))
--
2.40.1
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: Implemented initial tangle header merge framework from existing code template --]
[-- Type: text/x-patch, Size: 2565 bytes --]
From 366a120a394d7783bf1640037ade31f826ef0277 Mon Sep 17 00:00:00 2001
From: MT <mtekman89@gmail.com>
Date: Wed, 10 May 2023 17:41:37 +0200
Subject: [PATCH 2/3] * lisp/ob-core.el (org-babel-merge-params): Tangle header
with exclusive parameters can now be parsed, following the template of
:exports and :results
---
lisp/ob-core.el | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 013a37ce5..ed31a9de1 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -2803,6 +2803,9 @@ parameters when merging lists."
(exports-exclusive-groups
(mapcar (lambda (group) (mapcar #'symbol-name group))
(cdr (assq 'exports org-babel-common-header-args-w-values))))
+ (tangle-exclusive-groups
+ (mapcar (lambda (group) (mapcar #'symbol-name group))
+ (cdr (assq 'tangle org-babel-common-header-args-w-values))))
(merge
(lambda (exclusive-groups &rest result-params)
;; Maintain exclusivity of mutually exclusive parameters,
@@ -2822,7 +2825,7 @@ parameters when merging lists."
params ;Final parameters list.
;; Some keywords accept multiple values. We need to treat
;; them specially.
- vars results exports)
+ vars results exports tangle)
(dolist (plist plists)
(dolist (pair plist)
(pcase pair
@@ -2873,6 +2876,12 @@ parameters when merging lists."
(cond ((and value (functionp value)) (funcall value))
(value value)
(t ""))))))
+ (`(:tangle . ,value)
+ (setq tangle (funcall merge
+ tangle-exclusive-groups
+ tangle
+ (split-string
+ (or value "")))))
((or '(:dir . attach) '(:dir . "'attach"))
(unless (org-attach-dir nil t)
(error "No attachment directory for element (add :ID: or :DIR: property)"))
@@ -2898,7 +2907,8 @@ parameters when merging lists."
params)))))
;; Handle other special keywords, which accept multiple values.
(setq params (nconc (list (cons :results (mapconcat #'identity results " "))
- (cons :exports (mapconcat #'identity exports " ")))
+ (cons :exports (mapconcat #'identity exports " "))
+ (cons :tangle (mapconcat #'identity tangle " ")))
params))
;; Return merged params.
(org-babel-eval-headers params)))
--
2.40.1
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: Major rewrite of the merge function to handle :any keywords --]
[-- Type: text/x-patch, Size: 4186 bytes --]
From 6ce5313b7d3f0ab718072942f082bc259dccbae6 Mon Sep 17 00:00:00 2001
From: MT <mtekman89@gmail.com>
Date: Wed, 10 May 2023 17:44:42 +0200
Subject: [PATCH 3/3] * lisp/ob-core.el (org-babel-merge-params): Major rewrite
of the `merge' function, which adds the capability to process the :any
keyword when merging parameters with exclusive groups.
---
lisp/ob-core.el | 60 +++++++++++++++++++++++++++++++++++++++++++------
1 file changed, 53 insertions(+), 7 deletions(-)
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index ed31a9de1..3d9000efc 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -2811,15 +2811,61 @@ parameters when merging lists."
;; Maintain exclusivity of mutually exclusive parameters,
;; as defined in EXCLUSIVE-GROUPS while merging lists in
;; RESULT-PARAMS.
- (let (output)
+ (let (output group-any)
(dolist (new-params result-params (delete-dups output))
(dolist (new-param new-params)
- (dolist (exclusive-group exclusive-groups)
- (when (member new-param exclusive-group)
- (setq output (cl-remove-if
- (lambda (o) (member o exclusive-group))
- output))))
- (push new-param output))))))
+ (let (matched-param potential-any-param)
+ ;; new-param may be an :any value, so we check
+ ;; across all exclusive-groups.
+ ;; - If new-param matches one of groups, we
+ (dolist (exclusive-group exclusive-groups)
+ (if (member new-param exclusive-group)
+ (setq output (cl-remove-if
+ (lambda (o) (member o exclusive-group))
+ output)
+ ;; Cancel any potential matches if it's caught
+ matched-param t
+ potential-any-param nil)
+ ;; If not a direct match, flag it as a potential
+ ;; :any match This can happen multiple times for
+ ;; each new-param, but only once for each
+ ;; exclusive-group.
+ (if (and (not matched-param)
+ (member ":any" exclusive-group))
+ ;; At this point, the new-param has not yet matched
+ ;; anything in the N exclusive groups
+ ;; - We also assume that only 1 of these N groups
+ ;; has the :any keyword.
+ ;; - This point in the code can therefore be only
+ ;; reached once under this assumption.
+ ;; - We therefore setq instead of push
+ (setq potential-any-param (cons new-param
+ exclusive-group)))))
+
+ ;; At this point we know whether new-param and 1
+ ;; of the exclusive groups have an :any keyword -
+ ;; - Due to multiple new-params potentially being
+ ;; matches in the same group, we push these to a
+ ;; super group of "any" keywords, and process them
+ ;; later.
+ (if potential-any-param
+ (setq group-any potential-any-param)
+ ;; If the param isn't :any, add it to the output
+ ;; as a regular keyword
+ (push new-param output)))))
+
+ (when group-any
+ ;; Whatever is leftover at this point are :any candidates.
+ ;; - We assume that last added is the most relevant and
+ ;; that everything else should be ignored
+ ;; - We add the first, and reject everything else in that
+ ;; exclusion group.
+ (push (car group-any) output)
+ (setq output (cl-remove-if
+ (lambda (o) (member o (cdr group-any)))
+ output)))
+ output)))
+
(variable-index 0) ;Handle positional arguments.
clearnames
params ;Final parameters list.
--
2.40.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-10 16:20 ` Mehmet Tekman
@ 2023-05-12 12:33 ` Ihor Radchenko
2023-05-16 12:49 ` Mehmet Tekman
0 siblings, 1 reply; 30+ messages in thread
From: Ihor Radchenko @ 2023-05-12 12:33 UTC (permalink / raw)
To: Mehmet Tekman; +Cc: emacs-orgmode
Mehmet Tekman <mtekman89@gmail.com> writes:
>> It will be great if you could do it.
>> I have other things to work on.
>
> Of course! I'm just a little unfamiliar on how one coordinates active
> collaboration via mailing list :-)
Usually, we leave working on the patches to one person - whoever is the
most interested in the patch.
Maintainers and other interested users provide comments and suggestions.
> I've modified the `merge' function within `org-babel-merge-params' so
> that the main logic now accumulates a list of potential candidates that
> could be the :any keyword, and selects the last added candidate as the
> match.
I feel confused when reading the modified code.
> It all seems to be passing tests, though I would like to add my toy.org
> file to the org testing framework at some point.
I recommend trying various edge cases with your patch.
In particular:
1. Testing exclusive group inheritance when we inherit a header arg
value that matches :any:
#+PROPERTIES: header-args :tangle "foo.txt"
#+begin_src ... :tangle no
#+PROPERTIES: header-args :tangle no
#+begin_src ... :tangle "foo.txt"
#+PROPERTIES: header-args :tangle no
* Heading
:PROPERTIES:
:header-args: :tangle "foo.txt"
:END:
#+begin_src ... :tangle yes
2. :tangle "file with spaces.txt"
I feel that the following code is not reliable when we inherit exact and
:any exclusive group members in alternations.
> + (if (member new-param exclusive-group)
> ...
> + ;; Cancel any potential matches if it's caught
> + matched-param t
> + potential-any-param nil)
> + ;; If not a direct match, flag it as a potential
> + ;; :any match This can happen multiple times for
> + ;; each new-param, but only once for each
> + ;; exclusive-group.
> + (if (and (not matched-param)
> + (member ":any" exclusive-group))
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-12 12:33 ` Ihor Radchenko
@ 2023-05-16 12:49 ` Mehmet Tekman
2023-05-16 18:57 ` Ihor Radchenko
0 siblings, 1 reply; 30+ messages in thread
From: Mehmet Tekman @ 2023-05-16 12:49 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: emacs-orgmode
Hello, thanks for the last review and sorry for the lapse in
communication. May is proving to be a very strange month.
Ihor Radchenko <yantar92@posteo.net> writes:
> Mehmet Tekman <mtekman89@gmail.com> writes:
>>
>> I've modified the `merge' function within `org-babel-merge-params' so
>> that the main logic now accumulates a list of potential candidates that
>> could be the :any keyword, and selects the last added candidate as the
>> match.
>
> I feel confused when reading the modified code.
>
Okay, I guess comments are not my strong suit. I will try to remove any
unnecessary comments and use better variable names.
>> It all seems to be passing tests, though I would like to add my toy.org
>> file to the org testing framework at some point.
>
> I recommend trying various edge cases with your patch.
> In particular:
>
> 1. Testing exclusive group inheritance when we inherit a header arg
> value that matches :any:
>
> #+PROPERTIES: header-args :tangle "foo.txt"
> #+begin_src ... :tangle no
>
> #+PROPERTIES: header-args :tangle no
> #+begin_src ... :tangle "foo.txt"
>
> #+PROPERTIES: header-args :tangle no
> * Heading
>
> :PROPERTIES:
> :header-args: :tangle "foo.txt"
> :END:
> #+begin_src ... :tangle yes
>
> 2. :tangle "file with spaces.txt"
>
I think before I do any more disastrous changes, I will try to expand
the existing ert test suite to incorporate my toy org file and your
above examples.
That way I'd at least have a consistent framework to validate some of my
work.
> I feel that the following code is not reliable when we inherit exact and
> :any exclusive group members in alternations.
Ah, hmm - can you give me an example here? I thought the idea was that
an :any string would only be invokable once for a given header
parameter.
Ihor Radchenko <yantar92@posteo.net> writes:
>>> We should modify it. For example like the following:
>>>
>>> 1. We will assume that :any can only occur one time in the exclusive
>>> groups. (Otherwise, there is no single definite way to parse header
>>> arguments)
Best,
Mehmet
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-16 12:49 ` Mehmet Tekman
@ 2023-05-16 18:57 ` Ihor Radchenko
2023-05-17 13:45 ` Mehmet Tekman
0 siblings, 1 reply; 30+ messages in thread
From: Ihor Radchenko @ 2023-05-16 18:57 UTC (permalink / raw)
To: Mehmet Tekman; +Cc: emacs-orgmode
Mehmet Tekman <mtekman89@gmail.com> writes:
>> I feel confused when reading the modified code.
>>
>
> Okay, I guess comments are not my strong suit. I will try to remove any
> unnecessary comments and use better variable names.
No, comments were useful. They revealed that our understanding of the
expected behaviour might differ.
>> I recommend trying various edge cases with your patch.
>> In particular:
>>
>> 1. Testing exclusive group inheritance when we inherit a header arg
>> value that matches :any:
>>
>> #+PROPERTIES: header-args :tangle "foo.txt"
>> #+begin_src ... :tangle no
>>
>> #+PROPERTIES: header-args :tangle no
>> #+begin_src ... :tangle "foo.txt"
>>
>> #+PROPERTIES: header-args :tangle no
>> * Heading
>>
>> :PROPERTIES:
>> :header-args: :tangle "foo.txt"
>> :END:
>> #+begin_src ... :tangle yes
> ...
>> I feel that the following code is not reliable when we inherit exact and
>> :any exclusive group members in alternations.
>
> Ah, hmm - can you give me an example here? I thought the idea was that
> an :any string would only be invokable once for a given header
> parameter.
See the above 3 examples: (1) "foo.txt" shadowed by "no"; (2) "no"
shadowed by "foo.txt"; (3) "no" shadowed by "foo.txt" then shadowed by
"yes".
You can also consider "foo.txt" shadowed by "bar.txt" and other
variations.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-16 18:57 ` Ihor Radchenko
@ 2023-05-17 13:45 ` Mehmet Tekman
2023-05-18 10:30 ` Ihor Radchenko
0 siblings, 1 reply; 30+ messages in thread
From: Mehmet Tekman @ 2023-05-17 13:45 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 2441 bytes --]
Hello again,
Ihor Radchenko <yantar92@posteo.net> writes:
>>> I recommend trying various edge cases with your patch.
>>> In particular:
>>>
>>> 1. Testing exclusive group inheritance when we inherit a header arg
>>> value that matches :any:
>>>
>>> #+PROPERTIES: header-args :tangle "foo.txt"
>>> #+begin_src ... :tangle no
>>>
>>> #+PROPERTIES: header-args :tangle no
>>> #+begin_src ... :tangle "foo.txt"
>>>
>>> #+PROPERTIES: header-args :tangle no
>>> * Heading
>>>
>>> :PROPERTIES:
>>> :header-args: :tangle "foo.txt"
>>> :END:
>>> #+begin_src ... :tangle yes
>> ...
>>> I feel that the following code is not reliable when we inherit exact and
>>> :any exclusive group members in alternations.
>>
>> Ah, hmm - can you give me an example here? I thought the idea was that
>> an :any string would only be invokable once for a given header
>> parameter.
>
> See the above 3 examples: (1) "foo.txt" shadowed by "no"; (2) "no"
> shadowed by "foo.txt"; (3) "no" shadowed by "foo.txt" then shadowed by
> "yes".
>
> You can also consider "foo.txt" shadowed by "bar.txt" and other
> variations.
I've created a small patch that contains a single ert test function that
checks an example org file I made (based on the above as well as my own
toy file) to try to validate the `org-babel-merge-params' rewrite.
I've attached it below in case you want to test it...
...but there were a few cases where I wasn't entirely sure what the
result of the merge was supposed to be:
For example, a document with:
> #+TITLE: Header tests
> #+PROPERTY: header-args :tangle /tmp/default_tangle.txt
>
> * Inherit tangle header from document
>
> #+begin_src conf
> (:tangle . /tmp/default_tangle.txt)
> #+end_src
I would expect the output of:
(assoc :tangle (nth 2 (org-babel-get-src-block-info)))
within that block to evaluate to the contents written in that
block. Instead it evaluates to `(:tangle . no)' when run in a vanilla
Emacs. Is this expected?
Another example:
> * Header args overwritten by local header
> :PROPERTIES:
> :header-args: :tangle "foo.txt"
> :END:
> #+begin_src :tangle yes
> (:tangle . foo.txt)
> #+end_src
>
> ** Inherited header
>
> #+begin_src :tangle "file with spaces.txt"
> (:tangle . "file with spaces.txt")
> #+end_src
The first block correctly gives "foo.txt" under vanilla Emacs, but the
second block also gives "foo.txt". Is this expected behaviour?
Best,
Mehmet
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: test org file and associated ert test function --]
[-- Type: text/x-patch, Size: 7406 bytes --]
From 2f040db1197f835262d32e7ced857f2a47dd8ca8 Mon Sep 17 00:00:00 2001
From: MT <mtekman89@gmail.com>
Date: Wed, 17 May 2023 13:16:08 +0200
Subject: [PATCH 4/4] * testing/examples/header_test.org: New example test file
Contains several source blocks with document, heading, block header
arguments
* testing/lisp/test-ob.el (test-ob/merge-params): New test function
for `org-babel-merge-params' validation.
This function takes a list of all the ID properties for each of the
blocks in `header_test.org', along with a symbol or list of header
properties to test against. The expected value is written within
the block contents.
---
| 127 +++++++++++++++++++++++++++++++
testing/lisp/test-ob.el | 44 +++++++++++
2 files changed, 171 insertions(+)
create mode 100644 testing/examples/header_test.org
--git a/testing/examples/header_test.org b/testing/examples/header_test.org
new file mode 100644
index 000000000..9a33661be
--- /dev/null
+++ b/testing/examples/header_test.org
@@ -0,0 +1,127 @@
+#+TITLE: Header tests
+#+PROPERTY: header-args :tangle /tmp/default_tangle.txt
+
+The text contents in each block are tested against the output of
+=(assoc <prop> (nth 2 (org-babel-get-src-block-info)))=
+
+Multiple header properties can be tested by separating each property
+output with a **newline followed by exactly two spaces**.
+
+* Inherit tangle header from document
+:PROPERTIES:
+:ID: a41c3238-f457-4769-b10b-8d50e9d386a1
+:END:
+
+#+begin_src conf
+ (:tangle . /tmp/default_tangle.txt)
+#+end_src
+
+* Inherit tangle header but use local sync action
+:PROPERTIES:
+:ID: debf7bf8-e5eb-412d-9127-57950a27c390
+:END:
+
+#+begin_src conf :tangle skip
+ (:tangle . /tmp/default_tangle.txt skip)
+#+end_src
+
+* Use local tangle file and sync action
+:PROPERTIES:
+:ID: 1ca658c1-0dfd-42a5-bbe3-305582deeb00
+:END:
++ Ignore document header completely.
+#+begin_src conf :tangle randomfile sync
+ (:tangle . randomfile sync)
+#+end_src
+
+* Use local tangle file and sync action 2
+:PROPERTIES:
+:header-args: :tangle "newfile.txt" import
+:END:
+** Subheading
+:PROPERTIES:
+:ID: 602503b8-6657-49c6-9cac-7edac396f725
+:END:
++ Ignore document header and parent header completely.
+#+begin_src conf :tangle randomfile sync
+ (:tangle . randomfile sync)
+#+end_src
+
+
+* Test tangle and results param together
+:PROPERTIES:
+:ID: 4fb9938c-aec0-479f-bbc6-6b7a4228d9c2
+:END:
+#+begin_src conf :tangle randomfile
+ (:tangle . randomfile)
+ (:results . replace)
+#+end_src
+
+* Inherit the tangle file name, take the last sync action
+:PROPERTIES:
+:ID: 7a98b56d-e59f-426d-bd58-f93bb22cf57b
+:END:
++ Ignores import
+#+begin_src conf :tangle import export
+ (:tangle . /tmp/default_tangle.txt export)
+#+end_src
+
+* Take the last local tangle file name and the last sync action
+:PROPERTIES:
+:ID: cd85e03a-1a4c-45d5-ac33-90d96999b665
+:END:
++ Ignores fname1 and sync
+#+begin_src conf :tangle fname1 fname2 sync export
+ (:tangle . fname2 export)
+#+end_src
+
+* Merge document results param and local results param
+:PROPERTIES:
+:ID: f4e4e422-029b-4ef7-b594-cd70cff2d943
+:END:
+
+#+begin_src sh :results file wrap
+ (:results . wrap file replace)
+ (:exports . code)
+#+end_src
+
+* All tangle headers should be ignored (ITS FAILING HERE)
+:PROPERTIES:
+:ID: 9715d355-009c-4188-8b97-bcbebaeee86f
+:END:
+
+#+begin_src conf :tangle no
+ (:tangle . no)
+#+end_src
+
+* Tangle filename ignores document and heading args, inherits heading exports
+:PROPERTIES:
+:ID: 1a3b5565-27b5-450e-a2c5-7f95a8142f3b
+:header-args: :tangle no :exports verbatim
+:END:
+
+#+begin_src conf :tangle "foo.txt" :comments link
+ (:tangle . foo.txt)
+ (:exports . verbatim code)
+ (:comments . link)
+#+end_src
+
+* Heading tangle parameter is not overwritten by local "yes"
+:PROPERTIES:
+:ID: fe54b2be-d9f1-40b4-83df-49501e69d083
+:header-args: :tangle "foo.txt"
+:END:
+#+begin_src :tangle yes
+ (:tangle . foo.txt)
+#+end_src
+
+** Local tangle filename with spaces overwrites parent foo.txt
+:PROPERTIES:
+:ID: ab8af294-c586-4ec8-9f45-3c3baaeb184d
+:END:
++ The expected hierarchy is =/tmp/default_tangle.txt= is supplanted by
+ =foo.txt= which is supplanted by =file with spaces.txt=
+
+#+begin_src :tangle "file with spaces.txt"
+ (:tangle . "file with spaces.txt")
+#+end_src
diff --git a/testing/lisp/test-ob.el b/testing/lisp/test-ob.el
index c8dbd44f4..e05dd083a 100644
--- a/testing/lisp/test-ob.el
+++ b/testing/lisp/test-ob.el
@@ -314,6 +314,50 @@ this is simple"
(org-babel-next-src-block)
(should (= 14 (org-babel-execute-src-block)))))
+(ert-deftest test-ob/merge-params ()
+ "Test the output of merging multiple header parameters. The
+expected output is given in the contents of the source code block
+at each header. The desired test header parameters are given
+either as a symbol or a list in the `idtest-alist' variable.
+Multiple header parameters must be separated by a newline and
+exactly two spaces in the block contents."
+ (let ((idtest-alist '(("a41c3238-f457-4769-b10b-8d50e9d386a1" . :tangle)
+ ("debf7bf8-e5eb-412d-9127-57950a27c390" . :tangle)
+ ("1ca658c1-0dfd-42a5-bbe3-305582deeb00" . :tangle)
+ ("602503b8-6657-49c6-9cac-7edac396f725" . :tangle)
+ ("4fb9938c-aec0-479f-bbc6-6b7a4228d9c2" . (:tangle :results))
+ ("7a98b56d-e59f-426d-bd58-f93bb22cf57b" . :tangle)
+ ("cd85e03a-1a4c-45d5-ac33-90d96999b665" . :tangle)
+ ("f4e4e422-029b-4ef7-b594-cd70cff2d943" . (:results :exports))
+ ("9715d355-009c-4188-8b97-bcbebaeee86f" . :tangle)
+ ("1a3b5565-27b5-450e-a2c5-7f95a8142f3b" . (:tangle :exports :comments))
+ ("fe54b2be-d9f1-40b4-83df-49501e69d083" . :tangle)
+ ("ab8af294-c586-4ec8-9f45-3c3baaeb184d" . :tangle)))
+ buffer
+ failed-ids)
+ (unwind-protect
+ (org-test-in-example-file (expand-file-name "header_test.org" org-test-example-dir)
+ (setq buffer (current-buffer))
+ (dolist (testpair idtest-alist)
+ (let ((id (car testpair))
+ (prop (cdr testpair)))
+ (org-test-at-id id)
+ (org-babel-next-src-block)
+ (unless (string=
+ (if (string= "symbol" (type-of prop))
+ (format "%s" (assoc prop
+ (nth 2 (org-babel-get-src-block-info))))
+ (mapconcat (lambda (x) (format "%s"
+ (assoc x
+ (nth 2 (org-babel-get-src-block-info)))))
+ prop "\n ")) ;; newline with exactly two spaces.
+ (string-trim (org-element-property :value (org-element-at-point))))
+ (push id failed-ids))))
+ (kill-buffer buffer)
+ (if failed-ids
+ (user-error "%d Failed Blocks: %s" (length failed-ids) failed-ids))
+ (should (= 0 (length failed-ids) ))))))
+
(ert-deftest test-ob/inline-src-blocks ()
(should
(= 1
--
2.40.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-17 13:45 ` Mehmet Tekman
@ 2023-05-18 10:30 ` Ihor Radchenko
2023-05-19 7:10 ` Mehmet Tekman
0 siblings, 1 reply; 30+ messages in thread
From: Ihor Radchenko @ 2023-05-18 10:30 UTC (permalink / raw)
To: Mehmet Tekman; +Cc: emacs-orgmode
Mehmet Tekman <mtekman89@gmail.com> writes:
>> See the above 3 examples: (1) "foo.txt" shadowed by "no"; (2) "no"
>> shadowed by "foo.txt"; (3) "no" shadowed by "foo.txt" then shadowed by
>> "yes".
>>
>> You can also consider "foo.txt" shadowed by "bar.txt" and other
>> variations.
>
> I've created a small patch that contains a single ert test function that
> checks an example org file I made (based on the above as well as my own
> toy file) to try to validate the `org-babel-merge-params' rewrite.
>
> I've attached it below in case you want to test it...
Is the patch testing your code?
(Note on the ert tests: we generally prefer `org-test-with-temp-text' -
the old approach with test IDs is not very readable because one needs to
search those IDs manually when debugging test failures.)
> ...but there were a few cases where I wasn't entirely sure what the
> result of the merge was supposed to be:
>
> For example, a document with:
>
>> #+TITLE: Header tests
>> #+PROPERTY: header-args :tangle /tmp/default_tangle.txt
>>
>> * Inherit tangle header from document
>>
>> #+begin_src conf
>> (:tangle . /tmp/default_tangle.txt)
>> #+end_src
>
> I would expect the output of:
>
> (assoc :tangle (nth 2 (org-babel-get-src-block-info)))
>
> within that block to evaluate to the contents written in that
> block. Instead it evaluates to `(:tangle . no)' when run in a vanilla
> Emacs. Is this expected?
You forgot C-c C-c on the PROPERTY line, or re-opening the file.
#+PROPERTY lines are not automatically parsed when you add them. They
are only parsed upon explicit request (C-c C-c) or when opening the
file.
> Another example:
>
>> * Header args overwritten by local header
>> :PROPERTIES:
>> :header-args: :tangle "foo.txt"
>> :END:
>> #+begin_src :tangle yes
>> (:tangle . foo.txt)
>> #+end_src
>>
>> ** Inherited header
>>
>> #+begin_src :tangle "file with spaces.txt"
>> (:tangle . "file with spaces.txt")
>> #+end_src
>
> The first block correctly gives "foo.txt" under vanilla Emacs, but the
> second block also gives "foo.txt". Is this expected behaviour?
You forgot to specify the source block languages. Org sees your example
as two src blocks of language ":tangle" with no proper header arguments.
(See M-x org-lint)
If you add a language, like
#+begin_src conf :tangle yes
(:tangle . foo.txt)
#+end_src
The expected result is (:tangle . "yes") for the first block and
(:tangle . "file with spaces.txt") for the second block.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [ANN] lisp/ob-tangle-sync.el
2023-05-18 10:30 ` Ihor Radchenko
@ 2023-05-19 7:10 ` Mehmet Tekman
0 siblings, 0 replies; 30+ messages in thread
From: Mehmet Tekman @ 2023-05-19 7:10 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: emacs-orgmode
Ah, running `org-lint' and correcting the header line as well as adding
languages to the other blocks fixed the mistakes - thanks for this, I
thought I was losing my mind.
Ihor Radchenko <yantar92@posteo.net> writes:
> The expected result is (:tangle . "yes") for the first block and
> (:tangle . "file with spaces.txt") for the second block.
I also think I did not realise what "yes" did until now. I assumed it
defaulted to the last given filename in the hierarchy, but after reading
the manual I see that it just sets it to the name of the org file.
> Is the patch testing your code?
>
> (Note on the ert tests: we generally prefer `org-test-with-temp-text' -
> the old approach with test IDs is not very readable because one needs to
> search those IDs manually when debugging test failures.)
>
It is, but please ignore it for now. I will rewrite to to conform to the
`org-test-with-temp-text' examples.
Best,
Mehmet
^ permalink raw reply [flat|nested] 30+ messages in thread
end of thread, other threads:[~2023-05-19 7:11 UTC | newest]
Thread overview: 30+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-04-26 14:48 [ANN] lisp/ob-tangle-sync.el Mehmet Tekman
2023-04-26 16:43 ` John Wiegley
2023-04-26 18:43 ` Mehmet Tekman
2023-04-27 2:55 ` Ruijie Yu via General discussions about Org-mode.
2023-04-27 6:27 ` Mehmet Tekman
2023-04-28 10:57 ` Ruijie Yu via General discussions about Org-mode.
2023-04-28 11:28 ` Mehmet Tekman
2023-05-02 20:43 ` Mehmet Tekman
2023-05-03 2:31 ` Ruijie Yu via General discussions about Org-mode.
2023-05-03 7:53 ` Mehmet Tekman
2023-05-03 8:34 ` Mehmet Tekman
2023-05-03 8:44 ` Ihor Radchenko
2023-05-03 11:43 ` Ihor Radchenko
2023-05-03 13:54 ` Mehmet Tekman
2023-05-03 18:06 ` Ihor Radchenko
2023-05-03 15:05 ` Mehmet Tekman
2023-05-03 15:21 ` Ihor Radchenko
[not found] ` <87lei577g4.fsf@gmail.com>
[not found] ` <87lei5v1fg.fsf@localhost>
[not found] ` <87fs8duyae.fsf@localhost>
2023-05-09 14:03 ` Mehmet Tekman
2023-05-10 9:46 ` Ihor Radchenko
2023-05-10 11:06 ` mtekman89
2023-05-10 11:32 ` Ihor Radchenko
2023-05-10 16:20 ` Mehmet Tekman
2023-05-12 12:33 ` Ihor Radchenko
2023-05-16 12:49 ` Mehmet Tekman
2023-05-16 18:57 ` Ihor Radchenko
2023-05-17 13:45 ` Mehmet Tekman
2023-05-18 10:30 ` Ihor Radchenko
2023-05-19 7:10 ` Mehmet Tekman
2023-04-27 12:02 ` Ihor Radchenko
2023-04-27 13:01 ` Mehmet Tekman
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).