Re: [PATCH] org-babel-demarcate-block: split using element API

[gerard.vermeulen@posteo.net]

[-- Attachment #1: Type: text/plain, Size: 1697 bytes --]



On 14.01.2024 20:18, gerard.vermeulen@posteo.net wrote:
> On 14.01.2024 13:16, Ihor Radchenko wrote:
>> gerard.vermeulen@posteo.net writes:
> [...]
>> I tried make test and the tests are still failing with this new patch:
>> 
>> 1 unexpected results:
>>    FAILED  test-ob/demarcate-block  ((should (string= region-text
>> (org-trim (nth 1 info)))) :form (string= "mark this line as region"
>> "") :value nil :explanation (arrays-of-different-length 24 0 "mark
>> this line as region" "" first-mismatch-at 0))
> 
> This is a tough lesson: the tests pass always on my system.
> 
> I think the failure you see is related to a problem marking a region
> in my test code (wish: support in `org-test-with-temp-text' for
> "<mark>" besides "<point>", but maybe that depends on ERT), else
> the problem would have shown up while testing the patch interactively.
> 
> I think that I have improved my region marking code by using "<point>"
> in the temp-text as a start. Then, I only have to find where to set 
> mark,
> and eventually exchange point and mark.
> 
> The test code now checks (mark) in the 3 places where a region is 
> marked.
> This looks superfluous if the code is really robust, but at least it 
> checks
> whether the region marking is (or was) the problem.
> 

To converge faster, I have split the test into 6 tests:

test-ob/demarcate-block-split-duplication
test-ob/demarcate-block-split-prefix-point
test-ob/demarcate-block-split-prefix-region
test-ob/demarcate-block-split-user-errors
test-ob/demarcate-block-wrap-point
test-ob/demarcate-block-wrap-region

The test failure on your system was due to the sub-test that is now
test-ob/demarcate-block-wrap-region

Regards -- Gerard

[-- Attachment #2: 0001-org-babel-demarcate-block-split-using-element-API.patch --]
[-- Type: application/octet-stream, Size: 19672 bytes --]

From fa161b3793a210961aceb5f7c7d0ddc8efb0aa52 Mon Sep 17 00:00:00 2001
From: Gerard Vermeulen <gerard.vermeulen@posteo.net>
Date: Thu, 11 Jan 2024 20:20:01 +0100
Subject: [PATCH] org-babel-demarcate-block: split using element API

* lisp/ob-babel.el (org-babel-demarcate-block): Modify a copy
of (org-element-at-point) to replace the old source block with 2 or 3
new modified copies by means of `org-element-interpret-data'.  The 1st
source block contains the text from the body of the old block before
point or region, the 2nd block contains the body text after point or
body text within region, and in case of region, the 3rd block contains
the text after region.  The caption and the name are deleted from the
1 or 2 blocks below the upper source block.  Indent all blocks
immediately after insertion (pitfall, see link).  Use :post-blank to
control white lines between inserted blocks.  Leave point at the last
inserted block.  Take care to preserve (current-column) of text
after point (and mark) in the 2nd (and 3rd) block.  Trying to split
when point or region is not within the body of the old source block
raises an user-error.
* lisp/ob-babel (org-get-src-block-info): add the "within blank lines
after a source block" condition to the doc-string to match it with the
doc-string of and a comment in `org-babel-demarcate-block'.
* testing/lisp/test-ob.el (test-ob/demarcate-block-split-duplication)
(test-ob/demarcate-block-split-prefix-point)
(test-ob/demarcate-block-split-prefix-region)
(test-ob/demarcate-block-split-user-errors)
(test-ob/demarcate-block-wrap-point)
(test-ob/demarcate-block-wrap-region): New tests to check test cases
that broke earlier versions of this patch.

Link: https://list.orgmode.org/87ply6nyue.fsf@localhost/
---
 lisp/ob-core.el         |  94 +++++++++++-----
 testing/lisp/test-ob.el | 238 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 304 insertions(+), 28 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 1de3af6ad..e200f82c0 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -73,10 +73,12 @@
 (declare-function org-element-parent "org-element-ast" (node))
 (declare-function org-element-type "org-element-ast" (node &optional anonymous))
 (declare-function org-element-type-p "org-element-ast" (node &optional types))
+(declare-function org-element-interpret-data "org-element" (data))
 (declare-function org-entry-get "org" (pom property &optional inherit literal-nil))
 (declare-function org-escape-code-in-region "org-src" (beg end))
 (declare-function org-forward-heading-same-level "org" (arg &optional invisible-ok))
 (declare-function org-in-commented-heading-p "org" (&optional no-inheritance))
+(declare-function org-indent-block "org" ())
 (declare-function org-indent-line "org" ())
 (declare-function org-list-get-list-end "org-list" (item struct prevs))
 (declare-function org-list-prevs-alist "org-list" (struct))
@@ -700,8 +702,9 @@ By default, consider the block at point.  However, when optional
 argument DATUM is provided, extract information from that parsed
 object instead.
 
-Return nil if point is not on a source block.  Otherwise, return
-a list with the following pattern:
+Return nil if point is not on a source block (blank lines after a
+source block are considered a part of that source block).
+Otherwise, return a list with the following pattern:
 
   (language body arguments switches name start coderef)"
   (let* ((datum (or datum (org-element-context)))
@@ -2051,7 +2054,7 @@ With optional prefix argument ARG, jump backward ARG many source blocks."
       (goto-char (match-beginning 5)))))
 
 (defun org-babel-demarcate-block (&optional arg)
-  "Wrap or split the code in the region or on the point.
+  "Wrap or split the code in an active region or at point.
 
 With prefix argument ARG, also create a new heading at point.
 
@@ -2061,41 +2064,76 @@ is created.  In both cases if the region is demarcated and if the
 region is not active then the point is demarcated.
 
 When called within blank lines after a code block, create a new code
-block of the same language with the previous."
+block of the same language as the previous."
   (interactive "P")
   (let* ((info (org-babel-get-src-block-info 'no-eval))
 	 (start (org-babel-where-is-src-block-head))
          ;; `start' will be nil when within space lines after src block.
 	 (block (and start (match-string 0)))
-	 (headers (and start (match-string 4)))
+         (body-beg (and start (match-beginning 5)))
+         (body-end (and start (match-end 5)))
 	 (stars (concat (make-string (or (org-current-level) 1) ?*) " "))
 	 (upper-case-p (and block
 			    (let (case-fold-search)
 			      (string-match-p "#\\+BEGIN_SRC" block)))))
     (if (and info start) ;; At src block, but not within blank lines after it.
-        (mapc
-         (lambda (place)
-           (save-excursion
-             (goto-char place)
-             (let ((lang (nth 0 info))
-                   (indent (make-string (org-current-text-indentation) ?\s)))
-	       (when (string-match "^[[:space:]]*$"
-                                   (buffer-substring (line-beginning-position)
-                                                     (line-end-position)))
-                 (delete-region (line-beginning-position) (line-end-position)))
-               (insert (concat
-		        (if (looking-at "^") "" "\n")
-		        indent (if upper-case-p "#+END_SRC\n" "#+end_src\n")
-		        (if arg stars indent) "\n"
-		        indent (if upper-case-p "#+BEGIN_SRC " "#+begin_src ")
-		        lang
-		        (if (> (length headers) 1)
-			    (concat " " headers) headers)
-		        (if (looking-at "[\n\r]")
-			    ""
-			  (concat "\n" (make-string (current-column) ? )))))))
-	   (move-end-of-line 2))
-         (sort (if (org-region-active-p) (list (mark) (point)) (list (point))) #'>))
+        (let* ((copy (org-element-copy (org-element-at-point)))
+               (before (org-element-begin copy))
+               (beyond (org-element-end copy))
+               (parts
+                (if (org-region-active-p)
+                    (list body-beg (region-beginning) (region-end) body-end)
+                  (list body-beg (point) body-end)))
+               (pads ;; To calculate left-side white-space padding.
+                (if (org-region-active-p)
+                    (list (region-beginning) (region-end))
+                  (list (point))))
+               (n (- (length parts) 2)) ;; 1 or 2 parts in `dolist' below.
+               ;; `post-blank' caches the property before setting it to 0.
+               (post-blank (org-element-property :post-blank copy)))
+          ;; Point or region are within body when parts is in increasing order.
+          (unless (apply #'<= parts)
+            (user-error "Select within the source block body to split it"))
+          (setq parts (mapcar (lambda (p) (buffer-substring (car p) (cdr p)))
+                              (seq-mapn #'cons parts (cdr parts))))
+          ;; Map positions to columns for white-space padding.
+          (setq pads (mapcar (lambda (p) (save-excursion
+                                           (goto-char p)
+                                           (current-column)))
+                             pads))
+          (push 0 pads) ;; The 1st part never requires white-space padding.
+          (setq parts (mapcar (lambda (p) (string-join
+                                           (list (make-string (car p) ?\s)
+                                                 (cdr p))))
+                              (seq-mapn #'cons pads parts)))
+          (delete-region before beyond)
+          ;; Set `:post-blank' to 0.  We take care of spacing between blocks.
+          (org-element-put-property copy :post-blank 0)
+          (org-element-put-property copy :value (car parts))
+          (insert (org-element-interpret-data copy))
+          ;; `org-indent-block' may see another `org-element' (e.g. paragraph)
+          ;; immediately after the block.  Ensure to indent the inserted block
+          ;; and move point to its end.
+          (org-babel-previous-src-block 1)
+          (org-indent-block)
+          (goto-char (org-element-end (org-element-at-point)))
+          (org-element-put-property copy :caption nil)
+          (org-element-put-property copy :name nil)
+          ;; Insert the 2nd block, and the 3rd block when region is active.
+          (dolist (part (cdr parts))
+            (org-element-put-property copy :value part)
+            (insert (if arg (concat stars "\n") "\n"))
+            (cl-decf n)
+            (when (= n 0)
+              ;; Use `post-blank' to reset the property of the last block.
+              (org-element-put-property copy :post-blank post-blank))
+            (insert (org-element-interpret-data copy))
+            ;; Ensure to indent the inserted block and move point to its end.
+            (org-babel-previous-src-block 1)
+            (org-indent-block)
+            (goto-char (org-element-end (org-element-at-point))))
+          ;; Leave point at the last inserted block.
+          (goto-char (org-babel-previous-src-block 1)))
       (let ((start (point))
 	    (lang (or (car info) ; Reuse language from previous block.
                       (completing-read
diff --git a/testing/lisp/test-ob.el b/testing/lisp/test-ob.el
index 42c77ca56..0766fd2e2 100644
--- a/testing/lisp/test-ob.el
+++ b/testing/lisp/test-ob.el
@@ -2545,6 +2545,244 @@ abc
 		(lambda (&rest _) (error "No warnings should occur"))))
        (org-babel-import-elisp-from-file (buffer-file-name))))))
 
+(ert-deftest test-ob/demarcate-block-split-duplicatigon ()
+  "Test duplication of language, body, switches, and headers in splitting."
+  (let ((caption "#+caption: caption.")
+        (regexp (rx "#+caption: caption."))
+        (org-adapt-indentation nil))
+    (org-test-with-temp-text (format "
+%s
+#+header: :var edge=\"also duplicated\"
+#+header: :wrap \"src any-spanish -n\"
+#+name: Nobody
+#+begin_src any-english -i -n :var here=\"duplicated\" :wrap \"src any-english -n\"
+
+above split
+<point>
+below split
+
+#+end_src
+do not org-indent-block text here
+" caption)
+      (let ((wrap-val "src any-spanish -n") above below avars bvars)
+        (org-babel-demarcate-block)
+        (goto-char (point-min))
+        (org-babel-next-src-block) ;; upper source block
+        (setq above (org-babel-get-src-block-info))
+        (setq avars (org-babel--get-vars (nth 2 above)))
+        (org-babel-next-src-block) ;; lower source block
+        (setq below (org-babel-get-src-block-info))
+        (setq bvars (org-babel--get-vars (nth 2 below)))
+        ;; duplicated multi-line header arguments:
+        (should (string= "also duplicated" (cdr (assq 'edge avars))))
+        (should (string= "also duplicated" (cdr (assq 'edge bvars))))
+        (should (string= wrap-val (cdr (assq :wrap (nth 2 above)))))
+        (should (string= wrap-val (cdr (assq :wrap (nth 2 below)))))
+        ;; duplicated language, other header arguments, and switches:
+        (should (string= "any-english" (nth 0 above)))
+        (should (string= "any-english" (nth 0 below)))
+        (should (string= "above split" (org-trim (nth 1 above))))
+        (should (string= "below split" (org-trim (nth 1 below))))
+        (should (string= "duplicated" (cdr (assq 'here avars))))
+        (should (string= "duplicated" (cdr (assq 'here bvars))))
+        (should (string= "-i -n" (nth 3 above)))
+        (should (string= "-i -n" (nth 3 below)))
+        ;; non-duplication of name and caption, which is not in above/below.
+        (should (string= "Nobody" (nth 4 above)))
+        (should-not (string= "" (nth 4 below)))
+        (goto-char (point-min))
+        (should (re-search-forward regexp))
+        (should-not (re-search-forward regexp nil 'noerror))))))
+
+(ert-deftest test-ob/demarcate-block-split-prefix-point ()
+  "Test prefix argument point splitting."
+  (let ((org-adapt-indentation t)
+        (org-edit-src-content-indentation 2)
+        (org-src-preserve-indentation nil)
+        (ok-col 11)
+        (stars "^\\*\\*\\*\\*\\*\\*\\*\\*\\*\\*"))
+    (org-test-with-temp-text "
+********** 10 stars with point between two lines
+           #+begin_src emacs-lisp
+             ;; to upper block
+             <point>
+             ;; to lower block
+           #+end_src
+"
+      (org-babel-demarcate-block 'a-prefix-arg)
+      (goto-char (point-min))
+      (dolist (regexp `(,stars
+                        "#\\+beg" ";; to upper block" "#\\+end"
+                        ,stars
+                        "#\\+beg" ";; to lower block" "#\\+end"))
+        (should (re-search-forward regexp))
+        (goto-char (match-beginning 0))
+        (cond ((string= regexp stars)
+               (should (= 0 (current-column))))
+              ((string-prefix-p ";;" regexp)
+               (should (= (+ ok-col org-edit-src-content-indentation)
+                          (current-column))))
+              (t (should (= ok-col (current-column)))))))))
+
+(ert-deftest test-ob/demarcate-block-split-prefix-region ()
+  "Test prefix argument region splitting."
+  (let ((org-adapt-indentation t)
+        (org-edit-src-content-indentation 2)
+        (org-src-preserve-indentation nil)
+        (ok-col 11)
+        (stars "^\\*\\*\\*\\*\\*\\*\\*\\*\\*\\*")
+        (parts '("to upper block" "mark those words as region" "to lower block")))
+    (org-test-with-temp-text (format "
+********** 10 stars with region between two lines
+           #+header: :var b=\"also seen\"
+           #+begin_src any-language -i -n :var a=\"seen\"
+             %s
+             <point>%s
+             %s
+           #+end_src
+" (nth 0 parts) (nth 1 parts) (nth 2 parts))
+      (let ((n 0) info vars)
+        (set-mark-command nil)
+        (search-forward (nth 1 parts))
+        (exchange-point-and-mark)
+        ;; Check (mark).
+        (should (string= (nth 1 parts)
+                         (buffer-substring
+                          (- (mark) (length (nth 1 parts))) (mark))))
+        (org-babel-demarcate-block 'a-prefix-argument)
+        (goto-char (point-min))
+        (while (< n (length parts))
+          (org-babel-next-src-block)
+          (setq info (org-babel-get-src-block-info))
+          (setq vars (org-babel--get-vars (nth 2 info)))
+          (should (string= "any-language" (nth 0 info)))
+          (should (string= (nth n parts) (org-trim (nth 1 info))))
+          (should (string= "seen" (cdr (assq 'a vars))))
+          (should (string= "also seen" (cdr (assq 'b vars))))
+          (should (string= "-i -n" (nth 3 info)))
+          (cl-incf n)))
+      (goto-char (point-min))
+      (dolist (regexp `(,stars
+                        "#\\+beg" ,(nth 0 parts) "#\\+end"
+                        ,stars
+                        "#\\+beg" ,(nth 1 parts) "#\\+end"
+                        ,stars
+                        "#\\+beg" ,(nth 2 parts) "#\\+end"))
+        (should (re-search-forward regexp))
+        (goto-char (match-beginning 0))
+        (cond ((string= regexp stars)
+               (should (= 0 (current-column))))
+              ((memq regexp parts)
+               (should (= (+ ok-col org-edit-src-content-indentation)
+                          (current-column))))
+              (t (should (= ok-col (current-column)))))))
+    ))
+
+(ert-deftest test-ob/demarcate-block-split-user-errors ()
+  "Test for `user-error's in splitting"
+  (let ((org-adapt-indentation t)
+        (org-edit-src-content-indentation 2)
+        (org-src-preserve-indentation))
+    (let* ((caption "#+caption: caption.")
+           (within-body ";; within-body")
+           (below-block "# below block")
+           (template  "
+%s%s
+#+begin_src emacs-lisp
+
+  %s
+
+#+end_src
+
+%s%s
+"))
+      ;; Test point at caption.
+      (org-test-with-temp-text
+          (format template "<point>" caption within-body below-block "")
+        ;; Check (point).
+        (should (string= caption
+                         (buffer-substring
+                          (point) (+ (point) (length caption)))))
+        (should-error (org-babel-demarcate-block) :type 'user-error))
+      ;; Test region from below the block (mark) to within the body (point).
+      (org-test-with-temp-text
+          (format template "" caption within-body below-block "<point>")
+        ;; Set mark.
+        (set-mark-command nil)
+        ;; Check (mark).
+        (should (string= below-block
+                         (buffer-substring
+                          (- (mark) (length below-block)) (mark))))
+        ;; Set point.
+        (should (search-backward within-body nil 'noerror))
+        (goto-char (match-beginning 0))
+        ;; Check (point).
+        (should (string= within-body
+                         (buffer-substring
+                          (point) (+ (point) (length within-body)))))
+        (should-error (org-babel-demarcate-block) :type 'user-error)))))
+
+(ert-deftest test-ob/demarcate-block-wrap-point ()
+  "Test wrapping point in blank lines below a source block."
+  (org-test-with-temp-text "
+#+begin_src any-language -i -n :var here=\"not duplicated\"
+to upper block
+#+end_src
+<point>
+"
+    (let (info vars)
+      (org-babel-demarcate-block)
+      (goto-char (point-min))
+      (org-babel-next-src-block)
+      (setq info (org-babel-get-src-block-info))  ;; upper source block info
+      (setq vars (org-babel--get-vars (nth 2 info)))
+      (should (string= "any-language" (nth 0 info)))
+      (should (string= "to upper block" (org-trim (nth 1 info))))
+      (should (string= "not duplicated" (cdr (assq 'here vars))))
+      (should (string= "-i -n" (nth 3 info)))
+      (org-babel-next-src-block)
+      (setq info (org-babel-get-src-block-info)) ;; lower source block info
+      (setq vars (org-babel--get-vars (nth 2 info)))
+      (should (string= "any-language" (nth 0 info)))
+      (should (string= "" (org-trim (nth 1 info))))
+      (should-not vars)
+      (should (string= "" (nth 3 info))))))
+
+(ert-deftest test-ob/demarcate-block-wrap-region ()
+  "Test wrapping region in blank lines below a source block."
+  (let ((region-text "mark this line as region leaving point in blank lines"))
+    (org-test-with-temp-text (format "
+#+begin_src any-language -i -n :var here=\"not duplicated\"
+to upper block
+#+end_src
+<point>
+%s
+" region-text)
+      (let (info vars)
+        (set-mark-command nil)
+        (search-forward region-text)
+        (exchange-point-and-mark)
+        ;; Check (mark).
+        (should (string= region-text
+                         (buffer-substring
+                          (- (mark) (length region-text)) (mark))))
+        (org-babel-demarcate-block)
+        (goto-char (point-min))
+        (org-babel-next-src-block)
+        (setq info (org-babel-get-src-block-info))  ;; upper source block info
+        (setq vars (org-babel--get-vars (nth 2 info)))
+        (should (string= "any-language" (nth 0 info)))
+        (should (string= "to upper block" (org-trim (nth 1 info))))
+        (should (string= "not duplicated" (cdr (assq 'here vars))))
+        (should (string= "-i -n" (nth 3 info)))
+        (org-babel-next-src-block)
+        (setq info (org-babel-get-src-block-info)) ;; lower source block info
+        (setq vars (org-babel--get-vars (nth 2 info)))
+        (should (string= "any-language" (nth 0 info)))
+        (should (string= region-text (org-trim (nth 1 info))))
+        (should-not vars)
+        (should (string= "" (nth 3 info)))))))
+
 (provide 'test-ob)
 
 ;;; test-ob ends here
-- 
2.42.0