emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
From: Ryan Scott <ryan@vicarious-living.com>
To: Ihor Radchenko <yantar92@gmail.com>
Cc: numbchild@gmail.com, Greg Minshall <minshall@umich.edu>,
	Timothy <tecosaur@gmail.com>,
	emacs-orgmode@gnu.org
Subject: Re: [PATCH] Re: New source block results option for attaching file to node
Date: Tue, 14 Jun 2022 02:04:24 -0700	[thread overview]
Message-ID: <CAHBUXNwwMPEVUB-Bgs1S2xUaQQR4hoenGGvMv1k7zbiUWQG6+w@mail.gmail.com> (raw)
In-Reply-To: <CAHBUXNyierk4_5eWbAwXhB4hEbjRXs6ZyR_e84E2TrDDYOo3yQ@mail.gmail.com>


[-- Attachment #1.1: Type: text/plain, Size: 2845 bytes --]

I put together a clean setup in docker for running the tests and believe
I've gotten to the source of the problem.
However I've thought that a few times now.

This version of the patch has all tests passing.
The problem was in org-results-to-file and the attach dir detection. It
both had problems with an introduced assumption that the buffer-file-name
would be non-nil (causing several tests to fail) as well as being generally
overzealous in detecting file result paths as attachment links.


On Mon, Jun 13, 2022 at 10:55 PM Ryan Scott <ryan@vicarious-living.com>
wrote:

> Strange. I'll figure out a better setup for running the tests and get to
> the bottom of that.
> Thanks for the help.
>
> On Tue, Jun 14, 2022, 00:10 Ihor Radchenko <yantar92@gmail.com> wrote:
>
>> Ryan Scott <ryan@vicarious-living.com> writes:
>>
>> > Ah sorry about that. I'm on a windows laptop and didn't have make, so
>> was
>> > testing interactively and they were passing.
>> > I cleaned that up and remove the f-* usage and verified under Ubuntu (on
>> > WSL) that the new tests are passing. I was getting some failures with
>> > unrelated tests, but also get those in master as well.
>>
>> I am still getting test failures using Emacs 28.
>> All the tests are passing on main.
>>
>> Example backtrace:
>>
>> file-name-directory(nil)
>>   org-babel-result-to-file("/tmp/test.txt" nil attachment)
>>   org-babel-insert-result("/tmp/test.txt" ("replace" "value" "file" "g
>>   org-babel-execute-src-block()
>>   (progn (org-mode) (let ((point (string-match "<point>" inside-text))
>>   (unwind-protect (progn (org-mode) (let ((point (string-match "<point
>>   (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
>>   (let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current
>>   (let ((inside-text (if (stringp "\n<point>#+begin_src shell :results
>>   (let ((lexical-binding nil)) (let ((inside-text (if (stringp "\n<poi
>>   (lambda nil (let ((lexical-binding nil)) (let ((inside-text (if (str
>>   ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
>>   ert-run-test(#s(ert-test :name test-ob/result-graphics-link-type-hea
>>   ert-run-or-rerun-test(#s(ert--stats :selector "test-ob" :tests ... :
>>   ert-run-tests("test-ob" #f(compiled-function (event-type &rest event
>>   ert-run-tests-batch("test-ob")
>>   ert-run-tests-batch-and-exit("test-ob")
>>   (let ((org-id-track-globally t) (org-test-selector (if org-test-sele
>>   org-test-run-batch-tests("test-ob")
>>   command-line-1(("--eval" "(setq vc-handled-backends nil org-startup-
>>   command-line()
>>   normal-top-level()
>> Test test-ob/result-graphics-link-type-header-argument condition:
>>     (wrong-type-argument stringp nil)
>>    FAILED  129/140  test-ob/result-graphics-link-type-header-argument
>> (0.006189 sec)
>>
>> Best,
>> Ihor
>>
>

[-- Attachment #1.2: Type: text/html, Size: 3905 bytes --]

[-- Attachment #2: org-src-block-results-attach-dir.patch --]
[-- Type: application/octet-stream, Size: 11527 bytes --]

From 757762aebe25229016b5dc854e894fd9427c0e3c Mon Sep 17 00:00:00 2001
From: Ryan Scott <rscott@roblox.com>
Date: Fri, 10 Jun 2022 00:01:37 -0700
Subject: [PATCH] ob-core.el/babel: Special handling for attachment links in
 src block

* ob-core.el (org-babel-merge-params): Specifying the symbol 'attach`
or string "'attach" as the value of the `:dir' header now functions as
":dir (org-attach-dir nil t) :mkdirp t"
(org-babel-result-to-file): Optional `TYPE' argument accepts symbol
'attachment` to fixup up paths under `(org-attach-dir)' and use the
link type "attachment:" when that is detected.
(org-babel-insert-result): Pass symbol `attachment' as `TYPE' to
`org-babel-result-to-file'
* org-attach.el (org-attach-dir): Added autoload header to simplify
dependencies necessary to support this feature (called in
`org-babel-merge-params').
* test-ob.el (test-ob-core/dir-attach): Added unit test for new attach
feature
---
 doc/org-manual.org      |  6 ++++
 etc/ORG-NEWS            |  7 ++++
 lisp/ob-core.el         | 79 +++++++++++++++++++++++++++++------------
 lisp/org-attach.el      |  1 +
 testing/lisp/test-ob.el | 66 ++++++++++++++++++++++++++++++++++
 5 files changed, 136 insertions(+), 23 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 32a45f884..5f9b3353b 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -17541,6 +17541,12 @@ directory with {{{kbd(M-x cd RET DIRECTORY)}}}, and then not setting
 =dir=.  Under the surface, =dir= simply sets the value of the Emacs
 variable ~default-directory~.  Setting =mkdirp= header argument to
 a non-~nil~ value creates the directory, if necessary.
+Setting =dir= to the symbol ~attach~ or the string ~"'attach"~ will
+set =dir= to the directory returned by ~(org-attach-dir)~, set =:mkdir
+yes=, and insert any file paths, as when using =:results file=, which
+are under the node's attachment directory using =attachment:= links
+instead of the usual =file:= links. Any returned path outside of the
+attachment directory will use =file:= links as per usual.
 
 For example, to save the plot file in the =Work/= folder of the home
 directory---notice tilde is expanded:
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 35af94f92..0c88378d6 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -795,6 +795,13 @@ Finally, the closures are only evaluated if they're not overridden for
 a source block. This improves efficiency in cases where the result of
 a compute-expensive closure would otherwise be discarded.
 
+*** New special value ~'attach~ for src block =:dir= option
+
+Passing the symbol ~attach~ or string ="'attach"= (with quotes) to the =:dir=
+option of a src block is now equivalent to =:dir (org-attach-dir) :mkdir yes=
+and any file results with a path descended from the attachment directory will
+use =attachment:= style links instead of the standard =file:= link type.
+
 ** Miscellaneous
 *** =org-bibtex= includes =doi= and =url= entries when exporting to BiBTeX
 =doi= and =url= entries have been made optional for some publication
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 09d6adfe0..c2446daa6 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -801,7 +801,8 @@ block."
 		    (let ((*this* (if (not file) result
 				    (org-babel-result-to-file
 				     file
-				     (org-babel--file-desc params result)))))
+				     (org-babel--file-desc params result)
+                                     'attachment))))
 		      (setq result (org-babel-ref-resolve post))
 		      (when file
 			(setq result-params (remove "file" result-params))))))
@@ -2298,11 +2299,14 @@ INFO may provide the values of these header arguments (in the
   (cond ((stringp result)
 	 (setq result (org-no-properties result))
 	 (when (member "file" result-params)
-	   (setq result (org-babel-result-to-file
-			 result
-			 (org-babel--file-desc (nth 2 info) result)))))
+	   (setq result
+                 (org-babel-result-to-file
+		  result
+		  (org-babel--file-desc (nth 2 info) result)
+                  'attachment))))
 	((listp result))
 	(t (setq result (format "%S" result))))
+
   (if (and result-params (member "silent" result-params))
       (progn (message (replace-regexp-in-string "%" "%%" (format "%S" result)))
 	     result)
@@ -2605,27 +2609,49 @@ in the buffer."
 		 (line-beginning-position 2))
 	     (point))))))
 
-(defun org-babel-result-to-file (result &optional description)
+(defun org-babel-result-to-file (result &optional description type)
   "Convert RESULT into an Org link with optional DESCRIPTION.
 If the `default-directory' is different from the containing
-file's directory then expand relative links."
+file's directory then expand relative links.
+
+If the optional TYPE is passed as 'attachment` and the path is a
+descendant of the DEFAULT-DIRECTORY, the generated link will be
+specified as an an \"attachment:\" style link"
   (when (stringp result)
-    (let ((same-directory?
-	   (and (buffer-file-name (buffer-base-buffer))
-		(not (string= (expand-file-name default-directory)
-			      (expand-file-name
-			       (file-name-directory
-			        (buffer-file-name (buffer-base-buffer)))))))))
-      (format "[[file:%s]%s]"
-	      (if (and default-directory
-		       (buffer-file-name (buffer-base-buffer)) same-directory?)
-		  (if (eq org-link-file-path-type 'adaptive)
-		      (file-relative-name
-		       (expand-file-name result default-directory)
-		       (file-name-directory
-			(buffer-file-name (buffer-base-buffer))))
-		    (expand-file-name result default-directory))
-		result)
+    (let* ((result-file-name (expand-file-name result))
+           (base-file-name (buffer-file-name (buffer-base-buffer)))
+           (base-directory (and buffer-file-name
+                                (file-name-directory base-file-name)))
+           (same-directory?
+	    (and base-file-name
+	         (not (string= (expand-file-name default-directory)
+			       (expand-file-name
+			        base-directory)))))
+           (request-attachment (eq type 'attachment))
+           (attach-dir (let* ((default-directory base-directory)
+                              (dir (org-attach-dir nil t)))
+                         (when dir
+                           (expand-file-name dir))))
+           (in-attach-dir (and request-attachment
+                               attach-dir
+                               (string-prefix-p
+                                attach-dir
+                                result-file-name))))
+      (format "[[%s:%s]%s]"
+              (pcase type
+                ((and 'attachment (guard in-attach-dir)) "attachment")
+                (_ "file"))
+              (if (and request-attachment in-attach-dir)
+                  (file-relative-name result-file-name)
+	        (if (and default-directory
+		         base-file-name same-directory?)
+		    (if (eq org-link-file-path-type 'adaptive)
+		        (file-relative-name
+		         result-file-name
+                         (file-name-directory
+			  base-file-name))
+		      result-file-name)
+		  result))
 	      (if description (concat "[" description "]") "")))))
 
 (defun org-babel-examplify-region (beg end &optional results-switches inline)
@@ -2756,10 +2782,17 @@ parameters when merging lists."
 	   (setq exports (funcall merge
 				  exports-exclusive-groups
 				  exports
-				  (split-string
+                                  (split-string
                                    (cond ((and value (functionp value)) (funcall value))
                                          (value value)
                                          (t ""))))))
+          ((or '(:dir . attach) '(:dir . "'attach"))
+           (unless (org-attach-dir nil t)
+             (error "No attachment directory for element (add :ID: or :DIR: property)"))
+           (setq params (append
+                         `((:dir . ,(org-attach-dir nil t))
+                           (:mkdirp . "yes"))
+                         (assq-delete-all :dir (assq-delete-all :mkdir params)))))
 	  ;; Regular keywords: any value overwrites the previous one.
 	  (_ (setq params (cons pair (assq-delete-all (car pair) params)))))))
     ;; Handle `:var' and clear out colnames and rownames for replaced
diff --git a/lisp/org-attach.el b/lisp/org-attach.el
index 5ee2b84b2..6a061b2b3 100644
--- a/lisp/org-attach.el
+++ b/lisp/org-attach.el
@@ -324,6 +324,7 @@ Shows a list of commands and prompts for another key to execute a command."
 	    (command-execute command)
 	  (error "No such attachment command: %c" c))))))
 
+;;;###autoload
 (defun org-attach-dir (&optional create-if-not-exists-p no-fs-check)
   "Return the directory associated with the current outline node.
 First check for DIR property, then ID property.
diff --git a/testing/lisp/test-ob.el b/testing/lisp/test-ob.el
index aa05f87a3..a62bd56bf 100644
--- a/testing/lisp/test-ob.el
+++ b/testing/lisp/test-ob.el
@@ -1770,6 +1770,72 @@ nil
                 (file-modes "t.sh")
               (delete-file "t.sh"))))))
 
+(ert-deftest test-ob-core/dir-attach ()
+  "Test :dir header using special 'attach value"
+  (should
+   (org-test-with-temp-text-in-file
+    "* 'attach Symbol
+<point>#+begin_src elisp :dir 'attach :results file
+(with-temp-file \"test.txt\" (insert \"attachment testing\n\"))
+\"test.txt\"
+#+end_src"
+    (org-id-get-create)
+    (org-babel-execute-src-block)
+    (goto-char (org-babel-where-is-src-block-result))
+    (forward-line)
+    (and
+     (file-exists-p (format "%s/test.txt" (org-attach-dir nil t)))
+     (string= (buffer-substring-no-properties (point) (line-end-position))
+              "[[attachment:test.txt]]"))))
+  (should
+   (org-test-with-temp-text-in-file
+    "* 'attach String
+<point>#+begin_src elisp :dir \"'attach\" :results file
+(with-temp-file \"test.txt\" (insert \"attachment testing\n\"))
+\"test.txt\"
+#+end_src"
+    (org-id-get-create)
+    (org-babel-execute-src-block)
+    (goto-char (org-babel-where-is-src-block-result))
+    (forward-line)
+    (and
+     (file-exists-p (format "%s/test.txt" (org-attach-dir nil t)))
+     (string= (buffer-substring-no-properties (point) (line-end-position))
+              "[[attachment:test.txt]]"))))
+  (should
+   (org-test-with-temp-text-in-file
+    "* 'attach with Existing DIR property
+:PROPERTIES:
+:DIR:      custom-attach-dir
+:END:
+
+<point>#+begin_src elisp :dir 'attach :results file
+(with-temp-file \"test.txt\" (insert \"attachment testing\n\"))
+\"test.txt\"
+#+end_src"
+    (message "DIR: %s" (org-attach-dir t))
+    (org-babel-execute-src-block)
+    (goto-char (org-babel-where-is-src-block-result))
+    (forward-line)
+    (and
+     (file-exists-p (format "%s/test.txt" (org-attach-dir nil t)))
+     (string= (buffer-substring-no-properties (point) (line-end-position))
+              "[[attachment:test.txt]]"))))
+  (should-error
+   (org-test-with-temp-text-in-file
+    "* 'attach with no ID or DIR
+<point>#+begin_src elisp :dir 'attach :results file
+(with-temp-file \"test.txt\" (insert \"attachment testing\n\"))
+\"test.txt\"
+#+end_src"
+    (org-babel-execute-src-block)
+    (goto-char (org-babel-where-is-src-block-result))
+    (forward-line)
+    (and
+     (file-exists-p (format "%s/test.txt" (org-attach-dir nil t)))
+     (string= (buffer-substring-no-properties (point) (line-end-position))
+              "[[attachment:test.txt]]")))))
+
 (ert-deftest test-ob-core/dir-mkdirp ()
   "Test :mkdirp with :dir header combination."
   (should-not
-- 
2.33.0.windows.2


  reply	other threads:[~2022-06-14  9:08 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-26  8:48 New source block results option for attaching file to node Ryan Scott
2021-08-31 11:15 ` Timothy
2021-08-31 19:43   ` Ryan Scott
2021-09-01 14:45     ` Ihor Radchenko
2021-09-01 20:01       ` Ryan Scott
2021-09-02  7:40         ` [PATCH] " Ryan Scott
2021-09-02 13:44           ` Greg Minshall
2021-09-03  3:10             ` Ihor Radchenko
2021-09-03  3:28               ` Ryan Scott
2021-09-05 13:22                 ` Ihor Radchenko
2021-09-05 13:56                   ` Ryan Scott
2021-09-10  1:04                     ` Ryan Scott
2021-09-10  6:26                       ` Timothy
2021-10-02  8:32                       ` Ihor Radchenko
2021-10-02  9:39                         ` Ryan Scott
2021-10-05  0:04               ` Christopher M. Miles
2021-10-05  1:05                 ` Ryan Scott
2021-10-08  1:22                   ` Christopher M. Miles
2021-11-05  7:16                   ` Ryan Scott
2022-04-21 12:47                     ` Ihor Radchenko
2022-04-21 17:29                       ` Ryan Scott
2022-04-22  6:02                         ` Ihor Radchenko
2022-04-22  6:19                           ` Ryan Scott
2022-06-10  8:06                             ` Ryan Scott
2022-06-11  4:32                               ` Ihor Radchenko
2022-06-11  7:47                                 ` Ryan Scott
2022-06-11 12:49                                   ` Ihor Radchenko
2022-06-12  0:47                                     ` Ryan Scott
2022-06-14  4:11                                       ` Ihor Radchenko
2022-06-14  5:55                                         ` Ryan Scott
2022-06-14  9:04                                           ` Ryan Scott [this message]
2022-06-14 13:48                                             ` Ihor Radchenko
2022-06-14 18:23                                               ` Ryan Scott
2022-06-11 12:51                                   ` Ihor Radchenko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.orgmode.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAHBUXNwwMPEVUB-Bgs1S2xUaQQR4hoenGGvMv1k7zbiUWQG6+w@mail.gmail.com \
    --to=ryan@vicarious-living.com \
    --cc=emacs-orgmode@gnu.org \
    --cc=minshall@umich.edu \
    --cc=numbchild@gmail.com \
    --cc=tecosaur@gmail.com \
    --cc=yantar92@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).