* [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. @ 2014-04-22 19:54 Aaron Ecay 2014-04-22 21:22 ` Bastien 0 siblings, 1 reply; 17+ messages in thread From: Aaron Ecay @ 2014-04-22 19:54 UTC (permalink / raw) To: emacs-orgmode * lisp/ob-core.el (org-babel-generate-file-param): New function. (org-babel-get-src-block-info): Use it. --- lisp/ob-core.el | 33 ++++++++++++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-) This patch allows the autogeneration of result file names from source block names. Thus, instead of: #+name: one #+begin_src R :results file graphics :file one.png ... #+end_src One can write (with the same result, that one.png is generated): #+name: one #+begin_src R :results file graphics :file png ... #+end_src The benefit comes from the reduced duplication of information in the file. It also becomes possible to use the usual property inheritance mechanisms (via #+PROPERTY lines or :PROPERTY: drawers) to specify result file types for multiple source blocks at once. This patch also introduces the :output-dir property, which can be used to redirect all the output from blocks in a (file/subtree) to a subdirectory. The patch treats any :file args containing a period as before, so backwards compatibility with old files should in most cases be maintained. If this patch looks good, I can provide tests and documentation. Aaron diff --git a/lisp/ob-core.el b/lisp/ob-core.el index 1348f04..2defabf 100644 --- a/lisp/ob-core.el +++ b/lisp/ob-core.el @@ -282,7 +282,9 @@ Returns a list (setq info (org-babel-parse-inline-src-block-match)))) ;; resolve variable references and add summary parameters (when (and info (not light)) - (setf (nth 2 info) (org-babel-process-params (nth 2 info)))) + (setf (nth 2 info) (org-babel-generate-file-param + name + (org-babel-process-params (nth 2 info))))) (when info (append info (list name indent head))))) (defvar org-babel-exp-reference-buffer nil @@ -2890,6 +2892,35 @@ For the format of SAFE-LIST, see `org-babel-safe-header-args'." (member (cdr pair) (cdr entry))) (t nil))))))) +(defun org-babel-generate-file-param (src-name params) + "Calculate the filename for source block results. + +The directory is calculated from the :output-dir property of the +source block; if not specified, use the current directory. + +If the source block has a #+NAME and the :file parameter does not +contain any period characters, then the :file parameter is +treated as an extension, and the output file name is the +concatenation of the directory (as calculated above), the block +name, a period, and the parameter value as a file extension. +Otherwise, the :file parameter is treated as a full file name, +and the output file name is the directory (as calculated above) +plus the parameter value." + (let* ((file-cons (assq :file params)) + (file (cdr-safe file-cons)) + (dir (or (cdr-safe (assq :output-dir params)) ""))) + (when (and file (not (string= dir ""))) + (make-directory dir t) + (unless (string-match "/\\'" dir) + (setq dir (concat dir "/")))) + (cond + ((not file) nil) + ((or (string-match "\\." file) (not src-name)) + ;; Either the file name already has an extension, or there is no + ;; name for this source block -> we cannot do anything + (setcdr file-cons (concat dir file))) + (t (setcdr file-cons (concat dir src-name "." file)))) + params)) ;;; Used by backends: R, Maxima, Octave. (defun org-babel-graphical-output-file (params) -- 1.9.2 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. 2014-04-22 19:54 [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks Aaron Ecay @ 2014-04-22 21:22 ` Bastien 2014-04-23 0:04 ` Aaron Ecay 0 siblings, 1 reply; 17+ messages in thread From: Bastien @ 2014-04-22 21:22 UTC (permalink / raw) To: Aaron Ecay; +Cc: emacs-orgmode Hi Aaron, thanks for the patch. Aaron Ecay <aaronecay@gmail.com> writes: > #+name: one > #+begin_src R :results file graphics :file png > ... > #+end_src What happens when there is :file png with no #+name line? > The benefit comes from the reduced duplication of information in the > file. It also becomes possible to use the usual property inheritance > mechanisms (via #+PROPERTY lines or :PROPERTY: drawers) to specify > result file types for multiple source blocks at once. > > This patch also introduces the :output-dir property, which can be used > to redirect all the output from blocks in a (file/subtree) to a > subdirectory. Does :output-dir accept absolute or relative paths? I'm asking because you speak of "subdirectory", but both should be accepted IMHO. > The patch treats any :file args containing a period as before, so > backwards compatibility with old files should in most cases be > maintained. Maybe there are cases where the :file value does not take an extension but the user still want to write the output to this file? How would your patch handle this? > If this patch looks good, I can provide tests and documentation. Looks useful to me, but ":file png" looks wrong, with too much implicit. We should find something less confusing. Thanks, -- Bastien ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. 2014-04-22 21:22 ` Bastien @ 2014-04-23 0:04 ` Aaron Ecay 2014-04-23 1:35 ` Eric Schulte ` (2 more replies) 0 siblings, 3 replies; 17+ messages in thread From: Aaron Ecay @ 2014-04-23 0:04 UTC (permalink / raw) To: Bastien; +Cc: emacs-orgmode Hi Bastien, Thanks for your feedback. 2014ko apirilak 22an, Bastien-ek idatzi zuen: [...] >> #+name: one >> #+begin_src R :results file graphics :file png >> ... >> #+end_src > > What happens when there is :file png with no #+name line? This case will be treated as before the patch: output will go to the “png” file. (The :output-dir property, if present, will apply.) > Does :output-dir accept absolute or relative paths? I'm asking > because you speak of "subdirectory", but both should be accepted > IMHO. I agree. I think the current patch does this as long as :output-dir is an absolute pathname, but I have not tested that case. I will > > Maybe there are cases where the :file value does not take an extension > but the user still want to write the output to this file? How would > your patch handle this? At present, it doesn’t. > Looks useful to me, but ":file png" looks wrong, with too much > implicit. We should find something less confusing. One option would be to use :file-ext instead, to generate a :file parameter. I didn’t go this route because autogenerating :file from other parameters seemed like too much magic. But your points in the other direction are good. How does this sound as an algorithm: 1. if :file is present, behave exactly as we do now 2. if :file is absent but :file-ext and a #+name is present, generate a :file parameter from :output-dir, the #+name, and :file-ext. Open questions: 1. should :file-ext without a #+name be a no-op, or an error? 2. should :output-dir apply to the :file case as well? -- Aaron Ecay ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. 2014-04-23 0:04 ` Aaron Ecay @ 2014-04-23 1:35 ` Eric Schulte 2014-04-23 14:58 ` Bastien 2014-04-28 2:18 ` Aaron Ecay 2014-04-23 6:27 ` Bastien 2014-04-23 19:44 ` Achim Gratz 2 siblings, 2 replies; 17+ messages in thread From: Eric Schulte @ 2014-04-23 1:35 UTC (permalink / raw) To: Bastien; +Cc: emacs-orgmode >> Looks useful to me, but ":file png" looks wrong, with too much >> implicit. We should find something less confusing. > > One option would be to use :file-ext instead, to generate a :file > parameter. I didn’t go this route because autogenerating :file from > other parameters seemed like too much magic. But your points in the > other direction are good. > > How does this sound as an algorithm: > 1. if :file is present, behave exactly as we do now > 2. if :file is absent but :file-ext and a #+name is present, generate a > :file parameter from :output-dir, the #+name, and :file-ext. > This sounds like a great approach to me. > > Open questions: > 1. should :file-ext without a #+name be a no-op, or an error? One option might be to borrow naming behavior from the comment functionality in ob-tangle which looks like the following (from line 426 in ob-tangle.el). (let (... (source-name (intern (or (nth 4 info) ; explicit #+name: (format "%s:%d" ; constructed from header and position (or (ignore-errors (nth 4 (org-heading-components))) "No heading") block-counter)))) ...)) > > 2. should :output-dir apply to the :file case as well? > If you mean "should :output-dir be used as the base when :file is a relative pathname" then I'd say "yes", and I think if this isn't the current behavior then the current behavior should be changed. Thanks for this nice patch, Eric > > -- > Aaron Ecay > -- Eric Schulte https://cs.unm.edu/~eschulte PGP: 0x614CA05D ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. 2014-04-23 1:35 ` Eric Schulte @ 2014-04-23 14:58 ` Bastien 2014-04-28 2:18 ` Aaron Ecay 1 sibling, 0 replies; 17+ messages in thread From: Bastien @ 2014-04-23 14:58 UTC (permalink / raw) To: Eric Schulte; +Cc: emacs-orgmode Hi, Eric Schulte <schulte.eric@gmail.com> writes: > One option might be to borrow naming behavior from the comment > functionality in ob-tangle which looks like the following (from line 426 > in ob-tangle.el). Indeed, it's safer than just using the buffer file name. Aaron, please go ahead with the patch when you want, thanks again for it! -- Bastien ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. 2014-04-23 1:35 ` Eric Schulte 2014-04-23 14:58 ` Bastien @ 2014-04-28 2:18 ` Aaron Ecay 2014-04-28 6:20 ` Achim Gratz ` (2 more replies) 1 sibling, 3 replies; 17+ messages in thread From: Aaron Ecay @ 2014-04-28 2:18 UTC (permalink / raw) To: Eric Schulte, Bastien; +Cc: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 2452 bytes --] Hi Eric, Bastien, Achim, Thanks so much for the feedback. I’ve adopted the :file-ext approach suggested by Bastien, leaving the previous default behavior in place for blocks with a :file argument. 2014ko apirilak 22an, Eric Schulte-ek idatzi zuen: [...] > One option might be to borrow naming behavior from the comment > functionality in ob-tangle which looks like the following (from line 426 > in ob-tangle.el). > > (let (... > (source-name > (intern (or (nth 4 info) ; explicit #+name: > (format "%s:%d" ; constructed from header and position > (or (ignore-errors (nth 4 (org-heading-components))) > "No heading") > block-counter)))) > ...)) I’m not sure I like this approach. It relies on counting source blocks, so an addition/deletion of a block could change the index. I’m worried that this can lead to the accumulation of many output files: heading:1.ext, heading:2.ext, ... all with no clear indication of what block they were spawned by. It would also be possible for the result links in the buffer to become inconsistent with the actual block:auto-generated name mapping. I think I would prefer the code in this patch to do nothing in this case (not create a :file value), but for language-specific code that needs a :file to raise an error to prompt the user to add a name. > >> >> 2. should :output-dir apply to the :file case as well? >> > > If you mean "should :output-dir be used as the base when :file is a > relative pathname" then I'd say "yes", and I think if this isn't the > current behavior then the current behavior should be changed. Achim raises a backwards compatibility concern. I am not sure how serious it is: the default settings (no :output-dir) are backwards compatible, and if users set that arg we ought to just give them what they ask for. Nonetheless, the new version of the patch conservatively obeys Achim’s suggestion. I can change this to your suggestion, if that is the consensus. To address a comment from Bastien: :output-dir accepts absolute as well as relative directory names. Referring to a “subdirectory” was a mistake on my part; the docs in the new patch should be clearer. The updated patch (now with docs and tests) is attached to this email. Thanks again, -- Aaron Ecay [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-ob-core.el-allow-the-auto-generation-of-output-file-.patch --] [-- Type: text/x-diff, Size: 7429 bytes --] From 4b428820432752117c60b79da0a79fd4e50e4ba1 Mon Sep 17 00:00:00 2001 From: Aaron Ecay <aaronecay@gmail.com> Date: Tue, 22 Apr 2014 15:13:48 -0400 Subject: [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. * lisp/ob-core.el (org-babel-generate-file-param): New function. (org-babel-get-src-block-info): Use it. * testing/lisp/test-ob.el (test-org-babel/file-ext-and-output-dir): New test. * doc/org.texi (Specific header arguments): Add doc for :file-ext and :output-dir header args. --- doc/org.texi | 27 +++++++++++++++++++++++++++ lisp/ob-core.el | 34 ++++++++++++++++++++++++++++++++++ testing/examples/babel.org | 34 ++++++++++++++++++++++++++++++++++ testing/lisp/test-ob.el | 14 ++++++++++++++ 4 files changed, 109 insertions(+) diff --git a/doc/org.texi b/doc/org.texi index 2546be1..79cc044 100644 --- a/doc/org.texi +++ b/doc/org.texi @@ -14406,6 +14406,8 @@ argument in lowercase letters. The following header arguments are defined: be collected and handled * file:: Specify a path for file output * file-desc:: Specify a description for file results +* file-ext:: Specify an extension for file output +* output-dir:: Specify a directory to write file output to * dir:: Specify the default (possibly remote) directory for code block execution * exports:: Export code and/or results @@ -14840,6 +14842,31 @@ description for file code block results which are inserted as Org mode links with no value the link path will be placed in both the ``link'' and the ``description'' portion of the Org mode link. +@node file-ext +@subsubsection @code{:file-ext} +@cindex @code{:file-ext}, src header argument + +The value of the @code{:file-ext} header argument is used to provide an +extension to write the file output to. It is combined with the +@code{#+NAME:} of the source block and the value of the @ref{output-dir} +header argument to generate a complete file name. + +This header arg will be overridden by @code{:file}, and thus has no effect +when the latter is specified. + +@node output-dir +@subsubsection @code{:output-dir} +@cindex @code{:output-dir}, src header argument + +The value of the @code{:output-dir} header argument is used to provide a +directory to write the file output to. It may specify an absolute directory +(beginning with @code{/}) or a relative directory (without @code{/}). It is +combined with the @code{#+NAME:} of the source block and the value of the +@ref{output-dir} header argument to generate a complete file name. + +This header arg will be overridden by @code{:file}, and thus has no effect +when the latter is specified. + @node dir @subsubsection @code{:dir} and remote execution @cindex @code{:dir}, src header argument diff --git a/lisp/ob-core.el b/lisp/ob-core.el index 1348f04..4a3683d 100644 --- a/lisp/ob-core.el +++ b/lisp/ob-core.el @@ -283,6 +283,8 @@ Returns a list ;; resolve variable references and add summary parameters (when (and info (not light)) (setf (nth 2 info) (org-babel-process-params (nth 2 info)))) + (when info + (setf (nth 2 info) (org-babel-generate-file-param name (nth 2 info)))) (when info (append info (list name indent head))))) (defvar org-babel-exp-reference-buffer nil @@ -2890,6 +2892,38 @@ For the format of SAFE-LIST, see `org-babel-safe-header-args'." (member (cdr pair) (cdr entry))) (t nil))))))) +(defun org-babel-generate-file-param (src-name params) + "Calculate the filename for source block results. + +The directory is calculated from the :output-dir property of the +source block; if not specified, use the current directory. + +If the source block has a #+NAME and the :file parameter does not +contain any period characters, then the :file parameter is +treated as an extension, and the output file name is the +concatenation of the directory (as calculated above), the block +name, a period, and the parameter value as a file extension. +Otherwise, the :file parameter is treated as a full file name, +and the output file name is the directory (as calculated above) +plus the parameter value." + (let* ((file-cons (assq :file params)) + (file-ext-cons (assq :file-ext params)) + (file-ext (cdr-safe file-ext-cons)) + (dir (or (cdr-safe (assq :output-dir params)) ""))) + ;; Only operate if: + ;; 1. :file is not given + ;; 2. :file-ext is given + ;; 3. the source block has a #+name + (if (and (not file-cons) file-ext src-name) + (progn + ;; Create the :output-dir if it does not exist + (when (and file-ext (not (string= dir ""))) + (make-directory dir t) + (unless (string-match "/\\'" dir) + (setq dir (concat dir "/")))) + (cons (cons :file (concat dir src-name "." file-ext)) params)) + ;; Otherwise return params unmodified + params))) ;;; Used by backends: R, Maxima, Octave. (defun org-babel-graphical-output-file (params) diff --git a/testing/examples/babel.org b/testing/examples/babel.org index 449824f..68ba65a 100644 --- a/testing/examples/babel.org +++ b/testing/examples/babel.org @@ -458,3 +458,37 @@ function definition "0")))) (format "elisp a:%d, b:%d, c:%d, d:%d, e:%d" a b c d e) #+END_SRC + +* =:file-ext= and =:output-dir= header args + :PROPERTIES: + :ID: 93573e1d-6486-442e-b6d0-3fedbdc37c9b + :END: +#+name: file-ext-basic +#+BEGIN_SRC emacs-lisp :file-ext txt +nil +#+END_SRC + +#+name: file-ext-dir-relative +#+BEGIN_SRC emacs-lisp :file-ext txt :output-dir foo +nil +#+END_SRC + +#+name: file-ext-dir-relative-slash +#+BEGIN_SRC emacs-lisp :file-ext txt :output-dir foo/ +nil +#+END_SRC + +#+name: file-ext-dir-absolute +#+BEGIN_SRC emacs-lisp :file-ext txt :output-dir /tmp +nil +#+END_SRC + +#+name: file-ext-file-wins +#+BEGIN_SRC emacs-lisp :file-ext txt :file foo.bar +nil +#+END_SRC + +#+name: file-ext-file-wins2 +#+BEGIN_SRC emacs-lisp :output-dir xxx :file foo.bar +nil +#+END_SRC diff --git a/testing/lisp/test-ob.el b/testing/lisp/test-ob.el index c0ca493..fb791d6 100644 --- a/testing/lisp/test-ob.el +++ b/testing/lisp/test-ob.el @@ -1237,6 +1237,20 @@ echo \"$data\" (org-babel-execute-src-block))) (should (= noweb-expansions-in-cache-var 2))))) +(ert-deftest test-org-babel/file-ext-and-output-dir () + (org-test-at-id "93573e1d-6486-442e-b6d0-3fedbdc37c9b" + (macrolet ((at-next (&rest body) + `(progn + (org-babel-next-src-block) + (save-match-data ,@body)))) + (at-next (should (equal "file-ext-basic.txt" (cdr (assq :file (nth 2 (org-babel-get-src-block-info t))))))) + (at-next (should (equal "foo/file-ext-dir-relative.txt" (cdr (assq :file (nth 2 (org-babel-get-src-block-info t))))))) + (at-next (should (equal "foo/file-ext-dir-relative-slash.txt" (cdr (assq :file (nth 2 (org-babel-get-src-block-info t))))))) + (at-next (should (equal "/tmp/file-ext-dir-absolute.txt" (cdr (assq :file (nth 2 (org-babel-get-src-block-info t))))))) + (at-next (should (equal "foo.bar" (cdr (assq :file (nth 2 (org-babel-get-src-block-info t))))))) + (at-next (should (equal "foo.bar" (cdr (assq :file (nth 2 (org-babel-get-src-block-info t))))))) + ))) + (provide 'test-ob) ;;; test-ob ends here -- 1.9.2 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. 2014-04-28 2:18 ` Aaron Ecay @ 2014-04-28 6:20 ` Achim Gratz 2014-04-29 13:25 ` Bastien 2014-05-04 13:55 ` Eric Schulte 2 siblings, 0 replies; 17+ messages in thread From: Achim Gratz @ 2014-04-28 6:20 UTC (permalink / raw) To: emacs-orgmode Aaron Ecay writes: > Thanks so much for the feedback. I’ve adopted the :file-ext approach > suggested by Bastien, leaving the previous default behavior in place for > blocks with a :file argument. THanks for taking care, I'll not have time to look at this for the remainder of the week, though… Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Waldorf MIDI Implementation & additional documentation: http://Synth.Stromeko.net/Downloads.html#WaldorfDocs ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. 2014-04-28 2:18 ` Aaron Ecay 2014-04-28 6:20 ` Achim Gratz @ 2014-04-29 13:25 ` Bastien 2014-05-04 13:55 ` Eric Schulte 2 siblings, 0 replies; 17+ messages in thread From: Bastien @ 2014-04-29 13:25 UTC (permalink / raw) To: Aaron Ecay; +Cc: emacs-orgmode, Eric Schulte Hi Aaron, Aaron Ecay <aaronecay@gmail.com> writes: > Thanks so much for the feedback. I’ve adopted the :file-ext approach > suggested by Bastien, leaving the previous default behavior in place for > blocks with a :file argument. Looks good -- please apply in master. Thanks! PS: I assume you have commit and push access, otherwise send me your public key and I'll give it to you. -- Bastien ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. 2014-04-28 2:18 ` Aaron Ecay 2014-04-28 6:20 ` Achim Gratz 2014-04-29 13:25 ` Bastien @ 2014-05-04 13:55 ` Eric Schulte 2014-05-11 20:38 ` Aaron Ecay 2 siblings, 1 reply; 17+ messages in thread From: Eric Schulte @ 2014-05-04 13:55 UTC (permalink / raw) To: Aaron Ecay; +Cc: emacs-orgmode Hi Aaron, Thanks for this patch, and especially for including documentation. It looks good to me, and it (largely [1]) passes all tests. Aaron Ecay <aaronecay@gmail.com> writes: > Hi Eric, Bastien, Achim, > > Thanks so much for the feedback. I’ve adopted the :file-ext approach > suggested by Bastien, leaving the previous default behavior in place for > blocks with a :file argument. > > 2014ko apirilak 22an, Eric Schulte-ek idatzi zuen: > > [...] > > >> One option might be to borrow naming behavior from the comment >> functionality in ob-tangle which looks like the following (from line 426 >> in ob-tangle.el). >> >> (let (... >> (source-name >> (intern (or (nth 4 info) ; explicit #+name: >> (format "%s:%d" ; constructed from header and position >> (or (ignore-errors (nth 4 (org-heading-components))) >> "No heading") >> block-counter)))) >> ...)) > > I’m not sure I like this approach. It relies on counting source > blocks, so an addition/deletion of a block could change the index. > I’m worried that this can lead to the accumulation of many output > files: heading:1.ext, heading:2.ext, ... all with no clear indication > of what block they were spawned by. It would also be possible for > the result links in the buffer to become inconsistent with the actual > block:auto-generated name mapping. > > I think I would prefer the code in this patch to do nothing in this case > (not create a :file value), but for language-specific code that needs a > :file to raise an error to prompt the user to add a name. > Fair enough, especially given that this default will be applied to *all* code blocks, this seems like a reasonable approach. > >> >>> >>> 2. should :output-dir apply to the :file case as well? >>> >> >> If you mean "should :output-dir be used as the base when :file is a >> relative pathname" then I'd say "yes", and I think if this isn't the >> current behavior then the current behavior should be changed. > > Achim raises a backwards compatibility concern. I am not sure how > serious it is: the default settings (no :output-dir) are backwards > compatible, and if users set that arg we ought to just give them what > they ask for. > > Nonetheless, the new version of the patch conservatively obeys Achim’s > suggestion. I can change this to your suggestion, if that is the > consensus. > Please do make this change, I'd then be happy to apply the resulting patch. Thanks again! Eric > > To address a comment from Bastien: :output-dir accepts absolute as well > as relative directory names. Referring to a “subdirectory” was a > mistake on my part; the docs in the new patch should be clearer. > > The updated patch (now with docs and tests) is attached to this email. > > Thanks again, > > -- > Aaron Ecay > > From 4b428820432752117c60b79da0a79fd4e50e4ba1 Mon Sep 17 00:00:00 2001 > From: Aaron Ecay <aaronecay@gmail.com> > Date: Tue, 22 Apr 2014 15:13:48 -0400 > Subject: [PATCH] ob-core.el: allow the auto-generation of output file names > for src blocks. > > * lisp/ob-core.el (org-babel-generate-file-param): New function. > (org-babel-get-src-block-info): Use it. > * testing/lisp/test-ob.el (test-org-babel/file-ext-and-output-dir): > New test. > * doc/org.texi (Specific header arguments): Add doc for :file-ext and > :output-dir header args. > --- > doc/org.texi | 27 +++++++++++++++++++++++++++ > lisp/ob-core.el | 34 ++++++++++++++++++++++++++++++++++ > testing/examples/babel.org | 34 ++++++++++++++++++++++++++++++++++ > testing/lisp/test-ob.el | 14 ++++++++++++++ > 4 files changed, 109 insertions(+) > > diff --git a/doc/org.texi b/doc/org.texi > index 2546be1..79cc044 100644 > --- a/doc/org.texi > +++ b/doc/org.texi > @@ -14406,6 +14406,8 @@ argument in lowercase letters. The following header arguments are defined: > be collected and handled > * file:: Specify a path for file output > * file-desc:: Specify a description for file results > +* file-ext:: Specify an extension for file output > +* output-dir:: Specify a directory to write file output to > * dir:: Specify the default (possibly remote) > directory for code block execution > * exports:: Export code and/or results > @@ -14840,6 +14842,31 @@ description for file code block results which are inserted as Org mode links > with no value the link path will be placed in both the ``link'' and the > ``description'' portion of the Org mode link. > > +@node file-ext > +@subsubsection @code{:file-ext} > +@cindex @code{:file-ext}, src header argument > + > +The value of the @code{:file-ext} header argument is used to provide an > +extension to write the file output to. It is combined with the > +@code{#+NAME:} of the source block and the value of the @ref{output-dir} > +header argument to generate a complete file name. > + > +This header arg will be overridden by @code{:file}, and thus has no effect > +when the latter is specified. > + > +@node output-dir > +@subsubsection @code{:output-dir} > +@cindex @code{:output-dir}, src header argument > + > +The value of the @code{:output-dir} header argument is used to provide a > +directory to write the file output to. It may specify an absolute directory > +(beginning with @code{/}) or a relative directory (without @code{/}). It is > +combined with the @code{#+NAME:} of the source block and the value of the > +@ref{output-dir} header argument to generate a complete file name. > + > +This header arg will be overridden by @code{:file}, and thus has no effect > +when the latter is specified. > + > @node dir > @subsubsection @code{:dir} and remote execution > @cindex @code{:dir}, src header argument > diff --git a/lisp/ob-core.el b/lisp/ob-core.el > index 1348f04..4a3683d 100644 > --- a/lisp/ob-core.el > +++ b/lisp/ob-core.el > @@ -283,6 +283,8 @@ Returns a list > ;; resolve variable references and add summary parameters > (when (and info (not light)) > (setf (nth 2 info) (org-babel-process-params (nth 2 info)))) > + (when info > + (setf (nth 2 info) (org-babel-generate-file-param name (nth 2 info)))) > (when info (append info (list name indent head))))) > > (defvar org-babel-exp-reference-buffer nil > @@ -2890,6 +2892,38 @@ For the format of SAFE-LIST, see `org-babel-safe-header-args'." > (member (cdr pair) (cdr entry))) > (t nil))))))) > > +(defun org-babel-generate-file-param (src-name params) > + "Calculate the filename for source block results. > + > +The directory is calculated from the :output-dir property of the > +source block; if not specified, use the current directory. > + > +If the source block has a #+NAME and the :file parameter does not > +contain any period characters, then the :file parameter is > +treated as an extension, and the output file name is the > +concatenation of the directory (as calculated above), the block > +name, a period, and the parameter value as a file extension. > +Otherwise, the :file parameter is treated as a full file name, > +and the output file name is the directory (as calculated above) > +plus the parameter value." > + (let* ((file-cons (assq :file params)) > + (file-ext-cons (assq :file-ext params)) > + (file-ext (cdr-safe file-ext-cons)) > + (dir (or (cdr-safe (assq :output-dir params)) ""))) > + ;; Only operate if: > + ;; 1. :file is not given > + ;; 2. :file-ext is given > + ;; 3. the source block has a #+name > + (if (and (not file-cons) file-ext src-name) > + (progn > + ;; Create the :output-dir if it does not exist > + (when (and file-ext (not (string= dir ""))) > + (make-directory dir t) > + (unless (string-match "/\\'" dir) > + (setq dir (concat dir "/")))) > + (cons (cons :file (concat dir src-name "." file-ext)) params)) > + ;; Otherwise return params unmodified > + params))) > > ;;; Used by backends: R, Maxima, Octave. > (defun org-babel-graphical-output-file (params) > diff --git a/testing/examples/babel.org b/testing/examples/babel.org > index 449824f..68ba65a 100644 > --- a/testing/examples/babel.org > +++ b/testing/examples/babel.org > @@ -458,3 +458,37 @@ function definition > "0")))) > (format "elisp a:%d, b:%d, c:%d, d:%d, e:%d" a b c d e) > #+END_SRC > + > +* =:file-ext= and =:output-dir= header args > + :PROPERTIES: > + :ID: 93573e1d-6486-442e-b6d0-3fedbdc37c9b > + :END: > +#+name: file-ext-basic > +#+BEGIN_SRC emacs-lisp :file-ext txt > +nil > +#+END_SRC > + > +#+name: file-ext-dir-relative > +#+BEGIN_SRC emacs-lisp :file-ext txt :output-dir foo > +nil > +#+END_SRC > + > +#+name: file-ext-dir-relative-slash > +#+BEGIN_SRC emacs-lisp :file-ext txt :output-dir foo/ > +nil > +#+END_SRC > + > +#+name: file-ext-dir-absolute > +#+BEGIN_SRC emacs-lisp :file-ext txt :output-dir /tmp > +nil > +#+END_SRC > + > +#+name: file-ext-file-wins > +#+BEGIN_SRC emacs-lisp :file-ext txt :file foo.bar > +nil > +#+END_SRC > + > +#+name: file-ext-file-wins2 > +#+BEGIN_SRC emacs-lisp :output-dir xxx :file foo.bar > +nil > +#+END_SRC > diff --git a/testing/lisp/test-ob.el b/testing/lisp/test-ob.el > index c0ca493..fb791d6 100644 > --- a/testing/lisp/test-ob.el > +++ b/testing/lisp/test-ob.el > @@ -1237,6 +1237,20 @@ echo \"$data\" > (org-babel-execute-src-block))) > (should (= noweb-expansions-in-cache-var 2))))) > > +(ert-deftest test-org-babel/file-ext-and-output-dir () > + (org-test-at-id "93573e1d-6486-442e-b6d0-3fedbdc37c9b" > + (macrolet ((at-next (&rest body) > + `(progn > + (org-babel-next-src-block) > + (save-match-data ,@body)))) > + (at-next (should (equal "file-ext-basic.txt" (cdr (assq :file (nth 2 (org-babel-get-src-block-info t))))))) > + (at-next (should (equal "foo/file-ext-dir-relative.txt" (cdr (assq :file (nth 2 (org-babel-get-src-block-info t))))))) > + (at-next (should (equal "foo/file-ext-dir-relative-slash.txt" (cdr (assq :file (nth 2 (org-babel-get-src-block-info t))))))) > + (at-next (should (equal "/tmp/file-ext-dir-absolute.txt" (cdr (assq :file (nth 2 (org-babel-get-src-block-info t))))))) > + (at-next (should (equal "foo.bar" (cdr (assq :file (nth 2 (org-babel-get-src-block-info t))))))) > + (at-next (should (equal "foo.bar" (cdr (assq :file (nth 2 (org-babel-get-src-block-info t))))))) > + ))) > + > (provide 'test-ob) > > ;;; test-ob ends here Footnotes: [1] It took some extra work and manual refreshing of org-id's to get test-org-babel/file-ext-and-output-dir to pass. For some reason this tests still fails when I run "make test-dirty", but passes when run interactively. This could easily be particular to my system. -- Eric Schulte https://cs.unm.edu/~eschulte PGP: 0x614CA05D ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. 2014-05-04 13:55 ` Eric Schulte @ 2014-05-11 20:38 ` Aaron Ecay 2014-05-14 17:46 ` Achim Gratz 0 siblings, 1 reply; 17+ messages in thread From: Aaron Ecay @ 2014-05-11 20:38 UTC (permalink / raw) To: Eric Schulte; +Cc: emacs-orgmode Hi Eric, Bastien, Achim, et al., Thanks again for the feedback. I just pushed the revised patch to master. 2014ko maiatzak 4an, Eric Schulte-ek idatzi zuen: [...] >>> One option might be to borrow naming behavior from the comment >>> functionality in ob-tangle which looks like the following (from line 426 >>> in ob-tangle.el). >>> >>> (let (... >>> (source-name >>> (intern (or (nth 4 info) ; explicit #+name: >>> (format "%s:%d" ; constructed from header and position >>> (or (ignore-errors (nth 4 (org-heading-components))) >>> "No heading") >>> block-counter)))) >>> ...)) >> >> I’m not sure I like this approach. It relies on counting source >> blocks, so an addition/deletion of a block could change the index. >> I’m worried that this can lead to the accumulation of many output >> files: heading:1.ext, heading:2.ext, ... all with no clear indication >> of what block they were spawned by. It would also be possible for >> the result links in the buffer to become inconsistent with the actual >> block:auto-generated name mapping. >> >> I think I would prefer the code in this patch to do nothing in this case >> (not create a :file value), but for language-specific code that needs a >> :file to raise an error to prompt the user to add a name. >> > > Fair enough, especially given that this default will be applied to *all* > code blocks, this seems like a reasonable approach. I went ahead with my suggested approach here. [...] >> Achim raises a backwards compatibility concern. I am not sure how >> serious it is: the default settings (no :output-dir) are backwards >> compatible, and if users set that arg we ought to just give them what >> they ask for. >> >> Nonetheless, the new version of the patch conservatively obeys Achim’s >> suggestion. I can change this to your suggestion, if that is the >> consensus. >> > > Please do make this change, I'd then be happy to apply the resulting > patch. Done, as you and Bastien suggested. Thanks, -- Aaron Ecay ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. 2014-05-11 20:38 ` Aaron Ecay @ 2014-05-14 17:46 ` Achim Gratz 2014-05-15 10:05 ` Bastien 2014-05-16 3:28 ` Aaron Ecay 0 siblings, 2 replies; 17+ messages in thread From: Achim Gratz @ 2014-05-14 17:46 UTC (permalink / raw) To: emacs-orgmode Aaron Ecay writes: > Thanks again for the feedback. I just pushed the revised patch to master. > >>> I think I would prefer the code in this patch to do nothing in this case >>> (not create a :file value), but for language-specific code that needs a >>> :file to raise an error to prompt the user to add a name. >>> >> >> Fair enough, especially given that this default will be applied to *all* >> code blocks, this seems like a reasonable approach. > > I went ahead with my suggested approach here. That breaks the tests for Octave and Maxima; since you're intentionally not keeping backwards compatibility here this should be fixed in the tests, I'd think. Also, I'd think you should be using "user-error" instead of error to generate the messages. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. 2014-05-14 17:46 ` Achim Gratz @ 2014-05-15 10:05 ` Bastien 2014-05-16 3:28 ` Aaron Ecay 1 sibling, 0 replies; 17+ messages in thread From: Bastien @ 2014-05-15 10:05 UTC (permalink / raw) To: Achim Gratz; +Cc: emacs-orgmode Hi, Achim Gratz <Stromeko@nexgo.de> writes: > That breaks the tests for Octave and Maxima; since you're intentionally > not keeping backwards compatibility here this should be fixed in the > tests, I'd think. Agreed. Aaron, can you take care of this? > Also, I'd think you should be using "user-error" instead of error to > generate the messages. Agreed too. -- Bastien ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. 2014-05-14 17:46 ` Achim Gratz 2014-05-15 10:05 ` Bastien @ 2014-05-16 3:28 ` Aaron Ecay 2014-05-17 6:20 ` Achim Gratz 1 sibling, 1 reply; 17+ messages in thread From: Aaron Ecay @ 2014-05-16 3:28 UTC (permalink / raw) To: Achim Gratz, emacs-orgmode Hi Achim, 2014ko maiatzak 14an, Achim Gratz-ek idatzi zuen: > That breaks the tests for Octave and Maxima; since you're intentionally > not keeping backwards compatibility here this should be fixed in the > tests, I'd think. Fixed. (It actually required changes to the code, not the tests, since my commit made org-babel-graphical-output-file stricter). > > Also, I'd think you should be using "user-error" instead of error to > generate the messages. Good catch (especially since I recently pushed a patch changing some errors to user-errors *blush*). Fixed. Thanks, -- Aaron Ecay ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. 2014-05-16 3:28 ` Aaron Ecay @ 2014-05-17 6:20 ` Achim Gratz 0 siblings, 0 replies; 17+ messages in thread From: Achim Gratz @ 2014-05-17 6:20 UTC (permalink / raw) To: emacs-orgmode Aaron Ecay writes: > Fixed. (It actually required changes to the code, not the tests, since > my commit made org-babel-graphical-output-file stricter). OK. > Good catch (especially since I recently pushed a patch changing some > errors to user-errors *blush*). Fixed. Code review works. :-) Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Samples for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. 2014-04-23 0:04 ` Aaron Ecay 2014-04-23 1:35 ` Eric Schulte @ 2014-04-23 6:27 ` Bastien 2014-04-23 11:07 ` Eric Schulte 2014-04-23 19:44 ` Achim Gratz 2 siblings, 1 reply; 17+ messages in thread From: Bastien @ 2014-04-23 6:27 UTC (permalink / raw) To: Aaron Ecay; +Cc: emacs-orgmode Hi Aaron, Aaron Ecay <aaronecay@gmail.com> writes: > How does this sound as an algorithm: > 1. if :file is present, behave exactly as we do now > 2. if :file is absent but :file-ext and a #+name is present, generate a > :file parameter from :output-dir, the #+name, and :file-ext. I suggest this one: 1. if :file is present, behave exactly as we do now 2. if :file-ext is present: - if #+name is present, generate a :file parameter from :output-dir, #+name and :file-ext - otherwise, generate the :file parameter from :output-dir, the headline or the title or the current file name and :file-ext Just falling back on something sensible when :file is absent and :file-ext is specified. > Open questions: > 1. should :file-ext without a #+name be a no-op, or an error? See above. > 2. should :output-dir apply to the :file case as well? To me yes. In overall I think would be good, but I'd like Eric and other babelist around here to have a look before we commit this. So perhaps another round of patch testing will be good. Thanks! -- Bastien ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. 2014-04-23 6:27 ` Bastien @ 2014-04-23 11:07 ` Eric Schulte 0 siblings, 0 replies; 17+ messages in thread From: Eric Schulte @ 2014-04-23 11:07 UTC (permalink / raw) To: Bastien; +Cc: Aaron Ecay, emacs-orgmode Bastien <bzg@gnu.org> writes: > Hi Aaron, > > Aaron Ecay <aaronecay@gmail.com> writes: > >> How does this sound as an algorithm: >> 1. if :file is present, behave exactly as we do now >> 2. if :file is absent but :file-ext and a #+name is present, generate a >> :file parameter from :output-dir, the #+name, and :file-ext. > > I suggest this one: > > 1. if :file is present, behave exactly as we do now > > 2. if :file-ext is present: > > - if #+name is present, generate a :file parameter > from :output-dir, #+name and :file-ext > > - otherwise, generate the :file parameter from > :output-dir, the headline or the title or the > current file name and :file-ext > > Just falling back on something sensible when :file is absent > and :file-ext is specified. > FWIW I'm in full agreement here, see my other email for one sensible alternative. (Sorry to split up my reply, the hazards of an asynch/batch email pull/read/write/push setup.) > >> Open questions: >> 1. should :file-ext without a #+name be a no-op, or an error? > > See above. > >> 2. should :output-dir apply to the :file case as well? > > To me yes. > > In overall I think would be good, but I'd like Eric and other > babelist around here to have a look before we commit this. > So perhaps another round of patch testing will be good. > I agree here. Thanks again, > > Thanks! -- Eric Schulte https://cs.unm.edu/~eschulte PGP: 0x614CA05D ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks. 2014-04-23 0:04 ` Aaron Ecay 2014-04-23 1:35 ` Eric Schulte 2014-04-23 6:27 ` Bastien @ 2014-04-23 19:44 ` Achim Gratz 2 siblings, 0 replies; 17+ messages in thread From: Achim Gratz @ 2014-04-23 19:44 UTC (permalink / raw) To: emacs-orgmode Aaron Ecay writes: > How does this sound as an algorithm: > 1. if :file is present, behave exactly as we do now > 2. if :file is absent but :file-ext and a #+name is present, generate a > :file parameter from :output-dir, the #+name, and :file-ext. 3. If both :file and :file-ext are present, :file takes precedence and :file-ext is ignored. > 1. should :file-ext without a #+name be a no-op, or an error? Since you are hoping to inherit these via properties, this should be a nop if not applicable. > 2. should :output-dir apply to the :file case as well? I'd say no since it seems this would be backwards-incompatible. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf microQ V2.22R2: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2014-05-17 6:20 UTC | newest] Thread overview: 17+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-04-22 19:54 [RFC] [PATCH] ob-core.el: allow the auto-generation of output file names for src blocks Aaron Ecay 2014-04-22 21:22 ` Bastien 2014-04-23 0:04 ` Aaron Ecay 2014-04-23 1:35 ` Eric Schulte 2014-04-23 14:58 ` Bastien 2014-04-28 2:18 ` Aaron Ecay 2014-04-28 6:20 ` Achim Gratz 2014-04-29 13:25 ` Bastien 2014-05-04 13:55 ` Eric Schulte 2014-05-11 20:38 ` Aaron Ecay 2014-05-14 17:46 ` Achim Gratz 2014-05-15 10:05 ` Bastien 2014-05-16 3:28 ` Aaron Ecay 2014-05-17 6:20 ` Achim Gratz 2014-04-23 6:27 ` Bastien 2014-04-23 11:07 ` Eric Schulte 2014-04-23 19:44 ` Achim Gratz
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).