From: Dan Davison <dandavison7@gmail.com>
To: "Sébastien Vauban"
<public-wxhgmqzgwmuf-geNee64TY+gS+FvcfC7Uqw@lo.gmane.org>
Cc: emacs org-mode mailing list <emacs-orgmode@gnu.org>
Subject: Re: [babel] How to kill two birds with one stone?
Date: Fri, 04 Feb 2011 17:43:06 +0000 [thread overview]
Message-ID: <m11v3npufp.fsf@gmail.com> (raw)
In-Reply-To: <808vxv23j2.fsf@missioncriticalit.com> ("Sébastien Vauban"'s message of "Fri, 04 Feb 2011 17:00:33 +0100")
Hi Seb,
Cool post. I hope someone has some good ideas in this thread. Some quick
responses / questions below.
> #+TITLE: Document a shell script as separate blocks
> #+DATE: 2011-02-04
> #+LANGUAGE: en_US
>
> * Abstract
>
> When writing shell scripts, I'd like to kill *two* birds with one Babel stone:
>
> - Be able to *execute the script in situ*, so that I get a copy of the results
> stored in (and "versioned" with) my documentation.
>
> - Be able to *chain the code blocks*, so that I tangle a "natural" script into
> a file for later execution.
>
> I guess I currently miss some points in order to reach that goal in a clean
> way. Could you give me advice on how to get a step further down the route?
>
> * Sample code
>
> For the sake of clarity, let's take a simple problem: I'd like to generate a
> DOT graph of links between all files from a directory tree.
So if file a.org contains
-----
jhasg
[[file:b.org]]
[[file:c.org]]
hbjgv
-----
then you want an arrow from node a to node b and from node a to node c, right?
>
> The procedure:
>
> 1. (Recursively) list all files inside the directory.
>
> 2. For each file, (recursively) search for its name referenced in all the
> files.
>
> 3. Generate a DOT representation of the link between files.
>
> ** List all files under current directory
>
> #+srcname: dw-file-tree
> #+begin_src sh :results output
> cd ~/Some-Project-Dir
> find . -type f -print | grep -v .svn | head -n 5
> #+end_src
>
> #+results: dw-file-tree
> #+begin_example
> ./.cvsignore
> ./charge_dim
> ./charge_fct
> ./compte
> ./controle_config
> #+end_example
>
> Here, I voluntary limit the number of results to the first 5 files, for the
> compactness of this example. This sample does not include any file from
> subdirectories, but it doesn't matter.
>
> ** Search recursively for anything about a file
>
> Search through all files (ignoring =.svn= directories) for any reference to
> filename given as parameter.
>
> #+srcname: search-files-pointing-to-this-file
> #+begin_src sh :results output :var f="charge_dim"
> cd ~/Some-Project-Dir
> find . -not \( -name .svn -prune \) -type f -print0 |\
> xargs -0 grep -i --files-with-matches "$f"
> #+end_src
>
> #+results: search-files-pointing-to-this-file
> #+begin_example
> ./.cvsignore
> ./charge_dim
> ./compte
> ./IFP/Chrg_dim
> ./IFP/Chrg_dim.avant_simple_recovery_mode_2008_03_12
> ./principal.env
> ./29Aalst/Chrg_dim
> ./29Aalst/Chrg_dim_interactif
> ./29Aalst/Publ_dim_interactif
> #+end_example
>
> HERE, I'M GIVING A FILENAME AS DEFAULT VALUE OF =f= IN ORDER TO SEE A SAMPLE
> RESULTS.
>
> ** Convert to a DOT representation
>
> For every file pointing to the file given in parameter, generate an "arrow"
> (edge) in DOT representation.
>
> #+srcname: dot-arrow-from-files-pointing-to-this-file
> #+begin_src sh :results output :var f="charge_dim" :var data=search-files-pointing-to-this-file
> for i in $(echo "$data"); do echo " $(basename $i) -> $f"; done
> #+end_src
>
> #+results: dot-arrow-from-files-pointing-to-this-file
> #+begin_example
> .cvsignore -> charge_dim
> charge_dim -> charge_dim
> compte -> charge_dim
> Chrg_dim -> charge_dim
> Chrg_dim.avant_simple_recovery_mode_2008_03_12 -> charge_dim
> principal.env -> charge_dim
> Chrg_dim -> charge_dim
> Chrg_dim_interactif -> charge_dim
> Publ_dim_interactif -> charge_dim
> #+end_example
>
> HERE, I'M WORKING GIVING ONCE AGAIN THE SAME DEFAULT VALUE FOR TESTING (AND
> DOCUMENTATION) PURPOSE.
>
> * Problem
>
> All of the above nicely answer my first goal ("Be able to execute the script
> in situ, so that I get a copy of the results stored in my documentation").
>
> It does not allow for the second one: *how to chain the calls together*?
>
> For example, just for chaining steps 2 and 3 (the easier to chain, I think),
> I'd like to be able to write something like this:
>
> #+srcname: search-links-and-generate-dot-arrow
> #+begin_src sh :results output :var f="charge_dim" :var data=search-files-pointing-to-this-file :noweb yes
> for i in (
> <<search-files-pointing-to-this-file>>);
> do echo " $(basename $i) -> $f"; done
> #+end_src
Don't forget about C-c C-v C-v `org-babel-expand-src-block' when
debugging this problem. I've started using that a lot.
My shell scripting may be too weak to help here. (Which is one point,
I'd be tempted to bail out to another language on this one) What exactly
are you doing there? Executing the code from the other block in a
subshell and using the stdout as the set to loop over? You're sure you
don't want parentheses <<search-files-pointing-to-this-file>>() ? Or $(
... ) / backticks ? How are you going to pass arguments to the
search-files-pointing src block?
>
> #+results: search-links-and-generate-dot-arrow
>
> But it yields an error:
>
> #+begin_src text :eval no
> sh: line 14: syntax error near unexpected token `('
> sh: line 14: `for i in ('
> #+end_src
>
> Note, in the latter code block, that I did not even tried to really chain
> steps 2 and 3: I'm rewriting step 3, including step 2 inside it.
>
> *I certainly miss a smarter way* to achieve the above.
I think a relevant point here is that Org doesn't yet have the ability
to pass data on standard input to a code block. I.e. a :stdin header
arg. I don't think it's that hard, someone would just need to rework
`org-babel-eval' so that it puts the code into a temporary file, freeing
up stdin to be used for data (and therefore we would no longer be able
to use shell-command-on-region but some other command (call-process I
think?).)
Then you'd be able to do something like
#+srcname: search-links-and-generate-dot-arrow
#+header: :stdin search-files-pointing-to-this-file
#+begin_src sh :results output :var f="charge_dim"
while read f; do
echo " $(basename $i) -> $f";
done
#+end_src
I'll be interested to see the solution to all this.
Dan
>
> Quite important as well, is that the tangled file seems natural to read (for
> someone that would not read it from the Org file). This means we would have to
> minimize the use of "echo" and "tee" commands, among others -- but that's a
> nice to have.
>
> Do you see my point?
>
> Do you have ideas on how to go that way?
>
> Best regards,
> Seb
next prev parent reply other threads:[~2011-02-04 17:43 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-02-04 16:00 [babel] How to kill two birds with one stone? Sébastien Vauban
2011-02-04 17:43 ` Dan Davison [this message]
2011-02-04 22:23 ` Sébastien Vauban
2011-02-06 16:51 ` Sébastien Vauban
2011-02-20 8:57 ` Eric Schulte
2011-02-25 14:27 ` Sébastien Vauban
2011-02-25 22:44 ` Nick Dokos
2011-02-25 22:55 ` Nick Dokos
2011-02-28 13:59 ` Sébastien Vauban
2011-02-26 0:24 ` Eric Schulte
2011-02-26 9:56 ` Closing #+results: with #+end declaration? Bastien
2011-02-27 20:00 ` Eric Schulte
2011-02-28 13:54 ` Sébastien Vauban
2011-03-03 11:11 ` Bastien
2011-02-28 15:16 ` [babel] How to kill two birds with one stone? Sébastien Vauban
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=m11v3npufp.fsf@gmail.com \
--to=dandavison7@gmail.com \
--cc=emacs-orgmode@gnu.org \
--cc=public-wxhgmqzgwmuf-geNee64TY+gS+FvcfC7Uqw@lo.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).