From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Eric Schulte" Subject: Re: Re: [babel] How to kill two birds with one stone? Date: Sun, 20 Feb 2011 01:57:00 -0700 Message-ID: <87sjvj6oul.fsf@gmail.com> References: <808vxv23j2.fsf@missioncriticalit.com> <80mxm9yulk.fsf@missioncriticalit.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from [140.186.70.92] (port=56696 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Pr5WV-0007kF-94 for emacs-orgmode@gnu.org; Sun, 20 Feb 2011 04:24:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Pr5WS-0004DD-Cr for emacs-orgmode@gnu.org; Sun, 20 Feb 2011 04:24:09 -0500 Received: from mail-iy0-f169.google.com ([209.85.210.169]:48612) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Pr5WS-0004BV-6u for emacs-orgmode@gnu.org; Sun, 20 Feb 2011 04:24:08 -0500 Received: by mail-iy0-f169.google.com with SMTP id 13so1720525iyf.0 for ; Sun, 20 Feb 2011 01:24:08 -0800 (PST) List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: =?utf-8?Q?S=C3=A9bastien?= Vauban Cc: emacs-orgmode@gnu.org Hi, I haven't followed this discussion very closely, but I'm not sure why it would be necessary to pass data through STDIN rather than through a variable or an external file. I took a shot at the dot graph example you proposed, the following works for me over a simple example directory. Best -- Eric directory to search #+results: graph-dir : graph-dir list all files in dir #+source: graph-files #+begin_src sh :results vector :var dir=3Dgraph-dir find $dir -type f -exec basename {} \; #+end_src #+results: graph-files | other | | dan | | eric | | seb | association of files with mentions #+source: graph-associations #+begin_src sh :var dir=3Dgraph-dir :var files=3Dgraph-files for i in $files; do for j in `grep -l -r $i $dir`;do echo $i, `basename $j` done done #+end_src #+results: graph-associations | other | eric | | other | seb | | dan | eric | | eric | seb | | seb | dan | graphing with dot #+source: to-dot #+begin_src sh :var associations=3Dgraph-associations :results scalar echo "$associations"|awk '{print $1, "->", $2}' #+end_src #+results: to-dot : other -> eric : other -> seb : dan -> eric : eric -> seb : seb -> dan #+begin_src dot :var data=3Dto-dot :file files.png digraph G{ $data } #+end_src #+results: [[file:files.png]] S=C3=A9bastien Vauban writes: > Hi Dan, > > Dan Davison wrote: >> Cool post. I hope someone has some good ideas in this thread. Some quick >> responses / questions below. >> >>> Note, in the latter code block, that I did not even tried to really cha= in >>> steps 2 and 3: I'm rewriting step 3, including step 2 inside it. >>> >>> *I certainly miss a smarter way* to achieve the above. >> >> I think a relevant point here is that Org doesn't yet have the ability to >> pass data on standard input to a code block. I.e. a :stdin header arg. I >> don't think it's that hard, someone would just need to rework >> `org-babel-eval' so that it puts the code into a temporary file, freeing= up >> stdin to be used for data > > I think you exactly spotted the problem. > >> (and therefore we would no longer be able to use >> shell-command-on-region but some other command (call-process I think?).) > > I just don't understand this last sentence, by lack of knowledge on Babel= 's > internals. > >> Then you'd be able to do something like >> >> #+srcname: search-links-and-generate-dot-arrow >> #+header: :stdin search-files-pointing-to-this-file >> #+begin_src sh :results output :var f=3D"charge_dim" >> while read f; do >> echo " $(basename $i) -> $f"; >> done >> #+end_src >> >> I'll be interested to see the solution to all this. > > I've tried to rewrite the example in a much cleaner way, showing what it > should like in the best of the worlds, with a working code at hand... > > BTW, I think the following could be of use, with maybe slight modificatio= ns, > even for projects like Worg: identifying all relationships between files, > showing files that aren't referenced, etc. > > #+TITLE: Graph file dependencies > #+DATE: 2011-02-06 > #+BABEL: :dir ~/src/Worg > > * Context > > We want to demonstrate how to document a script in a very neat way (IMHO), > that is: > > - By defining and explaining multiple small code blocks, using them later= in a > tangle file for constructing the "full code". > > - By showing the effect of every small code block, that is what it returns > when applied on test input data. > > The latter is the problem, as the code has to be able to take a results s= et as > if it would come from =3Dstdin=3D. > > * Code > > For the sake of clarity, a real-life example that graph dependencies betw= een > files (based on their /basename/). > > ** List all files > > Simple file command, ignoring =3D.svn=3D directories. > > #+srcname: file-tree > #+begin_src sh :results output > find . -not \( -name .svn -prune \) -type f -print | head -n 5 > #+end_src > > #+results: file-tree > #+begin_example > ./digraph.dot > ./full-code.sh > ./graph-circo.pdf > ./graph-dot.pdf > ./graph-fdp.pdf > #+end_example > > ** Search recursively for anything about a file > > Grep-search through files, ignoring =3D.svn=3D directories. > > #+srcname: search-files-pointing-to-this-file > #+begin_src sh :results output :var toname=3D"charge_dim" > find . -not \( -name .svn -prune \) -type f -print0 |\ > xargs -0 grep -i --files-with-matches "$toname" > #+end_src > > #+results: search-files-pointing-to-this-file > : ./graph-file-dependencies.txt > > ** Convert to DOT > > In real life, the following block of code must read its input from =3Dstd= in=3D. > > For /in situ execution/, I should be able to say that =3Dstdin=3D is equa= l to any > results set (here: =3Dsearch-files-pointing-to-this-file=3D). > > #+srcname: make-dot-arrow-for-files-pointing-to-this-file > #+begin_src sh :results output :var toname=3D"charge_dim" > while read -r fromname > do > echo " \"${fromname##*/}\" -> \"$toname\"" > done > #+end_src > > #+results: make-dot-arrow-for-files-pointing-to-this-file > > ** Full code: generate the DOT file > > #+begin_src sh :results output :file digraph.dot :noweb yes :tangle full-= code.sh > echo 'digraph G {' > echo ' node [shape=3Ddiamond,style=3Dfilled,color=3Dlightgrey]; ".cvsi= gnore";' > echo ' node [shape=3Dbox,color=3Dyellow];' > echo '' > > <> |\ > while read -r fname > do > toname=3D"${fname##*/}" # basename of fname > echo "# Files pointing to \"$toname\"..." > <> |\ > <> > echo "" > done > > echo '}' > #+end_src > > #+results: > [[file:digraph.dot]] > > ** Draw multiple graphs > > #+srcname: draw-graphs > #+begin_src sh :dir ~/Projects :var infile=3D"digraph.dot" :var outfilepr= efix=3D"digraph" > for cmd in dot neato twopi circo fdp sfdp; > do > echo $cmd... > time $cmd -Gcharset=3Dlatin1 -Tpdf -o$outfileprefix-$cmd.pdf $infile > echo "" > done > #+end_src > > Best regards, > Seb