Re: Re: [babel] How to kill two birds with one stone?

["Eric Schulte" <schulte.eric@gmail.com>]

Hi,

I haven't followed this discussion very closely, but I'm not sure why it
would be necessary to pass data through STDIN rather than through a
variable or an external file.

I took a shot at the dot graph example you proposed, the following works
for me over a simple example directory.

Best -- Eric

directory to search
#+results: graph-dir
: graph-dir

list all files in dir
#+source: graph-files
#+begin_src sh :results vector :var dir=graph-dir
  find $dir -type f -exec basename {} \;
#+end_src

#+results: graph-files
| other |
| dan   |
| eric  |
| seb   |

association of files with mentions
#+source: graph-associations
#+begin_src sh :var dir=graph-dir :var files=graph-files
  for i in $files; do
      for j in `grep -l -r $i $dir`;do
          echo $i, `basename $j`
      done
  done
#+end_src

#+results: graph-associations
| other | eric |
| other | seb  |
| dan   | eric |
| eric  | seb  |
| seb   | dan  |

graphing with dot
#+source: to-dot
#+begin_src sh :var associations=graph-associations :results scalar
  echo "$associations"|awk '{print $1, "->", $2}'
#+end_src

#+results: to-dot
: other -> eric
: other -> seb
: dan -> eric
: eric -> seb
: seb -> dan

#+begin_src dot :var data=to-dot :file files.png
  digraph G{
    $data
  }
#+end_src

#+results:
[[file:files.png]]

Sébastien Vauban <wxhgmqzgwmuf@spammotel.com> writes:

> Hi Dan,
>
> Dan Davison wrote:
>> Cool post. I hope someone has some good ideas in this thread. Some quick
>> responses / questions below.
>>
>>> Note, in the latter code block, that I did not even tried to really chain
>>> steps 2 and 3: I'm rewriting step 3, including step 2 inside it.
>>>
>>> *I certainly miss a smarter way* to achieve the above.
>>
>> I think a relevant point here is that Org doesn't yet have the ability to
>> pass data on standard input to a code block. I.e. a :stdin header arg. I
>> don't think it's that hard, someone would just need to rework
>> `org-babel-eval' so that it puts the code into a temporary file, freeing up
>> stdin to be used for data
>
> I think you exactly spotted the problem.
>
>> (and therefore we would no longer be able to use
>> shell-command-on-region but some other command (call-process I think?).)
>
> I just don't understand this last sentence, by lack of knowledge on Babel's
> internals.
>
>> Then you'd be able to do something like
>>
>> #+srcname: search-links-and-generate-dot-arrow
>> #+header: :stdin search-files-pointing-to-this-file
>> #+begin_src sh :results output :var f="charge_dim"
>> while read f; do
>>     echo "    $(basename $i) -> $f";
>> done
>> #+end_src
>>
>> I'll be interested to see the solution to all this.
>
> I've tried to rewrite the example in a much cleaner way, showing what it
> should like in the best of the worlds, with a working code at hand...
>
> BTW, I think the following could be of use, with maybe slight modifications,
> even for projects like Worg: identifying all relationships between files,
> showing files that aren't referenced, etc.
>
> #+TITLE:     Graph file dependencies
> #+DATE:      2011-02-06
> #+BABEL:    :dir ~/src/Worg
>
> * Context
>
> We want to demonstrate how to document a script in a very neat way (IMHO),
> that is:
>
> - By defining and explaining multiple small code blocks, using them later in a
>   tangle file for constructing the "full code".
>
> - By showing the effect of every small code block, that is what it returns
>   when applied on test input data.
>
> The latter is the problem, as the code has to be able to take a results set as
> if it would come from =stdin=.
>
> * Code
>
> For the sake of clarity, a real-life example that graph dependencies between
> files (based on their /basename/).
>
> ** List all files
>
> Simple file command, ignoring =.svn= directories.
>
> #+srcname: file-tree
> #+begin_src sh :results output
> find . -not \( -name .svn -prune \) -type f -print | head -n 5
> #+end_src
>
> #+results: file-tree
> #+begin_example
> ./digraph.dot
> ./full-code.sh
> ./graph-circo.pdf
> ./graph-dot.pdf
> ./graph-fdp.pdf
> #+end_example
>
> ** Search recursively for anything about a file
>
> Grep-search through files, ignoring =.svn= directories.
>
> #+srcname: search-files-pointing-to-this-file
> #+begin_src sh :results output :var toname="charge_dim"
> find . -not \( -name .svn -prune \) -type f -print0 |\
> xargs -0 grep -i --files-with-matches "$toname"
> #+end_src
>
> #+results: search-files-pointing-to-this-file
> : ./graph-file-dependencies.txt
>
> ** Convert to DOT
>
> In real life, the following block of code must read its input from =stdin=.
>
> For /in situ execution/, I should be able to say that =stdin= is equal to any
> results set (here: =search-files-pointing-to-this-file=).
>
> #+srcname: make-dot-arrow-for-files-pointing-to-this-file
> #+begin_src sh :results output :var toname="charge_dim"
> while read -r fromname
> do
>     echo "    \"${fromname##*/}\" -> \"$toname\""
> done
> #+end_src
>
> #+results: make-dot-arrow-for-files-pointing-to-this-file
>
> ** Full code: generate the DOT file
>
> #+begin_src sh :results output :file digraph.dot :noweb yes :tangle full-code.sh
> echo 'digraph G {'
> echo '    node [shape=diamond,style=filled,color=lightgrey]; ".cvsignore";'
> echo '    node [shape=box,color=yellow];'
> echo ''
>
> <<file-tree>> |\
> while read -r fname
> do
>     toname="${fname##*/}" # basename of fname
>     echo "# Files pointing to \"$toname\"..."
>     <<search-files-pointing-to-this-file>> |\
>     <<make-dot-arrow-for-files-pointing-to-this-file>>
>     echo ""
> done
>
> echo '}'
> #+end_src
>
> #+results:
> [[file:digraph.dot]]
>
> ** Draw multiple graphs
>
> #+srcname: draw-graphs
> #+begin_src sh :dir ~/Projects :var infile="digraph.dot" :var outfileprefix="digraph"
> for cmd in dot neato twopi circo fdp sfdp;
> do
>     echo $cmd...
>     time $cmd -Gcharset=latin1 -Tpdf -o$outfileprefix-$cmd.pdf $infile
>     echo ""
> done
> #+end_src
>
> Best regards,
>   Seb