From: "Eric Schulte" <schulte.eric@gmail.com>
To: "Sébastien Vauban" <wxhgmqzgwmuf@spammotel.com>
Cc: emacs-orgmode@gnu.org
Subject: Re: Re: [babel] How to kill two birds with one stone?
Date: Sun, 20 Feb 2011 01:57:00 -0700 [thread overview]
Message-ID: <87sjvj6oul.fsf@gmail.com> (raw)
In-Reply-To: 80mxm9yulk.fsf@missioncriticalit.com
Hi,
I haven't followed this discussion very closely, but I'm not sure why it
would be necessary to pass data through STDIN rather than through a
variable or an external file.
I took a shot at the dot graph example you proposed, the following works
for me over a simple example directory.
Best -- Eric
directory to search
#+results: graph-dir
: graph-dir
list all files in dir
#+source: graph-files
#+begin_src sh :results vector :var dir=graph-dir
find $dir -type f -exec basename {} \;
#+end_src
#+results: graph-files
| other |
| dan |
| eric |
| seb |
association of files with mentions
#+source: graph-associations
#+begin_src sh :var dir=graph-dir :var files=graph-files
for i in $files; do
for j in `grep -l -r $i $dir`;do
echo $i, `basename $j`
done
done
#+end_src
#+results: graph-associations
| other | eric |
| other | seb |
| dan | eric |
| eric | seb |
| seb | dan |
graphing with dot
#+source: to-dot
#+begin_src sh :var associations=graph-associations :results scalar
echo "$associations"|awk '{print $1, "->", $2}'
#+end_src
#+results: to-dot
: other -> eric
: other -> seb
: dan -> eric
: eric -> seb
: seb -> dan
#+begin_src dot :var data=to-dot :file files.png
digraph G{
$data
}
#+end_src
#+results:
[[file:files.png]]
Sébastien Vauban <wxhgmqzgwmuf@spammotel.com> writes:
> Hi Dan,
>
> Dan Davison wrote:
>> Cool post. I hope someone has some good ideas in this thread. Some quick
>> responses / questions below.
>>
>>> Note, in the latter code block, that I did not even tried to really chain
>>> steps 2 and 3: I'm rewriting step 3, including step 2 inside it.
>>>
>>> *I certainly miss a smarter way* to achieve the above.
>>
>> I think a relevant point here is that Org doesn't yet have the ability to
>> pass data on standard input to a code block. I.e. a :stdin header arg. I
>> don't think it's that hard, someone would just need to rework
>> `org-babel-eval' so that it puts the code into a temporary file, freeing up
>> stdin to be used for data
>
> I think you exactly spotted the problem.
>
>> (and therefore we would no longer be able to use
>> shell-command-on-region but some other command (call-process I think?).)
>
> I just don't understand this last sentence, by lack of knowledge on Babel's
> internals.
>
>> Then you'd be able to do something like
>>
>> #+srcname: search-links-and-generate-dot-arrow
>> #+header: :stdin search-files-pointing-to-this-file
>> #+begin_src sh :results output :var f="charge_dim"
>> while read f; do
>> echo " $(basename $i) -> $f";
>> done
>> #+end_src
>>
>> I'll be interested to see the solution to all this.
>
> I've tried to rewrite the example in a much cleaner way, showing what it
> should like in the best of the worlds, with a working code at hand...
>
> BTW, I think the following could be of use, with maybe slight modifications,
> even for projects like Worg: identifying all relationships between files,
> showing files that aren't referenced, etc.
>
> #+TITLE: Graph file dependencies
> #+DATE: 2011-02-06
> #+BABEL: :dir ~/src/Worg
>
> * Context
>
> We want to demonstrate how to document a script in a very neat way (IMHO),
> that is:
>
> - By defining and explaining multiple small code blocks, using them later in a
> tangle file for constructing the "full code".
>
> - By showing the effect of every small code block, that is what it returns
> when applied on test input data.
>
> The latter is the problem, as the code has to be able to take a results set as
> if it would come from =stdin=.
>
> * Code
>
> For the sake of clarity, a real-life example that graph dependencies between
> files (based on their /basename/).
>
> ** List all files
>
> Simple file command, ignoring =.svn= directories.
>
> #+srcname: file-tree
> #+begin_src sh :results output
> find . -not \( -name .svn -prune \) -type f -print | head -n 5
> #+end_src
>
> #+results: file-tree
> #+begin_example
> ./digraph.dot
> ./full-code.sh
> ./graph-circo.pdf
> ./graph-dot.pdf
> ./graph-fdp.pdf
> #+end_example
>
> ** Search recursively for anything about a file
>
> Grep-search through files, ignoring =.svn= directories.
>
> #+srcname: search-files-pointing-to-this-file
> #+begin_src sh :results output :var toname="charge_dim"
> find . -not \( -name .svn -prune \) -type f -print0 |\
> xargs -0 grep -i --files-with-matches "$toname"
> #+end_src
>
> #+results: search-files-pointing-to-this-file
> : ./graph-file-dependencies.txt
>
> ** Convert to DOT
>
> In real life, the following block of code must read its input from =stdin=.
>
> For /in situ execution/, I should be able to say that =stdin= is equal to any
> results set (here: =search-files-pointing-to-this-file=).
>
> #+srcname: make-dot-arrow-for-files-pointing-to-this-file
> #+begin_src sh :results output :var toname="charge_dim"
> while read -r fromname
> do
> echo " \"${fromname##*/}\" -> \"$toname\""
> done
> #+end_src
>
> #+results: make-dot-arrow-for-files-pointing-to-this-file
>
> ** Full code: generate the DOT file
>
> #+begin_src sh :results output :file digraph.dot :noweb yes :tangle full-code.sh
> echo 'digraph G {'
> echo ' node [shape=diamond,style=filled,color=lightgrey]; ".cvsignore";'
> echo ' node [shape=box,color=yellow];'
> echo ''
>
> <<file-tree>> |\
> while read -r fname
> do
> toname="${fname##*/}" # basename of fname
> echo "# Files pointing to \"$toname\"..."
> <<search-files-pointing-to-this-file>> |\
> <<make-dot-arrow-for-files-pointing-to-this-file>>
> echo ""
> done
>
> echo '}'
> #+end_src
>
> #+results:
> [[file:digraph.dot]]
>
> ** Draw multiple graphs
>
> #+srcname: draw-graphs
> #+begin_src sh :dir ~/Projects :var infile="digraph.dot" :var outfileprefix="digraph"
> for cmd in dot neato twopi circo fdp sfdp;
> do
> echo $cmd...
> time $cmd -Gcharset=latin1 -Tpdf -o$outfileprefix-$cmd.pdf $infile
> echo ""
> done
> #+end_src
>
> Best regards,
> Seb
next prev parent reply other threads:[~2011-02-20 9:24 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-02-04 16:00 [babel] How to kill two birds with one stone? Sébastien Vauban
2011-02-04 17:43 ` Dan Davison
2011-02-04 22:23 ` Sébastien Vauban
2011-02-06 16:51 ` Sébastien Vauban
2011-02-20 8:57 ` Eric Schulte [this message]
2011-02-25 14:27 ` Sébastien Vauban
2011-02-25 22:44 ` Nick Dokos
2011-02-25 22:55 ` Nick Dokos
2011-02-28 13:59 ` Sébastien Vauban
2011-02-26 0:24 ` Eric Schulte
2011-02-26 9:56 ` Closing #+results: with #+end declaration? Bastien
2011-02-27 20:00 ` Eric Schulte
2011-02-28 13:54 ` Sébastien Vauban
2011-03-03 11:11 ` Bastien
2011-02-28 15:16 ` [babel] How to kill two birds with one stone? Sébastien Vauban
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87sjvj6oul.fsf@gmail.com \
--to=schulte.eric@gmail.com \
--cc=emacs-orgmode@gnu.org \
--cc=wxhgmqzgwmuf@spammotel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).