emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* example filter for code blocks?
@ 2013-09-26  0:03 John Kitchin
  2013-09-26  8:40 ` Daniele Pizzolli
                   ` (2 more replies)
  0 siblings, 3 replies; 21+ messages in thread
From: John Kitchin @ 2013-09-26  0:03 UTC (permalink / raw)
  To: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 1422 bytes --]

Hi everyone,

I have an idea for putting links in a pdf that would open python code
blocks in an editor. To do that, I need to modify what happens when an
org-file is published to latex.

Essentially I want it to do exactly what it already does in terms of
running pygments, and making nicely formatted and syntax highlighted code
blocks and output.

After that though, I want to tangle the code block to a file in a
directory, and insert a new link after the rendered code block. I would
prefer not to have to put :tangle headings in each code block because there
are many (e.g. hundreds) of them in course notes. It would be sufficient if
they were just sequentially numbered as dir/1.py, dir/2.py, etc... and it
is fine if these get overwritten on each export.

the link that would go after the code block in the latex export would be
something like:
\LaunchPython{dir/1.py}{Open code}

Then clicking on it would open dir/1.py in whatever editor your system is
configured for. \LaunchPython is a newcommand I have defined that works
already.

It seems like the new export engine should make this easy to do, but I am
not sure where to start. Could anyone point me to a starting place? Thanks!


John

-----------------------------------
John Kitchin
Associate Professor
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
http://kitchingroup.cheme.cmu.edu

[-- Attachment #2: Type: text/html, Size: 1795 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread
* Re: example filter for code blocks?
@ 2013-09-26 17:57 John Kitchin
  2013-09-26 19:43 ` Thomas S. Dye
                   ` (2 more replies)
  0 siblings, 3 replies; 21+ messages in thread
From: John Kitchin @ 2013-09-26 17:57 UTC (permalink / raw)
  To: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 3765 bytes --]

Hi everyone,
Thanks for  the tips in using export filters for code blocks. I thought I
would share my current solution. The goal was to export all the code blocks
in an org-file to files systematically named part1/script-%d.py where %d is
a number. I didnot want to tangle exactly, because I wanted to avoid naming
the code block tangle files.

Then, I wanted to insert a pdf link that would open the file, after the
syntax highlighted code.

I wanted this because it is not convenient to copy and paste the
syntax-highlighted code into an editor. I teach from the pdf that is
generated, and it would be convenient to just open the code, edit and rerun
to explore solutions.

So, here is the solution:

At the top of my orgfile, I have this definition which creates a pdf link.

#+LATEX_HEADER: \newcommand{\LaunchBinary}[2]{%
#+LATEX_HEADER:   % #1: layer name,
#+LATEX_HEADER:   % #2: link text
#+LATEX_HEADER:   \leavevmode%
#+LATEX_HEADER:   \pdfstartlink  attr{/C [0.9 0 0] /Border [0 0 2]} user {
#+LATEX_HEADER:     /Subtype /Link
#+LATEX_HEADER:     /A <<
#+LATEX_HEADER:       /F <<
#+LATEX_HEADER:          /DOS (#1)
#+LATEX_HEADER:       >>
#+LATEX_HEADER:       /S /Launch
#+LATEX_HEADER:     >>
#+LATEX_HEADER:   } #2%
#+LATEX_HEADER:   \pdfendlink%
#+LATEX_HEADER: }


Then, I use the code snippet below to export the file to latex. It is
stored in a noexport section at the end of the document. basically I set a
counter, and wrote a filter function for src blocks. the function captures
the lines between the first and last (first is \begin{minted}... and last
is \end{minted} in this case. I write those lines to a file named according
to the counter, and finally insert \LaunchBinary... into the string
returned by the filter. everything else in this let block is just
fine-tuning the latex packages, and export behavior.

(let (
      ;; these packages are loaded in the latex file
      (org-latex-default-packages-alist
       '(("utf8" "inputenc" nil)
     ("T1" "fontenc" nil)
     ("" "fixltx2e" nil)
     ("" "natbib" t)
     ("" "url" t)
     ("" "graphicx" t)
         ("" "textcomp" t)
         ("" "underscore" t)
     ("" "amsmath" t)
         ("version=3" "mhchem" t)
         ("tight,pdftex" "web" nil)
         ("" "exerquiz" nil)
         ("ImplMulti" "dljslib" nil)
        ))
      (async nil)
      (subtreep nil)
      (visible-only nil)
      (body-only nil))

(setq counter 0)

  (defun ox-mrkup-filter-src-block (text back-end info)
    (setq counter (+ counter 1))

    (let ((filename (format "part1-scripts/script-%d.py" counter)))
      (with-temp-buffer
        (insert (mapconcat 'identity (butlast (cdr (split-string text "\n"
t))) "\n"))
        (write-region (point-min) (point-max) filename))

      (format "%s

\\LaunchBinary{%s}{Open the python script (%s).}

" text filename filename)))

(let ((org-export-filter-src-block-functions '(ox-mrkup-filter-src-block)))
  (org-latex-export-to-latex async subtreep visible-only body-only
                 '(:with-author t
                                            :with-date t
                                            :with-title t
                                            :with-timestamps t
                                            :with-todo-keywords t
                                            :with-toc nil))))


After building the pdf with pdflatex, I get a link with a red box around it
that I can click on, and on my system it opens the python file in the
python editor I have configured to open the file!


Thanks again!

John

-----------------------------------
John Kitchin
Associate Professor
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
http://kitchingroup.cheme.cmu.edu

[-- Attachment #2: Type: text/html, Size: 4810 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread
* Re: example filter for code blocks?
@ 2013-09-27 12:59 John Kitchin
  2013-09-27 15:24 ` Alan Schmitt
  0 siblings, 1 reply; 21+ messages in thread
From: John Kitchin @ 2013-09-27 12:59 UTC (permalink / raw)
  To: Thomas S. Dye; +Cc: emacs-orgmode@gnu.org

[-- Attachment #1: Type: text/plain, Size: 6403 bytes --]

Actually, it turns out the publisher did put up the pdf with embedded
files, you just can't open them in a pdf in a browser. If you download the
file and open in a good pdf reader, you can open the embedded files. It
just made my day to figure that out!

This link:
http://pubs.acs.org/doi/suppl/10.1021/ie400582a/suppl_file/ie400582a_si_001.pdf

is a supporting information file created from org-mode, with embedded excel
sheets, and the org-file that was exported to the pdf! It contains all of
the data used to make the figures in the paper, along with the python
scripts we used to make the figures.  It is probably our most complete
supplemental file ever. And, this file is freely available (although the
manuscript requires a subscription to the journal).

We actually wrote the manuscript in org-mode too, and submitted the latex
source that was exported from the org-file. Now, if we can just get
publishers to let us embed the org-source of the manuscript in the
supporting information file so at least the content is available for free...


John

-----------------------------------
John Kitchin
Associate Professor
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
http://kitchingroup.cheme.cmu.edu



On Thu, Sep 26, 2013 at 4:58 PM, Thomas S. Dye <tsd@tsdye.com> wrote:

> Hi John,
>
> Terrific! There go my plans for the weekend ... :)
>
> Good luck with the heavy-handed journal processor.  It seems worth
> pursuing to me.
>
> All the best,
> Tom
>
> John Kitchin <jkitchin@andrew.cmu.edu> writes:
>
> > I think it is so interesting, we have actually tried a variation of it! I
> > set up an org-file that exported a supplementary information file, using
> > attachfile to embed data files and scripts in the pdf. Unfortunately, the
> > journal "processed" the pdf file, and stripped those files out ;( We
> > haven't tried to see if we can get our original supplemental pdf
> accepted.
> >
> >
> > John
> >
> > -----------------------------------
> > John Kitchin
> > Associate Professor
> > Doherty Hall A207F
> > Department of Chemical Engineering
> > Carnegie Mellon University
> > Pittsburgh, PA 15213
> > 412-268-7803
> > http://kitchingroup.cheme.cmu.edu
> >
> >
> >
> > On Thu, Sep 26, 2013 at 3:43 PM, Thomas S. Dye <tsd@tsdye.com> wrote:
> >
> >> Hi John,
> >>
> >> John Kitchin <jkitchin@andrew.cmu.edu> writes:
> >>
> >> > Hi everyone,
> >> > Thanks for  the tips in using export filters for code blocks. I
> thought I
> >> > would share my current solution. The goal was to export all the code
> >> blocks
> >> > in an org-file to files systematically named part1/script-%d.py where
> %d
> >> is
> >> > a number. I didnot want to tangle exactly, because I wanted to avoid
> >> naming
> >> > the code block tangle files.
> >> >
> >> > Then, I wanted to insert a pdf link that would open the file, after
> the
> >> > syntax highlighted code.
> >> >
> >> > I wanted this because it is not convenient to copy and paste the
> >> > syntax-highlighted code into an editor. I teach from the pdf that is
> >> > generated, and it would be convenient to just open the code, edit and
> >> rerun
> >> > to explore solutions.
> >>
> >> This seems like it might be an elegant way to distribute a piece of
> >> reproducible research. I suspect most readers would prefer to have a pdf
> >> entry point into a compendium over an Org-mode entry point. Instead of
> >> distributing the Org-mode file that is configured to make a pdf file,
> >> carry out calculations, draw figures, etc., one could distribute a
> >> ready-made pdf file with an appendix of Supplementary Material that has
> >> all the code for calculations, figures, etc.
> >>
> >> What do you think?
> >>
> >> All the best,
> >> Tom
> >>
> >> --
> >> Thomas S. Dye
> >> http://www.tsdye.com
> >>
> > I think it is so interesting, we have actually tried a variation of
> > it! I set up an org-file that exported a supplementary information
> > file, using attachfile to embed data files and scripts in the pdf.
> > Unfortunately, the journal "processed" the pdf file, and stripped
> > those files out ;( We haven't tried to see if we can get our original
> > supplemental pdf accepted.
> >
> > John
> >
> > -----------------------------------
> > John Kitchin
> > Associate Professor
> > Doherty Hall A207F
> > Department of Chemical Engineering
> > Carnegie Mellon University
> > Pittsburgh, PA 15213
> > 412-268-7803
> > http://kitchingroup.cheme.cmu.edu
> >
> > On Thu, Sep 26, 2013 at 3:43 PM, Thomas S. Dye <tsd@tsdye.com> wrote:
> >
> >     Hi John,
> >
> >
> >     John Kitchin <jkitchin@andrew.cmu.edu> writes:
> >
> >     > Hi everyone,
> >     > Thanks for the tips in using export filters for code blocks. I
> >     thought I
> >     > would share my current solution. The goal was to export all the
> >     code blocks
> >     > in an org-file to files systematically named part1/script-%d.py
> >     where %d is
> >     > a number. I didnot want to tangle exactly, because I wanted to
> >     avoid naming
> >     > the code block tangle files.
> >     >
> >     > Then, I wanted to insert a pdf link that would open the file,
> >     after the
> >     > syntax highlighted code.
> >     >
> >     > I wanted this because it is not convenient to copy and paste the
> >     > syntax-highlighted code into an editor. I teach from the pdf
> >     that is
> >     > generated, and it would be convenient to just open the code,
> >     edit and rerun
> >     > to explore solutions.
> >
> >
> >     This seems like it might be an elegant way to distribute a piece
> >     of
> >     reproducible research. I suspect most readers would prefer to have
> >     a pdf
> >     entry point into a compendium over an Org-mode entry point.
> >     Instead of
> >     distributing the Org-mode file that is configured to make a pdf
> >     file,
> >     carry out calculations, draw figures, etc., one could distribute a
> >     ready-made pdf file with an appendix of Supplementary Material
> >     that has
> >     all the code for calculations, figures, etc.
> >
> >     What do you think?
> >
> >     All the best,
> >     Tom
> >
> >     --
> >     Thomas S. Dye
> >     http://www.tsdye.com
> >
> >
>
> --
> T.S. Dye & Colleagues, Archaeologists
> 735 Bishop St, Suite 315, Honolulu, HI 96813
> Tel: 808-529-0866, Fax: 808-529-0884
> http://www.tsdye.com
>

[-- Attachment #2: Type: text/html, Size: 8946 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread
* Re: example filter for code blocks?
@ 2013-09-27 18:44 John Kitchin
  0 siblings, 0 replies; 21+ messages in thread
From: John Kitchin @ 2013-09-27 18:44 UTC (permalink / raw)
  To: emacs-orgmode@gnu.org

[-- Attachment #1: Type: text/plain, Size: 2177 bytes --]

That is a pretty interesting pdf. I cannot think of a way to do this
simply. I think that filters get an info argument, which contains the parse
tree. Maybe you could use that to build up a table of which blocks refer to
each other somehow. I guess you would need some syntax in the src-block to
indicate which blocks were referred to. then, use filters to modify the
output of the rendered src-block by inserting the links in.

From: "Sebastien Vauban" <sva-news@mygooglest.com>
To: emacs-orgmode@gnu.org
Cc:
Date: Fri, 27 Sep 2013 09:42:39 +0200
Subject: Re: [O] example filter for code blocks?
Hello,

John Kitchin wrote:
> Thanks for  the tips in using export filters for code blocks. I thought I
> would share my current solution. The goal was to export all the code
blocks
> in an org-file to files systematically named part1/script-%d.py where %d
is
> a number. I didnot want to tangle exactly, because I wanted to avoid
naming
> the code block tangle files.
>
> Then, I wanted to insert a pdf link that would open the file, after the
> syntax highlighted code.

Thanks for sharing your wonderful solution!

Related to such things, something I'd like to have for long (but never found
time investigating it more) is that the PDF produced by Org would resemble
more
the PDF produced by Noweb.

In the attached example file (sent on this ML in... 2009), you see that the
Noweb extension to LaTeX

- generated a code block number: page number + order (if there are multiple
  code blocks on the same page);

- for each code block, attached a reference to all code blocks using it.

In the example, you see that code block "sql-init.sql" (defined on page 2)
gets
the number 2a (in that export [1]), and is used in code blocks 2c and 3. And
all those references are hyperlinks.

I found that very handy for reading your LP (or RR) document.

Best regards,
  Seb

[1] It could change if text or code is added in the documentation.

--
Sebastien Vauban
John

-----------------------------------
John Kitchin
Associate Professor
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
http://kitchingroup.cheme.cmu.edu

[-- Attachment #2: Type: text/html, Size: 2685 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2013-09-30 17:49 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-09-26  0:03 example filter for code blocks? John Kitchin
2013-09-26  8:40 ` Daniele Pizzolli
2013-09-26  9:04 ` Marcin Borkowski
2013-09-26 12:18 ` Eric Schulte
2013-09-26 18:31   ` Cook, Malcolm
2013-09-26 19:34     ` John Kitchin
  -- strict thread matches above, loose matches on Subject: below --
2013-09-26 17:57 John Kitchin
2013-09-26 19:43 ` Thomas S. Dye
2013-09-26 19:48   ` John Kitchin
2013-09-26 20:58     ` Thomas S. Dye
2013-09-27  7:42 ` Sebastien Vauban
2013-09-30  0:33 ` Thomas S. Dye
2013-09-30  0:45   ` John Kitchin
2013-09-30  2:58     ` Thomas S. Dye
2013-09-30  7:34     ` Achim Gratz
2013-09-30 17:48       ` Thomas S. Dye
2013-09-27 12:59 John Kitchin
2013-09-27 15:24 ` Alan Schmitt
2013-09-27 15:34   ` John Kitchin
2013-09-30  7:49     ` Alan Schmitt
2013-09-27 18:44 John Kitchin

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).