Org-mode Code Blocks Manuscript: Request For Comments

Org-mode Code Blocks Manuscript: Request For Comments

[Eric Schulte]

Hi,

Dan Davison, Tom Dye, Carsten Dominik and myself have been working on a
paper introducing Org-mode's code block functionality.  We plan to
submit this paper to the Journal of Statistical Software.  As both
Org-mode and the code block functionality are largely products of this
mailing list community, and in the spirit of an open peer review process
we are releasing the current draft of the paper here to solicit your
review and comments.

Both the .org and .pdf formats of the paper are available at the
following locations.

http://cs.unm.edu/~eschulte/org-paper/babel.org

http://cs.unm.edu/~eschulte/org-paper/babel.pdf

Thanks -- Eric

Re: Org-mode Code Blocks Manuscript: Request For Comments

[Jeff Horn]

This made my afternoon. I'm sure I'll be spending the next hour deep
in the paper. Thanks for sharing!

Jeff

On Thu, Dec 2, 2010 at 2:28 PM, Eric Schulte <schulte.eric@gmail.com> wrote:
> Hi,
>
> Dan Davison, Tom Dye, Carsten Dominik and myself have been working on a
> paper introducing Org-mode's code block functionality.  We plan to
> submit this paper to the Journal of Statistical Software.  As both
> Org-mode and the code block functionality are largely products of this
> mailing list community, and in the spirit of an open peer review process
> we are releasing the current draft of the paper here to solicit your
> review and comments.
>
> Both the .org and .pdf formats of the paper are available at the
> following locations.
>
> http://cs.unm.edu/~eschulte/org-paper/babel.org
>
> http://cs.unm.edu/~eschulte/org-paper/babel.pdf
>
> Thanks -- Eric
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>



-- 
Jeffrey Horn
Graduate Lecturer and PhD Student in Economics
George Mason University

(704) 271-4797
jhorn@gmu.edu
jrhorn424@gmail.com

http://www.failuretorefrain.com/jeff/

Re: Org-mode Code Blocks Manuscript: Request For Comments

[Eric S Fraga]

"Eric Schulte" <schulte.eric@gmail.com> writes:

> Hi,
>
> Dan Davison, Tom Dye, Carsten Dominik and myself have been working on a
> paper introducing Org-mode's code block functionality.  We plan to
> submit this paper to the Journal of Statistical Software.  As both
> Org-mode and the code block functionality are largely products of this
> mailing list community, and in the spirit of an open peer review process
> we are releasing the current draft of the paper here to solicit your
> review and comments.
>
> Both the .org and .pdf formats of the paper are available at the
> following locations.
>
> http://cs.unm.edu/~eschulte/org-paper/babel.org
>
> http://cs.unm.edu/~eschulte/org-paper/babel.pdf
>
> Thanks -- Eric

Thanks for giving us early access to this.  Very nice article.  I
particularly like the range of examples you have included.  They are
comprehensive (both the languages illustrated and the types of problems)
and each is nicely self-contained.  I am a little surprised at the
choice of journal, mind you...  but best of luck in your submission!
Send us the citation information when it comes out as I'm sure I'll have
plenty of occasions to cite it (e.g. the paper I am currently writing
relies strongly on org for the generation of results from octave code
embedded in the document...).

eric
-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 23.2.1
: using Org-mode version 7.3 (release_7.3.163.g76877)

Re: Org-mode Code Blocks Manuscript: Request For Comments

[Thomas S. Dye]

Hi Eric,

On Dec 2, 2010, at 1:13 PM, Eric S Fraga wrote:

> "Eric Schulte" <schulte.eric@gmail.com> writes:
>
>> Hi,
>>
>> Dan Davison, Tom Dye, Carsten Dominik and myself have been working  
>> on a
>> paper introducing Org-mode's code block functionality.  We plan to
>> submit this paper to the Journal of Statistical Software.  As both
>> Org-mode and the code block functionality are largely products of  
>> this
>> mailing list community, and in the spirit of an open peer review  
>> process
>> we are releasing the current draft of the paper here to solicit your
>> review and comments.
>>
>> Both the .org and .pdf formats of the paper are available at the
>> following locations.
>>
>> http://cs.unm.edu/~eschulte/org-paper/babel.org
>>
>> http://cs.unm.edu/~eschulte/org-paper/babel.pdf
>>
>> Thanks -- Eric
>
> Thanks for giving us early access to this.  Very nice article.  I
> particularly like the range of examples you have included.  They are
> comprehensive (both the languages illustrated and the types of  
> problems)
> and each is nicely self-contained.  I am a little surprised at the
> choice of journal, mind you...  but best of luck in your submission!
> Send us the citation information when it comes out as I'm sure I'll  
> have
> plenty of occasions to cite it (e.g. the paper I am currently writing
> relies strongly on org for the generation of results from octave code
> embedded in the document...).
>
> eric
> -- 

Would you recommend publishing someplace else?  We hoped list members  
would have opinions on this and welcome comments and suggestions.

All the best,
Tom

Re: Org-mode Code Blocks Manuscript: Request For Comments

[Eric S Fraga]

"Thomas S. Dye" <tsd@tsdye.com> writes:

> Hi Eric,
>
> On Dec 2, 2010, at 1:13 PM, Eric S Fraga wrote:

[...]

>> and each is nicely self-contained.  I am a little surprised at the
>> choice of journal, mind you...  
[...]


>
> Would you recommend publishing someplace else?  We hoped list members
> would have opinions on this and welcome comments and suggestions.
>
> All the best,
> Tom

This is a little outside my field, even though I'm a computer scientist.
I work at the interface between computing and industrial engineering...

I was surprised at your choice simply because there appears to be
nothing relevant to statistics about what you have written.  A more
broad computer science journal may make sense, such as "Programming and
Computer Software" from Springer or maybe the "IEEE Transactions on
Software Engineering?"  But I really don't know which are the best
journals for something like this.

Hopefully somebody else from the list will have a better idea!

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 23.2.1
: using Org-mode version 7.3 (release_7.3.164.gcfd7)

Re: Org-mode Code Blocks Manuscript: Request For Comments

[Thomas S. Dye]

[-- Attachment #1.1: Type: text/plain, Size: 1928 bytes --]

Hi Eric,

On Dec 3, 2010, at 2:26 AM, Eric S Fraga wrote:

> "Thomas S. Dye" <tsd@tsdye.com> writes:
>
>> Hi Eric,
>>
>> On Dec 2, 2010, at 1:13 PM, Eric S Fraga wrote:
>
> [...]
>
>>> and each is nicely self-contained.  I am a little surprised at the
>>> choice of journal, mind you...
> [...]
>
>
>>
>> Would you recommend publishing someplace else?  We hoped list members
>> would have opinions on this and welcome comments and suggestions.
>>
>> All the best,
>> Tom
>
> This is a little outside my field, even though I'm a computer  
> scientist.
> I work at the interface between computing and industrial  
> engineering...

Yes, this is what makes the choice difficult.  Org-mode is tough to  
categorize.  It cuts across field boundaries in sometimes amazing ways.

> I was surprised at your choice simply because there appears to be
> nothing relevant to statistics about what you have written.  A more
> broad computer science journal may make sense, such as "Programming  
> and
> Computer Software" from Springer or maybe the "IEEE Transactions on
> Software Engineering?"  But I really don't know which are the best
> journals for something like this.

Thanks for the helpful suggestions, which we'll follow up.

I think the idea of submitting to JSS came from the functional overlap  
with Sweave, which is widely used by the R community, and the ease  
with which compendia can be implemented in Org-mode.  The compendium  
idea seems to have developed largely within the statistical computing  
community, with strong ties to R in particular.  JSS is one place the  
R community congregates, so we thought the article would get some  
attention there.

> Hopefully somebody else from the list will have a better idea!
>

As usual, you've provided good food for thought.

All the best,
Tom
> -- 
> : Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 23.2.1
> : using Org-mode version 7.3 (release_7.3.164.gcfd7)


[-- Attachment #1.2: Type: text/html, Size: 3345 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Org-mode Code Blocks Manuscript: Request For Comments

[Eric S Fraga]

"Thomas S. Dye" <tsd@tsdye.com> writes:

[...]

> Thanks for the helpful suggestions, which we'll follow up.  

You're very welcome.

> I think the idea of submitting to JSS came from the functional
> overlap with Sweave, which is widely used by the R community, and the
> ease with which compendia can be implemented in Org-mode.  The
> compendium idea seems to have developed largely within the
> statistical computing community, with strong ties to R in particular.
>  JSS is one place the R community congregates, so we thought the
> article would get some attention there. 

Ah, I see.  This is something I knew nothing about.  Very interesting!
I can now see the rationale for considering JSS.

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 23.2.1
: using Org-mode version 7.3 (release_7.3.172.g40812)

Re: Org-mode Code Blocks Manuscript: Request For Comments

[Nick Dokos]

Eric Schulte <schulte.eric@gmail.com> wrote:


> Dan Davison, Tom Dye, Carsten Dominik and myself have been working on a
> paper introducing Org-mode's code block functionality.  We plan to
> submit this paper to the Journal of Statistical Software.  As both
> Org-mode and the code block functionality are largely products of this
> mailing list community, and in the spirit of an open peer review process
> we are releasing the current draft of the paper here to solicit your
> review and comments.
> 
> Both the .org and .pdf formats of the paper are available at the
> following locations.
> 
> http://cs.unm.edu/~eschulte/org-paper/babel.org
> 
> http://cs.unm.edu/~eschulte/org-paper/babel.pdf
> 

I just finished reading the paper. I have watched babel go by without
doing much with it so far, so I knew a little of the motivation and the
functionality, but seeing the examples in action is another matter
altogether: beautiful stuff.

I haven't tried evaluating all the code blocks yet, but here are some
corrections to minor things and some questions, after this first
reading:

o p.9, Sec. 3.1.2: in the TODO example, a \texttt has sneaked into the
  output.

o p.14, Sec 3.3: second line of the "weaving" description, "...to export
  to HTML an number of other target formats,..." should probably be
  "...to export to HTML and a number of other target formats,..."
  perhaps?

o p. 17, Sec. 4.1.2, code block ps-to-fig, the #+ATT should
  be #+ATTR_LaTeX: I think.

o p. 20, Sec. 4.3, end of first paragraph: is "propagate" the right
  word? should it perhaps be "populate"?

o p.21, Sec. 4.3, sqlite code block: it does not have a #+source
  attribute (should it?) and the quoted code is out of sync with the
  "real" code, missing the "drop table" construct and some header args.
  I take it there is no concept of radio-code that would populate the
  ``#+begin_src org'' block with an up-to-date copy of the real code, is
  there?

o p.21, Sec. 4.3, paragraph after sqlite code block: refers to get-temps
  code block which does not exist. Even if omitted from the published
  version for brevity, it should be included in babel.org.

o p.23, Sec. 5: in the "Widely available" description, "... an wide
  variety..." should be "... a wide variety...".  In the "General and
  extensible" description, I would omit "pursuit of".

o There are widows/orphans in some examples (e.g. bottom of p.8, top of
  p.17) but I presume you'll wait to deal with those at the very end.

Finally, the questions: in Sec. 4.1.3, I may be missing something
fundamental, but I don't understand how that works at all. In
particular, is the table formula complete as it stands? How does it get
the diagonals? How are these things passed to the code block? Why does
the code block have values given to the variables? Is it correct as it
stands, or are things missing? This is the one section where I was
completely lost.  Could somebody explain?

Thanks,
Nick

Re: Org-mode Code Blocks Manuscript: Request For Comments

[Sébastien Vauban]

Hi Eric,

Let's see if I'm a good proof-reader. Here are my comments, looking at things
not already said by others:

Page 1 -- "... desirable to mix prose, (add input data?,) code, and
computational results."

Page 3 -- It'd be better not to have commas in front of the Org-mode block in
Figure 1.

Page 9 -- You say that "tags and properties of a node are inherited by its
sub-nodes". I agree for tags, not for properties (at least, by default).

Page 10 -- "Active code blocks are marked with a source line, followed by a
name unique within the document". Why don't you call such "named code blocks"
as in Babel's code base?

BTW, what happens if there is a name clash with other code blocks (in the same
document, or in the LOB)?  Though, this is not for your paper...

Page 12 -- When results is set to output, what do you mean by "collected from
STDOUT incrementally"?  Not sure about the added value of incrementally...

Page 16 -- I find the name of the code block "ps-to-dot" very badly chosen. PS
makes me think of PostScript, while you mean here Pascal's triangle...

Page 17 -- In the LaTeX ATTR, better use linewidth instead of textwidth. This
is a more secure setting.

Page 18 -- Is the default value of var (1 2 1) compliant with the "pass" table
beneath it?

Side comment -- Wouldn't you use a standard way of handling the acronyms in
LaTeX, so that they're expanded when required, and listed at the end of the
document?  Example of such acro: ESS.

For the rest, an excellent document, but not that good (IMHO) for publishing
to a statistics journal. For doing so, I find you'd have to only include R
examples, and show that you can do everything Sweave can do, and even much
more. But I would focus on stats a lot more than it is here.

But, the way it is written, it is much more general, and offers a much widen
view. So, this is excellent, but for another audience.

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: Org-mode Code Blocks Manuscript: Request For Comments

[Thomas S. Dye]

Aloha Seb,

Thanks for your detailed and thorough review.  Your comments will help  
us revise the paper.  Much appreciated.

All the best,
Tom

On Dec 7, 2010, at 12:55 PM, Sébastien Vauban wrote:

> Hi Eric,
>
> Let's see if I'm a good proof-reader. Here are my comments, looking  
> at things
> not already said by others:
>
> Page 1 -- "... desirable to mix prose, (add input data?,) code, and
> computational results."
>
> Page 3 -- It'd be better not to have commas in front of the Org-mode  
> block in
> Figure 1.
>
> Page 9 -- You say that "tags and properties of a node are inherited  
> by its
> sub-nodes". I agree for tags, not for properties (at least, by  
> default).
>
> Page 10 -- "Active code blocks are marked with a source line,  
> followed by a
> name unique within the document". Why don't you call such "named  
> code blocks"
> as in Babel's code base?
>
> BTW, what happens if there is a name clash with other code blocks  
> (in the same
> document, or in the LOB)?  Though, this is not for your paper...
>
> Page 12 -- When results is set to output, what do you mean by  
> "collected from
> STDOUT incrementally"?  Not sure about the added value of  
> incrementally...
>
> Page 16 -- I find the name of the code block "ps-to-dot" very badly  
> chosen. PS
> makes me think of PostScript, while you mean here Pascal's triangle...
>
> Page 17 -- In the LaTeX ATTR, better use linewidth instead of  
> textwidth. This
> is a more secure setting.
>
> Page 18 -- Is the default value of var (1 2 1) compliant with the  
> "pass" table
> beneath it?
>
> Side comment -- Wouldn't you use a standard way of handling the  
> acronyms in
> LaTeX, so that they're expanded when required, and listed at the end  
> of the
> document?  Example of such acro: ESS.
>
> For the rest, an excellent document, but not that good (IMHO) for  
> publishing
> to a statistics journal. For doing so, I find you'd have to only  
> include R
> examples, and show that you can do everything Sweave can do, and  
> even much
> more. But I would focus on stats a lot more than it is here.
>
> But, the way it is written, it is much more general, and offers a  
> much widen
> view. So, this is excellent, but for another audience.
>
> Best regards,
>  Seb
>
> -- 
> Sébastien Vauban
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: Org-mode Code Blocks Manuscript: Request For Comments

[Eric Schulte]

Hi Seb,

Thanks for the proof reading.  I have answers for some of your questions
below.

Sébastien Vauban <wxhgmqzgwmuf@spammotel.com> writes:

>
> Page 9 -- You say that "tags and properties of a node are inherited by its
> sub-nodes". I agree for tags, not for properties (at least, by default).
>

With respect to code blocks properties are inherited by subnodes, at
least all properties which can be used as header arguments are
inherited.

>
> BTW, what happens if there is a name clash with other code blocks (in the same
> document, or in the LOB)?  Though, this is not for your paper...
>

While no behavior is guaranteed in this case (meaning don't do it :)) I
believe that whichever code block is found first will be used, in
practice this would probably mean that local code blocks will override
lob code blocks, but I make no guarantees

>
> Side comment -- Wouldn't you use a standard way of handling the acronyms in
> LaTeX, so that they're expanded when required, and listed at the end of the
> document?  Example of such acro: ESS.
>

I don't understand what you mean by "standard acronyms" can you give a
specific location and how you would suggest it be changed?

Thanks -- Eric

Re: **: Re: Re: Org-mode Code Blocks Manuscript: Request For Comments

[Sébastien Vauban]

Hi Eric,

"Eric Schulte" wrote:
> Thanks for the proof reading.  I have answers for some of your questions
> below.

Sure!

>> Page 9 -- You say that "tags and properties of a node are inherited by its
>> sub-nodes". I agree for tags, not for properties (at least, by default).
>
> With respect to code blocks properties are inherited by subnodes, at
> least all properties which can be used as header arguments are
> inherited.

OK. You're talking now of the properties *of code blocks*. The, your paragraph
is a bit misleading, as you're also talking of tags -- which do not apply to
code blocks...

Having made the above distinction, I now understand your paragraph.

>> BTW, what happens if there is a name clash with other code blocks (in the same
>> document, or in the LOB)?  Though, this is not for your paper...
>
> While no behavior is guaranteed in this case (meaning don't do it :)) I
> believe that whichever code block is found first will be used, in
> practice this would probably mean that local code blocks will override
> lob code blocks, but I make no guarantees

Some ideas:

- report the conflict in a very visible way (at execution and export times)

- having the ability to look for potential clashes (some
  =list-code-block-shadows=)

- (why not?) being able to add the filename of the code block we want to use,
  to resolve the conflict (if we don't want to change the names...)

>> Side comment -- Wouldn't you use a standard way of handling the acronyms in
>> LaTeX, so that they're expanded when required, and listed at the end of the
>> document?  Example of such acro: ESS.
>
> I don't understand what you mean by "standard acronyms" can you give a
> specific location and how you would suggest it be changed?

#+TITLE:     Inserting proper acronyms in LaTeX
#+DATE:      2010-12-09
#+LANGUAGE:  en_US

#+LaTeX_HEADER: \usepackage[printonlyused]{acronym}% (not in medium TeX Live installation)

* Prologue

\ac{ESS} is a great add-on to Emacs.

* Epilogue

Emacs is made public by the \ac{FSF}. The second time the acronym \ac{FSF} is
used, it should not be expanded in the PDF...

All of that being taken care automagically by LaTeX itself, and the =acronym=
package. Plus you gain hyperlinks from every usage of acronym to its
definition table...

* Acronyms

\begin{acronym}[LONGEST]
    \acro{EEPROM} {Electrically Erasable Programmable \acs{ROM}}
    \acro{ESS}    {Emacs Speaks Statistics}
    \acro{FSF}    {Free Software Foundation}
    \acro{GNU}    {GNU is Not Unix}
\end{acronym}

** Note                                                            :noexport:

Unused acronyms won't be outputted in the final PDF... Out of the 4 defined
acronyms, only the 2 used will be listed at the end of the document... thanks
to the option =printonlyused=.

* Conclusion

Does this answer your question?

For me, this is one of the only missing piece that should be made more
standard into Org.

The real problem is: how do we have something clean for the HTML export, even
if ultra-minimal (like having no LaTeX symbols outputted in the middle of the
text, having always/never acronym expansion, printing all acronyms
independently of the fact they're used or not).

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: **: Re: Re: Org-mode Code Blocks Manuscript: Request For Comments

[Eric Schulte]

Hi Seb,

Sébastien Vauban <wxhgmqzgwmuf@spammotel.com> writes:

> Hi Eric,
>
> "Eric Schulte" wrote:
>> Thanks for the proof reading.  I have answers for some of your questions
>> below.
>
> Sure!
>
>>> Page 9 -- You say that "tags and properties of a node are inherited by its
>>> sub-nodes". I agree for tags, not for properties (at least, by default).
>>
>> With respect to code blocks properties are inherited by subnodes, at
>> least all properties which can be used as header arguments are
>> inherited.
>
> OK. You're talking now of the properties *of code blocks*. The, your paragraph
> is a bit misleading, as you're also talking of tags -- which do not apply to
> code blocks...
>
> Having made the above distinction, I now understand your paragraph.
>

Alright I'll take another look at this paragraph and see if I can
untangle the verbiage.

>
>>> BTW, what happens if there is a name clash with other code blocks (in the same
>>> document, or in the LOB)?  Though, this is not for your paper...
>>
>> While no behavior is guaranteed in this case (meaning don't do it :)) I
>> believe that whichever code block is found first will be used, in
>> practice this would probably mean that local code blocks will override
>> lob code blocks, but I make no guarantees
>
> Some ideas:
>
> - report the conflict in a very visible way (at execution and export times)
>
> - having the ability to look for potential clashes (some
>   =list-code-block-shadows=)
>
> - (why not?) being able to add the filename of the code block we want to use,
>   to resolve the conflict (if we don't want to change the names...)
>

actually this should already be implemented, you can specify another
file by constructing your reference as file-name:resource-name, e.g.

with the following saved to ~/Desktop/location.org

  #+source: year
  #+begin_src emacs-lisp
    2010
  #+end_src

  #+source: city
  #+begin_src emacs-lisp
    "Santa Fe"
  #+end_src

the following code block should resolve its references correctly from
*any* buffer

  #+headers: :var city=~/Desktop/location.org:city
  #+headers: :var year=~/Desktop/location.org:year
  #+begin_src emacs-lisp
    (message "I'm in %s in the year %d" city year)
  #+end_src

although apparently there is still some work to be done on this code as
for me it leaves the referenced file open.  This is code that apparently
hasn't been used or documented so I'm sure there will be some kinks to
work out

>
>>> Side comment -- Wouldn't you use a standard way of handling the acronyms in
>>> LaTeX, so that they're expanded when required, and listed at the end of the
>>> document?  Example of such acro: ESS.
>>
>> I don't understand what you mean by "standard acronyms" can you give a
>> specific location and how you would suggest it be changed?
>
> #+TITLE:     Inserting proper acronyms in LaTeX
> #+DATE:      2010-12-09
> #+LANGUAGE:  en_US
>
> #+LaTeX_HEADER: \usepackage[printonlyused]{acronym}% (not in medium TeX Live installation)
>
> * Prologue
>
> \ac{ESS} is a great add-on to Emacs.
>
> * Epilogue
>
> Emacs is made public by the \ac{FSF}. The second time the acronym \ac{FSF} is
> used, it should not be expanded in the PDF...
>
> All of that being taken care automagically by LaTeX itself, and the =acronym=
> package. Plus you gain hyperlinks from every usage of acronym to its
> definition table...
>
> * Acronyms
>
> \begin{acronym}[LONGEST]
>     \acro{EEPROM} {Electrically Erasable Programmable \acs{ROM}}
>     \acro{ESS}    {Emacs Speaks Statistics}
>     \acro{FSF}    {Free Software Foundation}
>     \acro{GNU}    {GNU is Not Unix}
> \end{acronym}
>
> ** Note                                                            :noexport:
>
> Unused acronyms won't be outputted in the final PDF... Out of the 4 defined
> acronyms, only the 2 used will be listed at the end of the document... thanks
> to the option =printonlyused=.
>
> * Conclusion
>
> Does this answer your question?
>

Yes, thanks for the explanation, this is the first I've heard of this
package but it certainly does seem useful.

>
> For me, this is one of the only missing piece that should be made more
> standard into Org.
>
> The real problem is: how do we have something clean for the HTML export, even
> if ultra-minimal (like having no LaTeX symbols outputted in the middle of the
> text, having always/never acronym expansion, printing all acronyms
> independently of the fact they're used or not).
>

Maybe this would be possible to implement using custom links?

Cheers -- Eric

>
> Best regards,
>   Seb

Re: Org-mode Code Blocks Manuscript: Request For Comments

[Sébastien Vauban]

Hi Eric,

"Eric Schulte" wrote:
> Thanks for the proof reading.  I have answers for some of your questions
> below.

Sure!

>> Page 9 -- You say that "tags and properties of a node are inherited by its
>> sub-nodes". I agree for tags, not for properties (at least, by default).
>
> With respect to code blocks properties are inherited by subnodes, at
> least all properties which can be used as header arguments are
> inherited.

OK. You're talking now of the properties *of code blocks*. The, your paragraph
is a bit misleading, as you're also talking of tags -- which do not apply to
code blocks...

Having made the above distinction, I now understand your paragraph.

>> BTW, what happens if there is a name clash with other code blocks (in the same
>> document, or in the LOB)?  Though, this is not for your paper...
>
> While no behavior is guaranteed in this case (meaning don't do it :)) I
> believe that whichever code block is found first will be used, in
> practice this would probably mean that local code blocks will override
> lob code blocks, but I make no guarantees

Some ideas:

- report the conflict in a very visible way (at execution and export times)

- having the ability to look for potential clashes (some
  =list-code-block-shadows=)

- (why not?) being able to add the filename of the code block we want to use,
  to resolve the conflict (if we don't want to change the names...)

>> Side comment -- Wouldn't you use a standard way of handling the acronyms in
>> LaTeX, so that they're expanded when required, and listed at the end of the
>> document?  Example of such acro: ESS.
>
> I don't understand what you mean by "standard acronyms" can you give a
> specific location and how you would suggest it be changed?

#+TITLE:     Inserting proper acronyms in LaTeX
#+DATE:      2010-12-09
#+LANGUAGE:  en_US

#+LaTeX_HEADER: \usepackage[printonlyused]{acronym}% (not in medium TeX Live installation)

* Prologue

\ac{ESS} is a great add-on to Emacs.

* Epilogue

Emacs is made public by the \ac{FSF}. The second time the acronym \ac{FSF} is
used, it should not be expanded in the PDF...

All of that being taken care automagically by LaTeX itself, and the =acronym=
package. Plus you gain hyperlinks from every usage of acronym to its
definition table...

* Acronyms

\begin{acronym}[LONGEST]
    \acro{EEPROM} {Electrically Erasable Programmable \acs{ROM}}
    \acro{ESS}    {Emacs Speaks Statistics}
    \acro{FSF}    {Free Software Foundation}
    \acro{GNU}    {GNU is Not Unix}
\end{acronym}

** Note                                                            :noexport:

Unused acronyms won't be outputted in the final PDF... Out of the 4 defined
acronyms, only the 2 used will be listed at the end of the document... thanks
to the option =printonlyused=.

* Conclusion

Does this answer your question?

For me, this is one of the only missing piece that should be made more
standard into Org.

The real problem is: how do we have something clean for the HTML export, even
if ultra-minimal (like having no LaTeX symbols outputted in the middle of the
text, having always/never acronym expansion, printing all acronyms
independently of the fact they're used or not).

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Org-mode Code Blocks Manuscript: Request For Comments

[Eric Schulte]

Hi Nick,

Thanks for the feedback.

Nick Dokos <nicholas.dokos@hp.com> writes:

>
> Finally, the questions: in Sec. 4.1.3, I may be missing something
> fundamental, but I don't understand how that works at all. In
> particular, is the table formula complete as it stands?  How does it
> get the diagonals? How are these things passed to the code block? Why
> does the code block have values given to the variables? Is it correct
> as it stands, or are things missing? This is the one section where I
> was completely lost.  Could somebody explain?
>

Thanks for bringing this up.  I truncated the table formulas so that
they wouldn't overrun the page.  This lead to an incomplete example
which didn't work.  In the current version of the paper (up at the old
link [1]) I have expanded these formulas and split them across multiple
lines.  Although Org-mode (unfortunately) does not (yet) support
multi-line table formulas, as least all of the required information is
now present so this example should no longer be quite so mystifying.

>
> Thanks,
> Nick

Footnotes: 
[1]  http://cs.unm.edu/~eschulte/org-paper/babel.pdf

Re: Org-mode Code Blocks Manuscript: Request For Comments

[Detlef Steuer]

Hi!

I very much appreciate your article as a nice introduction to org-babel
and its uses. As I'm going to introduce my colleagues into the nice
world of org-babel giving a talk sometime next term I'll shamelessly
steal from your work. (Of course giving attribution!) 

Some remarks: 
If you send it to Journal of _Statistical_ Software may be you should
be a little bit more focused on statistics. You article introduces
org-babel as a multi-language frontend to literate programming. What it
is, but there is little statistics in it.  

In their article Gentleman and Lang introduced the "statistical
compendium". In my opinion emacs + org-mode + babel +
all-programming-languages-we-know + LaTeX + HTML export build the first
incarnation of a tool to really create such a compendium, org-babel
being central in that chain. 
May be you can use some of Tom Dye's data to give an example of a
self-contained statistical workflow. I used his introduction given in
Worg to do my first steps in that direction. (Thx again Tom!) 
Doing everything beginning with data-cleaning over data analysis to
template generating and report publishing and presentation in one
text-file. 
That feature was, what caught me immediately as a statistician. 

If you want to focus on the simulation side (may be more focused on
academics) I would stress the "always-correctness" of graphs in
articles. You all know what I mean...

Just my 2 cents. Of course it is great as it stands  and surely I'm
biased by my own needs. 

Detlef 
(a statistician)

On Thu, 02 Dec 2010 12:28:27 -0700
"Eric Schulte" <schulte.eric@gmail.com> wrote:

> Hi,
> 
> Dan Davison, Tom Dye, Carsten Dominik and myself have been working on a
> paper introducing Org-mode's code block functionality.  We plan to
> submit this paper to the Journal of Statistical Software.  As both
> Org-mode and the code block functionality are largely products of this
> mailing list community, and in the spirit of an open peer review process
> we are releasing the current draft of the paper here to solicit your
> review and comments.
> 
> Both the .org and .pdf formats of the paper are available at the
> following locations.
> 
> http://cs.unm.edu/~eschulte/org-paper/babel.org
> 
> http://cs.unm.edu/~eschulte/org-paper/babel.pdf
> 
> Thanks -- Eric
> 
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
> 

Re: Re: Org-mode Code Blocks Manuscript: Request For Comments

[Thomas S. Dye]

Aloha Detlef,

On Dec 2, 2010, at 9:58 PM, Detlef Steuer wrote:

> Hi!
>
> I very much appreciate your article as a nice introduction to org- 
> babel
> and its uses. As I'm going to introduce my colleagues into the nice
> world of org-babel giving a talk sometime next term I'll shamelessly
> steal from your work. (Of course giving attribution!)
>
> Some remarks:
> If you send it to Journal of _Statistical_ Software may be you should
> be a little bit more focused on statistics. You article introduces
> org-babel as a multi-language frontend to literate programming. What  
> it
> is, but there is little statistics in it.
>
> In their article Gentleman and Lang introduced the "statistical
> compendium". In my opinion emacs + org-mode + babel +
> all-programming-languages-we-know + LaTeX + HTML export build the  
> first
> incarnation of a tool to really create such a compendium, org-babel
> being central in that chain.
> May be you can use some of Tom Dye's data to give an example of a
> self-contained statistical workflow. I used his introduction given in
> Worg to do my first steps in that direction. (Thx again Tom!)
> Doing everything beginning with data-cleaning over data analysis to
> template generating and report publishing and presentation in one
> text-file.
> That feature was, what caught me immediately as a statistician.
>
> If you want to focus on the simulation side (may be more focused on
> academics) I would stress the "always-correctness" of graphs in
> articles. You all know what I mean...
>
> Just my 2 cents. Of course it is great as it stands  and surely I'm
> biased by my own needs.
>
> Detlef
> (a statistician)
>

Thanks very much for the helpful comments and especially your  
perspective on the Journal of Statistical Software.

I'm interested to learn how you've developed a statistical workflow  
with Org-mode beyond my first tentative steps in that direction.  It  
would be great to have an example of your progress on Worg, if you can  
find the time.

All the best,
Tom

> On Thu, 02 Dec 2010 12:28:27 -0700
> "Eric Schulte" <schulte.eric@gmail.com> wrote:
>
>> Hi,
>>
>> Dan Davison, Tom Dye, Carsten Dominik and myself have been working  
>> on a
>> paper introducing Org-mode's code block functionality.  We plan to
>> submit this paper to the Journal of Statistical Software.  As both
>> Org-mode and the code block functionality are largely products of  
>> this
>> mailing list community, and in the spirit of an open peer review  
>> process
>> we are releasing the current draft of the paper here to solicit your
>> review and comments.
>>
>> Both the .org and .pdf formats of the paper are available at the
>> following locations.
>>
>> http://cs.unm.edu/~eschulte/org-paper/babel.org
>>
>> http://cs.unm.edu/~eschulte/org-paper/babel.pdf
>>
>> Thanks -- Eric
>>
>> _______________________________________________
>> Emacs-orgmode mailing list
>> Please use `Reply All' to send replies to the list.
>> Emacs-orgmode@gnu.org
>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>>
>
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: Org-mode Code Blocks Manuscript: Request For Comments

[Charles C. Berry]

On Sat, 4 Dec 2010, Thomas S. Dye wrote:

> Aloha Detlef,
>
> On Dec 2, 2010, at 9:58 PM, Detlef Steuer wrote:
>
>> Hi!
>> 
>> I very much appreciate your article as a nice introduction to org-babel
>> and its uses. As I'm going to introduce my colleagues into the nice
>> world of org-babel giving a talk sometime next term I'll shamelessly
>> steal from your work. (Of course giving attribution!)
>> 
>> Some remarks:
>> If you send it to Journal of _Statistical_ Software may be you should
>> be a little bit more focused on statistics. You article introduces
>> org-babel as a multi-language frontend to literate programming. What it
>> is, but there is little statistics in it.
>> 
>> In their article Gentleman and Lang introduced the "statistical
>> compendium". In my opinion emacs + org-mode + babel +
>> all-programming-languages-we-know + LaTeX + HTML export build the first
>> incarnation of a tool to really create such a compendium, org-babel
>> being central in that chain.
>> May be you can use some of Tom Dye's data to give an example of a
>> self-contained statistical workflow. I used his introduction given in
>> Worg to do my first steps in that direction. (Thx again Tom!)
>> Doing everything beginning with data-cleaning over data analysis to
>> template generating and report publishing and presentation in one
>> text-file.
>> That feature was, what caught me immediately as a statistician.
>> 
>> If you want to focus on the simulation side (may be more focused on
>> academics) I would stress the "always-correctness" of graphs in
>> articles. You all know what I mean...
>> 
>> Just my 2 cents. Of course it is great as it stands  and surely I'm
>> biased by my own needs.
>> 
>> Detlef
>> (a statistician)
>> 
>
> Thanks very much for the helpful comments and especially your perspective on 
> the Journal of Statistical Software.
>
> I'm interested to learn how you've developed a statistical workflow with 
> Org-mode beyond my first tentative steps in that direction.  It would be 
> great to have an example of your progress on Worg, if you can find the time.

Tom,

You might glean something from these links:

ESS and org-mode workflows are discussed here:


http://stackoverflow.com/questions/1429907/workflow-for-statistical-analysis-and-report-writing/

http://stackoverflow.com/questions/3027476/ess-workflow-for-r-project-package-development

https://github.com/Choens/LiterateR


CRAN's reproducible research 'task view' (with 'Related Links' of some
interest):

        http://cran.r-project.org/web/views/ReproducibleResearch.html

If you want to reach the R community, 'The R Journal' might be worth a try:

http://journal.r-project.org/

======

Let me just add my $0.02 worth to what others have already said and
say, that I really find org-babel useful in my R related work.

Currently, I am making use of it an environment for developing
R-packages. An org-mode file sits in the top level source directory of
an R package; it contains src blocks to fire up speedbar, list files
(for navigation w/o speedbar), do version control operations, check,
build, install, load the package, and do other routine tasks. Each
operation has its own headline, so I need only put the point on the
headline and 'C-c C-v C-s y' to run the subtree containing the block -
effectively making each operation a point - and - (a little more than
a) click.  Those source blocks are nearly the same for each package.

Additional blocks display help pages in the org file, load sample
data, let me work on new package features, and try out R idioms I
might want to use.

Then there are all the usual org-mode features that let me keep notes
and ideas and track the status of the package. org-mode has made this
part of my life a good deal simpler!

Chuck

>
> All the best,
> Tom
>
>> On Thu, 02 Dec 2010 12:28:27 -0700
>> "Eric Schulte" <schulte.eric@gmail.com> wrote:
>> 
>> > Hi,
>> > 
>> > Dan Davison, Tom Dye, Carsten Dominik and myself have been working on a
>> > paper introducing Org-mode's code block functionality.  We plan to
>> > submit this paper to the Journal of Statistical Software.  As both
>> > Org-mode and the code block functionality are largely products of this
>> > mailing list community, and in the spirit of an open peer review process
>> > we are releasing the current draft of the paper here to solicit your
>> > review and comments.
>> > 
>> > Both the .org and .pdf formats of the paper are available at the
>> > following locations.
>> > 
>> > http://cs.unm.edu/~eschulte/org-paper/babel.org
>> > 
>> > http://cs.unm.edu/~eschulte/org-paper/babel.pdf
>> > 
>> > Thanks -- Eric
>> > 
>> > _______________________________________________
>> > Emacs-orgmode mailing list
>> > Please use `Reply All' to send replies to the list.
>> > Emacs-orgmode@gnu.org
>> > http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>> > 
>> 
>> 
>> 
>> _______________________________________________
>> Emacs-orgmode mailing list
>> Please use `Reply All' to send replies to the list.
>> Emacs-orgmode@gnu.org
>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>

Charles C. Berry                            Dept of Family/Preventive Medicine
cberry@tajo.ucsd.edu			    UC San Diego
http://famprevmed.ucsd.edu/faculty/cberry/  La Jolla, San Diego 92093-0901

Re: Re: Org-mode Code Blocks Manuscript: Request For Comments

[Sunny Srivastava]

[-- Attachment #1.1: Type: text/plain, Size: 6308 bytes --]

Hello Chuck:

Your idea is very interesting. I am curious to make use of your ideas. If it
is not too much trouble, can you please share an example org file that you
use for package development? I completely understand if you can't share the
file.

Your help is highly appreciated.

Thank you in advance.

Best Regards,
S.

On Mon, Dec 6, 2010 at 2:52 PM, Charles C. Berry <cberry@tajo.ucsd.edu>wrote:

> On Sat, 4 Dec 2010, Thomas S. Dye wrote:
>
>  Aloha Detlef
>>
>> On Dec 2, 2010, at 9:58 PM, Detlef Steuer wrote:
>>
>>  Hi!
>>>
>>> I very much appreciate your article as a nice introduction to org-babel
>>> and its uses. As I'm going to introduce my colleagues into the nice
>>> world of org-babel giving a talk sometime next term I'll shamelessly
>>> steal from your work. (Of course giving attribution!)
>>>
>>> Some remarks:
>>> If you send it to Journal of _Statistical_ Software may be you should
>>> be a little bit more focused on statistics. You article introduces
>>> org-babel as a multi-language frontend to literate programming. What it
>>> is, but there is little statistics in it.
>>>
>>> In their article Gentleman and Lang introduced the "statistical
>>> compendium". In my opinion emacs + org-mode + babel +
>>> all-programming-languages-we-know + LaTeX + HTML export build the first
>>> incarnation of a tool to really create such a compendium, org-babel
>>> being central in that chain.
>>> May be you can use some of Tom Dye's data to give an example of a
>>> self-contained statistical workflow. I used his introduction given in
>>> Worg to do my first steps in that direction. (Thx again Tom!)
>>> Doing everything beginning with data-cleaning over data analysis to
>>> template generating and report publishing and presentation in one
>>> text-file.
>>> That feature was, what caught me immediately as a statistician.
>>>
>>> If you want to focus on the simulation side (may be more focused on
>>> academics) I would stress the "always-correctness" of graphs in
>>> articles. You all know what I mean...
>>>
>>> Just my 2 cents. Of course it is great as it stands  and surely I'm
>>> biased by my own needs.
>>>
>>> Detlef
>>> (a statistician)
>>>
>>>
>> Thanks very much for the helpful comments and especially your perspective
>> on the Journal of Statistical Software.
>>
>> I'm interested to learn how you've developed a statistical workflow with
>> Org-mode beyond my first tentative steps in that direction.  It would be
>> great to have an example of your progress on Worg, if you can find the time.
>>
>
> Tom,
>
> You might glean something from these links:
>
> ESS and org-mode workflows are discussed here:
>
>
>
> http://stackoverflow.com/questions/1429907/workflow-for-statistical-analysis-and-report-writing/
>
>
> http://stackoverflow.com/questions/3027476/ess-workflow-for-r-project-package-development
>
> https://github.com/Choens/LiterateR
>
>
> CRAN's reproducible research 'task view' (with 'Related Links' of some
> interest):
>
>       http://cran.r-project.org/web/views/ReproducibleResearch.html
>
> If you want to reach the R community, 'The R Journal' might be worth a try:
>
> http://journal.r-project.org/
>
> ======
>
> Let me just add my $0.02 worth to what others have already said and
> say, that I really find org-babel useful in my R related work.
>
> Currently, I am making use of it an environment for developing
> R-packages. An org-mode file sits in the top level source directory of
> an R package; it contains src blocks to fire up speedbar, list files
> (for navigation w/o speedbar), do version control operations, check,
> build, install, load the package, and do other routine tasks. Each
> operation has its own headline, so I need only put the point on the
> headline and 'C-c C-v C-s y' to run the subtree containing the block -
> effectively making each operation a point - and - (a little more than
> a) click.  Those source blocks are nearly the same for each package.
>
> Additional blocks display help pages in the org file, load sample
> data, let me work on new package features, and try out R idioms I
> might want to use.
>
> Then there are all the usual org-mode features that let me keep notes
> and ideas and track the status of the package. org-mode has made this
> part of my life a good deal simpler!
>
> Chuck
>
>
>
>> All the best,
>> Tom
>>
>>  On Thu, 02 Dec 2010 12:28:27 -0700
>>> "Eric Schulte" <schulte.eric@gmail.com> wrote:
>>>
>>> > Hi,
>>> > > Dan Davison, Tom Dye, Carsten Dominik and myself have been working on
>>> a
>>> > paper introducing Org-mode's code block functionality.  We plan to
>>> > submit this paper to the Journal of Statistical Software.  As both
>>> > Org-mode and the code block functionality are largely products of this
>>> > mailing list community, and in the spirit of an open peer review
>>> process
>>> > we are releasing the current draft of the paper here to solicit your
>>> > review and comments.
>>> > > Both the .org and .pdf formats of the paper are available at the
>>> > following locations.
>>> > > http://cs.unm.edu/~eschulte/org-paper/babel.org
>>> > > http://cs.unm.edu/~eschulte/org-paper/babel.pdf
>>> > > Thanks -- Eric
>>> > > _______________________________________________
>>> > Emacs-orgmode mailing list
>>> > Please use `Reply All' to send replies to the list.
>>> > Emacs-orgmode@gnu.org
>>> > http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>>> >
>>>
>>>
>>> _______________________________________________
>>> Emacs-orgmode mailing list
>>> Please use `Reply All' to send replies to the list.
>>> Emacs-orgmode@gnu.org
>>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>>>
>>
>>
>> _______________________________________________
>> Emacs-orgmode mailing list
>> Please use `Reply All' to send replies to the list.
>> Emacs-orgmode@gnu.org
>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>>
>>
> Charles C. Berry                            Dept of Family/Preventive
> Medicine
> cberry@tajo.ucsd.edu                        UC San Diego
> http://famprevmed.ucsd.edu/faculty/cberry/  La Jolla, San Diego 92093-0901
>
>
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>

[-- Attachment #1.2: Type: text/html, Size: 9173 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: Org-mode Code Blocks Manuscript: Request For Comments

[Charles C. Berry]

On Mon, 6 Dec 2010, Sunny Srivastava wrote:

> Hello Chuck:
>
> Your idea is very interesting. I am curious to make use of your ideas. If it
> is not too much trouble, can you please share an example org file that you
> use for package development? I completely understand if you can't share the
> file.
>

Sunny,

I posted the vanilla version of the file at

http://famprevmed.ucsd.edu/faculty/cberry/org-mode/RpkgExample.org

It has the src blocks I use in each package.

To use it, you set up a minimal package directory structure:

 	myPackage/
 	myPackage/DESCRIPTION
 	myPackage/man/
 	myPackage/R/

say, and (optionally) put it under version control.

Or use an existing package you are already working on.

Or download one from CRAN, and untar it.

Then copy RpkgExample.org to myPackage/

(or whatever the equivalent directory is)

and you are ready to start.

FWIW, if I have a good idea of what I am doing at the outset, I will
write functions in R/*.R files and create man/*.Rd files using
prompt() and then edit them, and then get around to checking,
installing, and trying out the package from the org file.

But usually, I have only a fuzzy idea of what how to organize the
code, so I start by writing a snippet of code in an R :session src
block that sets up some objects of the sort I would want my package to
work on. I run that block. Then I might write a script in another :session
src block to do some of the work I want the package to do, and
try it out. Once it works I wrap it into a function, and write another
:session src block to call the function. Once that works, I kill the
src block with the function in it and yank it into a fresh buffer
where it is saved as R/whatever.R. After using prompt() to make the
man/whatever.Rd and editting it, I am ready to run the package check,
install the package, restart my R session and load the package. Then I
can stitch together tests, examples, and more functions in the org
file, and test them and migrate them to the right places.


Comments welcome.

Best,

Chuck


> Your help is highly appreciated.
>
> Thank you in advance.
>
> Best Regards,
> S.
>
> On Mon, Dec 6, 2010 at 2:52 PM, Charles C. Berry <cberry@tajo.ucsd.edu>wrote:
>
>> On Sat, 4 Dec 2010, Thomas S. Dye wrote:
>>
>>  Aloha Detlef
>>>
>>> On Dec 2, 2010, at 9:58 PM, Detlef Steuer wrote:
>>>
>>>  Hi!

[rest deleted]


Charles C. Berry                            Dept of Family/Preventive Medicine
cberry@tajo.ucsd.edu			    UC San Diego
http://famprevmed.ucsd.edu/faculty/cberry/  La Jolla, San Diego 92093-0901

Re: Re: Org-mode Code Blocks Manuscript: Request For Comments

[Thomas S. Dye]

Aloha Chuck,
On Dec 6, 2010, at 6:48 PM, Charles C. Berry wrote:

> On Mon, 6 Dec 2010, Sunny Srivastava wrote:
>
>> Hello Chuck:
>>
>> Your idea is very interesting. I am curious to make use of your  
>> ideas. If it
>> is not too much trouble, can you please share an example org file  
>> that you
>> use for package development? I completely understand if you can't  
>> share the
>> file.
>>
>
> Sunny,
>
> I posted the vanilla version of the file at
>
> http://famprevmed.ucsd.edu/faculty/cberry/org-mode/RpkgExample.org
>
> It has the src blocks I use in each package.
>
> To use it, you set up a minimal package directory structure:
>
> 	myPackage/
> 	myPackage/DESCRIPTION
> 	myPackage/man/
> 	myPackage/R/
>
> say, and (optionally) put it under version control.
>
> Or use an existing package you are already working on.
>
> Or download one from CRAN, and untar it.
>
> Then copy RpkgExample.org to myPackage/
>
> (or whatever the equivalent directory is)
>
> and you are ready to start.
>
> FWIW, if I have a good idea of what I am doing at the outset, I will
> write functions in R/*.R files and create man/*.Rd files using
> prompt() and then edit them, and then get around to checking,
> installing, and trying out the package from the org file.
>
> But usually, I have only a fuzzy idea of what how to organize the
> code, so I start by writing a snippet of code in an R :session src
> block that sets up some objects of the sort I would want my package to
> work on. I run that block. Then I might write a script in  
> another :session
> src block to do some of the work I want the package to do, and
> try it out. Once it works I wrap it into a function, and write another
> :session src block to call the function. Once that works, I kill the
> src block with the function in it and yank it into a fresh buffer
> where it is saved as R/whatever.R. After using prompt() to make the
> man/whatever.Rd and editting it, I am ready to run the package check,
> install the package, restart my R session and load the package. Then I
> can stitch together tests, examples, and more functions in the org
> file, and test them and migrate them to the right places.
>
>
> Comments welcome.
>

Thanks for sharing this.  It looks useful.  Would you consider putting  
it on Worg with the other babel source block examples?

Have you thought about tangling the .R files directly from the Org- 
mode buffer?  With :tangle R/whatever.R you might save yourself having  
to kill the source block, yanking it to a fresh buffer, and saving.

My goal when designing these things, which might or might not appeal  
to you, is to hold the entire project in the Org-mode file.  In the  
end, exporting the Org-mode file to html or pdf can yield a rich  
description of the project, independent of its product, man files,  
etc.  Later, when I want to make changes, I know exactly where to go.

All the best,
Tom

> Best,
>
> Chuck
>
>
>> Your help is highly appreciated.
>>
>> Thank you in advance.
>>
>> Best Regards,
>> S.
>>
>> On Mon, Dec 6, 2010 at 2:52 PM, Charles C. Berry <cberry@tajo.ucsd.edu 
>> >wrote:
>>
>>> On Sat, 4 Dec 2010, Thomas S. Dye wrote:
>>>
>>> Aloha Detlef
>>>>
>>>> On Dec 2, 2010, at 9:58 PM, Detlef Steuer wrote:
>>>>
>>>> Hi!
>
> [rest deleted]
>
>
> Charles C. Berry                            Dept of Family/ 
> Preventive Medicine
> cberry@tajo.ucsd.edu			    UC San Diego
> http://famprevmed.ucsd.edu/faculty/cberry/  La Jolla, San Diego  
> 92093-0901
>
>

Re: Re: Org-mode Code Blocks Manuscript: Request For Comments

[Charles C. Berry]

On Tue, 7 Dec 2010, Thomas S. Dye wrote:

> Aloha Chuck,
> On Dec 6, 2010, at 6:48 PM, Charles C. Berry wrote:
>
>> On Mon, 6 Dec 2010, Sunny Srivastava wrote:

{snip]

>> 
>> I posted the vanilla version of the file at
>> 
>> http://famprevmed.ucsd.edu/faculty/cberry/org-mode/RpkgExample.org
>> 
>> It has the src blocks I use in each package.
>> 
>> To use it, you set up a minimal package directory structure:
>>
>>  myPackage/
>>  myPackage/DESCRIPTION
>>  myPackage/man/
>>  myPackage/R/
>> 
>> say, and (optionally) put it under version control.
>> 
>> Or use an existing package you are already working on.
>> 
>> Or download one from CRAN, and untar it.
>> 
>> Then copy RpkgExample.org to myPackage/
>> 
>> (or whatever the equivalent directory is)
>> 
>> and you are ready to start.
>> 
>> FWIW, if I have a good idea of what I am doing at the outset, I will
>> write functions in R/*.R files and create man/*.Rd files using
>> prompt() and then edit them, and then get around to checking,
>> installing, and trying out the package from the org file.
>> 
>> But usually, I have only a fuzzy idea of what how to organize the
>> code, so I start by writing a snippet of code in an R :session src
>> block that sets up some objects of the sort I would want my package to
>> work on. I run that block. Then I might write a script in another :session
>> src block to do some of the work I want the package to do, and
>> try it out. Once it works I wrap it into a function, and write another
>> : session src block to call the function. Once that works, I kill the
>> src block with the function in it and yank it into a fresh buffer
>> where it is saved as R/whatever.R. After using prompt() to make the
>> man/whatever.Rd and editting it, I am ready to run the package check,
>> install the package, restart my R session and load the package. Then I
>> can stitch together tests, examples, and more functions in the org
>> file, and test them and migrate them to the right places.
>> 
>> 
>> Comments welcome.
>> 
>
> Thanks for sharing this.  It looks useful.  Would you consider putting it on 
> Worg with the other babel source block examples?
>

Yes. I will clean it up a bit in the coming days and post it.


> Have you thought about tangling the .R files directly from the Org-mode 
> buffer?  With :tangle R/whatever.R you might save yourself having to kill the 
> source block, yanking it to a fresh buffer, and saving.

Well, yes and no. (There was a brief discussion of just his issue here
a couple of months back.)

I kinda assumed that the workflow would be a bit more awkward with the
package files all held in the org file:

   - run a test, find a bug
   - move to find src block with relevant .R or .Rd code
   - edit the .R or .Rd
   - tangle all the src blocks containing package code or files
   - check and/or install and then load the package
   - move back to the test src block
   - repeat

I was thinking that navigating thru the org file to find the .R or .Rd
src block and later back to the test src block would be slower than
jumping between a few code buffers, but maybe I overlooked the power
of folding, TODO's, and other org-built functionality. The speedbar
makes navigating the package directory pretty easy, but I suppose that
it could be used exclusively on the org file. I'll think a bit more
about this.

Also, I was thinking that tangling the whole package every time I
change a line of code is a bit over the top, but again maybe I should
just try it and see. Also, I suppose I could mark blocks that are not
under revision as ':tangle no' to suppress tangling them. If there is
an easy way to set the tangle arg to a filename if point is in the src
block or subtree and 'no' otherwise, that might make tangling just a
newly changed block easier.


Chuck

>
> My goal when designing these things, which might or might not appeal to you, 
> is to hold the entire project in the Org-mode file.  In the end, exporting 
> the Org-mode file to html or pdf can yield a rich description of the project, 
> independent of its product, man files, etc.  Later, when I want to make 
> changes, I know exactly where to go.
>
> All the best,
> Tom
>
>> Best,
>> 
>> Chuck
>> 
>> 
>> > Your help is highly appreciated.
>> > 
>> > Thank you in advance.
>> > 
>> > Best Regards,
>> > S.
>> > 
>> > On Mon, Dec 6, 2010 at 2:52 PM, Charles C. Berry 
>> > <cberry@tajo.ucsd.edu>wrote:
>> > 
>> > > On Sat, 4 Dec 2010, Thomas S. Dye wrote:
>> > > 
>> > > Aloha Detlef
>> > > > 
>> > > > On Dec 2, 2010, at 9:58 PM, Detlef Steuer wrote:
>> > > > 
>> > > > Hi!
>> 
>> [rest deleted]
>> 
>> 
>> Charles C. Berry                            Dept of Family/Preventive 
>> Medicine
>> cberry@tajo.ucsd.edu			    UC San Diego
>> http://famprevmed.ucsd.edu/faculty/cberry/  La Jolla, San Diego 92093-0901
>> 
>> 
>
>

Charles C. Berry                            Dept of Family/Preventive Medicine
cberry@tajo.ucsd.edu			    UC San Diego
http://famprevmed.ucsd.edu/faculty/cberry/  La Jolla, San Diego 92093-0901

Re: Re: Org-mode Code Blocks Manuscript: Request For Comments

[Charles C. Berry]

On Tue, 7 Dec 2010, Thomas S. Dye wrote:

> Aloha Chuck,
> On Dec 6, 2010, at 6:48 PM, Charles C. Berry wrote:
>

[stuff deleted]

>> 
>
> Thanks for sharing this.  It looks useful.  Would you consider putting it on 
> Worg with the other babel source block examples?
>

OK, I've put up a fresh version at

http://famprevmed.ucsd.edu/faculty/cberry/org-mode/Rpackage.org

I could not push it to Worg, in spite of my best try at following http://orgmode.org/worg/worg-git.php.
(Yes, I did email as directed.)

Chuck


Charles C. Berry                            Dept of Family/Preventive Medicine
cberry@tajo.ucsd.edu			    UC San Diego
http://famprevmed.ucsd.edu/faculty/cberry/  La Jolla, San Diego 92093-0901

Re: Re: Org-mode Code Blocks Manuscript: Request For Comments

[Thomas S. Dye]

Hi Chuck,

I put Rpackage.org up on Worg.  When you have the Worg setup worked  
out you might want to change uses.org, where your contribution is  
described.

Thanks for the contribution.

All the best,
Tom

On Dec 8, 2010, at 9:20 PM, Charles C. Berry wrote:

> On Tue, 7 Dec 2010, Thomas S. Dye wrote:
>
>> Aloha Chuck,
>> On Dec 6, 2010, at 6:48 PM, Charles C. Berry wrote:
>>
>
> [stuff deleted]
>
>>
>> Thanks for sharing this.  It looks useful.  Would you consider  
>> putting it on Worg with the other babel source block examples?
>>
>
> OK, I've put up a fresh version at
>
> http://famprevmed.ucsd.edu/faculty/cberry/org-mode/Rpackage.org
>
> I could not push it to Worg, in spite of my best try at following http://orgmode.org/worg/worg-git.php 
> .
> (Yes, I did email as directed.)
>
> Chuck
>
>
> Charles C. Berry                            Dept of Family/ 
> Preventive Medicine
> cberry@tajo.ucsd.edu			    UC San Diego
> http://famprevmed.ucsd.edu/faculty/cberry/  La Jolla, San Diego  
> 92093-0901
>
>

Re: Org-mode Code Blocks Manuscript: Request For Comments

[Christopher Allan Webber]

Hey Eric!

I've greatly enjoyed reading this paper.

One comment: Figure 1 shows its first src block as type "sh", but it's
clearly C (it's tangling to .c at the very least...)

Again, great and fun article!
 - cwebb

"Eric Schulte" <schulte.eric@gmail.com> writes:

> Hi,
>
> Dan Davison, Tom Dye, Carsten Dominik and myself have been working on a
> paper introducing Org-mode's code block functionality.  We plan to
> submit this paper to the Journal of Statistical Software.  As both
> Org-mode and the code block functionality are largely products of this
> mailing list community, and in the spirit of an open peer review process
> we are releasing the current draft of the paper here to solicit your
> review and comments.
>
> Both the .org and .pdf formats of the paper are available at the
> following locations.
>
> http://cs.unm.edu/~eschulte/org-paper/babel.org
>
> http://cs.unm.edu/~eschulte/org-paper/babel.pdf
>
> Thanks -- Eric
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Org-mode Code Blocks Manuscript: Request For Comments

[Eric Schulte]

Hi,

Sorry about my absence from this thread, I've run into a very busy week
and haven't been able to give this topic the attention it requires.

Many thanks to everyone who has given feedback, I've just finished
folding in your comments (updated copies of the .org and .pdf are now at
the original links below) and everything from misspellings to higher
level questions and suggestions were very helpful.

I'll respond to a couple of specific questions in separate threads,
however the question of venue seems to be of global interest so I'll
address that here.

The reproducible research community seems to be composed of a number of
only partially overlapping sub-communities, of these the ones of which I
am aware include biologists, physicists, economists, and statisticians.
I am not aware of any "reproducible research" publication which has
visibility into all of these sub-communities.  One of the only unifying
elements of the practice of Reproducible Research as I am aware of it is
the dominance of R and Sweave as the tools of choice.  Given these
points submitting a general paper to a journal (like JSS) with a history
of publishing RR and Sweave articles seems like a good bet.  Also JSS
has some very nice features like the fact that both the .pdf and .org
files comprising the paper could be made freely available for download
(I haven't talked to anyone at JSS, but this seems to be their policy).

It is very possible that there does exist a more appropriate venue,
however none that I have seen seem to hold significantly more promise
than JSS.  It may be the case that no single publication can
sufficiently introduce Babel (this is certainly true for Org-mode at
large), however hopefully this will serve as a good start, perhaps later
to be supplemented with smaller publications in venues targeting other
communities.

Thanks again for all the great feedback -- Eric

"Eric Schulte" <schulte.eric@gmail.com> writes:

> Hi,
>
> Dan Davison, Tom Dye, Carsten Dominik and myself have been working on a
> paper introducing Org-mode's code block functionality.  We plan to
> submit this paper to the Journal of Statistical Software.  As both
> Org-mode and the code block functionality are largely products of this
> mailing list community, and in the spirit of an open peer review process
> we are releasing the current draft of the paper here to solicit your
> review and comments.
>
> Both the .org and .pdf formats of the paper are available at the
> following locations.
>
> http://cs.unm.edu/~eschulte/org-paper/babel.org
>
> http://cs.unm.edu/~eschulte/org-paper/babel.pdf
>
> Thanks -- Eric