* Efficiency of Org v. LaTeX v. Word
@ 2014-12-26 22:47 Ken Mankoff
2014-12-26 23:36 ` Thomas S. Dye
` (7 more replies)
0 siblings, 8 replies; 21+ messages in thread
From: Ken Mankoff @ 2014-12-26 22:47 UTC (permalink / raw)
To: Org-mode mailing list
People here might be interested in a publication from [2014-12-19 Fri]
available at http://dx.doi.org/10.1371/journal.pone.0115069
Title: An Efficiency Comparison of Document Preparation Systems Used
in Academic Research and Development
Summary: Word users are more efficient and have less errors than even
experienced LaTeX users.
Someone here should repeat experiment and add Org into the mix, perhaps
Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
pandoc) beat straight Word?
-k.
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: Efficiency of Org v. LaTeX v. Word 2014-12-26 22:47 Efficiency of Org v. LaTeX v. Word Ken Mankoff @ 2014-12-26 23:36 ` Thomas S. Dye 2014-12-27 2:21 ` briangpowell . 2014-12-27 3:26 ` Christopher W. Ryan ` (6 subsequent siblings) 7 siblings, 1 reply; 21+ messages in thread From: Thomas S. Dye @ 2014-12-26 23:36 UTC (permalink / raw) To: Ken Mankoff; +Cc: Org-mode mailing list ,---------------------------------------------------------------------- | "One may also argue that given a well-designed LaTeX document class | file, document development speed and text and formatting accuracy are | significantly improved." `---------------------------------------------------------------------- Apparently, the LaTeX users didn't have the benefit of a document class. Hard to take a "study" like this seriously. ,----------------------------------------------------------------------- | "preventing researchers from producing documents in LaTeX would save | time and money to maximize the benefit of research and development for | both the research team and the public" `----------------------------------------------------------------------- All you have to lose is your freedom. All the best, Tom Ken Mankoff <mankoff@gmail.com> writes: > People here might be interested in a publication from [2014-12-19 Fri] > available at http://dx.doi.org/10.1371/journal.pone.0115069 > > Title: An Efficiency Comparison of Document Preparation Systems Used > in Academic Research and Development > > Summary: Word users are more efficient and have less errors than even > experienced LaTeX users. > > Someone here should repeat experiment and add Org into the mix, perhaps > Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume > Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via > pandoc) beat straight Word? > > -k. > > > -- Thomas S. Dye http://www.tsdye.com ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word 2014-12-26 23:36 ` Thomas S. Dye @ 2014-12-27 2:21 ` briangpowell . 2014-12-27 14:36 ` Eric S Fraga 0 siblings, 1 reply; 21+ messages in thread From: briangpowell . @ 2014-12-27 2:21 UTC (permalink / raw) To: Thomas S. Dye; +Cc: Org-mode mailing list, Ken Mankoff [-- Attachment #1: Type: text/plain, Size: 2726 bytes --] Word is a desktop publishing system. LaTeX is a macro language which lays on top of TeX=Tau-Epsilon-Chi~Art in Greek TeX is computerized typesetting that enables vector graphics--you can get TeX to draw anything you want--you can even create your own font. More Math journals and books you'd find in the library are created using TeX than any other software system. The poor kerning and severe limitations of Word are too many to number here. Word is in a different class of software, the 2 aren't comparable at all. Word is a poor WYSIWYG software package that is good for low quality desktop publishing, team collaboration but can be programmed and interacted with through VB--its useful to the general public. LaTeX provides precision and expression; there are things you can do with TeX that aren't possible with Word. Members of the Free Software community (which TeX has always been a part of) will never bow down to the Micro$oft tyranny which is so evil words can't express the depths of their corruption--the comparison is absurd. On Fri, Dec 26, 2014 at 6:36 PM, Thomas S. Dye <tsd@tsdye.com> wrote: > > ,---------------------------------------------------------------------- > | "One may also argue that given a well-designed LaTeX document class > | file, document development speed and text and formatting accuracy are > | significantly improved." > `---------------------------------------------------------------------- > > Apparently, the LaTeX users didn't have the benefit of a document > class. Hard to take a "study" like this seriously. > > ,----------------------------------------------------------------------- > | "preventing researchers from producing documents in LaTeX would save > | time and money to maximize the benefit of research and development for > | both the research team and the public" > `----------------------------------------------------------------------- > > All you have to lose is your freedom. > > All the best, > Tom > > Ken Mankoff <mankoff@gmail.com> writes: > > > People here might be interested in a publication from [2014-12-19 Fri] > > available at http://dx.doi.org/10.1371/journal.pone.0115069 > > > > Title: An Efficiency Comparison of Document Preparation Systems Used > > in Academic Research and Development > > > > Summary: Word users are more efficient and have less errors than even > > experienced LaTeX users. > > > > Someone here should repeat experiment and add Org into the mix, perhaps > > Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume > > Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via > > pandoc) beat straight Word? > > > > -k. > > > > > > > > -- > Thomas S. Dye > http://www.tsdye.com > > [-- Attachment #2: Type: text/html, Size: 3793 bytes --] ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word 2014-12-27 2:21 ` briangpowell . @ 2014-12-27 14:36 ` Eric S Fraga 0 siblings, 0 replies; 21+ messages in thread From: Eric S Fraga @ 2014-12-27 14:36 UTC (permalink / raw) To: Org-mode mailing list On Friday, 26 Dec 2014 at 21:21, briangpowell . wrote: [...] > Word is in a different class of software, the 2 aren't comparable at all. Indeed. Nonsense article, in my opinion. Comparing apples and oranges. In any case, as my writing is very highly equation based, I think I'll stick to LaTeX... (using org, of course) -- : Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.1, Org release_8.3beta-581-g0e52f0 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word 2014-12-26 22:47 Efficiency of Org v. LaTeX v. Word Ken Mankoff 2014-12-26 23:36 ` Thomas S. Dye @ 2014-12-27 3:26 ` Christopher W. Ryan 2014-12-28 22:45 ` Bob Newell 2014-12-27 4:27 ` Nick Dokos ` (5 subsequent siblings) 7 siblings, 1 reply; 21+ messages in thread From: Christopher W. Ryan @ 2014-12-27 3:26 UTC (permalink / raw) To: Org-mode mailing list This seems like more of a typing contest than anything else. Reproducing a single page of an already-typeset document is not what LaTeX is designed for, nor is it what scientists do for a living. The test selections were absurdly short relative to the typical scientic manuscript. Long and complex documents are where LaTeX excels. And this did not call upon some of the most important (IMHO) capabilities of LaTeX: managing citations with BibTex; changing the style to suit different journals; storing, revisiting, and reusing your document years later. --Chris Ryan Ken Mankoff wrote: > > People here might be interested in a publication from [2014-12-19 Fri] > available at http://dx.doi.org/10.1371/journal.pone.0115069 > > Title: An Efficiency Comparison of Document Preparation Systems Used > in Academic Research and Development > > Summary: Word users are more efficient and have less errors than even > experienced LaTeX users. > > Someone here should repeat experiment and add Org into the mix, perhaps > Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume > Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via > pandoc) beat straight Word? > > -k. > > ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word 2014-12-27 3:26 ` Christopher W. Ryan @ 2014-12-28 22:45 ` Bob Newell 0 siblings, 0 replies; 21+ messages in thread From: Bob Newell @ 2014-12-28 22:45 UTC (permalink / raw) To: Org-mode mailing list The "study" is an obvious diatribe couched in (poorly done) scientific method. It almost seems like these researchers have at some time been required to use LaTeX and are angry over it. I will agree that LaTeX is slower and less efficient than LibreOffice (I don't have Word on any of my computers but the argument I suppose is the same), if your only interest is pounding out text of a first draft. But the "study" didn't allow for the effects of proofreading, etc., and all the things that are always done when producing something for publication. I expect that the differences in grammar and orthographic errors will be insignificant. A 30-minute test is ridiculous. More meaningful would have been end-to-end time to complete a given document. Maybe then Word would have still been faster; I don't know. But that doesn't tell the whole story by any means, including the very important matter of long-term storage in a non-proprietary format. -- Bob Newell Honolulu, Hawai`i * Sent via Ma Gnus 0.12-Emacs 24.3-Linux Mint 17 * ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word 2014-12-26 22:47 Efficiency of Org v. LaTeX v. Word Ken Mankoff 2014-12-26 23:36 ` Thomas S. Dye 2014-12-27 3:26 ` Christopher W. Ryan @ 2014-12-27 4:27 ` Nick Dokos 2014-12-27 9:06 ` Peter Neilson 2014-12-27 9:48 ` Achim Gratz ` (4 subsequent siblings) 7 siblings, 1 reply; 21+ messages in thread From: Nick Dokos @ 2014-12-27 4:27 UTC (permalink / raw) To: emacs-orgmode Ken Mankoff <mankoff@gmail.com> writes: > People here might be interested in a publication from [2014-12-19 Fri] > available at http://dx.doi.org/10.1371/journal.pone.0115069 > > Title: An Efficiency Comparison of Document Preparation Systems Used > in Academic Research and Development > > Summary: Word users are more efficient and have less errors than even > experienced LaTeX users. > > Someone here should repeat experiment and add Org into the mix, perhaps > Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume > Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via > pandoc) beat straight Word? > The "study" is deeply flawed: Word users typed more text in 30 minutes than LaTeX users? 20% more? For straight text? I don't believe it: I think it's much more likely that the Word users were better typists on average and I didn't see any mention of normalizing the results by taking that into account. And for LaTeX, the editing environment is of paramount importance: did they mostly use vi, emacs, emacs+auctex? Other than a couple of vague sentences in the "Discussion" section, there is no mention of how this variable was (or was not) controlled. And did the LaTeX users have to type the preamble or were they allowed to use a template? It just seems unbelievable that there is such a big difference for straight text. I can believe perhaps that typing a table by hand into LaTeX is more error prone than typing it into Word (although to be honest, I have never done the latter, so I don't really know). With org and radio tables, this would be a non-issue on the LaTeX side. As for equations, even the authors admit that LaTeX is better, although they tend to minimize the differences as statistically insignificant (at least between the expert classes), which strikes me as somewhat suspect as well: there seems to be a 10% difference between the expert user averages and a bigger one for novices, although the error bars might overlap in the first case (although they don't look it). I don't think that even that would make the difference insignificant, but we'd have to analyze their raw results to make sure (which they do provide and which I took a look at, but afaict they don't provide answers to the questions I raised above; maybe we should suggest that the authors use org and reproducible results methods). Anyway, color me deeply suspicious of the "study". -- Nick ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word 2014-12-27 4:27 ` Nick Dokos @ 2014-12-27 9:06 ` Peter Neilson 2014-12-27 14:38 ` Eric S Fraga 0 siblings, 1 reply; 21+ messages in thread From: Peter Neilson @ 2014-12-27 9:06 UTC (permalink / raw) To: emacs-orgmode On Fri, 26 Dec 2014 23:27:37 -0500, Nick Dokos <ndokos@gmail.com> wrote: > Anyway, color me deeply suspicious of the "study". Indeed! The study touches only a few of the inherent difficulties in document production. Its major flaw is that it draws any conclusions at all recommending that authors produce documents one way or another. Personally I am always disappointed when someone requests a document in MS Word format, because that means I'll have to fire up Libre Office and shove my text through it, rather than using whatever other system I happen to have been using. I do not believe that I currently own a system with genuine MS Word. As well as having insufficient control of variables, and a flawed understanding of what is involved in "document preparation," the study also has a marginally small sample size. Any study for any purpose that presents "statistics" with sample sizes smaller than 30 is immediately suspect. I won't even begin to address the misinterpretation of correlation as causation that appears in the "softer" sciences, nor their necessity for sample sizes far larger than 100, nor the tendency in some fields to mistake a time series as a set of samples. MS Word works extremely well for "one-off" small papers. Little investment of effort is required for a naive person to produce adequate results, and as every user of emacs knows, that's pretty much the opposite of emacs. On the other hand, MS Word has historically been a terrible tool for producing large documents, or documents that are to be maintained by a group of people, or over several years or decades. Handling Word's "Master Document" provision without being crippled by corrupted documents is an art form unto itself. The standard advice among experienced users of Word has always been, "Don't Use Master Documents!" When a group of people are all editing versions of a document, any attempt to use standard formatting in Word requires substantial effort to prevent naive contributers from reformatting outside the established styles, or even breaking all the styles. Furthermore, Word documents are in general not amenable to incremental version control as commonly used by coding teams. My conclusions? If your paper is trivial and you are under pressure to produce it quickly, then MS Word might be the best tool. Established journals should attempt to allow contributions in more than one format, and restriction to MS Word format is a bad idea, no matter how much some people like the apparent ease-of-use that MS Word provides. Attempting to extend the "study" to include org mode would be a waste of effort. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word 2014-12-27 9:06 ` Peter Neilson @ 2014-12-27 14:38 ` Eric S Fraga 0 siblings, 0 replies; 21+ messages in thread From: Eric S Fraga @ 2014-12-27 14:38 UTC (permalink / raw) To: Peter Neilson; +Cc: emacs-orgmode On Saturday, 27 Dec 2014 at 04:06, Peter Neilson wrote: [...] > My conclusions? If your paper is trivial and you are under pressure to > produce it quickly, then MS Word might be the best tool. Actually, I don't think I can get any faster than using org for a trivial paper needed quickly... I'm in emacs already ;-) -- : Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.1, Org release_8.3beta-581-g0e52f0 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word 2014-12-26 22:47 Efficiency of Org v. LaTeX v. Word Ken Mankoff ` (2 preceding siblings ...) 2014-12-27 4:27 ` Nick Dokos @ 2014-12-27 9:48 ` Achim Gratz 2014-12-27 10:05 ` Paul Rudin ` (3 subsequent siblings) 7 siblings, 0 replies; 21+ messages in thread From: Achim Gratz @ 2014-12-27 9:48 UTC (permalink / raw) To: emacs-orgmode Am 26.12.2014 um 23:47 schrieb Ken Mankoff: > People here might be interested in a publication from [2014-12-19 Fri] > available at http://dx.doi.org/10.1371/journal.pone.0115069 > > Title: An Efficiency Comparison of Document Preparation Systems Used > in Academic Research and Development > > Summary: Word users are more efficient and have less errors than even > experienced LaTeX users. The way researcher efficiency is defined in that "study" completely misses the purpose of scientific publishing and it goes downhill from there. The statistics are pseudo-scientific smokes and mirrors, no control groups, no normalization and not a single hint of why it should be acceptable to use normal distributions for something that clearly isn't normally distributed other than the obvious convenience of drawing wild conclusions from a small sample size. I'm still not sure if this isn't an elaborate joke, but I'm afraid not. > Someone here should repeat experiment and add Org into the mix, perhaps > Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume > Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via > pandoc) beat straight Word? Repeating a deeply flawed "study" that seems designed to support some pre-conceived notion or preference of the authors isn't going to produce any new insights and I'm quite certain that there is better research into the differences of WYSIWIG vs. non-WYSIWIG publication systems and/or researcher efficiency. If a reasearcher is nothing more than a typist that needs to produce pages of texts, tables and equations in a prescribed format in the least amount of time motivated by a monetary prize, we wouldn't need researchers at all. That would incidentally save much more money than having them all switch from LaTeX to Word, so let's stop funding research. -- Achim. (on the road :-) ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word 2014-12-26 22:47 Efficiency of Org v. LaTeX v. Word Ken Mankoff ` (3 preceding siblings ...) 2014-12-27 9:48 ` Achim Gratz @ 2014-12-27 10:05 ` Paul Rudin 2014-12-27 10:36 ` M 2014-12-27 13:37 ` Daniele Pizzolli ` (2 subsequent siblings) 7 siblings, 1 reply; 21+ messages in thread From: Paul Rudin @ 2014-12-27 10:05 UTC (permalink / raw) To: emacs-orgmode-mXXj517/zsQ Ken Mankoff <mankoff-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes: > People here might be interested in a publication from [2014-12-19 Fri] > available at http://dx.doi.org/10.1371/journal.pone.0115069 > > Title: An Efficiency Comparison of Document Preparation Systems Used > in Academic Research and Development > > Summary: Word users are more efficient and have less errors than even > experienced LaTeX users. > > Someone here should repeat experiment and add Org into the mix, perhaps > Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume > Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via > pandoc) beat straight Word? > No mention of emacs... who uses anything else to prepare their LaTeX? ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word 2014-12-27 10:05 ` Paul Rudin @ 2014-12-27 10:36 ` M 2014-12-27 11:36 ` Fabrice Popineau 2014-12-31 18:19 ` Paul Rudin 0 siblings, 2 replies; 21+ messages in thread From: M @ 2014-12-27 10:36 UTC (permalink / raw) To: emacs orgmode-mailinglist; +Cc: Paul Rudin > Von: Paul Rudin <paul@rudin.co.uk> > Datum: Sat, 27 Dec 2014 10:05:19 +0000 > An: <emacs-orgmode@gnu.org> > Betreff: Re: [O] Efficiency of Org v. LaTeX v. Word > > Ken Mankoff <mankoff@gmail.com> writes: > >> People here might be interested in a publication from [2014-12-19 Fri] >> available at http://dx.doi.org/10.1371/journal.pone.0115069 >> >> Title: An Efficiency Comparison of Document Preparation Systems Used >> in Academic Research and Development >> >> Summary: Word users are more efficient and have less errors than even >> experienced LaTeX users. >> >> Someone here should repeat experiment and add Org into the mix, perhaps >> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume >> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via >> pandoc) beat straight Word? >> > > No mention of emacs... who uses anything else to prepare their LaTeX? > Did you forget the " ;-)" or are you serious? Emacs is for sure a very good one, but there are a lot of popular alternatives, if you have a look at the (for sure not representative) voting on the answers of this discussion here: http://tex.stackexchange.com/questions/339/latex-editors-ides (It's clear, that people may have voted for several of those editors, so that no valid statistics at all, but at least an idea...) Is there any real survey result about which editors LaTeX users use? Martin ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word 2014-12-27 10:36 ` M @ 2014-12-27 11:36 ` Fabrice Popineau 2014-12-28 22:43 ` Pascal Fleury 2014-12-31 18:19 ` Paul Rudin 1 sibling, 1 reply; 21+ messages in thread From: Fabrice Popineau @ 2014-12-27 11:36 UTC (permalink / raw) To: M; +Cc: Paul Rudin, emacs orgmode-mailinglist [-- Attachment #1: Type: text/plain, Size: 3744 bytes --] I agree that this study is certainly not large enough to draw strong conclusions, but it raises a couple of questions and some points may require attention. I have spent many years in the TeX world. I see how lots of people use TeX : students, professionals, researchers etc... and I would easily draw 2 categories of people : - those who are programmers "in their soul" (DEK once said that 2% or so of the whole human race is gifted with programming, the same way some people are gifted to play music etc.) - those who use LaTeX "because it is the best typesetting system" People who belong to the intersection of those 2 categories will certainly be very efficient in producing documents with LaTeX, much more than what this study shows. But people from the first category may also be efficient in producing documents with Word (Word is programmable too and the typesetting engine is fancier than most people would believe). The real problem is the guys from the second category who stick to use a tool they are not comfortable with but they don't want to admit it. Over the last years, I have seen more and more students come with LaTeX documents which had a very poor appearance. There has been a lot of pressure with the rise of Linux to use LaTeX. Unfortunately the results of using LaTeX may not be up to the expectations. The tool is too complex. It can produce beautiful documents when used right, but it can also easily produce awful documents. You can also spend a lot of time in fixing details, and it happens more frequently than even proficient LaTeX users would admit. In the end, I think the tendency is to a growing number of LaTeX users who use it poorly. Finally, today, my experience is that publishers charge much more for LaTeX documents than for Word (or similar tools) documents and they are reluctant to use LaTeX because of its complexity. That was my $0.02 Fabrice 2014-12-27 11:36 GMT+01:00 M <Elwood151@web.de>: > > Von: Paul Rudin <paul@rudin.co.uk> > > Datum: Sat, 27 Dec 2014 10:05:19 +0000 > > An: <emacs-orgmode@gnu.org> > > Betreff: Re: [O] Efficiency of Org v. LaTeX v. Word > > > > Ken Mankoff <mankoff@gmail.com> writes: > > > >> People here might be interested in a publication from [2014-12-19 Fri] > >> available at http://dx.doi.org/10.1371/journal.pone.0115069 > >> > >> Title: An Efficiency Comparison of Document Preparation Systems Used > >> in Academic Research and Development > >> > >> Summary: Word users are more efficient and have less errors than even > >> experienced LaTeX users. > >> > >> Someone here should repeat experiment and add Org into the mix, perhaps > >> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume > >> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via > >> pandoc) beat straight Word? > >> > > > > No mention of emacs... who uses anything else to prepare their LaTeX? > > > Did you forget the " ;-)" or are you serious? > > Emacs is for sure a very good one, but there are a lot of popular > alternatives, if you have a look at the (for sure not representative) > voting > on the answers of this discussion here: > > http://tex.stackexchange.com/questions/339/latex-editors-ides > > (It's clear, that people may have voted for several of those editors, so > that no valid statistics at all, but at least an idea...) > > Is there any real survey result about which editors LaTeX users use? > > Martin > > > > -- Fabrice Popineau ----------------------------- SUPELEC Département Informatique 3, rue Joliot Curie 91192 Gif/Yvette Cedex Tel direct : +33 (0) 169851950 Standard : +33 (0) 169851212 ------------------------------ [-- Attachment #2: Type: text/html, Size: 4980 bytes --] ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word 2014-12-27 11:36 ` Fabrice Popineau @ 2014-12-28 22:43 ` Pascal Fleury 0 siblings, 0 replies; 21+ messages in thread From: Pascal Fleury @ 2014-12-28 22:43 UTC (permalink / raw) To: Fabrice Popineau; +Cc: M, Paul Rudin, emacs orgmode-mailinglist [-- Attachment #1: Type: text/plain, Size: 4685 bytes --] On Sat, Dec 27, 2014 at 12:36 PM, Fabrice Popineau < fabrice.popineau@supelec.fr> wrote: > I agree that this study is certainly not large enough to draw strong > conclusions, but it raises a couple of questions > and some points may require attention. > > I have spent many years in the TeX world. I see how lots of people use TeX > : students, professionals, researchers etc... > and I would easily draw 2 categories of people : > - those who are programmers "in their soul" (DEK once said that 2% or so > of the whole human race is gifted with programming, the same way some > people are gifted to play music etc.) > - those who use LaTeX "because it is the best typesetting system" > People who belong to the intersection of those 2 categories will certainly > be very efficient in producing documents with LaTeX, much more than what > this study shows. > But people from the first category may also be efficient in producing > documents with Word (Word is programmable too and the typesetting engine is > fancier than most people would believe). > That is funny, as I still face regularly Word typeset documents that do not handle orphan lines properly, and have at least 2 fonts as "body text". Easy to fix, but a non-issue in Latex. As a researcher, handling references and cross-references is not something that is "amortized" on a one-off paper, it's something that pays off over a few documents. And in a publish-or-perish world, this does usually not take long. As a programmer, I like to be able to run one command (call it 'make' if you wish...) that will run some analysis and recompute both the figures and the document into a new version, possibly versionned. And now you know why I use orgmode too... --paf > The real problem is the guys from the second category who stick to use a > tool they are not comfortable with but they don't want to admit it. > Over the last years, I have seen more and more students come with LaTeX > documents which had a very poor appearance. > There has been a lot of pressure with the rise of Linux to use LaTeX. > Unfortunately the results of using LaTeX may not be up to the expectations. > The tool is too complex. It can produce beautiful documents when used > right, but it can also easily produce awful documents. > You can also spend a lot of time in fixing details, and it happens more > frequently than even proficient LaTeX users would admit. > In the end, I think the tendency is to a growing number of LaTeX users who > use it poorly. > > Finally, today, my experience is that publishers charge much more for > LaTeX documents than for Word (or similar tools) documents and they are > reluctant to use LaTeX because of its complexity. > > That was my $0.02 > > Fabrice > > 2014-12-27 11:36 GMT+01:00 M <Elwood151@web.de>: > >> > Von: Paul Rudin <paul@rudin.co.uk> >> > Datum: Sat, 27 Dec 2014 10:05:19 +0000 >> > An: <emacs-orgmode@gnu.org> >> > Betreff: Re: [O] Efficiency of Org v. LaTeX v. Word >> > >> > Ken Mankoff <mankoff@gmail.com> writes: >> > >> >> People here might be interested in a publication from [2014-12-19 Fri] >> >> available at http://dx.doi.org/10.1371/journal.pone.0115069 >> >> >> >> Title: An Efficiency Comparison of Document Preparation Systems Used >> >> in Academic Research and Development >> >> >> >> Summary: Word users are more efficient and have less errors than even >> >> experienced LaTeX users. >> >> >> >> Someone here should repeat experiment and add Org into the mix, perhaps >> >> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume >> >> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via >> >> pandoc) beat straight Word? >> >> >> > >> > No mention of emacs... who uses anything else to prepare their LaTeX? >> > >> Did you forget the " ;-)" or are you serious? >> >> Emacs is for sure a very good one, but there are a lot of popular >> alternatives, if you have a look at the (for sure not representative) >> voting >> on the answers of this discussion here: >> >> http://tex.stackexchange.com/questions/339/latex-editors-ides >> >> (It's clear, that people may have voted for several of those editors, so >> that no valid statistics at all, but at least an idea...) >> >> Is there any real survey result about which editors LaTeX users use? >> >> Martin >> >> >> >> > > > -- > Fabrice Popineau > ----------------------------- > SUPELEC > Département Informatique > 3, rue Joliot Curie > 91192 Gif/Yvette Cedex > Tel direct : +33 (0) 169851950 > Standard : +33 (0) 169851212 > ------------------------------ > > [-- Attachment #2: Type: text/html, Size: 6655 bytes --] ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word 2014-12-27 10:36 ` M 2014-12-27 11:36 ` Fabrice Popineau @ 2014-12-31 18:19 ` Paul Rudin 1 sibling, 0 replies; 21+ messages in thread From: Paul Rudin @ 2014-12-31 18:19 UTC (permalink / raw) To: emacs-orgmode-mXXj517/zsQ M <Elwood151-S0/GAf8tV78@public.gmane.org> writes: >> Von: Paul Rudin <paul-sqPYmOVXOov10XsdtD+oqA@public.gmane.org> >> >> No mention of emacs... who uses anything else to prepare their LaTeX? >> > Did you forget the " ;-)" or are you serious? I wasn't being entirely serious; but I was alluding to a serious point. You can't really compare a command line typesetting system alone with a word processor. To make a proper comparison you'd have to look at the complete toolchain. For example, some of the errors are typos. Word, of course, has a speil chucker. Did the LaTeX users use an editor that highlights such errors? But as others have pointed out the more fundamental problem with the study is that it tries to assess secretarial or copy-editing skills rather than authoring skills. (I haven't actually read the paper, just what has been said in this thread.) ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word 2014-12-26 22:47 Efficiency of Org v. LaTeX v. Word Ken Mankoff ` (4 preceding siblings ...) 2014-12-27 10:05 ` Paul Rudin @ 2014-12-27 13:37 ` Daniele Pizzolli 2014-12-28 21:40 ` Efficiency of Org v. LaTeX v. Word ---LOOK AT THE DATA! Christophe Pouzat 2015-01-04 20:38 ` Efficiency of Org v. LaTeX v. Word John Kitchin 7 siblings, 0 replies; 21+ messages in thread From: Daniele Pizzolli @ 2014-12-27 13:37 UTC (permalink / raw) To: Org-mode mailing list Hello, Ken Mankoff writes: > People here might be interested in a publication from [2014-12-19 Fri] > available at http://dx.doi.org/10.1371/journal.pone.0115069 > > Title: An Efficiency Comparison of Document Preparation Systems Used > in Academic Research and Development > > Summary: Word users are more efficient and have less errors than even > experienced LaTeX users. As other said, the efficiency in the paper is about the manual copy of a small portion of text, tables... This is a little bit different to publish a research, maybe a reproducible one, with the help of a team. > Someone here should repeat experiment and add Org into the mix, perhaps > Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume > Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via > pandoc) beat straight Word? Repeating a flawed experiment do not add a lot of value... Best, Daniele ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word ---LOOK AT THE DATA! 2014-12-26 22:47 Efficiency of Org v. LaTeX v. Word Ken Mankoff ` (5 preceding siblings ...) 2014-12-27 13:37 ` Daniele Pizzolli @ 2014-12-28 21:40 ` Christophe Pouzat 2014-12-29 19:47 ` Thomas S. Dye 2014-12-31 16:59 ` Colin Baxter 2015-01-04 20:38 ` Efficiency of Org v. LaTeX v. Word John Kitchin 7 siblings, 2 replies; 21+ messages in thread From: Christophe Pouzat @ 2014-12-28 21:40 UTC (permalink / raw) To: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 13854 bytes --] Hi all, After seeing Ken's mail: Le 26/12/2014 23:47, Ken Mankoff a écrit : > People here might be interested in a publication from [2014-12-19 Fri] > available at http://dx.doi.org/10.1371/journal.pone.0115069 > > Title: An Efficiency Comparison of Document Preparation Systems Used > in Academic Research and Development > > Summary: Word users are more efficient and have less errors than even > experienced LaTeX users. > > Someone here should repeat experiment and add Org into the mix, perhaps > Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume > Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via > pandoc) beat straight Word? > > -k. > > and some of replies it triggered on the list, I went to check the paper. As many of you guys I found some "results" puzzling in particular: 1. the use of bar graphs when the data would better be displayed directly (that qualifies immediately the paper as "low quality" for me). 2. the larger error bars observed for LaTeX when compared to Word. 3. the systematic inverse relationship between the blue and pink bars heights. So I went to figshare to download the data and looked at them. A quick and dirty "analysis" is attached to this mail in PDF format (generated with org, of course, and this awful software called LaTeX!) and the source org file can be found at the bottom of this mail. I used R to do the figures (and I'm sure the authors of the paper will then criticize me for not using Excel with which everyone knows errors are generated much more efficiently). I managed to understand the inverse relationship in point 3 above: the authors considered 3 types of mistakes / errors: 1. Formatting and typos error. 2. Orthographic and grammatical errors. 3. Missing words and signs. Clearly, following the mail of Tom (Dye) on the list and on the Plos web site, I would argue that formatting errors in LaTeX are bona fide bugs. But the point I want to make is that the third source accounts for 80% of the total errors (what's shown in pink bars in the paper) and clearly the authors counted what the subjects did not have time to type as an error of this type. Said differently, the blue and pink bars are showing systematically the same thing by construction! The second type of error in not a LaTeX issue (and in fact does not differ significantly from the Word case) but an "environment" issue (what spelling corrector had the LaTeX users access to?). There is another strange thing in the table copy case. For both the expert and novice group in LaTeX, there is one among 10 subjects that did produce 0% of the table but still manage to produce 22 typographic errors! The overall worst performance of LaTeX users remains to be explained and as mentioned in on the mails in the list, that does not make sense at least for the continuous text exercise. The method section of the paper is too vague but my guess is that some LaTeX users did attempt to reproduce the exact layout of the text they had to copy, something LaTeX is definitely not design to provide quickly. One more point: how many of you guys could specify their total number of hours of experience with LaTeX (or any other software you are currently using)? That what the subjects of this study had to specify... Let me know what you think, Christophe My org buffer: #+TITLE: An Efficiency Comparison of Document Preparation Systems Used in Academic Research and Development: A Re-analysis. #+DATE: <2014-12-28 dim.> #+AUTHOR: Christophe Pouzat #+EMAIL: christophe.pouzat@gmail.com #+OPTIONS: ':nil *:t -:t ::t <:t H:3 \n:nil ^:t arch:headline #+OPTIONS: author:t c:nil creator:comment d:(not "LOGBOOK") date:t #+OPTIONS: e:t email:nil f:t inline:t num:t p:nil pri:nil stat:t #+OPTIONS: tags:t tasks:t tex:t timestamp:t toc:nil todo:t |:t #+CREATOR: Emacs 24.4.1 (Org mode 8.2.10) #+DESCRIPTION: #+EXCLUDE_TAGS: noexport #+KEYWORDS: #+LANGUAGE: en #+SELECT_TAGS: export #+LaTeX_HEADER: \usepackage{alltt} #+LaTeX_HEADER: \usepackage[usenames,dvipsnames]{xcolor} #+LaTeX_HEADER: \renewenvironment{verbatim}{\begin{alltt} \scriptsize \color{Bittersweet} \vspace{0.2cm} }{\vspace{0.2cm} \end{alltt} \normalsize \color{black}} #+LaTeX_HEADER: \definecolor{lightcolor}{gray}{.55} #+LaTeX_HEADER: \definecolor{shadecolor}{gray}{.85} #+LaTeX_HEADER: \usepackage{minted} #+LaTeX_HEADER: \hypersetup{colorlinks=true} #+NAME: org-latex-set-up #+BEGIN_SRC emacs-lisp :results silent :exports none (setq org-latex-listings 'minted) (setq org-latex-minted-options '(("bgcolor" "shadecolor") ("fontsize" "\\scriptsize"))) (setq org-latex-pdf-process '("pdflatex -shell-escape -interaction nonstopmode -output-directory %o %f" "biber %b" "pdflatex -shell-escape -interaction nonstopmode -output-directory %o %f" "pdflatex -shell-escape -interaction nonstopmode -output-directory %o %f")) #+END_SRC * Introduction This is a re-analysis of the data presented in [[http://dx.doi.org/10.1371/journal.pone.0115069][An Efficiency Comparison of Document Preparation Systems Used in Academic Research and Development]]. My "interest" in this paper was triggered by a discussion on the [[http://article.gmane.org/gmane.emacs.orgmode/93655][emacs org mode mailing list]]. Ignoring the "message" of the paper, what stroke me was the systematic use of bar graphs: a way of displaying data that *should never be used* since when many observations are considered, a box plot is going to do a much better job and when, like in the present paper, few (10 in each of the 4 categories) observations are available, a direct display or even a simple table is going to do a *much better* job. Since it turns out that the data are available both on the Plos web site and on [[http://figshare.com/articles/_An_Efficiency_Comparison_of_Document_Preparation_Systems_Used_in_Academic_Research_and_Development_/1275631][figshare]], I decided to re-analyze them. * Getting the data, etc. We get the data with: #+BEGIN_SRC sh wget http://files.figshare.com/1849394/S1_Materials.xlsx #+END_SRC #+RESULTS: Using for instance [[http://dag.wiee.rs/home-made/unoconv/][unoconv]], we can convert the =Excel= file into a friendlier =csv= file: #+BEGIN_SRC sh unoconv -f csv S1_Materials.xlsx #+END_SRC #+RESULTS: We then get the data with =R= =read.csv= function: #+NAME: data-table #+BEGIN_SRC R :session *R* :results silent efficiency <- read.csv("S1_Materials.csv",header=TRUE,dec=",") #+END_SRC The description of this table is obtained with: #+BEGIN_SRC sh :exports both :results output wget http://files.figshare.com/1849395/S2_Materials.txt cat "S2_Materials.txt" #+END_SRC * Making some figures We can now make a figure out of the same data as figures 4, 5 and 6 of the paper but showing the actual data. We start with the "continuous text" exercise. We represent, in each of the four categories, each of the 10 individuals by a number between 0 and 9. Some horizontal jitter has been added to avoid overlaps. Category 1 corresponds to expert =Word= users; 2 to novice =Word= users; 3 to expert \LaTeX{} users; 4 to novice \LaTeX{} users: #+HEADER: :file continuous.png :width 1000 :height 1000 #+BEGIN_SRC R :session *R* :exports both :results output graphics layout(matrix(1:4,nc=2,byrow=TRUE)) par(cex=2) plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=c(0,100), xlab="User category",ylab="",main="Fraction of text") with(efficiency, sapply(1:4, function(k) points(runif(10,k-0.2,k+0.2), PROZENT1[Kenntnisse==k], pch = paste(0:9)))) with(efficiency, plot(c(1,4),c(0,100),type="n", xlim=c(0.5,4.5),ylim=range(FEHLERSFT),xlab="User category", ylab="",main="Formatting errors and typos")) with(efficiency, sapply(1:4, function(k) points(runif(10,k-0.2,k+0.2), FEHLERSFT[Kenntnisse==k], pch = paste(0:9)))) with(efficiency, plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5), ylim=range(FEHLEROFT),xlab="User category",ylab="", main="Orthographic and grammatical mistakes")) with(efficiency, sapply(1:4, function(k) points(runif(10,k-0.2,k+0.2), FEHLEROFT[Kenntnisse==k], pch = paste(0:9)))) with(efficiency, plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=range(FEHLENDFT), xlab="User category",ylab="",main="Missing words and signs")) with(efficiency, sapply(1:4, function(k) points(runif(10,k-0.2,k+0.2), FEHLENDFT[Kenntnisse==k], pch = paste(0:9)))) #+END_SRC Notice that the number of "missing words and signs" exactly mirrors the fraction of written text. We will see that this observation holds for the two following exercises. This "missing words and signs" is always roughly ten times as large as the two other sources of mistakes. This explains the inverse relationship between the blue and pink bars on each of the 3 figures. Let's keep going with the "table exercise": #+HEADER: :file table.png :width 1000 :height 1000 #+BEGIN_SRC R :session *R* :exports both :results output graphics layout(matrix(1:4,nc=2,byrow=TRUE)) par(cex=2) plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=c(0,100), xlab="User category",ylab="",main="Fraction of text") with(efficiency,sapply(1:4, function(k) points(runif(10,k-0.2,k+0.2), PROZENT2[Kenntnisse==k], pch = paste(0:9)))) with(efficiency,plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5), ylim=range(FEHLERST),xlab="User category", ylab="",main="Formatting errors and typos")) with(efficiency,sapply(1:4, function(k) points(runif(10,k-0.2,k+0.2), FEHLERST[Kenntnisse==k], pch = paste(0:9)))) with(efficiency,plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5), ylim=range(FEHLEROT),xlab="User category", ylab="",main="Orthographic and grammatical mistakes")) with(efficiency,sapply(1:4, function(k) points(runif(10,k-0.2,k+0.2), FEHLEROT[Kenntnisse==k], pch = paste(0:9)))) with(efficiency,plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5), ylim=range(FEHLENDT),xlab="User category",ylab="", main="Missing words and signs")) with(efficiency,sapply(1:4, function(k) points(runif(10,k-0.2,k+0.2), FEHLENDT[Kenntnisse==k], pch = paste(0:9)))) #+END_SRC We also see a strange thing here: in each of the expert \LaTeX{} and the novice \LaTeX{} users we have one individual who did not right anything but still manage to produce 22 "formatting errors and typos" (!) but luckily no orthographic or grammatical error... #+BEGIN_SRC R :session *R* :exports both with(efficiency,cbind(c(PROZENT2[Kenntnisse==3][10], FEHLERST[Kenntnisse==3][10], FEHLEROT[Kenntnisse==3][10], FEHLENDT[Kenntnisse==3][10]), c(PROZENT2[Kenntnisse==4][7], FEHLERST[Kenntnisse==4][7], FEHLEROT[Kenntnisse==4][7], FEHLENDT[Kenntnisse==4][7]))) #+END_SRC Now for the "equations" exercise: #+HEADER: :file equation.png :width 1000 :height 1000 #+BEGIN_SRC R :session *R* :exports both :results output graphics layout(matrix(1:4,nc=2,byrow=TRUE)) par(cex=2) plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=c(0,100), xlab="User category",ylab="",main="Fraction of text") with(efficiency, sapply(1:4, function(k) points(runif(10,k-0.2,k+0.2), PROZENT3[Kenntnisse==k], pch = paste(0:9)))) with(efficiency, plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5), ylim=range(FEHLERSFOR),xlab="User category",ylab="", main="Formatting errors and typos")) with(efficiency, sapply(1:4, function(k) points(runif(10,k-0.2,k+0.2), FEHLERSFOR[Kenntnisse==k], pch = paste(0:9)))) with(efficiency, plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=range(FEHLEROFOR), xlab="User category",ylab="", main="Orthographic and grammatical mistakes")) with(efficiency, sapply(1:4, function(k) points(runif(10,k-0.2,k+0.2), FEHLEROFOR[Kenntnisse==k], pch = paste(0:9)))) with(efficiency, plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5), ylim=range(FEHLENDFOR),xlab="User category",ylab="", main="Missing words and signs")) with(efficiency, sapply(1:4, function(k) points(runif(10,k-0.2,k+0.2), FEHLENDFOR[Kenntnisse==k], pch = paste(0:9)))) #+END_SRC -- A Master Carpenter has many tools and is expert with most of them. If you only know how to use a hammer, every problem starts to look like a nail. Stay away from that trap. Richard B Johnson. -- Christophe Pouzat MAP5 - Mathématiques Appliquées à Paris 5 CNRS UMR 8145 45, rue des Saints-Pères 75006 PARIS France tel: +33142863828 mobile: +33662941034 web: http://xtof.disque.math.cnrs.fr [-- Attachment #2: EfficiencyComparison.pdf --] [-- Type: application/pdf, Size: 218418 bytes --] ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word ---LOOK AT THE DATA! 2014-12-28 21:40 ` Efficiency of Org v. LaTeX v. Word ---LOOK AT THE DATA! Christophe Pouzat @ 2014-12-29 19:47 ` Thomas S. Dye 2014-12-31 16:59 ` Colin Baxter 1 sibling, 0 replies; 21+ messages in thread From: Thomas S. Dye @ 2014-12-29 19:47 UTC (permalink / raw) To: Christophe Pouzat; +Cc: emacs-orgmode Aloha Christophe, I think you make a good case for the authors' poor choice of metrics. These aren't well defined in the paper, so it is enlightening to see your graphics and learn how their metrics were ineptly designed. I hope you'll make your findings known to the PLOS audience. This looks to me like a clear case of peer review failure (see http://www.sciencemag.org/content/342/6154/60.summary for an eye-opening look at what's become of peer review.) All the best, Tom Christophe Pouzat <christophe.pouzat@gmail.com> writes: > Hi all, > > After seeing Ken's mail: > > Le 26/12/2014 23:47, Ken Mankoff a écrit : >> People here might be interested in a publication from [2014-12-19 Fri] >> available at http://dx.doi.org/10.1371/journal.pone.0115069 >> >> Title: An Efficiency Comparison of Document Preparation Systems Used >> in Academic Research and Development >> >> Summary: Word users are more efficient and have less errors than even >> experienced LaTeX users. >> >> Someone here should repeat experiment and add Org into the mix, perhaps >> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume >> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via >> pandoc) beat straight Word? >> >> -k. >> >> > and some of replies it triggered on the list, I went to check the paper. > As many of you guys I found some "results" puzzling in particular: > 1. the use of bar graphs when the data would better be displayed > directly (that qualifies immediately the paper as "low quality" for me). > 2. the larger error bars observed for LaTeX when compared to Word. > 3. the systematic inverse relationship between the blue and pink bars > heights. > > So I went to figshare to download the data and looked at them. A quick > and dirty "analysis" is attached to this mail in PDF format (generated > with org, of course, and this awful software called LaTeX!) and the > source org file can be found at the bottom of this mail. I used R to do > the figures (and I'm sure the authors of the paper will then criticize > me for not using Excel with which everyone knows errors are generated > much more efficiently). > > I managed to understand the inverse relationship in point 3 above: the > authors considered 3 types of mistakes / errors: > 1. Formatting and typos error. > 2. Orthographic and grammatical errors. > 3. Missing words and signs. > Clearly, following the mail of Tom (Dye) on the list and on the Plos web > site, I would argue that formatting errors in LaTeX are bona fide bugs. > But the point I want to make is that the third source accounts for 80% > of the total errors (what's shown in pink bars in the paper) and clearly > the authors counted what the subjects did not have time to type as an > error of this type. Said differently, the blue and pink bars are showing > systematically the same thing by construction! The second type of error > in not a LaTeX issue (and in fact does not differ significantly from the > Word case) but an "environment" issue (what spelling corrector had the > LaTeX users access to?). > > There is another strange thing in the table copy case. For both the > expert and novice group in LaTeX, there is one among 10 subjects that > did produce 0% of the table but still manage to produce 22 typographic > errors! > > The overall worst performance of LaTeX users remains to be explained and > as mentioned in on the mails in the list, that does not make sense at > least for the continuous text exercise. The method section of the paper > is too vague but my guess is that some LaTeX users did attempt to > reproduce the exact layout of the text they had to copy, something LaTeX > is definitely not design to provide quickly. > > One more point: how many of you guys could specify their total number of > hours of experience with LaTeX (or any other software you are currently > using)? That what the subjects of this study had to specify... > > Let me know what you think, > > Christophe > > My org buffer: > > #+TITLE: An Efficiency Comparison of Document Preparation Systems Used > in Academic Research and Development: A Re-analysis. > #+DATE: <2014-12-28 dim.> > #+AUTHOR: Christophe Pouzat > #+EMAIL: christophe.pouzat@gmail.com > #+OPTIONS: ':nil *:t -:t ::t <:t H:3 \n:nil ^:t arch:headline > #+OPTIONS: author:t c:nil creator:comment d:(not "LOGBOOK") date:t > #+OPTIONS: e:t email:nil f:t inline:t num:t p:nil pri:nil stat:t > #+OPTIONS: tags:t tasks:t tex:t timestamp:t toc:nil todo:t |:t > #+CREATOR: Emacs 24.4.1 (Org mode 8.2.10) > #+DESCRIPTION: > #+EXCLUDE_TAGS: noexport > #+KEYWORDS: > #+LANGUAGE: en > #+SELECT_TAGS: export > #+LaTeX_HEADER: \usepackage{alltt} > #+LaTeX_HEADER: \usepackage[usenames,dvipsnames]{xcolor} > #+LaTeX_HEADER: \renewenvironment{verbatim}{\begin{alltt} \scriptsize > \color{Bittersweet} \vspace{0.2cm} }{\vspace{0.2cm} \end{alltt} > \normalsize \color{black}} > #+LaTeX_HEADER: \definecolor{lightcolor}{gray}{.55} > #+LaTeX_HEADER: \definecolor{shadecolor}{gray}{.85} > #+LaTeX_HEADER: \usepackage{minted} > #+LaTeX_HEADER: \hypersetup{colorlinks=true} > > #+NAME: org-latex-set-up > #+BEGIN_SRC emacs-lisp :results silent :exports none > (setq org-latex-listings 'minted) > (setq org-latex-minted-options > '(("bgcolor" "shadecolor") > ("fontsize" "\\scriptsize"))) > (setq org-latex-pdf-process > '("pdflatex -shell-escape -interaction nonstopmode > -output-directory %o %f" > "biber %b" > "pdflatex -shell-escape -interaction nonstopmode -output-directory > %o %f" > "pdflatex -shell-escape -interaction nonstopmode -output-directory > %o %f")) > #+END_SRC > > * Introduction > This is a re-analysis of the data presented in > [[http://dx.doi.org/10.1371/journal.pone.0115069][An Efficiency > Comparison of Document Preparation Systems Used in Academic Research and > Development]]. My "interest" in this paper was triggered by a discussion > on the [[http://article.gmane.org/gmane.emacs.orgmode/93655][emacs org > mode mailing list]]. Ignoring the "message" of the paper, what stroke me > was the systematic use of bar graphs: a way of displaying data that > *should never be used* since when many observations are considered, a > box plot is going to do a much better job and when, like in the present > paper, few (10 in each of the 4 categories) observations are available, > a direct display or even a simple table is going to do a *much better* > job. Since it turns out that the data are available both on the Plos web > site and on > [[http://figshare.com/articles/_An_Efficiency_Comparison_of_Document_Preparation_Systems_Used_in_Academic_Research_and_Development_/1275631][figshare]], > I decided to re-analyze them. > > * Getting the data, etc. > > We get the data with: > > #+BEGIN_SRC sh > wget http://files.figshare.com/1849394/S1_Materials.xlsx > #+END_SRC > > #+RESULTS: > Using for instance [[http://dag.wiee.rs/home-made/unoconv/][unoconv]], > we can convert the =Excel= file into a friendlier =csv= file: > > #+BEGIN_SRC sh > unoconv -f csv S1_Materials.xlsx > #+END_SRC > > #+RESULTS: > We then get the data with =R= =read.csv= function: > > #+NAME: data-table > #+BEGIN_SRC R :session *R* :results silent > efficiency <- read.csv("S1_Materials.csv",header=TRUE,dec=",") > #+END_SRC > The description of this table is obtained with: > > #+BEGIN_SRC sh :exports both :results output > wget http://files.figshare.com/1849395/S2_Materials.txt > cat "S2_Materials.txt" > #+END_SRC > > * Making some figures > We can now make a figure out of the same data as figures 4, 5 and 6 of > the paper but showing the actual data. We start with the "continuous > text" exercise. We represent, in each of the four categories, each of > the 10 individuals by a number between 0 and 9. Some horizontal jitter > has been added to avoid overlaps. Category 1 corresponds to expert > =Word= users; 2 to novice =Word= users; 3 to expert \LaTeX{} users; 4 to > novice \LaTeX{} users: > > #+HEADER: :file continuous.png :width 1000 :height 1000 > #+BEGIN_SRC R :session *R* :exports both :results output graphics > layout(matrix(1:4,nc=2,byrow=TRUE)) > par(cex=2) > plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=c(0,100), > xlab="User category",ylab="",main="Fraction of text") > with(efficiency, > sapply(1:4, > function(k) points(runif(10,k-0.2,k+0.2), > PROZENT1[Kenntnisse==k], > pch = paste(0:9)))) > > with(efficiency, > plot(c(1,4),c(0,100),type="n", > xlim=c(0.5,4.5),ylim=range(FEHLERSFT),xlab="User category", > ylab="",main="Formatting errors and typos")) > with(efficiency, > sapply(1:4, > function(k) points(runif(10,k-0.2,k+0.2), > FEHLERSFT[Kenntnisse==k], > pch = paste(0:9)))) > > with(efficiency, > plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5), > ylim=range(FEHLEROFT),xlab="User category",ylab="", > main="Orthographic and grammatical mistakes")) > with(efficiency, > sapply(1:4, > function(k) points(runif(10,k-0.2,k+0.2), > FEHLEROFT[Kenntnisse==k], > pch = paste(0:9)))) > > with(efficiency, > plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=range(FEHLENDFT), > xlab="User category",ylab="",main="Missing words and signs")) > with(efficiency, > sapply(1:4, > function(k) points(runif(10,k-0.2,k+0.2), > FEHLENDFT[Kenntnisse==k], > pch = paste(0:9)))) > #+END_SRC > > > Notice that the number of "missing words and signs" exactly mirrors the > fraction of written text. We will see that this observation holds for > the two following exercises. This "missing words and signs" is always > roughly ten times as large as the two other sources of mistakes. This > explains the inverse relationship between the blue and pink bars on each > of the 3 figures. > > Let's keep going with the "table exercise": > > #+HEADER: :file table.png :width 1000 :height 1000 > #+BEGIN_SRC R :session *R* :exports both :results output graphics > layout(matrix(1:4,nc=2,byrow=TRUE)) > par(cex=2) > plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=c(0,100), > xlab="User category",ylab="",main="Fraction of text") > with(efficiency,sapply(1:4, > function(k) points(runif(10,k-0.2,k+0.2), > PROZENT2[Kenntnisse==k], > pch = paste(0:9)))) > > with(efficiency,plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5), > ylim=range(FEHLERST),xlab="User category", > ylab="",main="Formatting errors and typos")) > with(efficiency,sapply(1:4, > function(k) points(runif(10,k-0.2,k+0.2), > FEHLERST[Kenntnisse==k], > pch = paste(0:9)))) > > with(efficiency,plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5), > ylim=range(FEHLEROT),xlab="User category", > ylab="",main="Orthographic and grammatical mistakes")) > with(efficiency,sapply(1:4, > function(k) points(runif(10,k-0.2,k+0.2), > FEHLEROT[Kenntnisse==k], > pch = paste(0:9)))) > > with(efficiency,plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5), > ylim=range(FEHLENDT),xlab="User category",ylab="", > main="Missing words and signs")) > with(efficiency,sapply(1:4, > function(k) points(runif(10,k-0.2,k+0.2), > FEHLENDT[Kenntnisse==k], > pch = paste(0:9)))) > #+END_SRC > > We also see a strange thing here: in each of the expert \LaTeX{} and the > novice \LaTeX{} users we have one individual who did not right anything > but still manage to produce 22 "formatting errors and typos" (!) but > luckily no orthographic or grammatical error... > > #+BEGIN_SRC R :session *R* :exports both > with(efficiency,cbind(c(PROZENT2[Kenntnisse==3][10], > FEHLERST[Kenntnisse==3][10], > FEHLEROT[Kenntnisse==3][10], > FEHLENDT[Kenntnisse==3][10]), > c(PROZENT2[Kenntnisse==4][7], > FEHLERST[Kenntnisse==4][7], > FEHLEROT[Kenntnisse==4][7], > FEHLENDT[Kenntnisse==4][7]))) > #+END_SRC > > > Now for the "equations" exercise: > > #+HEADER: :file equation.png :width 1000 :height 1000 > #+BEGIN_SRC R :session *R* :exports both :results output graphics > layout(matrix(1:4,nc=2,byrow=TRUE)) > par(cex=2) > plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=c(0,100), > xlab="User category",ylab="",main="Fraction of text") > with(efficiency, > sapply(1:4, > function(k) points(runif(10,k-0.2,k+0.2), > PROZENT3[Kenntnisse==k], > pch = paste(0:9)))) > > with(efficiency, > plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5), > ylim=range(FEHLERSFOR),xlab="User category",ylab="", > main="Formatting errors and typos")) > with(efficiency, > sapply(1:4, > function(k) points(runif(10,k-0.2,k+0.2), > FEHLERSFOR[Kenntnisse==k], > pch = paste(0:9)))) > > with(efficiency, > plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=range(FEHLEROFOR), > xlab="User category",ylab="", > main="Orthographic and grammatical mistakes")) > with(efficiency, > sapply(1:4, > function(k) points(runif(10,k-0.2,k+0.2), > FEHLEROFOR[Kenntnisse==k], > pch = paste(0:9)))) > > with(efficiency, > plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5), > ylim=range(FEHLENDFOR),xlab="User category",ylab="", > main="Missing words and signs")) > with(efficiency, > sapply(1:4, > function(k) points(runif(10,k-0.2,k+0.2), > FEHLENDFOR[Kenntnisse==k], > pch = paste(0:9)))) > #+END_SRC -- Thomas S. Dye http://www.tsdye.com ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word ---LOOK AT THE DATA! 2014-12-28 21:40 ` Efficiency of Org v. LaTeX v. Word ---LOOK AT THE DATA! Christophe Pouzat 2014-12-29 19:47 ` Thomas S. Dye @ 2014-12-31 16:59 ` Colin Baxter 1 sibling, 0 replies; 21+ messages in thread From: Colin Baxter @ 2014-12-31 16:59 UTC (permalink / raw) To: emacs-orgmode Dear Christophe, Great work. You should submit it to http://www.plosone.org/ as a response. It would be interesting to see what the Referees make of it. Best wishes, Colin. > Hi all, > > After seeing Ken's mail: > > Le 26/12/2014 23:47, Ken Mankoff a écrit : >> People here might be interested in a publication from [2014-12-19 Fri] >> available at http://dx.doi.org/10.1371/journal.pone.0115069 >> >> Title: An Efficiency Comparison of Document Preparation Systems Used >> in Academic Research and Development >> >> Summary: Word users are more efficient and have less errors than even >> experienced LaTeX users. >> >> Someone here should repeat experiment and add Org into the mix, perhaps >> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume >> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via >> pandoc) beat straight Word? >> >> -k. >> >> > and some of replies it triggered on the list, I went to check the paper. > As many of you guys I found some "results" puzzling in particular: > 1. the use of bar graphs when the data would better be displayed > directly (that qualifies immediately the paper as "low quality" for me). > 2. the larger error bars observed for LaTeX when compared to Word. > 3. the systematic inverse relationship between the blue and pink bars > heights. > > So I went to figshare to download the data and looked at them. A quick > and dirty "analysis" is attached to this mail in PDF format (generated > with org, of course, and this awful software called LaTeX!) and the > source org file can be found at the bottom of this mail. I used R to do > the figures (and I'm sure the authors of the paper will then criticize > me for not using Excel with which everyone knows errors are generated > much more efficiently). > > I managed to understand the inverse relationship in point 3 above: the > authors considered 3 types of mistakes / errors: > 1. Formatting and typos error. > 2. Orthographic and grammatical errors. > 3. Missing words and signs. > Clearly, following the mail of Tom (Dye) on the list and on the Plos web > site, I would argue that formatting errors in LaTeX are bona fide bugs. > But the point I want to make is that the third source accounts for 80% > of the total errors (what's shown in pink bars in the paper) and clearly > the authors counted what the subjects did not have time to type as an > error of this type. Said differently, the blue and pink bars are showing > systematically the same thing by construction! The second type of error > in not a LaTeX issue (and in fact does not differ significantly from the > Word case) but an "environment" issue (what spelling corrector had the > LaTeX users access to?). > > There is another strange thing in the table copy case. For both the > expert and novice group in LaTeX, there is one among 10 subjects that > did produce 0% of the table but still manage to produce 22 typographic > errors! > > The overall worst performance of LaTeX users remains to be explained and > as mentioned in on the mails in the list, that does not make sense at > least for the continuous text exercise. The method section of the paper > is too vague but my guess is that some LaTeX users did attempt to > reproduce the exact layout of the text they had to copy, something LaTeX > is definitely not design to provide quickly. > > One more point: how many of you guys could specify their total number of > hours of experience with LaTeX (or any other software you are currently > using)? That what the subjects of this study had to specify... > > Let me know what you think, > > Christophe > > ----- Snip ----- ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word 2014-12-26 22:47 Efficiency of Org v. LaTeX v. Word Ken Mankoff ` (6 preceding siblings ...) 2014-12-28 21:40 ` Efficiency of Org v. LaTeX v. Word ---LOOK AT THE DATA! Christophe Pouzat @ 2015-01-04 20:38 ` John Kitchin 2015-01-04 21:15 ` Andreas Leha 7 siblings, 1 reply; 21+ messages in thread From: John Kitchin @ 2015-01-04 20:38 UTC (permalink / raw) To: Ken Mankoff; +Cc: Org-mode mailing list Ken Mankoff <mankoff@gmail.com> writes: Did anyone see the parody of this here: http://mjambon.github.io/vim-vs-emacs/ It is pretty funny! > People here might be interested in a publication from [2014-12-19 Fri] > available at http://dx.doi.org/10.1371/journal.pone.0115069 > > Title: An Efficiency Comparison of Document Preparation Systems Used > in Academic Research and Development > > Summary: Word users are more efficient and have less errors than even > experienced LaTeX users. > > Someone here should repeat experiment and add Org into the mix, perhaps > Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume > Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via > pandoc) beat straight Word? > > -k. > > > -- ----------------------------------- John Kitchin Professor Doherty Hall A207F Department of Chemical Engineering Carnegie Mellon University Pittsburgh, PA 15213 412-268-7803 http://kitchingroup.cheme.cmu.edu ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Efficiency of Org v. LaTeX v. Word 2015-01-04 20:38 ` Efficiency of Org v. LaTeX v. Word John Kitchin @ 2015-01-04 21:15 ` Andreas Leha 0 siblings, 0 replies; 21+ messages in thread From: Andreas Leha @ 2015-01-04 21:15 UTC (permalink / raw) To: emacs-orgmode Hi John, John Kitchin <johnrkitchin@gmail.com> writes: > Ken Mankoff <mankoff@gmail.com> writes: > > Did anyone see the parody of this here: > http://mjambon.github.io/vim-vs-emacs/ > > It is pretty funny! Indeed. Thanks for sharing. Regards, Andreas > >> People here might be interested in a publication from [2014-12-19 Fri] >> available at http://dx.doi.org/10.1371/journal.pone.0115069 >> >> Title: An Efficiency Comparison of Document Preparation Systems Used >> in Academic Research and Development >> >> Summary: Word users are more efficient and have less errors than even >> experienced LaTeX users. >> >> Someone here should repeat experiment and add Org into the mix, perhaps >> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume >> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via >> pandoc) beat straight Word? >> >> -k. >> >> >> ^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2015-01-04 21:16 UTC | newest] Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-12-26 22:47 Efficiency of Org v. LaTeX v. Word Ken Mankoff 2014-12-26 23:36 ` Thomas S. Dye 2014-12-27 2:21 ` briangpowell . 2014-12-27 14:36 ` Eric S Fraga 2014-12-27 3:26 ` Christopher W. Ryan 2014-12-28 22:45 ` Bob Newell 2014-12-27 4:27 ` Nick Dokos 2014-12-27 9:06 ` Peter Neilson 2014-12-27 14:38 ` Eric S Fraga 2014-12-27 9:48 ` Achim Gratz 2014-12-27 10:05 ` Paul Rudin 2014-12-27 10:36 ` M 2014-12-27 11:36 ` Fabrice Popineau 2014-12-28 22:43 ` Pascal Fleury 2014-12-31 18:19 ` Paul Rudin 2014-12-27 13:37 ` Daniele Pizzolli 2014-12-28 21:40 ` Efficiency of Org v. LaTeX v. Word ---LOOK AT THE DATA! Christophe Pouzat 2014-12-29 19:47 ` Thomas S. Dye 2014-12-31 16:59 ` Colin Baxter 2015-01-04 20:38 ` Efficiency of Org v. LaTeX v. Word John Kitchin 2015-01-04 21:15 ` Andreas Leha
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).