emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* Just sharing another orgmode use: usability tests
@ 2017-06-15 20:08 Eduardo Mercovich
  2017-08-02 15:03 ` Karl Voit
  0 siblings, 1 reply; 7+ messages in thread
From: Eduardo Mercovich @ 2017-06-15 20:08 UTC (permalink / raw)
  To: orgmode list

Dear all.

We all know that emacs+orgmode are not tools, but meta-tools. This 
is why it can be so daunting at first(*1) and totally awesome in 
the long run. ;)

This mail is just to share with you another great application of 
orgmode: usability tests.

For those of you who never experienced it yet, a usability test is 
basically a critical observation of certain people (Users) doing 
certain things (tasks) in a particular setting (context). And 
critical means that there are some defined things that you look 
specifically to, like task success of fail (what means "success" 
is previously defined), times, User observations, navigation, etc.

During a test there is
+ a User (usually one, but in certain specific cases can be more),
+ a Facilitator (who cares for the User and ensures that the 
experimental protocol is followed as it should) and
+ Observers (who takes notes of these defined criteria and many 
other observations). 

Test notes are usually comprised of task start and stop time, 
results (succes, failure or so-so), User quotes, navigation steps, 
intermediate steps results, Observer's hypothesis to be reviewed, 
etc. That is, the prescribed issues, plus any other notable fact. 

In some specific tools, like the industry standard setting Morae 
(https://www.techsmith.com/morae.html, US$ 1,995) you can take 
those notes with relative easy, because it automates the time 
counting and you can set codes (usually one letter) to specify the 
type of observation. For example, q for 'user quote' or v for 
'video' (something interesting to review after the test) or n for 
navigation, and so on. 

Handwrited notes are great because of their fluidity, except that 
is hard to count times (you can look at an external chronometer, 
of course, but it takes your attention away from the User) and in 
my case, my handwriting is fast but so bad that I can hardly read 
it after. ;)

And here comes orgmode. 

In my last test I finally tried to use orgmode for this (why not, 
I use it for almost any other important task) and the results 
where impressive, even in the 1st try, even without any 
customization or heavy data metabolism after. 

What do we have out of the box: 

+ integration of script and notes (I was the Facilitator, but took 
notes also) so I can read the user script and take notes 
integrated with each task and context. Each task is a heading, and 
notes are directly inside it with...

+ ... task results easily annotated, using the task state (TODO in 
red for failure, ENCURSO in brown for so-so and DONE in green for 
success). 

+ time stamping, not only start and end, but anything in between 
too. 

+ abbreviations allowed me to enter notes faster (chording may 
help to do it faster even, but I still don't use it). 

+ regular expression highlighting makes incredibly sweet to review 
the notes. For example, I use !! to mark something important and 
!!! for critical, so those go for line highlighting with yellow 
and red background respectively, and user quotes are between ", so 
those get separated from the normal text too. 


What I believe could be easely done with a bit of work:

+ automatic task clocking, including automated time stamping of 
every observation. 

+ automatic recollection of User quotes, like "let's see everyhing 
that Users said in task X". 

+ observation tagging (there are repeating patterns of use or 
interaction problems or sources that you could cath on the fly, so 
any observation could be related to those common issues). 

+ automatic filling of User/task-results-and-time matrix (the most 
fundamental metric of usability tests). 


In brief: orgmode is great for usability studies. :D

While I don't know if any of you will find these observations 
useful, I'm sure you all enjoy knowing that this incredible 
meta-tool many of you created and still make it grow has found 
another great use. 

Have a great orgmoding time... :)



*1: Yes, I know, not for us on this list but yes for many people 
 that's used to more specific, maket-centered tools. ;)
-- 
eduardo mercovich

 Donde se cruzan tus talentos 
 con las necesidades del mundo, 
 ahí está tu vocación. 
 (Anónimo)

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2017-08-03  1:27 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <8d18331fde644aa79d32d60d720a8d10@HE1PR01MB1898.eurprd01.prod.exchangelabs.com>
2017-06-16  8:25 ` Just sharing another orgmode use: usability tests Eric S Fraga
2017-06-16 14:19   ` Eduardo Mercovich
     [not found]   ` <e24c85840a894ce9bbe7c0b56cfc1d53@HE1PR01MB1898.eurprd01.prod.exchangelabs.com>
2017-06-17 13:37     ` Eric S Fraga
2017-06-18 22:21       ` Eduardo Mercovich
2017-06-15 20:08 Eduardo Mercovich
2017-08-02 15:03 ` Karl Voit
2017-08-03  1:26   ` Eduardo Mercovich

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).