* Just sharing another orgmode use: usability tests
@ 2017-06-15 20:08 Eduardo Mercovich
2017-08-02 15:03 ` Karl Voit
0 siblings, 1 reply; 7+ messages in thread
From: Eduardo Mercovich @ 2017-06-15 20:08 UTC (permalink / raw)
To: orgmode list
Dear all.
We all know that emacs+orgmode are not tools, but meta-tools. This
is why it can be so daunting at first(*1) and totally awesome in
the long run. ;)
This mail is just to share with you another great application of
orgmode: usability tests.
For those of you who never experienced it yet, a usability test is
basically a critical observation of certain people (Users) doing
certain things (tasks) in a particular setting (context). And
critical means that there are some defined things that you look
specifically to, like task success of fail (what means "success"
is previously defined), times, User observations, navigation, etc.
During a test there is
+ a User (usually one, but in certain specific cases can be more),
+ a Facilitator (who cares for the User and ensures that the
experimental protocol is followed as it should) and
+ Observers (who takes notes of these defined criteria and many
other observations).
Test notes are usually comprised of task start and stop time,
results (succes, failure or so-so), User quotes, navigation steps,
intermediate steps results, Observer's hypothesis to be reviewed,
etc. That is, the prescribed issues, plus any other notable fact.
In some specific tools, like the industry standard setting Morae
(https://www.techsmith.com/morae.html, US$ 1,995) you can take
those notes with relative easy, because it automates the time
counting and you can set codes (usually one letter) to specify the
type of observation. For example, q for 'user quote' or v for
'video' (something interesting to review after the test) or n for
navigation, and so on.
Handwrited notes are great because of their fluidity, except that
is hard to count times (you can look at an external chronometer,
of course, but it takes your attention away from the User) and in
my case, my handwriting is fast but so bad that I can hardly read
it after. ;)
And here comes orgmode.
In my last test I finally tried to use orgmode for this (why not,
I use it for almost any other important task) and the results
where impressive, even in the 1st try, even without any
customization or heavy data metabolism after.
What do we have out of the box:
+ integration of script and notes (I was the Facilitator, but took
notes also) so I can read the user script and take notes
integrated with each task and context. Each task is a heading, and
notes are directly inside it with...
+ ... task results easily annotated, using the task state (TODO in
red for failure, ENCURSO in brown for so-so and DONE in green for
success).
+ time stamping, not only start and end, but anything in between
too.
+ abbreviations allowed me to enter notes faster (chording may
help to do it faster even, but I still don't use it).
+ regular expression highlighting makes incredibly sweet to review
the notes. For example, I use !! to mark something important and
!!! for critical, so those go for line highlighting with yellow
and red background respectively, and user quotes are between ", so
those get separated from the normal text too.
What I believe could be easely done with a bit of work:
+ automatic task clocking, including automated time stamping of
every observation.
+ automatic recollection of User quotes, like "let's see everyhing
that Users said in task X".
+ observation tagging (there are repeating patterns of use or
interaction problems or sources that you could cath on the fly, so
any observation could be related to those common issues).
+ automatic filling of User/task-results-and-time matrix (the most
fundamental metric of usability tests).
In brief: orgmode is great for usability studies. :D
While I don't know if any of you will find these observations
useful, I'm sure you all enjoy knowing that this incredible
meta-tool many of you created and still make it grow has found
another great use.
Have a great orgmoding time... :)
*1: Yes, I know, not for us on this list but yes for many people
that's used to more specific, maket-centered tools. ;)
--
eduardo mercovich
Donde se cruzan tus talentos
con las necesidades del mundo,
ahí está tu vocación.
(Anónimo)
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Just sharing another orgmode use: usability tests
[not found] <8d18331fde644aa79d32d60d720a8d10@HE1PR01MB1898.eurprd01.prod.exchangelabs.com>
@ 2017-06-16 8:25 ` Eric S Fraga
2017-06-16 14:19 ` Eduardo Mercovich
[not found] ` <e24c85840a894ce9bbe7c0b56cfc1d53@HE1PR01MB1898.eurprd01.prod.exchangelabs.com>
0 siblings, 2 replies; 7+ messages in thread
From: Eric S Fraga @ 2017-06-16 8:25 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 735 bytes --]
On Thursday, 15 Jun 2017 at 20:08, Eduardo Mercovich wrote:
> This mail is just to share with you another great application of
> orgmode: usability tests.
Thanks. Very interesting.
[...]
> What I believe could be easely done with a bit of work:
>
> + automatic task clocking, including automated time stamping of
> every observation.
This should be trivial. Use a capture template for observations and you
can clock these automatically (I think) and/or time stamp as
required. In fact, many of your other suggested needs could be met with
capture templates.
But, being emacs, I am sure there will be plenty of alternatives!
Thanks again,
eric
--
: Eric S Fraga (0xFFFCF67D), Emacs 26.0.50, Org release_9.0.7-531-g530113
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Just sharing another orgmode use: usability tests
2017-06-16 8:25 ` Eric S Fraga
@ 2017-06-16 14:19 ` Eduardo Mercovich
[not found] ` <e24c85840a894ce9bbe7c0b56cfc1d53@HE1PR01MB1898.eurprd01.prod.exchangelabs.com>
1 sibling, 0 replies; 7+ messages in thread
From: Eduardo Mercovich @ 2017-06-16 14:19 UTC (permalink / raw)
To: Eric S Fraga; +Cc: emacs-orgmode
Hi Eric.
>> This mail is just to share with you another great application
>> of orgmode: usability tests.
[...]
>> What I believe could be easely done with a bit of work:
>> + automatic task clocking, including automated time stamping of
>> every observation.
> This should be trivial. Use a capture template for observations
> and you
> can clock these automatically (I think) and/or time stamp as
> required. In fact, many of your other suggested needs could be
> met with
> capture templates.
The standard clocking task mechanism will do it fine.
However, since there are many multiple small observations along
one task, maybe a capture template is a bit overdo...
> But, being emacs, I am sure there will be plenty of
> alternatives!
Yes, I do totally agree... :)
I will continue slowly to experiment with this. Clocking task
in-out is one of the 1st issues to be tried along with timers, and
also adding the seconds (which in a task could be important).
Best...
--
eduardo mercovich
Donde se cruzan tus talentos
con las necesidades del mundo,
ahí está tu vocación.
(Anónimo)
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Just sharing another orgmode use: usability tests
[not found] ` <e24c85840a894ce9bbe7c0b56cfc1d53@HE1PR01MB1898.eurprd01.prod.exchangelabs.com>
@ 2017-06-17 13:37 ` Eric S Fraga
2017-06-18 22:21 ` Eduardo Mercovich
0 siblings, 1 reply; 7+ messages in thread
From: Eric S Fraga @ 2017-06-17 13:37 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 660 bytes --]
On Friday, 16 Jun 2017 at 14:19, Eduardo Mercovich wrote:
[...]
> The standard clocking task mechanism will do it fine.
Great.
> However, since there are many multiple small observations along
> one task, maybe a capture template is a bit overdo...
I find that capturing is a lightweight activity. It all depends on what
you expect to have to do. For instance, I log some activities by simply
typing "C-c c l COUPLE OF KEYWORDS C-c C-c" and I'm done! The entry is
time stamped automatically and it simply allows me to note something
without really intruding into a workflow.
But, YMMV of course :-)
--
Eric S Fraga, ericsfraga@<most social media>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Just sharing another orgmode use: usability tests
2017-06-17 13:37 ` Eric S Fraga
@ 2017-06-18 22:21 ` Eduardo Mercovich
0 siblings, 0 replies; 7+ messages in thread
From: Eduardo Mercovich @ 2017-06-18 22:21 UTC (permalink / raw)
To: Eric S Fraga; +Cc: emacs-orgmode
Hi Eric.
>> The standard clocking task mechanism will do it fine.
>
> Great.
>
>> However, since there are many multiple small observations along
>> one task, maybe a capture template is a bit overdo...
> I find that capturing is a lightweight activity. It all depends
> on what
> you expect to have to do. For instance, I log some activities
> by simply
> typing "C-c c l COUPLE OF KEYWORDS C-c C-c" and I'm done! The
> entry is
> time stamped automatically and it simply allows me to note
> something
> without really intruding into a workflow.
It could be... maybe I feel it like too much it's because some obs
are really quite short (like 1 name/place) and I took them as
lines, so headings or list items are more than enough.
I'll have to try it. :)
OTOH, I just found the perfect timestamping method: org-timer.
:)))
Best...
--
eduardo mercovich
Donde se cruzan tus talentos
con las necesidades del mundo,
ahí está tu vocación.
(Anónimo)
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Just sharing another orgmode use: usability tests
2017-06-15 20:08 Just sharing another orgmode use: usability tests Eduardo Mercovich
@ 2017-08-02 15:03 ` Karl Voit
2017-08-03 1:26 ` Eduardo Mercovich
0 siblings, 1 reply; 7+ messages in thread
From: Karl Voit @ 2017-08-02 15:03 UTC (permalink / raw)
To: emacs-orgmode
* Eduardo Mercovich <eduardo@mercovich.net> wrote:
>
> + automatic task clocking, including automated time stamping of
> every observation.
You seem to use headings for any events.
If list items does the trick as well, you can take a look at timers
for doing the clocking thing: http://orgmode.org/manual/Timers.html
> + observation tagging (there are repeating patterns of use or
> interaction problems or sources that you could cath on the fly, so
> any observation could be related to those common issues).
You already mentioned templates. I'd go with yasnippets for that as
well.
> + automatic filling of User/task-results-and-time matrix (the most
> fundamental metric of usability tests).
Wow, this would be great.
--
get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode:
> get Memacs from https://github.com/novoid/Memacs <
Personal Information Management > http://Karl-Voit.at/tags/pim/
Emacs-related > http://Karl-Voit.at/tags/emacs/
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Just sharing another orgmode use: usability tests
2017-08-02 15:03 ` Karl Voit
@ 2017-08-03 1:26 ` Eduardo Mercovich
0 siblings, 0 replies; 7+ messages in thread
From: Eduardo Mercovich @ 2017-08-03 1:26 UTC (permalink / raw)
To: Karl Voit; +Cc: emacs-orgmode
Hi Karl.
>> + automatic task clocking, including automated time stamping of
>> every observation.
> You seem to use headings for any events.
> If list items does the trick as well, you can take a look at
> timers
> for doing the clocking thing:
> http://orgmode.org/manual/Timers.html
Totally agree, that's exactly what I use now. :)
Lists are used inside each task, at 1 item per (short)
observation.
>> + observation tagging (there are repeating patterns of use or
>> interaction problems or sources that you could catch on the
>> fly, so any observation could be related to those common
>> issues).
> You already mentioned templates. I'd go with yasnippets for that
> as well.
Sorry I wasn't clear, I meant more the use of tags related to
common observed patterns mark and select after (with sparse tree)
those in which that pattern or issue is relevant. Great to group
things (tasks, observations, etc.) related to the same issue.
>> + automatic filling of User/task-results-and-time matrix (the
>> most fundamental metric of usability tests).
> Wow, this would be great.
Yes, but for that I have to learn Lisp. ;)
I'm starting with "An Introduction To Programming in Emacs Lisp"
by Robert J. Chassell. Just minutes a week sadly, but this is what
I can do now. :)
In the document structure we have:
* Users,
** Pre-task interview
** tasks (with TODO states as indicators of success|failure|so-so;
maybe we can use properties).
- each one with one or more timer entry
- so total task time is the last item timer value,
** Post-task interview
To make the User/task-results-and-time matrix we'll have to walk
the hierarchy and get for each User the sequence of tasks, and of
each, the success (or not), time and tag (issue) related and put
that in the results matrix.
Since I couldn't find other references about orgmode used for
usability, I am organizing an informal workshop to share this
experience next week with 10 people or so from the usability
community in Buenos Aires.
If someone is interested, we can arrange another in English after
this 1st one. ;)
Best regards...
--
eduardo mercovich
Donde se cruzan tus talentos
con las necesidades del mundo,
ahí está tu vocación.
(Anónimo)
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-08-03 1:27 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-06-15 20:08 Just sharing another orgmode use: usability tests Eduardo Mercovich
2017-08-02 15:03 ` Karl Voit
2017-08-03 1:26 ` Eduardo Mercovich
[not found] <8d18331fde644aa79d32d60d720a8d10@HE1PR01MB1898.eurprd01.prod.exchangelabs.com>
2017-06-16 8:25 ` Eric S Fraga
2017-06-16 14:19 ` Eduardo Mercovich
[not found] ` <e24c85840a894ce9bbe7c0b56cfc1d53@HE1PR01MB1898.eurprd01.prod.exchangelabs.com>
2017-06-17 13:37 ` Eric S Fraga
2017-06-18 22:21 ` Eduardo Mercovich
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).