quartz/content/notes/running-in-person-experiments.md

title	aliases	tags	sr-due	sr-interval	sr-ease
running-in-person-experiments		info203 scott-video lecture	2022-06-23	20	250

in person

• planning
• execution
• analyzing

higher "bandwitdh" of engagement

Make clear goals

• scope
• purpose
  • what you hope to learn
• hypothesis
  • make a prediction
• schedule and location
  • use an approprate and relevant location
• participants
• scenarios

Plan out steps

• questions
• data to be collected
• set up
• roles
  • e.g., facilitator, recorder

Create concrete tasks

write them down

e.g.,

Ethical considerations

participants can feel pressured

get informed consent

remind them that your testing the site.. not them

paticipants feel good about finding issues, not bad about not being able to do something

Experiement details

• order
  • e.g., start simple, shuffle order
• training
  • depends on how th real system will be used
• DNF
  • set a time limit
  • maybe provide help if needed
• pilot
  • iron out the kinks in the study design
  • can also find very obvious issues that should be addressed so the actual participants aren't "wasted"

capturing results

• note down critical incidents
  • aha moments, stores to share, big problems
• record video
  • lets to grab moments easily and share them
• screen recording
  • depends if you are interested on their expressions or th interface
• interupptions
  • yes/no
  • provide help in necessary
  • think aloud
    • need to know what users are thinking not just what they are doing
    • ask users to talk while performing tasks
      • what they are;
        • thinking
        • trying to do
        • any questions as they work
        • reading
    • record or take notes
    • prompt them to keep talking
    • decide ahead of time about which things to help with
      • keep track of things you helped with
    • will thinking alout give the right answers
      • not always
      • avoid specific questions
      • if you ask a question, people with always give an answer even it it has nothing to do with the facts
      • talking while doing the tasks may change how you do the task

Greeting participants

welcome, explain setup, scenarios,

collecting data

• process data
  • observations (qualittative)
• bottom line data
  • numbers
  • i.e., dependent variables

measuring bottom line usability

useful:

• time requirements
• success/fail
  • define in advance what this is
• compare speed or # of errors

dont combine with think aloud

debrief

tell them what you goals are.

learn more holistically what they are thinking