quartz/content/notes/designing-studies.md

title	aliases	tags
designing-studies		info203

Need to be more specific than "Do you like my interface". Need to avoid experimenter bias

Terms

• Comparison: What is good
  • "Yark stick"
• Baserate: how often does Y occur
  • requires measurin Y
• Correlation: Do X and Y co-vary
  • Requires measuring X and Y
• Causes: Does X cause Y
  • Requires measuring X and Y and manipulating X
• Manipulations: Indepenedent variables
• Measures: Dependent variables
  • e.g., task completion time, recall, accuracy, emotional response
• Precision: Internal Validity
  • remove confounding factors
  • large sample size
• Generlisability: External Validity
  • does this apply in the real world

Strategies for fairer comparisons

Two main things differ between prototype and Production: Fidelity (how "polished the design is", Approach (the actual design)

• insert your new approach into the production setting
• recreate the production approach in you new setting
• scale down so youre just looking at a piece of a larger system
• when expertise is relevant, train people up

Bad example

manipulation: input style measure: words per minute external validity: not so much

benefits and drawbacks

• not a fair example: novices vs experts
• are the results significant

Good example

manipulation: input style measure: words per minute and error rate