2.5 KiB
| title | aliases | tags | |
|---|---|---|---|
| assigning-participants |
|
Methods
Between Subjects
The participants are split into equal groups, each which reviews one of the alternatives
Within Subjects
The participants are split into two grops, one group review the first alternative first, the other review the seconds alternative first.
Latin Square
order of each group is different
Counterbalanced assignments
e.g., typing speed might affect interface usage you can use a pretest to assign participants to that typing speed is roughly balanced
many techniques: key is to have equal chance of each participant in each group
offline counterbalancing
pre-testing then forming matches pairs, which are split between groups]
online counterbalancing
-
of ther is no pre-test pick a threshold that is likely to be about the middle
-
as they come in
-
dont need to ensure even nubmer of high and low typers.
-
do need to ensure the same number of high/low typers in a and b
dangers
regression:
- find heady coins
- first flip them all (pre-test)
- if they land heads more than half, call them heady
- "feed them a snack"
- does snacking increase the natural tendency of coins
both regress towards the mean
how to avoid
if the pretest is used to counterbalance, and assignment is random, then the error goes away
should every participant use every alternative
three major srategies
- Within
- everyone tries all options
- good when not fussed about learning/practise/exposure isssues
- between participants
- each person tries only only of the options
- requires more people, and more attention to fair assignment
- has the benefit that each participant is uncorrupted
- most common forthings like web studies
-
Counterbalancing
hawthorne effect
results are a result of the act of exmperimenting itself not as a resyult of the manupulations of the experiment
can be avoided with random assignment
vaccum cleaner example
-
manipulation
- vacuum type
-
measures
- speed
- cleanliness
-
between subjects design: assign half the participants to each type.
- worried about individual differences
-
within subjects design: everyone uses both interfaces:
- worried about ordering effects
-
half try one first, the other try the other first (counterbalancing)
- each of the tasks should be difference e.g., clean differnt buildings/rooms
individual differnces: go based of intuition of if it will make a difference. Random assignment is importantg