mirror of
https://github.com/jackyzha0/quartz.git
synced 2025-12-24 05:14:06 -06:00
62 lines
1.4 KiB
Markdown
62 lines
1.4 KiB
Markdown
---
|
|
title: "analyzing-experiments"
|
|
aliases:
|
|
tags:
|
|
- info203
|
|
- lecture
|
|
- scott-video
|
|
sr-due: 2022-06-01
|
|
sr-interval: 7
|
|
sr-ease: 250
|
|
---
|
|
|
|
# 3 questions
|
|
- what does my data look like
|
|
- graphs, plots, differnent summary plots
|
|
- what are the overall numbers
|
|
- aggregate stats e.g., mean average std dev
|
|
- are the differences "real"?
|
|
- significance p-value
|
|
- likihood that results are due to chance
|
|
|
|
## p value
|
|
pearsons chi-squared test. comparing rate of expected value vs observed value
|
|
|
|
$$
|
|
\chi^{2}=\frac{(observed-expected)^2}{expected}
|
|
$$
|
|
|
|
"normal" outcome variance follow normal/gaussian distribution.
|
|
|
|
as chi squared gets bigger it is less likey that the coin is unbiased
|
|
|
|
e.g., 20 tosses, we got 13 heads. at p<0.05 can we reject the null that the coin is unbiased
|
|
|
|
|
|
|
|
|
|
|
|
degrees of freedom num possibilites n-1 = 2-1 = 1
|
|
|
|
we cannot reject the null
|
|
|
|
 reject the null
|
|
|
|
\
|
|
|
|
formalieses: "were pretty sure". helps generalize from small samples
|
|
|
|
for normal continiuous data
|
|
- t-tests (compare 2)
|
|
- annova (compare more than 2)
|
|
|
|
data is not always normal.
|
|
- bi modal - 2 peaks
|
|
- skewed
|
|
- e.g., time: can be infiniely slow, but not infinitely fast
|
|
|
|
|
|
non-normal data
|
|
- knowing is half tha battle
|
|
- run A/A tests
|
|
- use randomised testing |