Econ 378 Lecture Notes
Joseph McMurray
L0	Introduction
1.	Opening Prayer
2.	About me
a.	Raised in Salt Lake City, mission in Seoul Korea, Economics major at BYU, met my wife at BYU, PhD at University of Rochester, research in political economics (also teach Econ 477), 4 kids, sincerely believe in the Gospel of Jesus Christ and the mission of BYU.
b.	I enjoy teaching Econ 378 because the material is so useful for students, which is rewarding.  It is also hard, so making it interesting and easy is a fun challenge.
3.	Data analysis in Economics
a.	Scientific method: observe patterns, theorize, test theories, policy implications, policy calibration
b.	Theory: Econ 110, 380-382, 400+
c.	Evidence: Econ 378, 388, 400+, Research, internships, jobs (big data industrial transition)
d.	Economics is both (recommend Econ 210 for exploring careers in Economics, also MATH 213/215 linear algebra)
4.	Probability and statistics
a.	Often care about population but observe only sample.
Can’t know what’s true, but can know what’s probably true
b.	Probability is the language of uncertainty
c.	Probability theory also useful in theoretical models of risk/uncertainty (e.g. insurance, investments, search, asymmetric information)
5.	Syllabus
a.	Materials, participation, homework, exams, project
b.	How to choose a research topic
c.	Finish part 1 (data collection) on time!  After the midterm, homework will include data analysis from your own projects.
L1	Math Preview
1.	Spiritual thought: like Joseph in Egypt, your time at BYU is 7 years of plenty (spiritual abundance).  Likely less so when you go to graduate school or workforce.  Store up all you can now (e.g. devotionals, religion classes, student ward, ministering), like wise virgins with oil lamps, to sustain you as you “go forth to serve”
2.	In a similar (but temporal) way, this lecture and HW 1 seek to fill your “math lamps” in preparation for the rest of the semester.
3.	Factorials
a.	
b.	
4.	Exponents
a.	 denotes growth
i.	$1 invested at 100% interest, compound annually, equals $2 a year later
ii.	$1 invested at 100% interest, compound continuously, equals $2.72 a year later
Expression	Simplified / Rewritten	

		

		

		

5.	Logarithms
a.	 (How many powers of  give you ?)
i.	
b.	 (How many powers of  give you ?)
i.	
c.	Logs makes huge numbers smaller, miniscule numbers (e.g. probabilities) bigger
d.	Inverse of exponents
i.	 (How many powers of  does it take to reach ?)
ii.	 (It takes  powers of  to make ; what if we take  to that many powers?)
Expression	Simplified / Rewritten	

		

	

	Can’t simplify

	

	

6.	Summation
a.	
b.	Column of  observations can be denoted by , with 
c.	 denotes the average
d.	
e.	
f.	
g.	Does ?  No!
i.	e.g. 
7.	Derivatives
a.	Intuition: limit of slope
b.	Finding maximum/minimum
i.	First-order condition: 
ii.	Second-order condition:  negative for max (slope is decreasing, function makes a frown), positive for min (slope is increasing, function makes a smile)
c.	Simple derivatives

		

		

d.	Product rule: 
i.	
ii.	Same pattern for products of 100 terms
e.	Chain rule: 
i.	Example:  
ii.	Example: 
iii.	Same pattern for longer chains
f.	[The Quotient rule is useful as well, but I won’t require it here.]
8.	Integrals
g.	Intuition
i.	“sum”/area under  (negative if )
ii.	Anti-derivative: add up  to get 
	Anti-derivative of 	

		

		

h.	Definite integral 
 
i.	
i.	Useful techniques that I won’t cover (or expect you to know)
i.	-substitution (chain rule in reverse)
ii.	Integration by Parts (product rule in reverse)
j.	Double Integrals
i.	Simple: inside integral then outside integral

ii.	Note: for rectangular bounds (i.e. bounds of  don’t depend on , and vice versa), can integrate in reverse order

iii.	Practice 
L2	Statistics preview
Introduction
1.	We recently revised the Econ 378 curriculum.  Formerly, we started with basic theory and the basic tools based on it, introduced complex theory with complex tools, then more complex theory and more complex tools.  This seemed reasonable, but I realized: “Still to this day, I’ve never learned to build a car, but even without knowing how to build one, I managed to learn to drive one.”
2.	Now: learn basic, complex, and more complex tools upfront.  Then go learn the underlying theory.
Spreadsheets
1.	Unit of observation
2.	Quantitative variables
3.	Binary variables
a.	Categorical variables as binary (or “dummy”) variables
Data Visualization
1.	Single variables
a.	Binary variables: Pie charts
b.	Quantitative variables: Histograms
2.	Interactions
a.	Two binary variables: Double pie charts
b.	Binary and quantitative: bar chart
c.	Two quantitative: scatter chart
i.	Quantitative & time: line graph
d.	Three variables
i.	Two binary & quantitative: clustered bar chart
ii.	Two quantitative & binary: color-coded scatter chart
iii.	Three quantitative: bubble chart
Summary statistics
1.	Proportions
2.	Mean
a.	From histogram, eyeball center of gravity
3.	Median/percentiles
4.	Mode
5.	Standard deviation
a.	Rule of thumb: two standard deviations
b.	Chebyshev’s inequality: % of population outside  standard deviations can’t exceed 
6.	Correlation coefficient
a.	
b.	 gives fraction of variance in  that coincides with variation in 
7.	Regressions
a.	Slope & intercept
i.	Predict  for any 
ii.	Predict future!
iii.	Counterfactual “experiments” (way less costly than real experiments)
b.	 (coefficient of determination)
c.	Error terms / detrended data
d.	Multiple regression
Estimation
1.	Population / samples
a.	Importance of representative sample
2.	Point estimates, interval estimates / margin of error
3.	Hypothesis test
L3	Excel
Excel basics
1.	Intro FRED website: maintained by Federal Reserve Bank of St. Louis, 800,000+ macroeconomic variable series over time; virtually everything collected by federal agencies, and many variables collected by foreign governments and central banks.
2.	Downloading variables
a.	Choose variable 1: real GPD
b.	Edit chart > Add variable > population (quarterly, not monthly)
c.	Adjust years (1960 onward)
d.	Download .xlsx (explain .csv, etc.)
3.	Spreadsheet management
a.	Basic calculator (e.g.  displays )
b.	Navigation shortcuts: Ctrl+arrows, Ctrl+Shift+arrows, Ctrl+Home
c.	Adjust column widths, center cells, format color code
d.	Freeze pains
e.	Hide labels
i.	Show how to uhide
f.	Cut, paste, copy, slide cells, reverse cells
g.	Sort
4.	Construct variables
a.	Real GDP per capita = real GDP/pop, copy formula
i.	Always label variables!
b.	Real GDP per cap (absolute) growth = real GDP per cap minus previous
c.	Percentage real GDP per cap growth = real GDP per cap growth / previous
i.	Suppress digits / format percentage
d.	Pos growth (binary / Boolean / dummy variable) = 1*(growth > 0)
e.	Percentage population growth = (pop-previous)/previous
5.	Summary statistics
a.	Histogram of % GDP per cap growth, % pop growth
i.	File > Options > Add-ins > Analysis ToolPak + Go > Analysis Toolpak + OK
ii.	Data Analysis Toolbar > Histogram
iii.	GDP Bins from -10% to +10% by 1% increments (excludes one outlier)
1.	Reduce typing by extrapolating (“plus” cursor in bottom right-hand corner of cell)
iv.	Pop bins from 0% to .75% by .05%
b.	Min, max, mean, median, standard deviation
i.	Eyeball first
ii.	Compute for % real GDP per cap growth
iii.	Copy formulas for % pop growth
c.	Correlation % pop growth, % real GDP per cap growth
i.	Reasons to expect positive/negative?
ii.	Compute
d.	Line graph: % pop growth, % real GDP per cap growth
e.	Conditional mean
i.	Sort by pos growth
ii.	Compute mean % real pop growth (does pop growth accelerate/slow down in recessions?)
6.	Regressions
a.	Scatter % real GDP per cap growth over time
b.	Scatter real GDP per cap (not growth) and time
c.	Regress real GDP per cap (not growth) on time
i.	First create time index variable?
d.	Create predicted real GDP
i.	Use $ sign to pin formula
e.	Detrend to create business cycle variable: actual real GDP minus predicted real GDP
7.	Task: test whether pop growth is higher or lower on average in booms than busts
a.	Create boom dummy: residual > 0
b.	Sort boom, average % pop growth
8.	Filter
a.	Decompose date
b.	Filter by quarters
c.	Create dummy for first half of year (Q1-2)
d.	Is % real GDP per cap growth higher in first half or second half of year, on average?
Learning Statistics
1.	We just finished semester (overview).  You can now do everything by computer.  Rest of semester, we’ll go back and do them by hand.
2.	Why work by hand?  Important to know what computer is doing.  (GIGO)
a.	Pushing a button works great unless a situation arises when the standard button is the wrong one to use.  We need to know the limitations of the standard techniques and how to modify them appropriately.
b.	Car analogy: why insist on building cars when we could just drive them?  Analogy incomplete: I can objectively verify that I’ve correctly driven a car; I can’t objectively verify that I’ve correctly used statistics.  In the real world of research projects, internships, and jobs, there is no answer key!  We only know our answers are correct if we know we’ve done them correctly, and that’s only possible if we understand deeply what theoretical basis underlies the tools we’re using.
3.	Simple things (e.g. margin of error) mask extremely complex background.  Understanding entire background is essential for confidence that we’re using statistical formulas appropriately.  (Sometimes, tweaks are necessary.)
4.	Spiritual analogy: the “why” of the gospel.  If atheist friend is kind and righteous already, why need doctrine?  Even more happiness.  Example: doctrine of eternal marriage informs decisions to resolve conflicts, versus divorce.
L4	Stata: Basics
Stata
1.	Statistical programming
a.	Data sets get too big for spreadsheet software
b.	Many operations, nice to save work and save steps
i.	Helps with collaboration
ii.	Helps with finding and correcting errors (without redoing work)
iii.	Commands efficiently operate on large sections of data simultaneously
2.	Introduction
a.	Stata especially well-suited to economics (regression analysis).
b.	But expensive.
c.	Other stats packages are available (e.g. R, SAS, SPSS), can program in Python, C, Matlab, Java).  But learning new stats package or programming language is like using a Casio calculator when you’re familiar with TI—all the buttons do all the same things, they’re just located in different places.  So learning one language helps you pick up others quickly, as needed.  (Stata has specific value for Economics research assistants, future PhDs.)
3.	Basic user interface
a.	Open Stata.  Very different GUI (graphical user interface) than Excel, but same idea.  Feels more like computer programming software, for good reason.
b.	Click on Data Editor (Edit), enter data by hand: numbers 1-10 in column 1, make up ten values in column 2.
i.	Row entries are called observations (numbered automatically on left side)
ii.	Column entries are called variables, default named “var1” and “var2”
c.	Close data editor.  Notice in main screen, a running list of individual commands (and their results), based on what we just did manually.
i.	With our first action, we actually implemented three commands: set “obs” from 0 to 1, generate variable “var1”, and set the value of var1 for observation 1.
ii.	As we added more data, we set “obs” to 2, 3, …, 10, replaced var1 for observations 2, 3, …, 10, and then generated a second variable, “var2”, and replaced its values for observations 1-10.  (Note, we no longer needed to expand “obs” at that point.)
d.	Command line
i.	We can use the command line as a calculator: type display sqrt(8*2)
ii.	We can add additional commands using the command line.  Type: generate varthree
1.	Note: in Stata commands, shortcuts are indicated by underlining the minimal number of letters to convey the same meaning: “gen” or “gene” or “gener” or “genera” or “generat” or “generate” are all equivalent, but “ge” is ambiguous, and will therefore not work.
iii.	Error: when we generate a new variable, we need to define its values.  Let’s try again: generate varthree=3
iv.	Now open Data Editor again to see the result.  We have defined a new variable “varthree” to equal 3, not just for a particular observation, but for every observation.  This is the power of the programming approach to data: you can do lots of things at once!  (In Excel, there are shortcuts to copy and fill, but nothing this quick.)  Most of the time, we won’t interact with the spreadsheet directly; we’ll just be programming.
v.	To see this again, close Data Editor and type: replace varthree=2*var2 if var1>7.  We see that three real changes were made, and if we open the Data Editor again, we can see what that looks like.
vi.	What if we meant to type varthree=2*var2 if var1>5?  We could type over again, or we can push “PgUp” to repeat the previous command (and edit it before hitting return).  Pushing this multiple times allows us to repeat earlier commands.
e.	Left and right panels
i.	On top right-hand side is a list of the variables we have so far: var1, var2, and newvar.
ii.	The bottom right-hand side summarizes our data: we have three variables and 10 observations.
iii.	(If you ever need to, you can resize these panels by dragging their borders.)
iv.	Notice on left-hand side is list of commands we’ve used so far (red for the ones that returned error codes).  Let’s highlight the first 26 lines (click on the first, then shift-click on line 26) and push Ctrl+C to copy these.  (This creates var1 and var2 and replaces all 10 values of var1 but only the first 5 values of var2.)
v.	Break.  If you send the computer on an impossible task, it may think for a really long time.  To get it to stop so you can revise its instructions, click on “Break” (red X button).
4.	Do Files
a.	Click on “New Do File Editor” at the top of the screen (looks like a Word document, with a pencil).  This opens a new window, with a text editor.
b.	Type Ctrl+V to paste the command lines that we copied earlier.  This becomes a short computer program that, once we execute, will create two variables and replace their values.  (In computer programming, a “command” tells the computer to do something.  A “script” is a list of commands, to be executed in order.  Scripts in Stata are called Do files.)
c.	Note: some programming languages require a semicolon or other punctuation to denote the end of one command and the beginning of the next.  “Return” plays that role in Stata, so writing on separate commands on separate lines is sufficient.
d.	Always use a script, not the command line!
i.	Often, after many steps of manipulating data, you realize that you should have done step 3 differently.  In Excel, you might be able to hit “undo” repeatedly (if you haven’t saved and closed the program yet), but once you correct step 3, you’ll then have to redo all subsequent steps.  With a script, you can edit step 3 and recompile in moments.
ii.	A script is also useful for collaborators and replicators, as well as yourself when you come back to a project after a long pause.  I learned this the hard way: I downloaded data, manipulated it repeatedly using the command line, found results that were really interesting, and copied and pasted them into my research paper.  A more urgent project came up, so it was months before I came back to this.  When I came back, I did (what I thought were) the same manipulations but got totally different results.  I couldn’t figure out how to reproduce the results that I had recorded earlier.  Maybe I had been making a mistake?  Maybe I’m making a mistake now, but was previously correct?  If I had a paper trail of all my data manipulations at the time, I could compare my work now and then to see how they differ, and which is more reliable.  …But I don’t, and may never publish that paper.
iii.	For your data project, you will need to submit your Do file, not just your results.
iv.	Related programming tip: Never overwrite your original data set.  Use a script to open your original data set, make whatever edits need to made, and then save the revised data as a new data set with a different file name.  That way, every time you run your script, it pulls the same original data.
e.	Use comments to keep things neat, organized
i.	*Comments begin with an asterisk.
ii.	/* Or they can be enclosed (even mid-line) in asterisks and slashes */
f.	Save .do file using Save icon.
Data files
5.	Clear
6.	Computer programs are designed to recognize and interpret information, but only in specific formats, specified by file type (e.g. .pdf, .docx, .mp3).
7.	For example, Spreadsheets often saved as .csv (“comma separated values”)
a.	year,growth;2021,2%;2020,1%;2019,1.5% can be understood to be a 3x2 table
8.	Excel recognizes .csv or .xlsx, which adds formatting information.
9.	Stata stores data using .dta and stores scripts using .do
10.	Example: search Google for “Our World in Data Covid github”, download xslx of covid data.  Save as FileName (maybe FileName_raw).  (If you save in OneDrive, you should be able to access it from other computers.)  Good idea to save in its own folder, so that additional files all stay together.
11.	Directories
a.	Open file location and Shift-right click to “Copy as path”.  In Stata .do file, type import delimited “FilePath”, clear (with quotation marks around FilePath).
i.	The “delimited” command lets Stata know it’s a .csv file.
ii.	Note: “clear” option replaces data in Stata memory, if any (can only hold one dataset at a time).
b.	We’ll often be creating multiple files: the original .xlsx or .csv, the converted original .dta, a .do, a final .dta, and possibly intermediate .dta or other files.  To keep things organized, it’s a good idea to make a directory for each project.
c.	Stata is set to work within a “working directory”.  From any working directory, you can call a file that has a complete file path name.  But you can also set the working directory to be the file you want to work in, and not have to write out the complete file path each time.
d.	To see what directory is the current working directory, type pwd in the command line (for print working directory).
e.	cd command: can change working directory.  Then can rewrite earlier import command with shorter file name instead of complete file path.
12.	Resave as .dta
a.	export excel Filename_raw.xlsx, replace firstline
i.	firstline option recognizes first line of data set as labeling variable names.
b.	save Filename_raw.dta
c.	use Filename_raw(.dta), clear
13.	Save edited.dta at end of .do file
a.	save FileName_edited.dta, replace
b.	Note: replace option only important when re-running code.
Exploring data
1.	Browse
2.	Describe
3.	Lookfor
4.	Sort
5.	Order
6.	Summarize
a.	Summarize icu_patients_per_million hosp_patients_per_million
b.	Summarize tot* [where * is a place holder for any letter or combination of letters]
c.	Summarize *
d.	Doesn’t work with strings; notice location isn’t summarized.
e.	Summarize, detail gives medians and percentiles
7.	Generate
a.	Generate rich = gdp_per_capita > 11840.85
8.	Tabulate
a.	tabulate rich
b.	tab continent
9.	Histogram
10.	Correlate
11.	Scatter
L5	Stata: Advanced
Covid data:
Histogram
	Hist total_deaths_per_million if date==”2021-09-08”
Scatter
	Scatter total_deaths_per_million gdp
Regress
	regress new_deaths_per_million people_vaccinated_per_hundred if date=="2021-09-08"
Predict
	predict predictable_new_deaths if date=="2021-09-08"
	predict excess_new_deaths if date=="2021-09-08", resid
Twoway graphs
	graph twoway (scatter new_deaths_per_million people_vaccinated_per_hundred) (line predictable_new_deaths people_vaccinated_per_hundred) if date==”2021-09-08”
	twoway scatter new_deaths_per_million people_vaccinated_per_hundred || line predictable_new_deaths people_vaccinated_per_hundred
Collapse
	gen US=1 if location==”United States”
	gen US_newdeaths_percap = us*new_deaths_per_million
collapse (mean) new_cases_per_million new_deaths_per_million new_vaccinations_smoothed_per_mi [fweight=population], by(date)
Variable types
	help datatypes
	destring date, gen(date2)
	generate int numdate = date(date,"YMD",2021)

Graphs
	twoway line us_new_deaths_percap numdate || line new_deaths_per_million numdate
twoway line new_deaths_per_million numdate || line new_cases_per_million numdate
graph save filename.gph, replace
Time Series
	tsset numdate
	gen rising = new_deaths_per_million > new_deaths_per_million
Log files
	log using filename.og, replace
	log close
Review stata commands
L6	Probability, Combinatorics (WMS 2.1-6)
1.	How many have already collected data?  How many know what data to collect?
2.	Spiritual thought: many of our greatest priorities are not time sensitive, so it’s easy to procrastinate till “later”, but often this postpones blessings.  Let your daily activities match your eternal priorities!  (e.g. Don’t delay repentance, marriage/children, promptings, family history, making life plans, for movies/hobbies or even school/work)
3.	Probability
a.	Language of uncertainty
b.	Economic applications: risk/insurance, investments, shopping/learning
c.	Data analysis (huge): infer population characteristics from sample
d.	Fundamentally, probability is merely ratio
e.	Sample space / Universe
f.	Subset / Event
i.	
ii.	
g.	Probability function 
i.	A function is like a machine; in this case, output is number but input is set
ii.	
h.	Categorical descriptions: 50/500 unemployed = 10% unemployment rate / probability
4.	Set Notation
a.	Element
i.	 , 
b.	Subset
i.	 
c.	Empty set 
d.	Complement
i.	
ii.	
iii.	
iv.	Note: this is a simple but useful tool.  Sometimes it’s easier to derive  than to derive .  For example: the probability that two or more students in Econ 378 share birthdays is very difficult to find directly, but the complementary event (i.e. that no two students share a birthday) is not as bad.
e.	Intersection (“and”)
i.	
ii.	
f.	Union (“or”, “at least one”)
i.	
ii.	
g.	Mutually exclusive
i.	
ii.	 
h.	Collectively exhaustive
i.	
ii.	 
5.	Total probability: 
a.	Including both  and  “double counts” 
b.	Example: among set  of workers in particular industry, unemployment rate , women , intersection ; 
6.	Independence: 
a.	Example: 
b.	Not the same as mutually exclusive!  (Mutually exclusive events are highly negatively correlated)
7.	Combinatorics
a.	“ rule”
i.	6 pants (2 brown), 10 shirts (3 green);  probability that random outfit consists of brown pants and green shirt is 
ii.	Equivalently, since independent, 
b.	Number of ways to permute (i.e. order) 10 students: 
c.	Number of ways to choose 3 out of 10 students: 
i.	Numerator: there are  ways to order the 10 students
ii.	Denominator: but this double-counts ( times) orderings which shuffle the first three and last seven, but don’t change the identity of the chosen three
d.	Number of ways to assign 10 students into bins of 3, 5, 2: 
8.	Applications: discrimination lawsuit after 9 workers (3 immigrants, 6 natives) assigned to 4 dangerous jobs + 5 safe jobs
a.	All 3 immigrants assigned to dangerous jobs
i.	
ii.	Alternatively, can think sequentially: 
iii.	Alternatively, can assign workers to safe jobs: 
iv.	Alternatively, can assign jobs to workers: 
b.	2 out of 3 immigrants assigned to dangerous jobs
i.	
ii.	
c.	24 workers assigned to 10 safe and 14 dangerous jobs, lawsuit because 6 immigrants all assigned dangerous job
i.	
ii.	
iii.	If 5 out of 6 assigned dangerous job: , 
L7	Conditional Probability (WMS 2.7-10)
1.	If possible, be prepared next lecture with idea for research project
2.	Typically, don’t count to determine ; estimate from sample
Conditional probability
1.	Definition: 
2.	This is how online stores (e.g. Ebay, Amazon, Google) figure out what to advertise: given that you purchased a textbook, how likely are you to want a Lego set or motorcycle helmet?
3.	Story problem keywords: “given”, “conditional on”, “among”, or “out of”
4.	Example 1: Among set  of workers in particular industry, unemployment rate , women , intersection 
d.	Rectangular Venn diagram
e.	Unemployment rate among women 
f.	Fraction of unemployed who are women 
g.	Practice:
i.	Unemployment rate among men 
ii.	Fraction of unemployed who are men 
Independence
1.	Definition: ,  (equivalent to )
2.	What is the probability of a person being unemployed?  ; what if it’s raining outside?  Then the probability of being unemployed is .
3.	Surgeon joke (failing to account for independence): the bad news is that this type of surgery is successful only 25% of the time.  The good news is that the last three patients all died.
Event decomposition:
1.	If  are mutually exclusive and collectively exhaustive then 
2.	Example 1: 30% of web traffic comes from a Google add (), 30% from online newspaper (), and 40% from a product reviewer’s blog ().  40% of Google traffic, 20% of newspaper traffic, and 30% of reviewer traffic end in a sale ().  What fraction of overall traffic ends in a purchase?
a.	Step 1: draw event tree (first web source, then purchase decision)
b.	Step 2: translate question into notation.  , , , , , , wish to find 
c.	
 
d.	 and  are independent, since .  Is  independent of ?  Of ?
3.	Bayes’ Rule
a.	
b.	Therefore, can derive  from , or vice versa.
c.	
d.	Practice: find and interpret , ,  (mere coincidence that )
4.	Warning: think carefully about difference between , , and .  Be sure you know which you really want. 
5.	Note: It’s possible for composite probabilities and conditional probabilities to tell rather opposite stories
a.	Charig et al. (1986): Kidney stone treatment B looked more effective, but A was more actually effective more effective both with small stones and large stones (but stone size matters, and treatments A and B had been used disproportionately on large and small stones, respectively) 
Kidney stone size	Treatment A	Treatment B	
Small	81/87=93%	234/270=87%	
Large	192/263=73%	55/80=69%	
Both	273/350=78%	289/350=83%	

b.	MLB batting averages: David Justice was better in 1995 and 1996 but Derek Jeter was better in 1995-96.  Who is better batter?
Batter	1995	1996	1995-96	
Derek Jeter	12/48=.250	183/582=.314	195/630=.310	
David Justice	104/411=.253	45/140=.321	149/551=.270	
Either could be.  Likely depends on which is more predictive of 1997 (depends on other assumptions)
c.	Israel covid data: August 2021 (https://www.covid-datascience.com/post/israeli-data-how-can-efficacy-vs-severe-disease-be-strong-when-60-of-hospitalized-are-vaccinated)
i.	When covid Delta variant hit, Israeli hospitals filled up with covid cases: 214 that were unvaccinated and 301 that were vaccinated.  Since 60% were vaccinated, superficial conclusion is that vaccines make covid worse, not better!
ii.	But 60%=P(vax|cv).  We really want to know P(cv|vax) (actually, want to compare P(cv|vax) and P(cv|no vax))
iii.	
 
Vaccinated only catch covid  as often (i.e. vaccine 68% effective)
iv.	Nearly 80% of Israelis over age 12 were vaccinated against covid, so if it were unrelated random draw, 80% of covid patients should have been vaccinated; lower rate than 80% supports hypothesis that treatment helped.
v.	Put differently, so many Israelis were vaccinated that even though those vaccinated only got covid 68% as often, there were more vaccinated covid cases than unvaccinated covid cases.

L?	Research Design
Research questions
1.	There are two important ingredients to a good research study: a good question, and a good methodology for finding an answer
2.	Question selection
a.	What ideas do you already have for data analysis projects?
b.	What (topic) are you excited / passionate about?
c.	If you had a crystal ball, what would you ask?
d.	What if you had a crystal ball that could answer anything but that?  What would you need to ask so that you can figure out your own answer to the main question?
e.	Continue until so narrow you can collect your own data (the more specific, the better)
f.	Given (time and money) budget constraints, your project may need to settle for similar data
i.	Similar variables
ii.	A few observations
iii.	“Pilot study”: this is often what is done in real world
iv.	Proof of concept (consulting sales pitch): can even use fake data
3.	Data mining
a.	Given data (e.g. from a private business, a consulting client, etc.), ask, “what can we learn?” and “who is interested?”
b.	Example: private business data
c.	Typically needs to be paired with research question process above
d.	Example: what would CEO want to know?
Correlation may not mean causation!
1.	Three possibilities
2.	Causation: 
Theory: cell phones  distraction  accidents
Policy implication: banning cell phones will reduce car accidents
3.	Reverse causation: 
Not likely in this case (car accidents cause cell phone use?)
4.	Lurking variable: 
Example: careless (teenage?) drivers are prone independently both to use cell phones and (regardless of cell phone use) to get in accidents
Policy implication: banning cell phones will not reduce car accidents
5.	Historic instances of conflating correlation with causation
a.	The “Phillips curve” documented a negative correlation between inflation and unemployment, suggesting to policy makers that monetary policy could only avoid one problem by embracing the other.  They printed more money in the 1970s, hoping to lower unemployment, but discovered “stagflation”: the coincidence of high unemployment and high inflation.
b.	Documenting a positive correlation between on-the-job computer use and income, Krueger (QJE 1993) concluded that computers increase productivity, and recommended policies to increase computer use.  Using similar data, however, DiNardo and Pischke (QJE 1997) showed that income is also correlated with pencil use on the job and argued (tongue-in-cheek) that subsidizing pencils would be a much more cost-effective intervention.
c.	Can also conflate lack of correlation with lack of causation: in yesterday’s covid example, we derived that  and ,  so vaccine is 68% effective.  Correlation of  and  is negative, but weak.  If further condition on age (<50 vs. >50):
i.	
 
Vaccine 92% effective for this group.
ii.	
 
Vaccine 85% effective for this group.
iii.	If condition further, vaccine efficacy by age:
Age	Vaccine efficacy	Age	Vaccine efficacy	
12-15	100%	50-59	93%	
16-19	100%	60-69	89%	
20-29	100%	70-79	90%	
30-39	97%	80-89	81%	
40-49	94%	90+	92%	
iv.	Biggest determinant of covid is age (overall, 90+ over 1000 times more likely than 12-15 to be hospitalized with covid), not vaccine.  Since people of all ages got vaccinated, corr(vax,cv) gets weaker when not conditioning in age than when conditioning on age.  But even for oldest groups (where most “breakthrough” cases are occurring), vaccinated do way better than unvaccinated.
d.	These examples underscore importance of careful data work, understanding statistics!  Good intentions can easily be led astray by statistical subtleties.
Establishing causation (this is most of the work in economics)
1.	Random experiment
a.	Best method
b.	Example: force group A to drive with cell phone, group B to not
c.	Often impractical, ethically and/or logistically (e.g. only one national economy; no control group) or even impossible (e.g. race/gender)
d.	Natural experiment: wait for nature to run experiments
i.	These are rare, might wait a long time
ii.	Government policy randomly allocates permits for some drivers to use cell phones.
iii.	Angrist and Evans (1998): Does having more children affect mother’s income?  Lurking variables and reverse causation both likely.  But parents whose second child was (randomly) same gender as first child were more likely to have third child, (temporarily) reduced (poor) mother’s income
iv.	Angrist (1990): What impact (positive or negative) did military service have on men with (randomly) high Vietnam draft numbers had 15% lower incomes years later.
v.	Clever researchers keep eyes open for natural randomness, ask “what can we learn?”
vi.	Sources of randomness: lottery numbers (e.g. gambling, school choice, scarce social program), random executive decisions (e.g. dorm rooms, judge assignment, advertising), weather, earthquakes, accidents, terrorist attacks
2.	Second best: quasi-experiment
a.	Example: cell phones legal in one state, illegal in another
b.	Problem: other reasons for differential accidents (e.g. speed limits, enforcement, roads, recklessness?)
c.	Refinement: large number of states; before/after cell phone law change
d.	Pope (1989, BYU): Geneva Steel closed then reopened six months later, concomitant decrease then increase in local hospitalizations for pneumonia, pleurisy, bronchitis, asthma.  (Landmark study in air pollution.)
e.	Sargeant et al. (2004): Restaurant smoking ban in Montana, repealed after six months.  Heart attacks dropped 40%, then went back up.
f.	Lee et al. (2004): How does politician (Democrat/Republican) affect policy outcomes?  Random election?  No.  But in close elections (e.g. 48-52% votes), winning or losing was plausibly random.
g.	Possible sources of quasi-randomness: cutoffs (e.g. grades, income thresholds, performance thresholds, birth date), bureaucratic decisions that are not literally random but seem arbitrary (e.g. regulatory decisions, tax levels, regularly/tax hike timing, pre-/post-construction project, mission assignment)
3.	Controls
a.	Compare sub-populations to “control” for lurking variables
b.	Most common method (since others infeasible)
c.	Example: compare cell phone use and accidents among teenagers/adults
vii.	Other proxies for recklessness: grades? debt?  Often imperfect
d.	Econ 378: restrict sample (Econ 388: regressions)
Prediction
1.	If correlation does not reflect causation,  cannot be used to control , but still can be used to predict 
a.	Example: reduced car insurance premiums for good grades, females, good driving history, yellow cars
b.	Ethics of “statistical discrimination” (e.g. traffic stops for blacks, airport scrutiny for Arabs)
c.	Role of theory is to posit reasons for correlation; essential if anything changes (e.g. cell phones get cheaper). 
Research Design for Causal Inference
1.	Many of the topics we’re interested in seek to establish cause/effect relationships.
a.	What examples did you come up with?  (e.g. Do masks reduce covid spread?)
b.	Were there any topics you came up with that do not involve cause/effect relationships?  (Probably not.)
2.	What is a cause/effect relationship you would like to discover?
3.	Which variables might have a simple correlation that suggests the relationship above?
4.	Are there any competing forces that might produce the opposite correlation?  If the correlation turns out to be consistent with a hypothesized cause/effect relationship, the hypothesized relationship might outweigh any competing forces.
5.	But are there other mechanisms that could produce the same correlation?  If so, finding a correlation where you expected it does not guarantee that the hypothesized cause/effect relationship is valid.
6.	This raises a new question: where could we look for evidence of the hypothesized cause/effect relationship that would not pick up correlations for these other reasons?
7.	This is the key question of research design.  Note that you can (and should!) think through and plan out your response to these issues before you ever look at the data.
Research design in the quest for spiritual truth
1.	A friend, skeptical of spiritual things, recommended the following experiment: go to a hospital, pray for people in every other room.  See if they recover more quickly/fully than the others.  (His prediction: no.)  Is this a valid statistical test of the validity of prayer?  Why or why not?
2.	Research design is important in answering spiritual questions, too:
a.	What do the scriptures say about experiments to uncover spiritual truth?
b.	“If any of you lack wisdom, let him ask of God, who giveth to all men liberally, and upbraideth not; and it shall be given him.  But let him ask in faith, nothing wavering.” (James 1:5-6, emphasis added)
c.	“And when ye shall receive these things, I would exhort you that ye would ask God, the Eternal Father, in the name of Christ, if these things are not true; and if ye shall ask with a sincere heart, with real intent, having faith in Christ, he will manifest the truth of it unto you, by the power of the Holy Ghost.” (Moroni 10:4, emphasis added)
d.	“Now, we will compare the word unto a seed.  Now, if ye give place, that a seed may be planted in your heart, behold, if it be a true seed, or a good seed, if ye do not cast it out by your unbelief, that ye will resist the Spirit of the Lord,	 behold, it will begin to swell within your breasts; and when you feel these swelling motions, ye will begin to say within yourselves—It must needs be that this is a good seed, or that the word is good, for it beginneth to enlarge my soul; yea, it beginneth to enlighten my understanding, yea, it beginneth to be delicious to me.” (Alma 32:28, emphasis added)
e.	To me, asking in faith means an honest willingness to follow the promptings received.  If I don’t honestly intend to follow impressions that are given, the experiment is void.
f.	“If any man will do his will, he shall know of the doctrine, whether it be of God, or whether I speak of myself.” (John 7:17, emphasis added)
g.	Mission friend: finally prayed about the Book of Mormon but “nothing happened”.  Zone leader: real intent might mean praying more than once.  After continued prayer, he received confirming witness.

L9	Probability Distributions (WMS 3.1-3)
1.	Events are useful for describing binary/categorical information.  Next, we’ll consider random variables, which are useful for describing quantitative information.
2.	Random variables
a.	Distribution Function: Number of cars  owned by a family 
b.	Mean (i.e. average) “mu” 
i.	We don’t know total population size.  If we knew there were  families, .  If population size  is unknown then  but this reduces to …
ii.	Expected value (or “expectation”) 
1.	Note: if all realizations of  are equally likely then  for all  so  reduces to familiar formula
c.	Expected value of functions of 
i.	Example: expected utility )
ii.	Formula: 
iii.	Example: 
iv.	Algebra shortcuts
1.	Linear functions, e.g. car maintenance cost ; average maintenance cost 
2.	Shortcuts: 
  
a.	Summations: 
b.	Pull out coefficients
c.	For constant , 
d.	Variance, standard deviation
i.	Variance 
ii.	Standard deviation 
1.	Interpretation: by rule of thumb, “most” families own between -.1 and 3.5 cars
2.	Note: variance, by itself, is difficult to interpret (e.g. units is “cars squared”), but is easier to work with algebraically, because it’s technically and average of something, whereas standard deviation is the square root of an average of something.
iii.	Simpler formula: 
1.	Show equivalent, as homework problem
2.	Remember this formula, as we’ll use it repeatedly
iv.	Algebra shortcuts
1.	 doesn’t have any easy algebra shortcut;  
2.	Shortcut: 
a.	 gets added and subtracted: 
b.	For constant , 
c.	Pull out coefficients, ... but square them! (because Variance is a quadratic function of a random variable)
3.	Practice example: number  of car accidents 
a.	
b.	
c.	
d.	
e.	Insurance profit 
i.	
ii.	
iii.	
f.	Review this one more time before attempting your homework!
L10	Correlation (WMS 3.1-8)
[Long lecture; use time efficiently]
1.	Correlation coefficient 
a.	Positive correlation means variables  and  tend to move in same direction (e.g. temperature  and ice cream sales )
b.	Negative correlation means variables  and  tend to move in opposite direction (e.g. frequency of exercise  and body mass index)
c.	Magnitude indicates strength of relationship
d.	Independence implies 
2.	Joint distribution function
a.	Employee hours per week  and hourly wage 

	.2	.1	
	.1	.2	
	.1	.3	

Illustrate: 
3.	Marginal distributions
a.	Sum rows or columns:  and 
b.	Summary statistics (quick recap)
i.	
ii.	
1.	
2.	
3.	
iii.	
iv.	
1.	
2.	
3.	
c.	Independence
i.	Definition:  for every  pair
ii.	Not independent here, since 
4.	Expectations of functions of  and 
a.	Average weekly payment 
b.	Can do any function 
5.	Correlation
a.	Covariance

 

 
 
b.	Simpler formula: 
c.	Correlation 
i.	Positive, but fairly weak (again, not independent)
ii.	Later:  measures % of variation in  that covaries with  
6.	Algebra shortcuts
a.	Covariance of a sum
 
b.	Correlation of a sum
 
c.	Variance of a sum
 
d.	Variance of a larger sum
  (with 100 variables,   terms)
i.	Importance of independent samples
7.	Application: financial diversification
a.	Assume two stocks have same average return  and same standard deviation .
b.	You could buy two shares of , or one share of  and one share of .  Since you are indifferent between  and , it might seem that you should be indifferent between these two stock portfolios.
c.	However, on your homework, you will prove that  but , as long as  and  are not perfectly correlated (i.e. ).  In fact, if they are perfectly negatively correlated then !
d.	Thus, the common financial advice to “Diversify your portfolio”.
8.	Practice [if time]: Cell phone use  and number  of car accidents
				
	.60	.08	.02	
	.10	.12	.08	

a.	Note: numerical values can be assigned to binary categorical variables, so that notion of correlation is still meaningful.
b.	Marginal distribution , mean , variance , standard deviation 
c.	Marginal distribution , mean , variance , standard deviation 
d.	Not independent since 
e.	
f.	Covariance 
g.	Correlation  positive and moderately strong
L11	Continuous Distributions (WMS 4.1-3)
1.	Continuous random variables
a.	Infinite domain, e.g. sleep hours 
b.	Philosophical view: continuous functions conveniently approximate discrete world, or world is truly infinite but measurement is imprecise
2.	Probability density function (pdf) 
a.	Measures relative likelihood of individual  values
b.	Individual  values occur with zero probability (and  is possible); to find probabilities, must take definite integral 
c.	Density must be non-negative and integrate to one over domain (just like probabilities sum to one)
d.	Example ; 
i.	Not directly from (finite) data; maybe from calibrated theory
ii.	Find 
1.	 requires that 
2.	That is, ; 
iii.	Mode solves ; solution at 
1.	Note: if  everywhere positive/negative then maximum is at highest/lowest  in range
2.	Note: second-order condition  satisfied as long as 
iv.	Probabilities: 
3.	Cumulative distribution function (cdf) 
a.	
 
(This assumes ; if  then  and if then )
b.	Percentiles
Median ; solving by computer, 
75th percentile 
90th percentile 
c.	Easier probabilities 
 
d.	From cdf, get pdf ; , else 
4.	Moments
a.	Mean
  (just like )

b.	Standard deviation
i.	
 
ii.	
iii.	
c.	Note: algebra tricks still work (e.g. lost wages while sleeping)
i.	
ii.	
5.	Practice describing steps to classmate: Warehouse stock (as fraction of capacity) 
a.	Find 
b.	mode 
c.	Draw and interpret pdf (upside-down parabola; warehouse full more often than empty)
d.	Find cdf 
e.	Find  from 
f.	
g.	median , 75th percentile 
h.	mean 
i.	standard deviation 
j.	Insurance payout 
i.	 
L12	Continuous Joint Distributions (WMS 5.1-8)
1.	Joint Density
a.	Compare discrete/continuous pdf and joint pdf 
b.	Warehouse stocks up to two pallets of cereal  and one pallet of cereal , with density .
c.	Height of joint pdf represents likelihood of particular  pairs.  Must integrate to one (double integral).
i.	 requires ,
or .
d.	Mode: since upward sloping in both dimensions, mode at 
2.	Marginal densities
a.	Analogous to margins of table in discrete joint distribution: total probability of particular realization of  is the sum of all joint probabilities of  pairs, with that particular  value, but any  value in domain.
b.	
c.	
d.	Subscript simply distinguishes  from 
e.	Moments: means, standard deviations
i.	
 
ii.	
 
iii.	
iv.	
v.	
 
vi.	
 
vii.	
viii.	
ix.	Could also derive mode, median, cdf, percentiles of  or 
f.	Independence requires  for all  pairs.
i.	 and  not independent here, since  (e.g. when )
3.	Correlation
a.	
 
b.	
 
c.	
4.	Practice example:  for 
a.	Find :   implies 
b.	Find marginal densities , :  for ; symmetrically,  for 
c.	Find means  and  and standard deviations  and :

 
 
Symmetrically, , 
d.	Correlation :

 
L13	Conditional Distributions (WMS 5.3, 11)
1.	Recall distribution of cell phone use and car accidents:
				
	.60	.08	.02	
	.10	.12	.08	

a.	Recall 
2.	Conditional probability
a.	Cell phone use among those with two accidents  versus those with no accidents 
b.	Practice: find , ,, , , 
3.	Conditional distribution
a.	 and  are legitimate distribution functions (numerators sum to denominator)
b.	Plot, and compare with 
c.	Conditional mean
i.	 
ii.	Practice 
d.	Average number of car accidents is higher for cell phone users than for non-users.  This still doesn’t imply causation!
e.	Conditional standard deviation
i.	Just like ,
 
ii.	(If time) Example :  , so 
In this case, , and  
By a similar derivation, ; cell phone use increases variance.
4.	In an effort to establish causation, could find  or , and then .  For example, find correlation between cell phone use and car accidents among teenagers.
5.	Continuous densities
a.	Recall joint density of cereal inventory, 
b.	Recall marginal densities  and , means , ,  standard deviations , 
c.	Conditional density .  For example, 
d.	Conditional mean and standard deviation

.
 
Thus, when : density of  is steeper, mean of  is higher, variance is lower.
e.	More generically, for arbitrary  value,

 	For example,  as before
Practice: 

 
 	
 .  For example,  as before, 
Thus, when , density of  is less steep, mean is lower, variance is higher.

6.	With three variables, can derive joint distribution of  and  conditional on 
a.	Israel covid data
i.	When covid Delta variant hit, Israeli hospitals filled up with covid patients.  60% of these patients had already been vaccinated.
ii.	Natural (but erroneous) conclusion: vaccines make things worse, not better!
iii.	Nearly 80% of Israelis over age 12 were vaccinated against covid, so if “random draw,” 80% of hospital patients should have been vaccinated; lower rate means treatment helped.
iv.	
7.	
L14	Regressions (WMS 5.3, 11)
1.	Regressions
a.	Sir Francis Galton 1886 (cousin of Darwin)
b.	Use data to determine (average) linear relationship .  Once relationship is known, we can predict  for any  value (even out of sample), as if through a crystal ball!
c.	Examples:
i.	Income  as function of education 
ii.	Unemployment  next year as function of (e.g. fiscal or monetary) policy 
iii.	Stock price tomorrow  as function of today’s earnings/price 
iv.	Consultant’s “secret formula” predicting sales, etc.
d.	Puts units on correlation (“education and income are strongly correlated” vs. “each year of education is associated with an additional $4k of income”)
e.	Working example: education  years;  years; income ; ; correlation 
f.	Any  and  determine line , implying an error term  such that the data satisfies .  We can choose  and  so that the resulting line is as useful as possible.
g.	“Least squares” regression: choose  and  to minimize 
i.	Equivalently, choose  so that  and  to minimize 
ii.	Can use other criteria (e.g. least absolute deviation ), but less common
2.	Intercept
a.	If  is high, most  will be negative; if  is low, most  will be positive
b.	 implies that .
Easier: , so regression line passes through 
c.	Example: 
3.	Slope
a.	
b.	To minimize, 
c.	Solution 
d.	Slope is simply (normalized) correlation coefficient
e.	Example: /yr. (e.g. four-year degree provides extra $16,000/yr)
f.	Equivalently,  solves  (see homework)
4.	Predictions
a.	High school grad () expects 
b.	College grad () expects 
c.	Three PhDs () expects 
i.	This assumes linear trend holds up, constant returns to scale (which may not be reasonable); in econometrics (Econ 388), learn nonlinear regressions
d.	Standardized
i.	 (since , , and ).
ii.	If  is  or  or  standard deviation above  then  is  or  or  standard deviations above 
e.	Stay in school to get rich?
i.	Caveat 1: I made these numbers up.  Before making important financial decisions, you should collect the true numbers.
ii.	Caveat 2: We’ve modeled this as straight line, implying constant marginal returns to education; if decreasing marginal returns, might be better to use parabola (take Econometrics first).
iii.	Caveat 3: Regressions just express correlation, still not causation (despite popular terminology of “dependent” and “independent” variables).
1.	Maybe causation: school teaches useful skills that generate income.
2.	Maybe reverse causation: schooling is pure consumption, and wealthy individuals can afford more.
3.	Maybe spurious correlation: smart kids enjoy school (just as athletes enjoy sports) but would earn high incomes with or without school.
iv.	Either way, predict higher incomes for those who do stay in school: going to school increases my prediction of your income, even if it doesn’t increase your income.
f.	Reverse predictions
i.	What if worker makes $100k income and asks for you to guess education?
ii.	Could read regression backward, but it was designed to minimize errors in income not errors in education
iii.	Better to start over, with income as  and education as 
5.	Errors
a.	
b.	De-trend data (e.g. “skill” or “luck”, above and beyond education)
c.	Example: who is more genius (or luckier):  or ?
i.	
ii.	
d.	Variance  of error distribution tells us how far people are off the regression line

6.	Explanatory power (%)
a.	Partition 
i.	Note:  (see homework) because optimal slope extracts all correlation
ii.	This decomposes  into “explained” 144 plus “unexplained” 256 (e.g. talent, luck, or some other mystery).  (Warning: terminology sounds like causation, but isn’t; more accurately, variation is “related to education” and “unrelated to education”.)
b.	“Explained” portion is  fraction of 
i.	“Explained” portion is 
ii.	Generically, ; thus, “explained” variation is always  (sometimes called “coefficient of determination”) fraction of .  In this case 
c.	“Unexplained” portion is 
i.	In this case, , so 
d.	Implicit linearity of 
i.	Fundamentally, what does  measure?
ii.	 is perfectly predictable from , but linear regression produces 
iii.	Thus, 
iv.	 fundamentally measures linear relationship (see homework)
Exam 1 Review
1.	Spiritual thought: prayer through life’s trials, faith without works is dead, obedience gives confidence
2.	Exam info
a.	Any calculator
b.	No time limit, predict 2-3 hours
c.	Handout provided
d.	Hard: typically C average
3.	Study tips
a.	Take it seriously: equal weight with final exam
b.	Start with study guide
c.	Practice exams (first without solutions, then with)
d.	Extra homework problems (or repeat homework problems)
e.	Time saver: talk through steps, don’t work out algebra
4.	Exam strategies
a.	Don’t forget to pray!
b.	Extend familiar material to unfamiliar settings (good practice for real world)
c.	Difficulty is similar to homework, but no TAs or books, so fewer A’s than homework
d.	Average score is typically C, which averaged with A- homework gives B- overall.
e.	Show work and list what you know for partial credit (e.g. , even if you can’t figure out )
5.	Key formulas
a.	Binary events
i.	
ii.	
iii.	
iv.	Independent events:  (or )
v.	
vi.	
b.	Random variables
i.	Legitimate distribution?  
ii.	Mode maximizes  or  (i.e.  and )
iii.	
iv.	
v.	; 
vi.	, 
vii.	
viii.	Percentile: solve 
ix.	
c.	Joint distributions
i.	Legitimate joint distribution? 
ii.	Marginal distribution

iii.	Independent variables

iv.	
v.	
vi.	
vii.	Conditional distribution

viii.	
ix.	
d.	Regressions
i.	
ii.	
iii.	
iv.	
6.	Algebra tricks
a.	
b.	
c.	
d.	
7.	Distributional relationships
a.	If  then  and 
b.	If  then 
c.	If  then 
d.	If  independent then  and 
e.	If  and  independent then 
8.	Rejoice in how much we’ve learned!
L15	Bernoulli, Uniform, Standard Normal (WMS 4.4-4.5)
Spiritual thought
1.	Dealing with disappointment
a.	In grad school, we took two years of courses, then two qualifying exams.  If pass, four years of research; if fail, retake or exit with Masters degree.  I prepped hard, but on day of exam, got hung up on one really hard question, lost track of time, didn’t finish, and failed!
b.	I benefitted from a friend’s experience, who had previously been preparing for graduation (robes, parents in town, etc.), when checked grades: E!  Couldn’t graduate.
i.	First reaction: denial.  Must be a mistake!
ii.	Second reaction: blame.  Grading is unfair!
iii.	Third reaction: dejection.  I’m a failure.
iv.	Fourth reaction: hope.  I’m not a failure, I just failed at this thing.  I can move forward productively to the next step.  Retook class, found a summer internship that turned out to be career altering.
c.	Scriptures
i.	Joseph Smith in Liberty jail: “My son, peace be unto they soul; thine adversity and thine afflictions shall be but a small moment; and then, if thou endure it well, God shall exalt thee on high; thou shalt triumph over all thy foes” (D&C 121:7-8).
ii.	“Search diligently, pray always, and be believing, and all things shall work together for your good, if ye walk uprightly and remember the covenant wherewith ye have covenanted one with another” (D&C 90:24).
iii.	“…All things work together for good for them that love God…” (Romans 8:28).
d.	Midterm exam: If you performed less well than you hoped, press forward with a perfect brightness of hope!  Help the Lord make it work toward your good.
i.	Learn what went wrong (like spelling bee mistakes, may always remember).  Final exam not cumulative per se, but does repeat concepts.
ii.	Reassess study habits (e.g. understand every step of every question; don’t just trust TA or study group).
2.	 Bernoulli() (after Swiss mathematician Jacob Bernoulli, 1713)
a.	Recall cell phone use 
b.	Mean 
c.	Variance 
d.	Pattern: ,  for “success” parameter 
3.	Uniform()
a.	
b.	
c.	
d.	
e.	Example: 90 second stop light
i.	Average wait time 
ii.	Standard deviation 
iii.	Wait less than 30 seconds with probability 
1.	Standard normal 
a.	 (integrate using polar coordinates or trig substitutions)
b.	 (u substitution)
c.	 (integration by parts)
d.	Practice reading Table A
i.	Excel: NORM.S.DIST(x, cdf?) or NORM.S.INV(percentile)
ii.	
iii.	
iv.	
e.	Symmetric: 
L16	Normal, Chi Square, t Distributions (WMS 4.5-4.6)
1.	Standardization (for later reference)
a.	If  and  then you can always change units to create a new random variable  such that  and 
i.	
ii.	
2.	Normal (or Gaussian, after German mathematician Carl Friedrich Gauss, 1809) 
a.	 (integrate using polar coordinates or trig substitutions)
b.	 (u substitution)
c.	 (integration by parts)
d.	No analytical cdf; instead, approximate numerically
i.	Excel: NORM.DIST(x, mu, sd, cdf?)
ii.	Percentiles: NORM.INV(percentile, mu, sd)
e.	Special Properties
i.	
In other words, adding a constant changes the precise distribution of  but keeps it in the normal family
1.	Note: this is true of some other families of random variables (e.g. uniform) but not all (e.g. Bernoulli, binomial, exponential)
ii.	
In other words, multiplying by a constant keeps  in the normal family
1.	Note: this is true of some other families of random variables (e.g. uniform, exponential) but not all (e.g. Bernoulli, binomial)
iii.	
That is, if  and 
then 
In other words, the sum of two normally distributed random variables is a normally distributed random variable
1.	Note: this is true of some other families of random variables (e.g. independent binomials), but not all (e.g. Bernoulli, correlated binomials, uniform, exponential)
3.	Standard normal 
a.	Practice reading Table A
i.	Excel: NORM.S.DIST(x, cdf?) or NORM.S.INV(percentile)
ii.	
iii.	
iv.	
b.	Symmetric: 
c.	Standardized normal  is standard normal  (because of special properties of normal )
d.	Example 1:  to find 

e.	Example 2: costs 
i.	Budget  so that 
ii.	 (from Table A)
iii.	If  then 
f.	Example 3: costs ) and revenue  are independent; how often are profits  positive?
i.	
ii.	
iii.	
iv.	So 
v.	
4.	 (German statistican Friedrich Robert Helmert, 1875)
a.	Domain is , roughly bell-shaped (but asymmetric, unlike Normal distribution)
b.	 is often called “degrees of freedom”, because in the most common application, it corresponds to how many 
c.	 and 
d.	 (I won’t expect you to know or use)
e.	CDF  approximated on Table 6
i.	 represents a realization of the random variable, where  is the probability to the right of that value (i.e., )
ii.	Example: suppose sales follow Chi-square distribution, with average of 30 units
1.	Degrees of freedom 
2.	10th percentile is , 90th percentile is 
3.	Putting these together, 
iii.	Note: Table 6 only gives 10 points on the cdf.  With a computer, you can get the rest.  Excel: CHISQ.DIST(x,df, cdf?), CHISQ.INV(percentile, df)
f.	Facts
i.	If  then 
ii.	If  and  independent then 
iii.	Variance is a quadratic function of a random variable, so when we estimate the variance of a random variable that has a normal distribution (in lecture L19), our estimates will follow a  distribution.
5.	 distribution (Friedrich Robert Helmert 1876, Karl Pearson 1900)
a.	; as in Chi-square distribution,  is called “degrees of freedom” 
b.	Similar to standard normal, but with higher variance (i.e. thicker tails)
c.	Approaches  as 
d.	 (I won’t expect you to know or use)
e.	, 
f.	CDF  approximated on Table C
i.	Table is oriented so that probability  lies between  and .
ii.	Example: if  find 90th percentile
1.	Following  (fifth column) for  leads to .
2.	In other words,  of the distribution is left of ,  is between  and , and  is above .
3.	Since  of the distribution is below  and  is above,  is the 90th percentile of the distribution.
4.	Alternatively, can come up from a one-sided p-value of  or a two-sided p-value of  (bottom of the table) to reach the same conclusion.
iii.	For degrees of freedom greater than , can read  row of the table, which corresponds to a standard normal distribution (i.e.,  degrees of freedom).
iv.	Note: Table C only gives 12 points on CDF.  With a computer, you can get the rest.  Excel: T.DIST(x, df, cdf?) and T.INV(percentile, df)
g.	Fact
i.	If  and  independent then 
ii.	If we knew the population variance, then estimates of the mean would follow a normal distribution.  Since we have to estimate the population variance, and estimates follow a  distribution, our estimates of the mean follow a  distribution
6.	Other distributions
a.	The distributions we’ve gone over are some of the most common; there are many others, with various shapes, properties, and uses.
b.	Illustrated: https://www.itl.nist.gov/div898/handbook/eda/section3/eda366.htm
c.	Discrete
i.	Uniform
ii.	Binomial
iii.	Geometric
iv.	Poisson
v.	Hypergeometric
d.	Continuous
i.	Exponential
ii.	F
iii.	Beta
iv.	Gamma
v.	Log-normal
vi.	Pareto
vii.	Weibull
L17	Confidence Intervals (WMS 7.2-3,8.5-9)
Project note
1.	If you didn’t turn the project in on time, get it in ASAP!  Items from part 2 of the project will show up on homework; if you do them with your homework, your project will be finished by the end of the semester.
2.	From now on, must start on project as part of homework
3.	Keep results, to submit as project
4.	Note: If you have a population instead of a sample from a population (e.g. all 50 states), just pretend this is a sample from a larger population (i.e. 50 draws from a population of thousands of U.S. states).
Samples
1.	Population vs. sample
a.	So far, our discussion of distributions has presumed an entire population.  Often, information is only available from a sample.
i.	Surveys are costly, populations are often huge
ii.	Some of you might have whole populations (e.g. all 50 states, all teams, every week of a company’s sales data); for projects, pretend sample even if you actually have population.  But be careful:
1.	Sometimes population of interest includes future generations (e.g. NBA rookies, stock returns).
2.	Similarly, population of things that actually happened can in some cases be viewed as a sample from the larger set of things that potentially could have occurred instead.
b.	Unless entire population is observed, can’t know what is true, only what is probably true
2.	Random sample
a.	i.i.d. (Independently and Identically Distributed): survey answers are independent from each other, and identical to population of interest
b.	Convenience survey (e.g. urban survey of wages): expand sample or limit scope of inference
c.	Selection bias (e.g. survey participation, program participation): administrative records, measurements before participation decided, interpret results narrowly (e.g. benefit of college for those who chose to attend)
d.	Time trends (e.g. daily/weekly sales) – rare “spot check” observations, econometric corrections
3.	Estimation
a.	Example: Suppose we wish to estimate the average family size  of BYU students, along with the standard deviation  and the correlation  between family size and GPA.  What pieces of data should be used, and how should they be combined?
b.	Population parameter  (i.e. generic proxy for , etc.), seek “estimator” function  (commonly denoted by “hat” variable)
i.	Evaluating this “estimator” function with our data provides point estimates; next two lectures we’ll talk about interval estimates, or margin of error
c.	An estimator is a tool for producing estimates.  We’ll spend most of the semester talking about a variety of such tools (i.e. estimators for different parameters) but first we need some tool-building tools (i.e. techniques for developing estimators in new settings of interest).
Estimators vs. estimates
1.	Example: suppose distribution of income among last year’s  BYU graduates has mean  and standard deviation 
But we can’t observe this, so we survey  graduates and estimate  and 
2.	Sampling distribution
a.	Every survey of students yields different estimates .  Sampling with replacement, there are  such samples.
(Sampling without replacement is more common in practice, and violates i.i.d. but only slightly, as long as population size is large.)
b.	Before we conduct interviews, survey responses  can be viewed as random variables, each drawn from the population of BYU grads
3.	Estimates and estimators
a.	Once we conduct survey,  provides estimate of parameter .  Before we conduct survey, estimator  is random.
b.	To evaluate estimation procedure, we must think about entire distribution of estimates (in other words, evaluate estimator), not individual estimate.
c.	Therefore,  is random variable with mean  and variance 
d.	Similarly,  is random variable with mean  and variance 
Margin of error
1.	Recall that

2.	Previous estimates are point estimates; margin of error (e.g. ) measures precision, gives interval estimate
3.	Example
a.	Income  of 8,500 BYU graduates has unknown mean  and known standard deviation .
b.	If  then  has same mean , and standard error 
c.	Rule of thumb:  typically within ,  typically within ; thus,  is “margin of error”
d.	Dog and leash principle: 3 ft. leash keeps dog within 3 ft. of owner; symmetrically keeps owner within 3 ft. of dog
e.	Observe 
Maybe  as low as  and we overestimated
Maybe  as high as  and we underestimated.
4.	If  unknown, can estimate margin of error using 
Confidence Intervals
1.	How often is  in interval ?  To compute probability, we need the cdf of .
2.	Normality of 
a.	If population distribution of  is normal then  is normal too.  Specifically, .
b.	Standardizing, .
Confidence interval for 
1.	Construction
a.	We want  and from Table A we know (still assumingand)

 
 
b.	(Can also construct one-sided confidence intervals)
c.	Example:  (still assume ; later we’ll estimate)
i.	90% confidence interval 
ii.	95% confidence interval 
iii.	99% confidence interval 
2.	Distribution of 
a.	If  and  then .
b.	Intuitively, expectation of  is , expectation of  is .
3.	Confidence interval for  when  unknown
a.	If we replace with  then .
i.	This is because , which is  divided by 
ii.	Note:  if  large then .
b.	Example: average weekly income , , , 
i.	90% confidence interval 
ii.	95% confidence interval 
iii.	99% confidence interval 
Central Limit Theorem (de Moivre 1733, Laplace 1812, Lyapunov 1901)
4.	 (and therefore ) no matter what the distribution of 
5.	Dice example
a.	Distribution of  is bell-shaped, even though  is (discrete) uniform
b.	Intuition: centrist values frequent (e.g. moderate  and , or  low  high, or vice versa), but extreme values rare (e.g.  and  both low)
c.	; tails become exponentially less likely (key feature of normal distribution) as  increases
6.	Skewed example
a.	Bernoulli unemployment 
b.	Average of two: 
c.	Average of four: 5,
7.	CLT explains why normal distribution is so prevalent in nature: one attribute is sum total of many, smaller, independent attributes
L18	Hypothesis Tests (WMS 10.2-8)
1.	Hypothesis test: old profit , new management; keep or fire?
a.	Null hypothesis (benefit of doubt) 
b.	Alternative hypothesis (burden of proof) 
c.	Level 
d.	Test statistic 
e.	Critical value , rejection region to left
f.	Data:  over 8 weeks
g.	If  true, ; reject 
h.	“Significantly less than 400” (statistical vs. practical significance)
i.	Type 1 error: convict innocent (probability )
j.	Type 2 error: acquit guilty (probability )
k.	Repeat for ; critical value , fail to reject
l.	P-value = smallest  such that (for this data) reject ;  in this case
m.	Practice: Reject if ?
2.	Variations
a.	 (expect and tolerate adjustment cost $20 for new); test statistic increases to , p-value increases to .  (At  level, $350 is significantly less than $400, but not significantly less than $380)
b.	; if still  then critical value ; test statistic negative, so (really) fail to reject
c.	What if  unknown, and  instead?  Use t-distribution with 7 degrees of freedom; critical value if  is ; reject null hypothesis.  (p-value not on chart, but by computer is )
d.	; critical values at , now fail to reject; p-value 
3.	Relationship to confidence intervals
a.	In two-sided  level hypothesis test, reject if .  In other words, if  more than  units from 400.
b.	Two-sided 95% confidence interval consists of 
c.	In other words,  level hypothesis test merely asks whether  lies inside the 95% confidence interval.
L19	Bias and Consistency (WMS 7.2-7.4, 9.1-9.3)
[What if you used median to estimate mean, say in income distribution?  Biased.]
Properties of estimators
1.	Evaluating  amounts to evaluating distribution of  relative to true unknown value 
2.	Though  is unknown, we know how  relates to  and how  relates to , so can know (probabilistically) how  relates to 
3.	We’ll use this to evaluate estimator goodness and to define margin of error/interval estimates, and do hypothesis test
4.	Moments of 
a.	
Thus, thought we don’t know what  is, we know that average realization of  and average realization of  are same
b.	
Variance of  is much smaller than variance of 
c.	Standard error (i.e. standard deviation of estimator) 
i.	In population (), incomes typically between  ] 
ii.	For , sample average  typically between   
iii.	For , sample average  typically between   
iv.	For ,  typically between   
Consistency
1.	Best imaginable case:  degenerate with  and 
2.	As ,  approaches ideal distribution
a.	That is,  and 
Put differently,  (“in probability”)
3.	Example:  is consistent estimator of 
a.	 for all 
b.	
4.	Law of large numbers (Jacob Bernoulli, 1713)
a.	Sample means converge to population means
b.	Higher order moments
i.	
ii.	
iii.	Sample moments converge to population moments (justification for MOM)
5.	Fact: continuous functions of consistent estimators are consistent
6.	Fact: MLE are always consistent
Bias
1.	Bias 
2.	On average, does  produces estimates that are higher or lower than ?
3.	Unbiased estimator: 
4.	 is unbiased estimator of  because 
5.	Example of biased estimation procedure: sample max from uniform distribution
6.	When bias can be measured, can sometimes correct (target analogy)
(Relative) Efficiency
5.	Given two estimators, the one with lower variance is more efficient.
6.	An estimator cannot be efficient, per se, but only more efficient than another estimator.  In some cases in Econ 388, however, it is possible to prove categorically that a particular unbiased estimator is more efficient than any other unbiased estimator.
7.	Example: consider throwing out one observation, computing sample average of  observations
a.	 still
b.	
c.	Still unbiased, still consistent, but less efficient than using all available data
Sample Variance
1.	 is biased
a.	
b.	
 
 (since  and )
 
c.	
d.	Still consistent:  (and can show that )
2.	“Sample variance” 
a.	To eliminates bias: 
b.	So if then 
i.	Example: sample of  student wages, , , 
 
ii.	Excel: use VAR.S or STDEV.S, not =VAR.P or =STDEV.P
c.	Correcting bias actually sacrifices some efficiency
L20	Difference in Means, Proportions (WMS 8.3-8,10.3)
1.	[Long lecture; use time efficiently.]
2.	Difference in means
a.	Relating quantitative and binary variables: conditional distributions, conditional means , 
b.	Wages gap between men and women:
, , , .
c.	95% confidence intervals for men  and women  overlap, making it difficult to tell true size of wage gap (if any)
d.	Trick (used a lot in more advanced settings): combine into single parameter
; MOM estimator 
i.	; unbiased!
ii.	; consistent (as long as both sample sizes grow large)!
iii.	 and , so…
iv.	
Standardizing, 
v.	Note: if estimate  and  then 
1.	(e.g. If  and  then )
2.	For this class, just use , which is appropriate when  and  are both large
3.	(df between minimum and sum of  and )
e.	Margin of error: 
f.	95% confidence interval for : 
g.	Test : test statistic ; p-value 0.0217
h.	Test : p-value 
i.	Test : test statistic ; p-value 0.1562
j.	Note: if we estimated  instead of , rejection region would be on left instead of right, and test statistics would be negative instead of positive, but produce same p-values
3.	Binary data (i.e. Bernoulli())
a.	Intuitive estimator: proportion , where  = # of 1’s in data
i.	Example: election survey, , 
b.	MOM estimator: ; actually same, since  (for zeros and ones, adding is the same as counting)
c.	Since ,
; unbiased!
; consistent!
d.	By Central Limit Theorem, 
Note: this is not actually different from ; just special case with , , and 
Note: does not follow  distribution for small , because  not normal
e.	Example: election survey, , 
i.	Margin of error: 
ii.	95% Confidence interval 
iii.	Test  against : test statistic ; p-value 0.3446
4.	Difference in proportions: unemployment in U.S. and France (2% difference?)
a.	, ; , 
b.	95% confidence intervals ]
c.	Estimate  with MOM estimator 
i.	; unbiased!
ii.	; consistent!
iii.	
d.	95% Confidence interval 
e.	Test , test statistic ; p-value .0162
f.	Test , test statistic ; p-value .2005
L21	Variance Estimation (WMS 8.9,10.9)
Review
1.	
2.	
Variance estimation
1.	Applications: inequality/heterogeneity, quality control, estimation error
2.	
3.	Sample variance: 
a.	Intuition 1: , so ; ; we lose one degree of freedom because we’re measuring deviations from  rather than deviations from 
b.	Intuition 2: a single observation conveys information about  but not , so if  then we have 100 pieces of information about  but only 99 pieces of information about 
c.	Intuition 3: , so 
d.	 [Skip] Formal derivation: 

 		
 		
e.	Recall that, in estimating , using  instead of  required the use of a t distribution instead of a normal.  This is because it can be shown that  and  are independent, implying that .
4.	Example: variance among  sales representatives is 
a.	Confidence interval
i.	Seek  and know , so from Table 6,
ii.	

b.	Hypothesis test
i.	, 
ii.	Critical value 
iii.	Test statistic , reject  (from Excel, p-value is )
L22	Regression Estimation (WMS 11.1-3)
1.	Recall from Lecture 9
a.	Relationship between  and  can be represented by  or by regression line 
b.	 can be guaranteed by choosing intercept to solve 
c.	Crystal ball: can predict  for any  value (even out of sample)
d.	 can be minimized by choosing slope coefficient  
e.	Fraction of variation in  associated with  is 
2.	Estimation
a.	
b.	
c.	
d.	
e.	
f.	
g.	Income predictions 
h.	Individual errors 
i.	
3.	Can also use “sums of squares”, rather than variance (i.e. “total” not “average” deviations)
a.	Let 
b.	Let 
c.	With this notation, 
d.	Let 
4.	Example : Regress Income  (in $ thousands) on Education  (in years)

				120	37.2	2.8	
				10	78.2	1.8	
				0	78.2	-8.2	
				10	61.8	-1.8	
				90	94.6	5.4	

							

							

a.	We’ll use this example again in subsequent lecture
b.	Predictions
i.	High school graduate 
ii.	College graduate 
iii.	PhD graduate 
c.	Estimated errors
i.	Predict income  for each individual in sample
ii.	
1.	Individual 
iii.	
1.	Alternatively, , so  (useful if only know summary statistics for  and ).
iv.	, 
d.	Illustrate with Excel: Data>Data Analysis>Regression, using education & income data above
5.	Preliminaries
a.	Algebra trick 1: It can be shown that 
 			      
Similarly, 
b.	Algebra trick 2: It can be shown that

 
 
Similarly,   and 
c.	Deterministic 
i.	You can think of  and  as being random (i.e. they depend on who you interview), and this is what we did when we derived the population regression parameters.  But for simplicity, assume in the estimation that  are known.  That is, we are only considering various samples of  individuals who have the same education levels as the people we sampled today (and incomes that potentially differ from the people we interviewed).
ii.	If an estimator is unbiased conditional on these ’s, it is also unbiased unconditionally.  For example, if  for every sample of ’s then, averaging across all such samples,  as well.
iii.	 is still random because  and  is random.
L23	Regression Inference (WMS 11.4-9)
Introduction
1.	Long lecture (use time efficiently)
2.	We’ve derived estimators , , , but so far all we have are point estimates.  Are these good estimators (i.e. unbiased and consistent)?  What are the margins of errors?  To get confidence intervals or do hypothesis tests, we need to know their distributions.
3.	Estimator distributions: if  (which is plausible, by Central Limit Theorem, if each error term is viewed as the sum total of a lot of smaller, independent factors) then
a.	
b.	
c.	
d.	
e.	
f.	
g.	Estimation error 
h.	 (essentially because estimating  requires estimating two parameters   and , leaving only  pieces of information)
i.	Could compare  from two regressions to see which better explains , using  distribution
Slope estimator
1.	It can be shown that 

2.	; consistent ☺!

Note: most noisy when incomes more varied (conditional on education); least noisy when education more varied ( in denominator)
3.	; therefore, 
4.	Example
a.	From previous lecture, , , 
b.	95% Confidence interval: 
c.	Hypothesis Test  at  level
i.	Critical value 2.353
ii.	Test statistic ; reject 
Intercept estimator
1.	It can be shown that ; unbiased ☺!
It can be shown that ; consistent ☺!
a.	[For those curious, 

 

Note:  because…

 
 
					]
2.	Note two pieces: small error in identifying  and larger error in identifying slope (which matters more when  high).
3.	; therefore, 
Prediction estimator
1.	
2.	; unbiased ☺!
3.	; consistent ☺!
[
  (since ) ]
Note: most precise close to ; can still make predictions far away from , but more noisy
4.	
 
Error variance
1.	, 
2.	95% confidence interval for 
a.	
3.	Test  at  level
a.	Critical value 9.49 (from Table 6)
b.	Test statistic ; not significant
4.	If you had two regressions and wanted to know which has better predictive power (i.e. lower residual error variance) you could compare  and  using F distribution
Review
1.	Thanks for a great semester!
2.	Thanks TAs!
3.	Recommend Econ 388: regression with multiple variables
a.	We’re on the brink of knowledge explosion
b.	Also Econ 210 for exploring careers in Economics
c.	For advanced statistics/econometrics, recommend linear algebra (Math 213)
4.	Student project findings
a.	Wide variety of projects
b.	Value of statistics for consumers, not just producers
5.	Key concepts
a.	We’ve seen several trees, now let’s notice the forest
b.	Pre-midterm, we discussed population distributions (discrete or continuous), including moments such as mean, variance, and covariance.
c.	From sample, we seek to estimate population parameter: , , , , , , , , , , or  from another distribution function (e.g. ,  from uniform,  from exponential), including distributions we haven’t encountered yet (e.g.  from geometric,  from Poisson,  from quadratic regression)
d.	Deriving estimators: MOM and ML
e.	Properties of estimators: bias, efficiency, consistency
f.	Margin of error, confidence intervals, hypothesis tests
g.	Matrix algebra
i.	This class has focused on the correlation  between two variables, which also underlies linear regression line: slope coefficient  and coefficient of determination 
ii.	Matrix algebra allows us to generalize everything to multiple variables (e.g.  instead of 
iii.	Individual slope coefficient  then reflects partial correlation between education and income, holding experience, age, race, gender, etc. fixed.
iv.	Controlling for more variable makes stronger case for correlation as causation
6.	Deriving estimator distributions
a.	Estimators depend on , and so are random variables with distributions
b.	
c.	
d.	Distribution  or  (by normality of  or Central Limit Theorem)
e.	CLT also implies that 
f.	Difference estimators  and 
g.	 normal implies 
and therefore  and 
h.	Similarly, , 
and  normal implies that  or 
i.	, implying that 
7.	Matrix algebra True/False question tip
a.	Check simple cases (e.g. , ), but not special cases
b.	Example: “All symmetric matrices are idempotent”
i.	Try simple example: not special matrix like identity matrix or , but something with no special properties other than symmetry, such as  (and show that )
ii.	Can also try really simple examples, such as scalar matrices: .
c.	T/F questions are not “trick” questions, but be careful.  In real world, much of statistics is simply a matter of careful attention to details, and knowing exactly which inferences are legitimate under exactly which circumstances.
8.	Example problems
a.	Confidence interval for 
b.	Hypothesis test for