Probability Theory

This is a full set of lecture notes and problem sets for a Probability Theory class (AMS 311) taught by me in Spring 2005 at the Department of Applied Mathematics and Statistics at Stony Brook University. There probably are some typos. Feel free to contact me any time if you see any, or just with your comments/suggestions!

Lecture Notes

Lecture 01	Introduction. Set Theory. Probabilistic Models.
Lecture 02	Probabilistic Models: Examples. Probability Law. Conditional Probability.
Lecture 03	Multiplication Rule. Total Probability Theorem. Bayes’ Rule.
Lecture 04	Independence.
Lecture 05	Conditional Independence. Counting. Bernoulli Trials.
Lecture 06	Counting: Partitions. Discreet Random Variables. Probability Mass Function.
Lecture 07	Functions of Random Variables. Expectation and Variance.
Lecture 08	Expectation and Variance. Joint PMF.
Lecture 09	Conditioning. Conditional Expectation.
Lecture 10-11	Conditional Expectation. Independence of Random Variables.
Lecture 12	General Random Variables. Probability Density Function. Expectation. Exponential Random Variable. Cumulative Density.
Lecture 13	Distributions of Maximum and Minimum. Normal Random Variable. Conditioning on Event.
Lecture 14	Conditional Expectation.
Lecture 15	Multiple Random Variables. Conditioning. Expectation. Continuous Bayes’ Rule.
Lecture 16	Independence. Joint CDF. Derived Distributions.
Lecture 17	Convolutions.
Lecture 18	Transforms. Moments. Inversion Property.
Lecture 19	Transform of the Sum. Conditional Expectation as a Random Variable.
Lecture 20	Conditional Variance as a Random Variable. Sum of Random Number of Random Variables.
Lecture 21	Sum of Random Number of Random Variables. Covariance.
Lecture 22	Covariance and Correlation. Estimation.
Lecture 23-24	Stochastic Processes. Bernoulli Process.
Lecture 25	Poisson Processes.
Lecture 26	Poisson Processes: Time of the First Arrival, Interarrival Times.
Lecture 27	Merging Processes. Competing Exponents. Incidence Paradox.
Lecture 28	Markov Chains.
Lecture 29	Markov Chains: Classification of States, Periodicity, Steady-State Behavior.
Lecture 30	Birth-Death Processes.
Lecture 31	Absorption Probabilities.
Lecture 32	Expected Times to Absorption. Mean First Passage Time.
Lecture 33	Limit Theorems. Inequalities: Markov, Chebyshev.
Lecture 34	Weak Law of Large Numbers.
Lecture 35	Central Limit Theorem.
Lecture 36	Polling.

Problem Sets

Problem Set 01	Set Theory. Probability Axioms. Basics of Conditional Probability.
Problem Set 02	Conditional Probability. Bayes’ Rule. Independence.
Problem Set 03	Counting. Discrete Random Variables. PMFs. Expectations.
Problem Set 04	Expectations. Variance. Joint PMFs. Conditioning. Independence.
Problem Set 05	Continuous Random Variables. PDFs. Expectation. Variance.
Problem Set 06	Continuous Random Variables. Derived Distributions. Conditional Expectation and Variance. Transforms.
Problem Set 07	Bernoulli Processes. Poisson Processes.
Problem Set 08	Markov Chains.
Problem Set 09	Limit Theorems.