Teaching

Sample space, events, probability measure. Permutations and combinations, sampling with or without replacement. Conditional probability, partitions of the sample space, law of total probability, Bayes' Theorem. Independence. Discrete random variables, probability mass functions, examples: Bernoulli, binomial, Poisson, geometric. Expectation, expectation of a function of a discrete random variable, variance. Joint distributions of several discrete random variables. Marginal and conditional distributions. Independence. Conditional expectation, law of total probability for expectations. Expectations of functions of more than one discrete random variable, covariance, variance of a sum of dependent discrete random variables. Solution of first and second order linear difference equations. Random walks (finite state space only). Probability generating functions, use in calculating expectations. Examples including random sums and branching processes. Continuous random variables, cumulative distribution functions, probability density functions, examples: uniform, exponential, gamma, normal. Expectation, expectation of a function of a continuous random variable, variance. Distribution of a function of a single continuous random variable. Joint probability density functions of several continuous random variables (rectangular regions only). Marginal distributions. Independence. Expectations of functions of jointly continuous random variables, covariance, variance of a sum of dependent jointly continuous random variables. Random sample, sums of independent random variables. Markov's inequality, Chebyshev's inequality, Weak Law of Large Numbers.

Continuous random variables. Jointly continuous random variables, independence, conditioning, functions of one or more random variables, change of variables. Examples including some with later applications in statistics. Moment generating functions and applications. Statements of the continuity and uniqueness theorems for moment generating functions. Characteristic functions (definition only). Convergence in distribution and convergence in probability. Weak law of large numbers and central limit theorem for independent identically distributed random variables. Strong law of large numbers. Discrete-time Markov chains: definition, transition matrix, n-step transition probabilities, communicating classes, absorption, irreducibility, periodicity, calculation of hitting probabilities and mean hitting times. Recurrence and transience. Invariant distributions, mean return time, positive recurrence, convergence to equilibrium (proof not examinable), ergodic theorem. Random walks. Poisson processes in one dimension, Poisson counts, thinning and superposition.

Measure spaces. Outer measure, null set, measurable set. The Cantor set. Lebesgue measure on the real line. Counting measure. Probability measures. Construction of a non-measurable set. Simple function, measurable function, integrable function. Reconciliation with the integral introduced in Prelims. A simple comparison theorem. Integrability of polynomial and exponential functions over suitable intervals. Monotone Convergence Theorem. Fatou's Lemma. Dominated Convergence Theorem. Corollaries and applications of the Convergence Theorems (including term-by-term integration of series). Theorems of Fubini and Tonelli (proofs not examinable). Differentiation under the integral sign. Change of variables. Brief introduction to Lp spaces. Hölder and Minkowski inequalities.

Motivation for a "function'' with the properties the Dirac delta-function. Test functions. Continuous functions. Distributions and delta as a distribution. Differentiating distributions. Theory of Fourier and Laplace transforms, inversion, convolution. Inversion of some standard Fourier and Laplace transforms via contour integration. Use of Fourier and Laplace transforms in solving ordinary differential equations, with some examples. Use of Fourier and Laplace transforms in solving partial differential equations; in particular, use of Fourier transform in solving Laplace's equation and the Heat equation.

Order statistics, probability plots. Estimation: observed and expected information, statement of large sample properties of maximum likelihood estimators in the regular case, methods for calculating maximum likelihood estimates, large sample distribution of sample estimators using the delta method. Hypothesis testing: simple and composite hypotheses, size, power and p-values, Neyman-Pearson lemma, distribution theory for testing means and variances in the normal model, generalized likelihood ratio, statement of its large sample distribution under the null hypothesis, analysis of count data. Confidence intervals: exact intervals, approximate intervals using large sample theory, relationship to hypothesis testing. Probability and Bayesian Inference. Posterior and prior probability densities. Constructing priors including conjugate priors, subjective priors, Jeffreys priors. Bayes estimators and credible intervals. Statement of asymptotic normality of the posterior. Model choice via posterior probabilities and Bayes factors. Examples: statistical techniques will be illustrated with relevant datasets in the lectures.

Concept of likelihood; examples of likelihood for simple distributions. Estimation for a single unknown parameter by maximising likelihood. Examples drawn from: Bernoulli, binomial, geometric, Poisson, exponential (parametrized by mean), normal (mean only, variance known). Data to include simple surveys, opinion polls, archaeological studies, etc. Properties of estimators---unbiasedness, Mean Squared Error. Statement of Central Limit Theorem. Confidence intervals using CLT.