Statistical Signal Processing & Inference
A rigorous foundation in statistical estimation theory for the design of signal processing and machine learning algorithms, covering random signals, linear stochastic models, optimal estimators, and adaptive filters.
Instructor: Prof. Danilo Mandic
Term: Spring
Location: Imperial College London, Dept. of EEE
Course Overview
This course provides a rigorous foundation in statistical inference and estimation, equipping students with the mathematical tools and practical expertise to design optimal learning machines for real-world data — guided by the principle: “Use your knowledge and not brute force.”
Role: Graduate Teaching Assistant (2024–2026, Spring terms)
Learning Outcomes
- Understand random signals, their properties, and statistical descriptors
- Build and analyse linear stochastic models (AR, MA, ARMA)
- Derive optimal estimators and rigorous performance bounds (CRLB)
- Apply Least Squares, BLUE, and Maximum Likelihood methods
- Design adaptive filters for nonstationary and streaming data
- Gain practical experience with real-world signals: speech, physiology, finance
Prerequisites
- No formal prerequisites required
- Digital Signal Processing (helpful)
- Basic probability & statistics (helpful)
- Working knowledge of MATLAB (important for coursework)
Textbooks
- Fundamentals of Statistical Signal Processing — S. Kay
- Time Series Analysis — Box & Jenkins
- Statistical Digital Signal Processing — M. Hayes
- Adaptive Filter Theory — S. Haykin
- Recurrent Neural Networks — Mandic & Chambers
Assessment
- 5 MATLAB-based coursework assignments: 100%
Schedule
| Week | Date | Topic | Materials |
|---|---|---|---|
| 1 | Background on Random Signals Ensemble averages, PDFs, mean and variance; introduction to the statistical inference paradigm. | ||
| 2 | Time Series Analysis & Linear Stochastic Models AR, MA, ARMA models; sunspot estimation example; representing long signals with few parameters. | ||
| 3 | Introduction to Estimation Theory Statistical estimation paradigm; applications in radar, speech, biomedicine, communications, and seismics. | ||
| 4 | Bias-Variance Dilemma & Performance Bounds Minimum Variance Unbiased (MVU) estimation; Cramér-Rao Lower Bound (CRLB); sufficient statistics. | ||
| 5 | BLUE & Maximum Likelihood Estimation Best Linear Unbiased Estimator; MLE motivation via GPS, generative AI, HDD controllers; sinusoidal frequency estimation. | ||
| 6 | The Method of Least Squares Block & sequential LS; linear and logistic regression; CAPM estimation; orthogonality principle. | ||
| 7 | Adaptive Learning & Inference Wiener filter; adaptive filters for nonstationary data; noise-cancelling headphones; foetal ECG extraction; artificial neuron model. |