Applied maths 15.1: Bias, variance and overfitting
Why the model that fits your data best is often the worst at predicting new data, and how test error splits into bias, variance and noise. With 10 data points, a polynomial of degree 9 passesโฆ
154 posts match
Why the model that fits your data best is often the worst at predicting new data, and how test error splits into bias, variance and noise. With 10 data points, a polynomial of degree 9 passesโฆ
Ridge and lasso: penalise large coefficients to trade a little bias for a lot less variance. A model with 500 inputs fitted to 200 patients will happily find 'effects' that are pure noiseโฆ
Logistic regression turns a score into a probability; the confusion matrix, precision and recall say how good the classifier really is. A test for a rare disease that simply says 'healthy' toโฆ
Estimate test error without wasting data, choose models and tuning parameters honestly, and avoid data leakage. Many published models fail in the real world for one reason: somewhere, informationโฆ
Decision trees split data into boxes by asking yes/no questions; random forests and boosting combine many trees into one strong model. The models that win most competitions on tables of data areโฆ
Gradient descent fits a model by walking downhill on its loss; a neural network is layers of simple units fitted that way. A language model has billions of numbers to fit, and no formula givesโฆ
k-means groups unlabelled data into clusters; principal component analysis finds the few directions that carry most of the variation. A shop has a million customers and no labels. Which of themโฆ
Model a system that hops between states at random: transition probabilities, predictions a few steps ahead, the long-run share of time in each state, and how long a state lasts. A factory machineโฆ
Model random arrivals in time: counts in an interval, waiting times, the memoryless property, and merging streams of arrivals. Calls to a help desk, insurance claims, radioactive decays and websiteโฆ
Add up many random steps: the mean and spread of a random walk, the square-root law, and Brownian motion with drift as the continuous version. A pollen grain in water jiggles because millions ofโฆ
Differentiate functions of Brownian motion with Itรด's lemma, and use it on geometric Brownian motion, the standard model of share prices. For Brownian motion $W$, the average of $W(t)$ is 0 for allโฆ
Price options by replication: payoffs, the one-step binomial tree and risk-neutral probabilities, the Black-Scholes formula, and put-call parity. An airline wants protection if fuel becomesโฆ
Estimate averages, probabilities and prices by simulation: the estimate, its standard error, the number of runs needed, and a simple way to need fewer. Banks price exotic options on fifty shares atโฆ
Measure and forecast how much prices move: log returns, historical volatility, EWMA and GARCH updates, long-run volatility, and implied volatility. Black-Scholes assumes volatility is constantโฆ
Understand any one-variable system x' = f(x) from the graph of f: where it settles, how fast, and why it can never oscillate. A marble dropped into a bowl of honey never overshoots the bottom: itโฆ
Follow fixed points as a parameter changes: saddle-node, transcritical and pitchfork bifurcations, critical harvests, epidemic thresholds and hysteresis. A fishery is harvested a little harder eachโฆ
Sustained oscillations of a fixed size: nullclines, limit cycles, the Hopf bifurcation, and why flows in the plane cannot be chaotic. Your heart beats with the same rhythm and strength whether youโฆ
Systems that move in steps, x(n+1) = g(x(n)): fixed points, cobwebs, the stability test |g'| < 1, and the period-doubling route of the logistic map. Insects that breed once a year, a bank balanceโฆ
Deterministic but unpredictable: sensitive dependence, the Lyapunov exponent as the average stretching, and how far ahead a chaotic system can be predicted. In 1961 Edward Lorenz restarted aโฆ
Shapes with detail at every scale: self-similarity, the similarity and box-counting dimensions, the coastline paradox, and why strange attractors are fractals. How long is the coast of Britain?โฆ
Split a PDE into ODEs, let the boundary conditions choose the modes, and build the solution for any starting state from a Fourier sine series. Lesson 11.3 solved the heat equation only when the rodโฆ
Read the wave equation, use travelling and standing waves, and find the frequencies of a vibrating string. Pluck a guitar string and it sings at one pitch; press it at the 12th fret and the pitchโฆ
Recognise harmonic functions, use the mean value and maximum principles, and solve Laplace's equation on a rectangle. Wait long enough and the temperature in a metal plate stops changing. So doesโฆ
Compute with complex numbers: arithmetic, the complex plane, modulus and argument, Euler's formula, powers and roots. An electrical engineer writes an alternating voltage as $V e^{i\omega t}$, andโฆ