The mixing-board analogy
Picture a recording studio mixing board with billions of tiny knobs. Each knob nudges the sound by a hair. Set them randomly and you get noise. Spend years carefully turning them while a song plays back, and eventually the board reproduces the song.
An AI model is that mixing board. The knobs are called parameters (or weights). Training is the slow, automated process of turning every knob a hair at a time so the output matches the data we showed it.
- The board (model) is just a function: input goes in, output comes out.
- The knobs (parameters) are what gets learned.
- Training is search: nudge knobs, measure error, nudge again.
- Inference is playback: knobs are frozen, run input through, read output.
What "model" actually means
Strip the hype and a model is a mathematical function y = f(x; θ) where:
xis the input (a sentence, an image, a row of features).yis the output (a next token, a label, a number).θis the parameters — the billions of numbers that define what the function does.
GPT-class models have hundreds of billions of parameters. A logistic regression has a handful. Same shape of math, different scale.
Training vs inference
| Phase | What changes | What it costs |
|---|---|---|
| Training | Parameters move every step | Huge — GPUs for weeks |
| Inference | Parameters frozen, input flows through | Small — milliseconds per call |
Training is a one-time (expensive) investment. Inference is what your users pay for, per request, forever.
Why bigger isn't always better
More parameters = more capacity to memorise patterns, but also:
- More compute and memory at inference time.
- More data needed to actually learn (a billion-knob model on a thousand examples just memorises noise).
- More chance of overfitting — fitting the training data so closely it fails on anything new.
A right-sized model on quality data beats a giant model on garbage data, every time.
What a senior engineer asks first
- What's the input distribution? — train and prod must look alike.
- What's the eval metric? — "looks good" is not a metric.
- What's the inference budget? — milliseconds and dollars per call.
- How does it fail? — every model is wrong sometimes; design for the failure mode.