Computer Science › Specialization
A language model never sees characters or words, it sees integer token ids drawn from a fixed vocabulary. Byte-pair encoding builds that vocabulary by greedily merging the most frequent adjacent pair of symbols, landing between a character vocabulary (tiny, but sequences too long) and a word vocabulary (short sequences, but every unseen word is unknown).
"You shall know a word by the company it keeps": word2vec turns that linguistic hypothesis into a training objective, learning dense vectors whose dot products predict which words co-occur, and the resulting geometry (similar words close together, consistent offsets for analogies) is the ancestor of the embedding layer at the bottom of every modern language model.
Self-supervised pretraining manufactures labels from raw text: a causal model (GPT) predicts each next token from the ones before it, a masked model (BERT) predicts hidden tokens from both sides. The choice of objective decides what the model can do afterwards, generate text left to right, or encode a whole sentence for classification.
Test loss of a language model falls as a smooth power law in parameters, data, and compute, which turns "how big should the model be" into an engineering calculation. Chinchilla showed the earlier recipe was badly unbalanced: for a fixed compute budget, parameters and training tokens should grow together, roughly 20 tokens per parameter.
Full fine-tuning updates and stores every one of a model's billions of weights per task. LoRA freezes the pretrained weights and learns only a low-rank correction BA added to each target matrix, cutting trainable parameters by orders of magnitude, and it works because the update a task actually needs has low intrinsic rank.
A pretrained model imitates the internet, not a helpful assistant. RLHF collects human comparisons between pairs of answers, fits a reward model to them with the Bradley-Terry model, and optimizes the policy against that reward under a KL leash; DPO reaches the same optimum with a single classification-style loss and no explicit reward model or RL loop.
A model's weights are a frozen, lossy snapshot of its training data. Retrieval-augmented generation fetches relevant passages at query time (by dense embedding similarity, keyword search, or both) and puts them in the context window, so answers can cite current, private, or long-tail facts the weights never memorized.
Classification answers "what is in this image"; detection also answers "where", with a box per object, and segmentation answers it per pixel. Faster R-CNN proposes candidate boxes and classifies them, U-Net upsamples a CNN's coarse features back to full resolution with skip connections, and both are trained and scored with intersection over union.
A Vision Transformer cuts an image into 16x16 patches and treats them as tokens, trading the CNN's built-in locality for flexibility that pays off with enough data. CLIP trains an image encoder and a text encoder together so matching image-caption pairs land close in a shared space, which is what makes zero-shot classification by text prompt possible.
A plain autoencoder's latent space has holes: decoding a random point gives garbage. A VAE makes the encoder output a distribution, regularizes it toward a standard Gaussian with a KL term, and trains by maximizing the evidence lower bound, using the reparameterization trick so gradients can flow through the sampling step.
A diffusion model learns to undo a fixed process that gradually turns data into Gaussian noise: training is just regressing the noise that was added (a plain mean-squared error), and generation runs the learned denoiser backwards from pure noise. Flow matching recasts the same idea as learning a velocity field that transports noise to data along simple paths.
Value iteration needs the transition model and a table over every state; Q-learning drops the model by learning from sampled transitions, and DQN drops the table by approximating Q with a neural network. Two engineering fixes, a replay buffer and a slowly updated target network, are what keep that combination from diverging.
Instead of learning values and acting greedily, a policy-gradient method adjusts the policy's parameters directly in the direction that makes high-return actions more likely. The REINFORCE estimator makes that gradient computable from samples, a baseline tames its variance, and PPO's clipped objective stops each update from moving the policy too far at once.
Given three realistic requests (a support assistant over a private, changing knowledge base, a visual product search with no labeled training set, and a warehouse controller that must learn from trial and error), choose and justify a design from this track for each, and name what each rejected alternative would have cost.
Part of the Computer Science curriculum.
A language model never sees characters or words, it sees integer token ids drawn from a fixed vocabulary. Byte-pair encoding builds that vocabulary by greedily merging the most frequent adjacent pair of symbols, landing between a character vocabulary (tiny, but sequences too long) and a word vocabulary (short sequences, but every unseen word is unknown).
"You shall know a word by the company it keeps": word2vec turns that linguistic hypothesis into a training objective, learning dense vectors whose dot products predict which words co-occur, and the resulting geometry (similar words close together, consistent offsets for analogies) is the ancestor of the embedding layer at the bottom of every modern language model.
Self-supervised pretraining manufactures labels from raw text: a causal model (GPT) predicts each next token from the ones before it, a masked model (BERT) predicts hidden tokens from both sides. The choice of objective decides what the model can do afterwards, generate text left to right, or encode a whole sentence for classification.
Test loss of a language model falls as a smooth power law in parameters, data, and compute, which turns "how big should the model be" into an engineering calculation. Chinchilla showed the earlier recipe was badly unbalanced: for a fixed compute budget, parameters and training tokens should grow together, roughly 20 tokens per parameter.
Full fine-tuning updates and stores every one of a model's billions of weights per task. LoRA freezes the pretrained weights and learns only a low-rank correction BA added to each target matrix, cutting trainable parameters by orders of magnitude, and it works because the update a task actually needs has low intrinsic rank.
A pretrained model imitates the internet, not a helpful assistant. RLHF collects human comparisons between pairs of answers, fits a reward model to them with the Bradley-Terry model, and optimizes the policy against that reward under a KL leash; DPO reaches the same optimum with a single classification-style loss and no explicit reward model or RL loop.
A model's weights are a frozen, lossy snapshot of its training data. Retrieval-augmented generation fetches relevant passages at query time (by dense embedding similarity, keyword search, or both) and puts them in the context window, so answers can cite current, private, or long-tail facts the weights never memorized.
Classification answers "what is in this image"; detection also answers "where", with a box per object, and segmentation answers it per pixel. Faster R-CNN proposes candidate boxes and classifies them, U-Net upsamples a CNN's coarse features back to full resolution with skip connections, and both are trained and scored with intersection over union.
A Vision Transformer cuts an image into 16x16 patches and treats them as tokens, trading the CNN's built-in locality for flexibility that pays off with enough data. CLIP trains an image encoder and a text encoder together so matching image-caption pairs land close in a shared space, which is what makes zero-shot classification by text prompt possible.
A plain autoencoder's latent space has holes: decoding a random point gives garbage. A VAE makes the encoder output a distribution, regularizes it toward a standard Gaussian with a KL term, and trains by maximizing the evidence lower bound, using the reparameterization trick so gradients can flow through the sampling step.
A diffusion model learns to undo a fixed process that gradually turns data into Gaussian noise: training is just regressing the noise that was added (a plain mean-squared error), and generation runs the learned denoiser backwards from pure noise. Flow matching recasts the same idea as learning a velocity field that transports noise to data along simple paths.
Value iteration needs the transition model and a table over every state; Q-learning drops the model by learning from sampled transitions, and DQN drops the table by approximating Q with a neural network. Two engineering fixes, a replay buffer and a slowly updated target network, are what keep that combination from diverging.
Instead of learning values and acting greedily, a policy-gradient method adjusts the policy's parameters directly in the direction that makes high-return actions more likely. The REINFORCE estimator makes that gradient computable from samples, a baseline tames its variance, and PPO's clipped objective stops each update from moving the policy too far at once.
Given three realistic requests (a support assistant over a private, changing knowledge base, a visual product search with no labeled training set, and a warehouse controller that must learn from trial and error), choose and justify a design from this track for each, and name what each rejected alternative would have cost.