LLM alignment Archives - machinelearningplus

machine learning + RLHF & DPO Explained: Simulate Alignment in Python machinelearningplus.com

23 min

RLHF & DPO Explained: Simulate Alignment in Python

Build a reward model, PPO loop, and DPO training from scratch in NumPy. Compare RLHF vs DPO side-by-side with runnable code.

machine learning + DPO (Direct Preference Optimization) — A Simpler Alternative to RLHF machinelearningplus.com

42 min

Align LLMs with human preferences using one loss function -- no reward model, no RL. Complete guide with derivation, PyTorch code, and DPO variants.