Scales
"Athletes train. Musicians train. Performers train. But knowledge workers don't." — David Perell
Scales are short form notes I write on topics I've learned previously. I'm uploading them here as a form of accountability. Some of these may be typed up, and some may be handwritten notes. The goal is to continuously and iteratively reinforce topics I've studied but don't reach for on a daily basis.
KL Divergence
May 2026
Handwritten notes on KL divergence.
The Dreaded F-word
November 1, 2025
F-divergences, Fenchel duality, and how variational bounds connect to GANs.
Exponential race trick for categorical sampling
October 21, 2025
A fast, CUDA-sync-free sampling trick using exponential random variables.
Proceedings of Dead Ends: tokenizer-free autoregressive models
October 17, 2025
A failed but instructive approach to autoregressive protein generation without tokenization.
Kabsch me if you can
October 11, 2025
Deriving the Kabsch algorithm and why RMSD makes a poor loss function.
Coordinates need the full float32
October 10, 2025
Why bfloat16 precision causes training instabilities for coordinate-based networks.
A very quick intro to generative biology
October 5, 2025
Proteins, folding, and the landscape of generative structure modeling.
Everything is an ODE
October 5, 2025
Diffusion models as parameter-efficient ODEs, and why Kanzi works with only 30M params.