tinyGPT

A real GPT running in your browser. Port of Karpathy's pure-Python GPT to TypeScript. Train it, generate names, inspect attention heads, watch tokens flow.

✨ New to transformers? Take the animated tour — playful enough for kids, real math under the hood →
params 4,192
heads 4
layers 1
embed 16
vocab 27
ctx 16
try this:① Auto-train until loss < 1.0→② Generate names→③ Click H0–H3 in the Attention Heads panel to see what each head learned

Architecture

4,192 params

Click any block to learn more. Train to see data flow.

Training

step 0loss —
stop at loss
Click "Step" to begin training

Token Flow

Input
Embed
Attn
MLP
Output

Type a name and click Flow to watch tokens pass through each layer of the GPT. After the pipeline completes, the model generates a continuation.

Temperature Control

conservative0.50 (warm)creative
Low = predictable, common names. High = wild, creative names.

Attention Heads

layer 0
Train or generate to light these up

Same algorithm as the big ones

tinyGPTGPT-4
Parameters~5,000~1,800,000,000,000
Training data200 namesThe internet
GeneratesSingle namesEssays, code, poetry
Runs onYour browserMassive GPU clusters

Same architecture — embeddings, multi-head attention, MLP, softmax. Just a few orders of magnitude smaller.

Built with Next.js 16 + React 19 + TypeScript + Tailwind v4 + Motion 12 + Recharts

Based on Karpathy's pure-Python GPT — the most atomic way to train a transformer