In this video, Andrej Karpathy demonstrates how to build a Generatively Pretrained Transformer (GPT), following the paper "Attention is All You Need" and OpenAI's GPT-2 / GPT-3, and much more. Make sure that you watch at least parts of it!
The Qwen team has released Qwen3, a new family of open-weight language models comprising two MoE and six dense models. The Qwen3 model family features hybrid thinking capabilities, support for 119 languages, and competitive performance.
Microsoft has introduced three new small language models—Phi-4-reasoning, Phi-4-reasoning-plus, and Phi-4-mini-reasoning—that reportedly deliver complex reasoning capabilities comparable to much larger models while maintaining efficiency for deployment across various computing environments.
AI2's new Olmo 2 1B model outperforms similar-sized offerings from Google, Meta, and Alibaba across multiple key benchmarks, including GSM8K, TruthfulQA, and DROP, while maintaining a small enough size to run on resource-limited hardware.
Comments