Uncategorized
Chinchilla and the Rethinking of Scaling Laws
DeepMind showed in 2022 that many large models were undertrained.
Uncategorized
Uncategorized
Learning From Human Feedback: The InstructGPT Shift
OpenAI used reinforcement learning from human feedback to align models in 2022.
Uncategorized
BERT and the Bidirectional Turn in Language Understanding
Google's BERT read text in both directions at once, released in 2018.
Uncategorized
AlphaZero: Mastering Chess and Go From Scratch
AlphaZero learned chess, shogi, and Go through self-play alone in 2017.
Uncategorized
CLIP: Teaching Machines to Link Images and Words
OpenAI's CLIP learned from 400 million image-text pairs in 2021.
Uncategorized
Denoising the Future: The Rise of Diffusion Models
Denoising diffusion models learn to reverse a gradual noising process.
Uncategorized
GPT-3 and the Arrival of Few-Shot Language Models
OpenAI's GPT-3, with 175 billion parameters, debuted in 2020.
Uncategorized
AlphaFold and the 50-Year Protein-Folding Problem
DeepMind's AlphaFold2 predicted 3D protein structures at near-experimental accuracy in 2021.
Uncategorized
Deeper by Design: How ResNet Broke the Depth Barrier
Residual networks introduced skip connections so gradients could flow through very deep stacks.
Uncategorized
AlphaGo and the Fall of a 2,500-Year-Old Game
DeepMind's AlphaGo beat world champion Lee Sedol 4-1 in 2016.