Reward Hacking in Reinforcement Learning
Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the reward function to achieve high…
Why We Think
Special thanks to John Schulman for a lot of super valuable feedback and direct edits on this post.
Scaling Laws, Carefully
Scaling laws are one of the most critical empirical findings in deep learning. The observation is simple in form: the…
Harness Engineering for Self-Improvement
The concept of recursive self-improvement (RSI) dates back to I. J. Good (1965), where he defined an “ultraintelligent machine” as…
Building an AI Text Detector From Scratch
Substack recently launched its AI detector feature in the UI, which is super interesting.
How Claude Watermarks AI-Generated Text
I recently posted a Substack note about Claude’s new watermarking process and implementation. Since it’s such a popular topic and…
GPT-6 Astra, Looped Transformers, and Hidden Reasoning
A lot has happened in the last few weeks. I am sure that OpenAI’s GPT-6 Astra is top of mind…
Language Models for Text Classification: From Bag-of-Words to Jev
The recently released Jev AI model has been quite a cultural phenomenon in technical communities in the past 2 weeks.
Building Reasoning Models from the Ground Up: A Full Video Course
I have written a lot about reasoning models here over the last two years on Ahead of AI, including a…
How to Train Your own Decision Model like Jev using Unsloth
In this article, you will learn how to turn an open-weight language model into a decision model that classifies user…