After repeating myself for the $n^text{th}$ time on how to build product evals, I figured I should write it down. There are three basic steps: (i) labeling a small dataset, (ii) aligning our LLM evaluators, and (iii) running the experiment + evaluation harness with each config change.
Read original on Eugene Yan ↗
2025 was an eventful year of progress. I doubled down on my health, had some pleasant career surprises, and my wife and I made time for two vacations.
Read original on Eugene Yan ↗
How can we work effectively with AI? What’s the workflow, how does it scale, and how do we improve our systems over time? And ideally, it should compound. Every finished artifact—code, docs, analysis, decisions—becomes context for the next session. And each correction updates a config that reduces future errors. While I’m still learning, I’ve repeated my answers often enough that I’m writing it here so the next time I’m asked I can share a link instead.
Read original on Eugene Yan ↗
Build a threat model, discover vulnerabilities, verify, triage, and patch.
Read original on Eugene Yan ↗
How do we evaluate if a model can find and exploit security vulnerabilities? How do we know when agents become useful for defenders, and when they cross the threshold into uplifting attackers? Here, we discuss some benchmarks that measure this, from capture-the-flag exercises to data exfiltration on a 50-host network.
Read original on Eugene Yan ↗
Hallucination in large language models usually refers to the model generating unfaithful, fabricated, inconsistent, or nonsensical content. As a term, hallucination has been somewhat generalized to cases when the model makes mistakes. Here, I would like to narrow down the problem of hallucination to cases where the model output is fabricated and not grounded by either the provided context or world knowledge.
Read original on Lilian Weng - Lil'Log ↗
Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the reward function to achieve high rewards, without genuinely learning or completing the intended task. Reward hacking exists because RL environments are often imperfect, and it is fundamentally challenging to accurately specify a reward function.
Read original on Lilian Weng - Lil'Log ↗
Special thanks to John Schulman for a lot of super valuable feedback and direct edits on this post.
Read original on Lilian Weng - Lil'Log ↗
Scaling laws are one of the most critical empirical findings in deep learning. The observation is simple in form: the training loss $L$ decreases predictably as we scale up model size $N$, dataset size $D$, and compute $C$, following a power-law curve, which appears as a straight line on a log-log plot. We can view scaling laws as a framework for describing the relationship between compute, loss, model size and data; at its core, it is about how to allocate precious compute optimally between $N$ and $D$.
Read original on Lilian Weng - Lil'Log ↗
The concept of recursive self-improvement (RSI) dates back to I. J. Good (1965), where he defined an “ultraintelligent machine” as a system that can surpass humans in all intellectual activities and design better machines to improve itself. Yudkowsky (2008) used the phrase “recursive self-improvement” for a specific feedback loop: an AI uses its current intelligence to improve the cognitive machinery that produces its intelligence.
Read original on Lilian Weng - Lil'Log ↗
Substack recently launched its AI detector feature in the UI, which is super interesting.
Read original on Sebastian Raschka - Ahead of AI ↗
I recently posted a Substack note about Claude’s new watermarking process and implementation. Since it’s such a popular topic and sparked such a lively discussion, I thought it might be interesting to go into a bit more detail when explaining how it works.
Read original on Sebastian Raschka - Ahead of AI ↗
A lot has happened in the last few weeks. I am sure that OpenAI’s GPT-6 Astra is top of mind for everyone right now. In particular, thoughts on its performance, the looped transformer/recurrent depth aspects, and rumors that Astra is “hiding” its reasoning trace (i.e., chain of thought).
Read original on Sebastian Raschka - Ahead of AI ↗
The recently released Jev AI model has been quite a cultural phenomenon in technical communities in the past 2 weeks.
Read original on Sebastian Raschka - Ahead of AI ↗
I have written a lot about reasoning models here over the last two years on Ahead of AI, including a recent article on mechanisms to control reasoning effort levels in contemporary LLMs. Reasoning models are still as relevant as ever. In fact, almost all current-generation LLMs today are reasoning models or offer a reasoning mode.
Read original on Sebastian Raschka - Ahead of AI ↗
