I decided the NCR region needed a proper machine learning reading club. Not a talk series, not a networking mixer, but a small room where a handful of people actually read the paper, line by line, and argue about it. So I started one. This post is about the first session, the talk I gave, and the paper that made me question the single biggest training trend of 2025.
Session 01, hosted with venue partner Lyric · see the event on Luma
My talk was Talk 2 of 2 that evening, and I picked a paper I had a personal stake in. I use PPO in my own work (a hallucination aware reranker at the IIT Patna AIML Lab), so a paper arguing that RL might not do what everyone thinks it does deserved a careful thirty minutes.
The question
Through 2025, the recipe behind the reasoning boom was RL with verifiable rewards, or RLVR. You take a base model, give it math or code problems where an answer can be checked by a program, and reward it for being correct. DeepSeek-R1 made this famous. The story everyone told was: RL taught the model to reason.
The paper I presented asks the uncomfortable version of that: did RLVR actually teach the model anything new, or did it just make the model better at showing what the base model could already do? The honest test is simple to state. Take the base model and the RL-tuned model, and don't just ask each one once. Ask them a thousand times.
pass@k, and the crossover
The tool is pass@k: give the model k attempts at a problem and count it solved if any attempt is correct. Think of it as an exam. pass@1 is exam day, one shot. pass@1024 locks the student in a room with 1,024 tries and asks a different question entirely: does the knowledge exist in there at all?
Here is the result that turns the room quiet.
This holds across Qwen and LLaMA families, at 7B through 32B, across six RL algorithms and ten benchmarks. It shows up in code too, where hidden unit tests mean a correct answer cannot be a lucky guess. Same shape everywhere.
Sharpening, not expanding
So what is RL actually doing? It is sharpening the distribution, not expanding it. RL concentrates probability onto the paths that get rewarded. That is a real win at pass@1, which is the number products actually ship on. But the flip side is that the unrewarded paths fade, and the model loses coverage. On some problems the base model could find a solution and the RL-tuned model effectively cannot anymore. In the paper's tables, the set of problems only RL can solve and the base cannot is, strikingly, about zero.
My favourite moment from the deck was reading a raw base-model transcript out loud. Before any RL, the base model sets up a problem, derives a negative count of people, writes "the number of residents cannot be negative, there is an error in our calculations, let us re-evaluate," switches approach, and boxes the correct answer. Reflection and self-correction, from the base model, with no RL anywhere in sight.
The invisible leash
The clean intuition for why is a one-liner about gradients. On-policy RL (REINFORCE, PPO, GRPO) can only learn from the model's own samples. If the model never once produces a particular solution, there is no term in the update for it, so the training can never create it from nothing. A companion paper, The Invisible Leash, makes this a theorem: the trained policy's reachable set stays inside the base model's. If the base assigns a correct solution probability zero, RLVR cannot discover it.
The lovely historical kicker: the very paper that introduced GRPO, DeepSeekMath, quietly said this back in February 2024. It reported that its RL improved Maj@K but not Pass@K, and that the gain came from making correct answers more likely to surface, not from new capability. Eleven months before R1. The lesson I left the room with: read section five of papers, not just the abstract.
So how did 2025 and 2026 models get so good?
This is the part I did not want to skip, because "RL is a mirage" is too cynical and also wrong. Models genuinely got better, and the theory tells you exactly where the real gains came from: from methods that add capability from outside the model's own samples.
- Distillation. Training on a teacher's traces places probability on sequences the student would never have sampled. It is the one operation the RL gradient cannot do, and it is why DeepSeek found that distilling R1 into small models beat running RL on them directly.
- Search that keeps novelty outside the weights. FunSearch and AlphaEvolve found genuinely new mathematics, including the first improvement to 4x4 matrix multiplication in 56 years, by putting an LLM inside an evolutionary loop where good ideas accumulate outside the model.
- Heavy parallel search plus verification. The IMO 2025 gold results and GPT-5 tightening a proof in convex optimization came from wide sampling, process rewards, and expert or automated verification.
Every one of those is a route the theory leaves wide open. My closing line: the ceiling was real, and the answer was not more GRPO. The field changed the game.
What I actually take away
RL is not pointless. pass@1 is the metric your product lives on, and packaging capability into a single reliable attempt is genuine value. The claim is narrower and sharper than the hype: RL buys you reliability, not new boundaries. If your task has a checker, price base-model-plus-sampling against an RL model before you assume RL wins. And if you want new capability, reach for a teacher or search, not another RL run.
The full deck
All 34 slides, scroll straight through them here.
If you are in Delhi NCR and this is your idea of a good evening, the ML Reading Club NCR runs again soon. Thanks to Ankit for building it with me, and to Lyric (and Anjanay there) for sorting the venue.