Recently, I’ve been thinking about a fundamental tension that every mathematician faces: the conflict between systematic goal-setting and open-ended exploration. This tension became particularly clear to me after reading Kenneth Stanley and Joel Lehman’s book “Why Greatness Cannot Be Planned: The Myth of the Objective.”
The Stepping Stone Problem
Stanley and Lehman argue that truly great discoveries emerge not from pursuing specific objectives directly, but as byproducts of curiosity-driven exploration. They introduce the compelling idea of “proof space” as a vast landscape to explore rather than a direct path to follow. The most elegant proofs often aren’t found by attacking a theorem head-on, but by wandering through related mathematical territory, building up techniques and insights that eventually converge on the target from unexpected directions.
Consider the development of harmonic analysis and the proof of the Prime Number Theorem. The theorem itself is about the distribution of prime numbers, but its proof required venturing deep into complex analysis—exploring the Riemann zeta function, understanding its zeros, and developing techniques for analytic continuation. Mathematicians like Hadamard and de la Vallée Poussin weren’t just attacking the prime counting problem directly; they were exploring the rich landscape of complex function theory, building tools that seemed tangentially related to number theory. The breakthrough came from recognizing that the behavior of primes was intimately connected to the analytic properties of the zeta function—a connection that wasn’t obvious from the original problem.
The Practical Reality
This all sounds wonderful in theory, but the reality of mathematical research is more complicated. We work within systems—funding agencies, academic institutions, industry—that demand concrete deliverables, milestones, and objectives. You can’t exactly tell your advisor or funding committee “I’m going to explore interesting things and maybe something will happen.”
My own approach has evolved into operating on two levels simultaneously:
The collaborative/systematic level: I work with many people on projects that have more concrete deliverables. These collaborations very often produce publishable mathematics. There’s something satisfying about having clear problems to solve and making steady progress.
The exploratory level: I tackle hard problems with new ideas and explorations. These are much more hit-or-miss. Sometimes I find beautiful results and make exciting progress; other times they lead nowhere. But this is where the potential for breakthrough discoveries lies.
The Emotional Roller Coaster
What’s rarely discussed is the emotional side of this tension. Both approaches can be incredibly motivating—there’s nothing quite like the feeling of making progress on a difficult problem or seeing a new connection emerge. But the frustration is equally real and challenging to overcome.
The non-linear nature of research means you can spend months or years seemingly making no progress, then suddenly have a breakthrough that makes everything click. Sometimes getting through these stuck periods requires nothing more than stubborn perseverance—just grinding through until something gives way.
But letting things “percolate” is harder. It goes against the instinct to keep pushing, and it can feel like giving up. There’s always the worry that if you step back, you’ll lose the thread entirely.
The Randomness of Insight
Perhaps the most honest thing I can say about mathematical research is that inspiration and ideas strike randomly. They come during collaborative work, during exploratory phases, sometimes while brushing your teeth or in the middle of a completely different conversation. You can’t summon these moments of insight, but you also can’t schedule them.
This randomness is both the blessing and curse of research. It’s maddening from a planning perspective because you can’t build a research timeline around “wait for random inspiration.” But it’s also wonderful that the human mind works this way—that understanding can emerge from unpredictable moments of connection and synthesis.
Finding Balance
I don’t think this tension ever fully resolves, and maybe that’s okay. The systematic work provides the foundation and skills that make exploratory work possible. The exploratory work suggests new systematic directions. The collaborative projects keep you connected to the broader mathematical community and expose you to different techniques that might unexpectedly connect to your solo explorations.
The key insight from Stanley and Lehman’s work is that we might be most effective not when directly targeting specific unsolved problems, but when we’re free to explore interesting mathematical territories and follow novel directions. An AGI system that discovers beautiful, unexpected mathematical structures might stumble upon proofs for problems we’ve been attacking unsuccessfully for decades—and perhaps the same is true for human mathematicians.
Embracing the Process
What I’ve learned is that both approaches are necessary. The collaborative work gives you breathing room to take bigger risks on the exploratory side. The exploratory work keeps you engaged with the deep questions that drew you to mathematics in the first place. And yes, the frustration and uncertainty are part of the process, not a sign that you’re doing something wrong.
The stepping stones to mathematical greatness are often deceptive—they don’t obviously lead toward the ultimate achievement and may even appear to lead away from it. But by staying engaged with interesting problems, maintaining broad collaborations, and being ready for those moments when inspiration strikes, we create the conditions for serendipity.
After all, the most significant mathematical breakthroughs often come not from optimizing toward specific targets, but from following what’s novel, interesting, or personally meaningful—even when we don’t know where it leads.