Invention as Exploration

Richard Sutton's Bitter Lesson is that general methods leveraged by more computation beat specialized intuition encoded in domain-specific algorithms. The Bitter Lesson gets lots of attention these days because of its implications for artificial intelligence. But for a patent lawyer like me, it's more interesting for what it says about invention — that at its heart, invention is about exploring combinatorial complexity, not cleverness.

What Sutton concisely observes is that methods leveraging raw computation consistently outperform those incorporating human expertise. Chess programs evolved from expert systems encoding grandmaster strategies to search algorithms evaluating millions of positions. Computer vision abandoned hand-crafted features for networks learning representations through countless parameter combinations.

This pattern exposes a fundamental truth: the spaces where optimal solutions hide are combinatorially vast and sparse. Human intuition, no matter how sophisticated, becomes inadequate and often counterproductive in such domains. Systematic exploration beats clever inuition.

The Combinatorial Challenge

At the frontiers of knowledge, nobody (and no model) possesses superior insight into where best solutions reside. The parameter spaces — whether chess positions, neural network weights, or scientific theories — are so high-dimensional that human intuition cannot meaningfully guide the search.

Consider the numbers: Go has roughly 10^170 possible positions. Neural network architecture spaces are effectively infinite. In these vast territories, optimal solutions occupy infinitesimally small regions, scattered like rare gems in a cosmic desert. Human experts attempt to predict where these gems lie, but mostly succeed only where previous generations have already explored extensively.

The Speed of Systematic Exploration

The greatest inventors do not distinguish themselves with superior intelligence, but with their compulsive drive to explore. Thomas Edison, Luther Burbank, and Henry Ford — shown in the photo — demonstrated this as well as anybody.

Edison didn't invent the light bulb through brilliant theoretical insight. He systematically tested over 6,000 plant fibers and 1,600 materials before finding a viable filament. His laboratory was essentially an industrial-scale search engine, generating and testing hypotheses at unprecedented speed. Edison's genius lay not in knowing where solutions were, but in creating systems to find them faster than anyone else. What looked like prescience was the result of having seen more of nature. For example, his "etheric force" — rejected by scientists, and never formalized into a rigorous scientific theory — was later understood as a form of wireless transmission.

Luther Burbank revolutionized agriculture through exhaustive combinatorial exploration of plant genetics. He tested over 100,000 plant variations, crossbreeding relentlessly to discover new varieties. Burbank developed the Russet Burbank potato—now the foundation of America's potato industry—not through botanical theory but through systematic exploration of genetic combinations. His 55-year career produced over 800 new plant varieties through pure exploratory persistence. He was terrible at documenting and communicating his experiments, and therefore cannot be understood as a scientist. But Burbank understood epigenetics from his direct experience with plant hybrids in an era in which Mendelian genetics alone was accepted by scientists.

Henry Ford did not transform manufacturing through engineering brilliance, but through relentless experimentation with assembly line configurations. He tested countless arrangements of workers, tools, and processes, discarding failures immediately when better solutions emerged. Ford's moving assembly line emerged from systematic exploration of manufacturing possibilities, reducing Model T production time from 12 hours to 90 minutes through iterative optimization. Like Edison and Burbank, Ford's systematic experiments gave him insights that were not available to his contemporaries — including an understanding that paying his employees high wages would lead to higher aggregate demand in an era in which economists rejected that possibility.

What Edison, Burbank, and Ford had in common was speed — how quickly they could generate hypotheses, test them, and discard failures. They understood intuitively that in vast solution spaces, the winner is whomever can explore fastest.

Intuition from Experiment

Beyond what it says about how to build AI, the Bitter Lesson challenges our fundamental assumptions about invention. Traditional approaches assume human insight can meaningfully constrain search spaces, that expertise provides reliable shortcuts. But in complex domains, human expertise often points toward already-explored regions, missing the revolutionary solutions hiding in unexplored territories.

This is also why breakthrough inventions frequently come from young people or outsiders unconstrained by expert knowledge. They're free to explore combinatorial possibilities that experts have learned to dismiss as unpromising or impossible.

What makes exceptional inventors special isn't raw intelligence, it's their refusal to be satisfied with existing solutions. This restlessness reflects intuitive understanding of combinatorial spaces: better solutions always await discovery, but only for those willing to explore beyond the boundaries of current knowledge.

Embracing Systematic Search

The Bitter Lesson teaches us that innovation in complex domains emerges from systematic exploration of combinatorial complexity, not brilliant insights or domain expertise. The systems capable of searching fastest and most comprehensively through vast solution spaces will consistently outperform those constrained by human intuition.

This realization diminishes the role of human cleverness in favor of systematic exploration. But it's also liberating: Our most challenging problems may be solvable if humans and machines work together in systems that explore solution spaces at larger scale and higher speed. The future belongs not to the smartest humans or models, but to the combination of humans and machines that are the most efficient explorers of combinatorial complexity.