The Architecture of Probabilities and Possibilities: Inside Nobel Prize Winner Demis Hassabis' Three-Stage Framework for AI Creativity

The Move That Changed The World

In a hushed Seoul conference room in March 2016, eighteen-time world Go champion Lee Sedol stared at the board in disbelief. On move 37 of his second game against DeepMind's AlphaGo, the AI had placed a white stone in a position so unconventional that expert commentators initially assumed it was a mistake. Fan Hui, the European Go champion watching from the sidelines, whispered, "It's not a human move. I've never seen a human play this move."

This moment—forever immortalized as "Move 37"—marked a profound shift in our understanding of artificial intelligence. It wasn't just that a machine had defeated a human champion; it was that the machine had done so by making a creative leap beyond human conventional wisdom.

For Demis Hassabis, DeepMind's founder and now Nobel laureate, this moment crystallized what would become a transformative framework for AI creativity—a three-stage progression from familiar pattern recognition to groundbreaking innovation that has since guided breakthroughs across domains from drug discovery to climate science.

"What fascinates me about Move 37," Hassabis later reflected, "is that it represents a genuine creative insight. The system wasn't just calculating odds but recognizing a possibility that humans had overlooked for centuries."

Today, as researchers and innovators seek to apply AI creativity across fields, Hassabis' framework offers not just theoretical insight but a practical methodology—a roadmap for harnessing machine creativity to expand the boundaries of human knowledge.

Part I: The Probability Landscapes of Intelligence

The Statistician's Canvas

To understand Hassabis' framework, we must first understand how modern AI systems perceive the world: not as discrete objects and concepts, but as vast multidimensional probability distributions.

"Our brains naturally categorize the world into neat buckets," explains Dr. Sara Chen, who leads computational biology at the Broad Institute. "But reality—especially at the frontiers of knowledge—is fuzzier. Every protein structure, every chemical compound, every possible move in a game of Go exists within a continuous landscape of possibility."

When scientists talk about AI "learning" from data, what's really happening is statistical pattern recognition. The system builds an internal model of which configurations are likely (given the data it's seen) and which are unlikely. This probability landscape becomes the foundation for all further creativity.

This understanding forms the backbone of Hassabis' three-stage framework:

Interpolation: Operating within the high-probability regions of known data. Extrapolation: Venturing into lower-probability regions that remain consistent with underlying patterns. Invention: Discovering entirely new probability landscapes by connecting previously unrelated domains.

Each stage represents a deeper form of creativity, requiring different approaches to data collection, model architecture, and evaluation.

Stage 1: Interpolation - Working Within the Known

Interpolation is where most AI systems operate—combining existing patterns in familiar ways. When GPT generates a Shakespearean sonnet or DALL-E creates an image of a cat wearing a top hat, they're working within the statistical patterns of their training data.

"Think of interpolation like a skilled chef following a recipe," suggests Dr. Michael Ramirez, who studies computational creativity at MIT. "The chef may add a personal touch here or there, but they're working within well-established culinary traditions. They're unlikely to create anything revolutionary, but the results will be reliably good."

In mathematical terms, interpolation can be understood through conditional probability. For language models, this looks like:

p(w₁, w₂, ..., wₙ) = p(w₁) × p(w₂|w₁) × p(w₃|w₁,w₂) × ... × p(wₙ|w₁,w₂,...,wₙ₋₁)

Where each term represents the probability of a word given the preceding words. The AI's creativity is limited to sampling from high-probability regions of this distribution—areas where it has seen many examples during training.

Building the Dataset for Interpolation

The data requirements for interpolation-stage systems are substantial but straightforward:

Volume: Typically 10⁴-10⁶ examples depending on domain complexity
Breadth: Coverage across the full distribution of desired outputs
Quality: Clean, well-labeled data with minimal noise or bias
Granularity: Appropriate level of detail for the intended application

AlphaGo's training, for example, included:

160,000 games played by human experts
30 million positions extracted from those games
Millions of self-play games where the system played against itself

"People underestimate the importance of this foundational data work," notes Dr. Chen. "We spent almost a year just curating our protein structure dataset—filtering experimental artifacts, resolving inconsistencies in the data, and ensuring proper annotations. That unglamorous work made everything else possible."

The Methodology of Interpolation

Implementing the interpolation stage follows a structured process:

Domain Mapping: Identify the core elements and relationships in your field. For a medical diagnosis system, this means mapping symptoms to conditions, tracking how conditions progress over time, and understanding how treatments affect outcomes.
Data Harmonization: Ensure consistency across your dataset. For climate models, this might involve normalizing measurements from different sensors, accounting for changes in measurement technology over time, and establishing common units.
Distribution Modeling: Train systems to capture the full probability distribution of valid examples. For creative text generation, this means learning not just what words typically follow others, but the higher-level structure of narratives, arguments, and stylistic elements.
Validation Against Ground Truth: Test by comparing outputs to known examples not seen during training. For a medical imaging system, this means evaluating diagnostic accuracy against radiologist-confirmed cases.

Alex Turner, who led AI research at Recursion Pharmaceuticals, emphasizes the importance of thorough validation: "Before we even consider creative applications, we verify that our system has mastered the basics. Can it accurately predict how standard compounds will affect cell cultures? If not, we go back to the drawing board."

Stage 2: Extrapolation - Finding Your Move 37

If interpolation is about operating within the crowded center of probability distributions, extrapolation ventures into the sparser regions—areas where valid examples might exist but are rarely or never seen in training data.

This is where AlphaGo's Move 37 lives—a move that had a vanishingly small probability in the distribution of human play, yet proved brilliantly effective.

"Extrapolation is where AI starts to get interesting," explains Dr. Ramirez. "It's like a chef who understands food chemistry so well that they can create entirely new flavor combinations that shouldn't work on paper but are delicious in practice."

In statistical terms, extrapolation involves sampling from the tails of learned distributions while maintaining consistency with underlying patterns. For a system like AlphaGo, this means finding moves that might have a low probability p(move|historical_games) based on human play, but high expected value E[outcome|move] based on its own evaluation.

The Dataset for Extrapolation

Extrapolation builds on the foundation of interpolation but requires additional data elements:

Boundary Examples: Rare but valid cases from the edges of the distribution
Synthetic Data: Computer-generated examples that explore sparse regions
Constraint Definitions: Formal rules defining validity boundaries
Negative Examples: Invalid cases that help define what's outside the distribution

AlphaFold's protein structure prediction, for example, supplemented its training data with:

Computationally simulated protein structures for rare configurations
Physics-based rules defining molecular forces and chemical constraints
Protein sequences with known invalid folding patterns

"The key is defining your validity boundaries clearly," explains Dr. Chen. "In protein folding, a structure might be novel but physically impossible. Our system needs to know the difference between creative extrapolation and physically impossible hallucination."

The Novelty Parameter: Tuning AI Creativity

At the heart of extrapolation is what Hassabis calls the "novelty parameter" (λ)—essentially a dial that controls how far from high-probability regions the system should explore.

This parameter modifies the objective function that guides exploration:

Objective = (1-λ) × Expected_Value + λ × Novelty_Reward

Setting λ = 0 results in pure exploitation (sticking to well-known solutions), while increasing λ encourages progressively more creative exploration. Different domains require different settings:

Physics/Chemistry: λ = 0.15-0.25 (stronger physical constraints)
Biology: λ = 0.25-0.35 (complex but rule-bound)
Social Sciences: λ = 0.30-0.40 (more interpretive flexibility)
Creative Arts: λ = 0.35-0.50 (higher tolerance for novelty)

"Finding the right λ value is more art than science," admits Turner. "Too low, and your outputs will be boring; too high, and they'll be fascinating but useless. We typically start around 0.25 and gradually adjust based on expert feedback."

The Exploration Algorithms

Different algorithms excel at different types of exploration:

Monte Carlo Tree Search (MCTS): Used by AlphaGo, MCTS balances exploitation and exploration by:

Selecting promising paths based on both value and uncertainty
Expanding the search tree by adding new nodes
Simulating outcomes through random playouts
Backpropagating results to update node values

The balance between exploitation and exploration is typically managed through the Upper Confidence Bound formula:

UCB = Q(s,a) + c × √(ln(N(s))/N(s,a))

Where Q(s,a) is the estimated value, N(s) is the parent node visit count, N(s,a) is the action visit count, and c is an exploration constant (analogous to our novelty parameter λ).

Variational Autoencoders (VAEs): For generating novel outputs in high-dimensional spaces like images or molecules, VAEs learn a compressed latent space Z that captures the underlying structure of valid examples.

The objective function balances reconstruction accuracy with latent space regularization:

L = Reconstruction_Loss + β × KL_Divergence

Where β controls the trade-off between accuracy and exploration.

Bayesian Optimization: When evaluations are expensive (as with wet-lab experiments), Bayesian optimization efficiently explores by:

Building a probabilistic model of the objective function
Computing an acquisition function that balances exploitation and exploration
Selecting the next point to evaluate based on this acquisition function
Updating the model with new observations

The Human-in-the-Loop Filter

No computational creativity system is complete without human evaluation. In Hassabis' framework, domain experts play a crucial role in:

Filtering out "hallucinations" or invalid outputs
Prioritizing the most promising candidates
Providing feedback to refine evaluation functions

"We expect high rejection rates," notes Turner. "In our drug discovery projects, experts typically reject 70-90% of AI-generated molecule candidates. That's not failure—it's part of the process of separating creative insights from interesting hallucinations."

Stage 3: Invention - Creating New Probability Landscapes

The rarest and most profound form of creativity in Hassabis' framework is invention—the creation of entirely new conceptual frameworks that transform how we understand problems.

"This is where we're trying to 'jump out of the system,' to use Gödel's phrase," explains Dr. Maya Rodriguez, an AI researcher at MIT. "Instead of working within existing probability distributions, we're discovering new ones that make different assumptions about what variables matter and how they relate."

In statistical terms, invention involves discovering new probability distributions that better explain phenomena by connecting previously separate domains. If we represent a domain's knowledge as a probabilistic graphical model G = (V, E), where V represents variables and E represents their relationships, invention involves creating a new model G' = (V', E') that either:

Connects previously separate graphs: G' = G₁ ∪ G₂ ∪ new_edges
Simplifies existing graphs by identifying higher-order variables
Introduces entirely new variables that better explain observed phenomena

The Dataset for Invention

Invention-stage creativity requires fundamentally different data:

Multi-Domain Data: Datasets spanning different fields or perspectives
Meta-Knowledge: Information about how knowledge itself is structured
Historical Innovations: Examples of past paradigm shifts
Cross-Domain Correlations: Unexpected patterns that span fields

DeepMind's GNoME (Graph Networks for Materials Exploration) system, for example, combines:

Quantum mechanical simulation data
Experimental results from materials science
Crystallography databases
Manufacturing process information

"The key insight is that invention often happens at the boundaries between fields," notes Dr. Rodriguez. "When Einstein connected space and time into spacetime, he was creating a new mathematical framework that unified previously separate domains."

The Contrastive Learning Approach

For invention-stage creativity, contrastive learning helps identify patterns across domains by learning representations that:

Minimize distances between related concepts across domains
Maximize distances between unrelated concepts
Preserve the internal structure of each domain

The contrastive loss function often takes the form:

L = -log(exp(sim(a, a⁺)/τ) / Σ exp(sim(a, a_i)/τ))

Where sim is a similarity function, a⁺ is a positive example (related concept), a_i are negative examples (unrelated concepts), and τ is a temperature parameter.

The Invention Methodology

The invention stage follows a structured process:

Multi-Domain Integration: Gather datasets and models from different but potentially related fields. Example: Combining genomic data with clinical outcomes and environmental factors to discover new disease mechanisms.
Abstraction Mining: Search for higher-order patterns that simplify complex relationships. Example: Using dimensionality reduction techniques to identify the minimal sufficient statistics that capture a system's behavior.
Framework Evaluation: Test new conceptual frameworks against both existing data and novel predictions. Example: Einstein's general relativity was confirmed not just by explaining existing astronomical observations but by predicting new phenomena like gravitational lensing.

Dr. Rodriguez emphasizes that invention-level creativity remains aspirational for current AI systems: "Today's systems might contribute to human invention, but true paradigm shifts still require that creative leap that comes from human intuition combined with machine computation."

Part II: Implementing the Framework - A Practical Guide

The DHACM-7 Workflow

Translating Hassabis' theoretical framework into practice involves a seven-step workflow that guides projects from initial conception through creative exploration to validation and release.

Step 1: Create a Problem Charter

Before any technical work begins, draft a 1-2 page document that defines:

The specific problem you're addressing
Who benefits from solving it
Your success metrics
Ethical constraints and considerations

"This step is non-negotiable," insists Dr. Rodriguez. "Without clear success metrics and ethical guidelines, creative AI can easily go off the rails."

Example charter elements for a drug discovery project:

Research Question: Can we identify novel chemical scaffolds with antibacterial activity against multi-drug resistant (MDR) Gram-negative pathogens?

Success Metrics: 
- ≥3 structurally distinct chemical families identified
- MIC ≤32 μg/mL against priority pathogens
- Selectivity index ≥10 (bacterial vs. human cells)
- Activity against resistant strains with known escape mutations

Ethical Constraints:
- All data from ethically approved sources
- No use of private patient data
- Environmental impact of compounds assessed
- Dual-use potential evaluated by biosecurity committee

Step 2: Build Your Foundation Model

Develop a system that demonstrates basic competence in your domain:

Select an appropriate architecture for your data type
Gather comprehensive training data
Train with appropriate regularization
Evaluate against established benchmarks
Document all hyperparameters and design decisions

This baseline model should achieve performance comparable to current state-of-the-art in your field.

For the drug discovery example:

Architecture: Graph Neural Network for molecular representation + Transformer for contextual embedding

Training Data:
- 120,000 molecules with known antibacterial activity
- 2.8 million molecules with general bioactivity data
- 50,000 bacterial protein structures and ligand interactions

Baseline Performance:
- MIC prediction: R² = 0.71
- Toxicity prediction: R² = 0.68
- Overall multi-property prediction accuracy: 74%

Step 3: Conduct a Ground-Truth Audit

Rigorously assess your model's factual accuracy and potential biases:

Set aside 10-20% of data as a validation set
Test performance across relevant subgroups
Identify systematic errors or weaknesses
Establish a factual accuracy threshold (typically ≥95%)

Example audit findings:

Bias Detection:
- Performance dropped by 18% for natural product-like molecules
- Predictions for Gram-negative permeability less reliable (R² = 0.58)
- Molecules with uncommon functional groups showed higher error rates

Mitigation Actions:
- Augmented training with additional natural product examples
- Added permeability-focused auxiliary task with higher weighting
- Implemented confidence scoring to flag unusual structures

Step 4: Implement the Exploration Loop

This is the "Move 37 Engine" at the heart of the framework:

Choose an exploration algorithm appropriate to your domain
Set your novelty parameter based on domain constraints
Generate a diverse pool of candidates (typically 100-1000)
Score each candidate on novelty and estimated validity

Example implementation:

Exploration Strategy: Multi-objective optimization combining:
- Monte Carlo Tree Search for efficient space exploration
- Genetic algorithm component for scaffold evolution
- Reinforcement learning with novelty rewards

Novelty Parameter: λ = 0.32

Output: 3,800 initial candidates, filtered to 950 meeting basic criteria, clustered into 72 distinct chemical families

Step 5: Incorporate Human Expertise

No computational system can fully evaluate creative outputs without human judgment:

Assemble a diverse panel of domain experts
Create standardized evaluation forms
Document reasons for rejection
Use feedback to refine future exploration

Example review process:

Expert Panel: 3 medicinal chemists, 2 microbiologists, 1 pharmacologist

Assessment Criteria (1-5 scale):
- Chemical synthesizability
- Predicted metabolic stability
- Novelty of structure
- Plausibility of predicted activity

Rejection Patterns: Of 950 candidates:
- 412 rejected for synthetic infeasibility
- 208 rejected for obvious toxicity concerns
- 125 rejected for predicted instability
- 73 rejected for mechanism implausibility

Selected Candidates: 132 molecules passing all expert filters, clustered into 18 distinct scaffold families

Step 6: Pursue Cross-Domain Synthesis

To reach the invention stage, actively seek connections across domains:

Identify secondary knowledge bases relevant to your problem
Map your candidates into these secondary domains
Apply contrastive learning to identify cross-domain patterns
Evaluate potential new frameworks for explanatory power

Example synthesis:

Secondary Integration:
- Bacterial transcriptomics data from resistance studies
- Protein-protein interaction networks
- Metabolic pathway models for target pathogens

Emergent Framework:
- Novel relationship discovered between siderophore pathways and membrane permeability
- Unexpected connection between scaffold elements and ribosomal protein interactions
- New paradigm for antibiotic design targeting stress response coordination

Step 7: Validate and Document Extensively

Ensure your creative outputs can be independently verified:

Conduct rigorous testing against held-out data
Have separate teams attempt to reproduce key findings
Publish comprehensive methodology
Create accessible tools for broader application

Example validation:

Experimental Validation:
- 22 molecules synthesized by contract chemistry lab
- 18 showed activity in preliminary screening
- 7 showed activity against resistant clinical isolates
- 3 demonstrated novel mechanisms through transcriptomic studies

Documentation:
- Full computational notebooks with exact parameters
- Molecular generation traces for reproducibility
- Expert assessment records
- Synthesized molecule characterization data

Case Study: Reimagining Cancer Treatment

To see how this framework translates into practice, consider how Recursion Pharmaceuticals applied it to cancer drug discovery.

Interpolation Stage:

The team began by building a comprehensive foundation:

"We collected images of 20 million cells treated with 40,000 compounds," explains Turner. "Each image captured 1,000+ cellular features across multiple cell types. We trained deep learning models to recognize how healthy and cancerous cells respond to known drugs."

This established the baseline probability distribution of how cells respond to treatment.

Extrapolation Stage:

With this foundation, they moved to creative exploration:

"We set our novelty parameter to λ = 0.28, balancing novelty with biological plausibility. We used evolutionary algorithms to explore chemical space, generating 200,000 novel molecular structures. Each structure was evaluated for synthesizability, predicted safety, and unique mechanism of action."

Human experts filtered the top candidates, eliminating 85% based on chemical expertise.

"We weren't just looking for compounds similar to existing drugs—those have already been extensively explored by pharmaceutical companies. We were looking for new mechanisms of action that operate on cellular pathways differently than existing treatments."

Invention Stage:

The most surprising breakthrough came when they connected their cellular imaging data with genomic databases:

"We discovered that certain molecular structures affected cells differently depending on specific gene variants. This led to a new framework for personalized cancer treatment that predicts which drugs will work for which genetic profiles."

The framework connected previously separate domains: chemical screening, cellular imaging, and genomic medicine.

"What's remarkable is that this framework wasn't explicitly programmed—it emerged from the data. By connecting these different domains, we discovered patterns that weren't visible within any single domain."

The result: two drug candidates now in clinical trials that would likely never have been discovered through traditional approaches.

Part III: Extending the Framework

Data Collection Best Practices

The quality and structure of data fundamentally shapes what kinds of creativity are possible. Hassabis' framework prescribes different data collection strategies for each creative stage.

For Interpolation:

Focus on comprehensive coverage with attention to:

Volume: Typically 10⁴-10⁶ examples depending on domain complexity
Diversity: Samples from across the full distribution
Quality: Clean, well-annotated data with minimal noise
Coverage: Representation of all relevant subgroups

"Don't skimp on data preparation," advises Dr. Chen. "In protein folding, we spent months curating our datasets—filtering experimental artifacts, resolving inconsistencies, and ensuring proper annotations. That unglamorous work made everything else possible."

For Extrapolation:

Supplement comprehensive datasets with:

Edge Cases: Rare but valid examples from distribution tails
Synthetic Data: Simulated examples from sparse regions
Constraint Definitions: Formal rules defining validity boundaries
Evaluation Protocols: Methods to assess novel outputs

"The key is defining your validity boundaries clearly," explains Turner. "In drug discovery, we use computational chemistry tools to verify that our AI-generated molecules follow physical laws, even when they look nothing like existing drugs."

For Invention:

Gather cross-domain datasets with emphasis on:

Multiple Perspectives: Data representing different conceptualizations of related phenomena
Historical Innovations: Examples of past paradigm shifts
Meta-Knowledge: Information about the relationships between domains
Evaluation Frameworks: Methods to assess explanatory power across contexts

"This is where traditional machine learning approaches often fall short," notes Dr. Rodriguez. "Invention requires not just more data but fundamentally different data—data about how knowledge itself is structured."

Mathematical Tools for Creative Exploration

Several mathematical frameworks prove particularly valuable when implementing Hassabis' approach:

1. Probabilistic Graphical Models

Represent knowledge as a network where nodes are variables and edges are relationships. This allows for explicit modeling of conditional dependencies and enables reasoning about causal structures.

For example, in healthcare, a graphical model might connect:

Genetic variants → Protein expression → Metabolic pathways → Disease symptoms

This structured representation makes it easier to identify where new knowledge might fit or where current understanding is incomplete.

2. Information Bottleneck Methods

These techniques identify the minimal sufficient statistics that capture relevant information about a target variable Y from an input X:

min I(X;Z) - β I(Z;Y)

Where I is mutual information, Z is a compressed representation, and β controls the trade-off between compression and prediction.

This approach helps identify the core variables that matter in a complex system, enabling more efficient exploration and better abstraction.

3. Disentangled Representations

Learning representations where different dimensions correspond to meaningful, independent factors of variation. This makes it easier to:

Identify which factors matter for a particular task
Manipulate specific attributes while keeping others constant
Transfer knowledge across domains by mapping similar factors

For example, in molecular design, disentangled representations might separate aspects like:

Solubility (hydrophilic vs. hydrophobic)
Size (small molecule vs. macromolecule)
Charge distribution (polar vs. nonpolar)
Structural flexibility (rigid vs. flexible)

This makes creative exploration more tractable by allowing targeted modifications along specific dimensions.

Beyond Technical Implementation: The Human Element

Despite its technical sophistication, what's most striking about Hassabis' framework is how deeply it integrates human and machine cognition. It recognizes that AI creativity works best not as a replacement for human creativity but as a collaborator in the creative process.

"The most powerful applications of this framework involve human-AI co-creativity," emphasizes Dr. Rodriguez. "The machine proposes possibilities humans wouldn't have considered, while humans provide judgment about which possibilities matter."

This collaboration takes different forms across the creative stages:

In Interpolation: Humans define problems and evaluate solutions, while AI handles the computational heavy lifting of pattern recognition and synthesis.

In Extrapolation: Humans set constraints and novelty parameters, while AI systematically explores possibility spaces too vast for human cognition.

In Invention: AI identifies unexpected connections between domains, while humans interpret these connections and develop new conceptual frameworks.

Dr. Chen puts it simply: "AlphaGo couldn't have made Move 37 without Lee Sedol to play against. And Lee couldn't have seen the beauty of that move without AlphaGo to play it."

Epilogue: The Legacy of Move 37

Five years after AlphaGo's historic match, Lee Sedol made a surprising announcement: he was retiring from professional Go. "With the debut of AI in Go games, I've realized that I'm not at the top even if I become the number one," he explained.

Yet rather than despair, Lee found a new purpose. "Now I enjoy more playing against AI than human Go players," he said. "I found that there are moves that humans have not discovered yet."

This perspective—seeing AI not as a competitor but as a collaborator in expanding human creativity—captures the true promise of Hassabis' framework. The legacy of Move 37 isn't just a better way to play an ancient board game. It's a new approach to thinking itself—one that embraces machine creativity not as a threat but as a tool for pushing the boundaries of human knowledge.

In fields from drug discovery to climate science, from materials research to urban planning, this framework offers a structured approach to harnessing creativity, whether human or machine. And in a world facing unprecedented challenges, this creative partnership between human and machine intelligence may be our most valuable resource—not because machines will solve our problems for us, but because they might help us see those problems in new ways.

As Hassabis himself puts it: "The true potential of AI isn't to replace human creativity, but to expand it—to help us find our own Move 37s in science, medicine, and beyond."