The Knowledge We Cannot Know

When AI Discovers What Humans Can't Explain

Across multiple scientific domains, we are witnessing an inversion of the traditional knowledge-creation process.

Instead of theory-guided discovery, scientific insights are coming from patterns that AI systems detect but humans cannot initially comprehend. AI is producing operational knowledge that works in practice before humans can explain why it works in theory.

The implication is that reality may contain aspects fundamentally inaccessible to human cognition alone.

If these insights exist outside of the human theoretical corpus, how do we evaluate the veracity of this type of knowledge? What does this mean for our understanding of knowledge itself?

In this article, I argue that in oder to navigate a world where transformative discoveries emerge from patterns beyond human comprehension, we need a new kind of theory of knowledge a.k.a epistemology: a negotiated-epistemology. An epistemology that shifts from “what is true?” to “whose truth matters?” A concept that doesn’t demand complete human comprehension but still preserves human agency and values in deciding which AI-discovered patterns to adopt.

AI gets us from essence to resemblance

For millennia, we've operated under the rationalist paradigm of scientific discovery: that there is something to learn about reality. An objective truth lying somewhere and accessible through human reasoning. Objectivity, the cornerstone of rationalist thinking, propelled scientific progress by assuming that things have essential properties waiting to be discovered.

In philosophical terms, this is the classical paradigm that things have a form of essence (remember Plato’s Cave) and that our job as humans is to uncover these truths. Our quest is to forever get closer to that truth.

AI ditches this paradigm and embraces a view more closer to Wittgenstein: that of a family of resemblances. AI systems in the form of Large Language Models (LLMs) map resemblances. They group things based on statistical similarities and predict what comes next based on that, and not on any essential property.

This resemblance-based approach is producing knowledge that functions effectively in the world while existing partially beyond human comprehension.

The Fracturing of Objectivity: AI Might See More than We Can

Let me trace this shift from theory to pattern recognition with three examples of groundbreaking discoveries:

1. Drug discovery: Halicin - The Antibiotic Human Theory Missed

Halicin is an antibiotic drug discovered in partnership with AI.

MIT researchers trained an AI on 2,000 molecules to learn about the ability of molecules to inhibit bacterial growth. The system then evaluated over 60,000 candidate molecules from a chemical library, looking for compounds with potential antibiotic properties that wouldn't be toxic to humans. It found one: Halicin.

What struck me about this finding is that the AI identified relationships that were outside the concepts and theories humans have devised. It didn’t need to know why the molecule could work. It merely had to make a prediction based on the fabric of relationships it had encoded during training.

“The AI that MIT researchers trained did not simply recapitulate conclusions derived from the previously observed qualities of the molecules. Rather, it detected new molecular qualities - relationships between aspects of their structure and their antibiotic capacity that humans had neither perceived nor defined. Even after the antibiotic was discovered, humans could not articulate precisely why it worked” (Source: Kissinger, Schmidt & Huttenlocher, 2021)

2. Weather prediction

In 2023, DeepMind's AI weather forecasting models GraphCast was released to the public displaying “10-day weather predictions at unprecedented accuracy in under one minute.” (Source: Google DeepMind)

The most interesting aspect (beyond the improved accuracy) was that the AI had identified atmospheric relationships that weren't accounted for in existing meteorological theories.

Climate scientists found themselves in the position of using predictions they couldn't fully explain theoretically due to the “black box” nature of LLMs.

“Despite their predictive capabilities, most advanced ML models used for meteorology are usually regarded as "black boxes", lacking inherent transparency in their underlying logic and feature attributions (Du et al., 2019; Deng et al., 2021; Xiong et al., 2024). This lack of interpretability poses major challenges. First, it reduces trust from domain experts, such as meteorologists, who may be reluctant to rely on unexplained model outputs for high-stakes decision making. Second, it hinders further model refinement, as developers cannot easily diagnose errors or identify which relationships the models have captured. Third, opaque ML models provide limited insight into the fundamental atmospheric processes that lead to their predictions.” Source: (Yang et al., 2024, p. 2)

Even in domains where we believe our theoretical understanding is strong, AI can detect patterns that exceed our current frameworks.

3. AI even won a Nobel prize

In 2024, Hassabis and Jumper, the creators of the AI model Alphafold2 were awarded the Nobel Prize in Chemistry for developing an

“AI model to solve a 50-year-old problem: predicting proteins’ complex structures.” (Source: Nobel Prize Press Release)

Understanding how proteins fold is crucial for drug design and disease treatment. Yet despite decades of effort, before this scientists could only determine structures through expensive, time-consuming experiments.

Alphafold2 had succeeded in predicting the structures of over 200 million proteins, accomplishing more in a few months than scientists had in the previous 150 years.

Yet like Halicin and GraphCast before it, there was a catch: not even AlphaFold2's creators could fully explain how it arrived at its predictions.

While they understand the architecture —the neural networks, attention mechanisms, and training procedures—the actual decision-making process that emerges from billions of parameters remains opaque. As illustrated in this document, the precise way the model calculates probabilities and determines protein structures from amino acid sequences remains, an unsolved mystery.

The Nobel committee essentially awarded science's highest honor to a discovery process that defied scientific explanation.

AI models are built to learn by themselves and once they do, they are black boxes: interconnected parameters whose logic and inner working are impenetrable. Yet these black boxes are producing knowledge that works better than our transparent theories.

Navigating a negotiated epistemology

The realization that AI finds structures, connections, and relationships that humans might never independently discover challenges our position at the pedestal of scientific truth.