Vedant's Notes

Search

SearchSearch

Recently Created

AI Invention vs Discovery The Black Box Paradox

Created on May 11th 2026

AI: Invention vs. Discovery (The “Black Box” Paradox)

Part 2: The Transformer & The Black Box A Deep Dive

🧠 The Core Concept

There is a growing consensus among top tech leaders (like Jeff Bezos) and AI researchers that Large Language Models (LLMs) are fundamentally different from traditional software.

  • Traditional Software (Engineering): We build it like a bridge or a Boeing 787. We know exactly what every line of code does. If it works, it’s because we engineered it to work.
  • Generative AI (Discovery): We built the “container” (the algorithm), but the “intelligence” inside grew on its own. We are constantly surprised by what it can do.

🗣️ The Argument

Jeff Bezos’s Take:
LLMs are not inventions; they are discoveries.

“The telescope was an invention. But looking through it at Jupiter, knowing that it had moons, was a discovery.”

The “Alien Brain” Theory:
We didn’t “code” the AI to think. We discovered a mathematical structure (The Transformer) that, when fed enough data, develops a digital version of cognition. It feels less like engineering a machine and more like uncovering a biological entity built from math.

🔍 Is This Actually True?

Yes, with a nuance.

Technically, humans did write the code for the Transformer architecture (Vaswani et al., 2017). However, the statement is functionally true regarding how the model behaves.

1. The Problem of “Emergence”

In a Boeing 787, the wings provide lift because we calculated the aerodynamics. In an LLM, the model can explain a joke or write Python code, but we never explicitly programmed it to do those things.

  • We just taught it to predict the next word.
  • Emergence: The ability to reason, code, and translate emerged spontaneously as a side effect of learning to predict words. This was a discovery, not a plan.

2. The “Black Box” (Interpretability Crisis)

If you open the brain of ChatGPT, you don’t find code that says if user asks for poem, write poem. You find billions of floating-point numbers (matrices).
We know how the math works (multiplication), but we don’t know why a specific arrangement of numbers results in the concept of “love” or “democracy.” We are currently trying to reverse-engineer our own creation.

💡 Key Analogies for Understanding

The Telescope vs. Jupiter

  • The Invention: The Transformer architecture (the code/math) is the Telescope. We built the lens.
  • The Discovery: The resulting intelligence (GPT-4, Claude) is Jupiter. We looked through the lens we built and were shocked to find a sophisticated world we didn’t design.

The 787 vs. The Petri Dish

  • The 787 (Old Tech): An engineered object. 100% predictable. If a plane crashes, we can trace it to a specific bolt or line of code. We want zero surprises.
  • The Petri Dish (AI): We created the environment (the algorithm and data) and applied heat (compute power). Something complex grew out of it. We are now poking it to see what it does.

📌 Summary

We invented the mechanism (Transformers), but we discovered the phenomenon (digital intelligence). We are currently in a phase of scientific observation of a tool we built, trying to understand the “second kind of brain” that evolved out of our math.

Backlinks