Talk about AI today, and the star of the show is always the Large Language Model (LLM) like ChatGPT, Claude, or Gemini. They can write code, compose poems, and even hand out dating advice. At first glance, it feels like we are very close to AGI (Artificial General Intelligence, AI whose intelligence matches ours).
But have you noticed that this AI sometimes "hallucinates" and makes mistakes even a schoolkid would not make?
Among AI experts, that is a lively argument right now. One of the biggest names in the field, Yann LeCun (Chief AI Scientist at Meta), says plainly that the LLM architecture we have will eventually hit a wall. The road to AGI cannot lean on LLMs alone. The answer, according to LeCun, is a new architecture called JEPA (Joint Embedding Predictive Architecture).
The Curse of the "Stochastic Parrot"
Why do people keep saying LLMs will hit a wall? The main reason sits in how they work. The LLMs you use today are essentially word-guessing machines (next-token prediction). They are trained on trillions of words from the internet to guess the probability of which word should come next.
The trouble is that this approach traps LLMs in what researchers call a "Stochastic Parrot".
Picture a parrot that is very good at imitating a human voice. An LLM is brilliant at assembling words that look smart, but it does not really understand the meaning behind them. It has no foundation for understanding the physical world (reality). So when you ask it to explain how two objects interact, or a complex chain of cause and effect, it often errors out or hallucinates, because its knowledge is text probability, not basic physics.
Why Is Yann LeCun Sure LLMs Will Hit a Wall?
Across many forums, Yann LeCun keeps stressing that autoregressive models (systems that predict one word at a time, over and over) carry a fatal weakness: error accumulation.
If the AI makes one small mistake at the start of a sentence, that mistake carries over and gets worse in the sentences that follow. LeCun also argues that language (text) is already a heavily compressed form of information. A lot of what happens in the real world (gravity, the way objects fall, the way people move) cannot be explained through text alone. Humans and animals learn by seeing and touching the physical world, not only by reading books.
So however much text data you feed an LLM, it will never reach AGI, because it is blind to how the physical world works. It is an interface that talks well, with no strong backend (logic brain) behind it.
Enter JEPA: Predicting Meaning in "Latent Space"
If an LLM is guessing words, and image Generative AI (like DALL-E or Nano Banana) is guessing color pixels, JEPA arrives with a completely different way of thinking. JEPA does not care about tokens (words) or pixels. It predicts inside Representation Space or Latent Space (an abstract, conceptual space).
Put simply: imagine you want to tell a friend about a photo. Does your brain pick the first word, then think about the second, then the third? No. Your brain builds the whole concept first (say, "an orange cat asleep on the sofa"), and only then turns it into words.
That is the core of recent research like VL-JEPA (Vision-Language JEPA). Instead of burning compute to guess every word or redraw every pixel detail (like a cat hair nobody needs), JEPA is trained to guess the "meaning" or the "embedding". That makes it far more efficient, lighter on compute, and much closer to how human cognition works.
V-JEPA: Teaching AI Physics Through Video
One of the coolest breakthroughs from this architecture is V-JEPA (Video JEPA). If an LLM learns from text, V-JEPA learns purely from watching millions of hours of unlabeled video.
Because it works in latent space, V-JEPA learns to understand "intuitive physics". It can predict what happens next in a scene without re-rendering the video. It grasps that a glass falling to the floor will break. This is the foundation of common sense, the thing that has been kryptonite for conventional LLMs.
With a strong World Model from V-JEPA, future AI will not only chat. It can drive robotics systems that plan actions in the real world without being fed rigid instructions.
The Future: Marrying the "Mouth" and the "Brain"?
The road to AGI will not be walked by leaning on AI that is all talk and no substance, haha. This debate hands us a new perspective: LLMs are remarkable as a communication tool, but to be truly smart? AI still needs a foundation for understanding the real world.
Going forward, the architecture of the future may be a marriage of both. LLMs become the flexible "mouth" (communication interface), while something like JEPA becomes the "brain" (backend/world model) that guarantees the AI has common sense and understands the laws of reality.
It is not impossible that in the next 3-5 years, the next-token prediction paradigm starts to be left behind, and we enter an era where AI actually "thinks" before it talks. We'll see :D
So what do you think? Do you believe LLMs will hit a wall, or are you sure ChatGPT and friends can still evolve into AGI?