What is the most significant technical limitation you have encountered when working with large language models, and why?
Most answers highlight challenges with context handling and coherence in large language models, while others point to compute limits, real-world reasoning, catastrophic forgetting, and uncertainty modeling.
7 answers
Answers


Yann LeCun
VP & Chief AI Scientist at Meta · Facebook
Real-World Data Handling
Yann LeCun says LLMs succeed where language is the substrate of reasoning but struggle with the real world, which he describes as messy, continuous and unpredictable rather than made of words.
TheAIGRID · Watch on YouTube

ali dasdan
Visiting Research Scientist · New York University
Limited Context Window
Ali Dasdan says the limited context window is the problem: with too much context the model doesn't know what to focus on and may ignore the middle part.
MIT Sloan Management Review · Watch on YouTube
Richard Socher
CEO · Recursive
Compute Constraints
Richard Socher says compute is the biggest constraint, as companies will have to decide how much compute to give to solving different problems.
The MAD Podcast with Matt Turck · Watch on YouTube

Max Spero
Cofounder & CEO · Pangram Labs
Long-Context Coherence
Max Spero says the limitation is long-context writing: while AI may be superhuman at short passages, it can't maintain that over the course of a novel.
Derek Thompson · Derek Thompson · Read

Itamar Friedman
Co-Founder & CEO · Qodo
Context Fragmentation
Itamar Friedman says the key issue is context: given the right context, models can already reason well about code issues, but without it even the best model gives generic feedback like asking about error handling, and he notes context is currently spread across agents, cloud and skills MD files with differing standards.
AI Engineer · Watch on YouTube

Ang Li
CEO & Co-Founder · Simular
Catastrophic Forgetting
Ang Li says catastrophic forgetting is an important problem in training neural networks that no one has solved, where models' weight changes cause them to forget what they learned from past data, and says continual learning approaches aim to fix this.
Analyse Podcast · Analyse Podcast · Listen at 9:38
Ruchir Puri
Chief Scientist · IBM Research
Modeling Uncertainty
Ruchir Puri identifies modeling uncertainty as one of two fundamental issues, saying these models are still learning to model uncertainty and always tell you an answer rather than say “I don’t know.”
The Deep View: Conversations · The Deep View: Conversations · Listen at 11:22
More Data Science & Machine Learning questions
View all →- Why does AI need continual learning beyond static training?6 answers
- What practice best gives AI models the right context?9 answers
- What practice advances reinforcement learning beyond tradition?7 answers
- What is key to building effective domain-specific AI models?9 answers
- What matters most when evaluating autonomous AI agents?6 answers


















