What is the most significant technical limitation you have encountered when working with large language models, and why?

Most answers highlight challenges with context handling and coherence in large language models, while others point to compute limits, real-world reasoning, catastrophic forgetting, and uncertainty modeling.

7 answers

Answers

  • Yann LeCun

    VP & Chief AI Scientist at Meta · Facebook

    Real-World Data Handling

    Yann LeCun says LLMs succeed where language is the substrate of reasoning but struggle with the real world, which he describes as messy, continuous and unpredictable rather than made of words.

    TheAIGRID · Watch on YouTube
  • ali dasdan

    Visiting Research Scientist · New York University

    Limited Context Window

    Ali Dasdan says the limited context window is the problem: with too much context the model doesn't know what to focus on and may ignore the middle part.

    MIT Sloan Management Review · Watch on YouTube
  • Richard Socher

    CEO · Recursive

    Compute Constraints

    Richard Socher says compute is the biggest constraint, as companies will have to decide how much compute to give to solving different problems.

    The MAD Podcast with Matt Turck · Watch on YouTube
  • Max Spero

    Cofounder & CEO · Pangram Labs

    Long-Context Coherence

    Max Spero says the limitation is long-context writing: while AI may be superhuman at short passages, it can't maintain that over the course of a novel.

    Derek Thompson · Derek Thompson · Read
  • Itamar Friedman

    Co-Founder & CEO · Qodo

    Context Fragmentation

    Itamar Friedman says the key issue is context: given the right context, models can already reason well about code issues, but without it even the best model gives generic feedback like asking about error handling, and he notes context is currently spread across agents, cloud and skills MD files with differing standards.

    AI Engineer · Watch on YouTube
  • Ang Li

    CEO & Co-Founder · Simular

    Catastrophic Forgetting

    Ang Li says catastrophic forgetting is an important problem in training neural networks that no one has solved, where models' weight changes cause them to forget what they learned from past data, and says continual learning approaches aim to fix this.

    Analyse Podcast · Analyse Podcast · Listen at 9:38
  • Ruchir Puri

    Chief Scientist · IBM Research

    Modeling Uncertainty

    Ruchir Puri identifies modeling uncertainty as one of two fundamental issues, saying these models are still learning to model uncertainty and always tell you an answer rather than say “I don’t know.”

    The Deep View: Conversations · The Deep View: Conversations · Listen at 11:22

More Data Science & Machine Learning questions

View all →