Imagine training a model on trillions of words, only to find that it suddenly starts solving math problems it was never explicitly taught. This is the core mystery behind emergent capabilities, which are skills or behaviors in AI systems that appear abruptly in larger models but are absent in smaller ones. The term gained traction after Jason Wei and colleagues published their foundational paper in 2022, arguing that certain abilities cannot be predicted by simply looking at how smaller models perform. As of 2026, this concept remains central to understanding where artificial intelligence is heading, yet it also sparks intense debate about whether these jumps represent genuine breakthroughs or just measurement artifacts.

The Core Definition and Historical Context

To understand the current state of research, we first need to pin down what researchers actually mean when they say a capability is "emergent." According to the most widely accepted academic definition, an ability qualifies as emergent if it demonstrates unpredictability and a significant magnitude of performance increase. Specifically, models must perform better than random chance upon reaching specific scale thresholds. Before this threshold, performance often looks stagnant or even random. Once crossed, however, proficiency unlocks dramatically. This mirrors principles from complexity science, where quantitative increases in inputs lead to qualitative shifts in outcomes. In the context of Large Language Models (LLMs), this phenomenon is amplified by transformer architectures' self-attention mechanisms, which allow the system to capture long-range dependencies across vast datasets.

The historical record provides concrete examples of this shift. For instance, on the BIG-Bench "emoji_movie" task, models predict movie titles from emoji sequences only after scaling up significantly. Below a certain parameter count, log-probabilities for correct tokens remain flat. Above that threshold, they diverge sharply. This pattern suggests that while predictable loss reduction via more compute sets the stage, downstream emergence defies simple extrapolation. It is not just about having more data; it is about crossing a critical point where the architecture can leverage that data effectively.

Documented Examples of Emergence

Research has cataloged over 137 distinct instances of emergent capabilities, spanning diverse domains. These examples illustrate the breadth of what scaling alone can unlock without explicit programming. Some of the most notable include:

• Instruction-following: First observed in FLAN 68B models, where fine-tuning on instructions enabled zero-shot generalization to unseen tasks.
• Chain-of-thought prompting: Observed in LaMDA 68B, enabling multi-step reasoning by generating intermediate thoughts before final answers.
• Zero-shot chain-of-thought: Demonstrated in GPT-3 175B, allowing the solution of math word problems with the simple prompt "Let's think step-by-step."
• Multilingual reasoning: PaLM 62B solved multi-step math problems in under-represented languages, showing that reasoning capabilities transfer across linguistic boundaries.

These cases highlight a key theme: the model isn't just memorizing patterns; it is developing new ways to process information. For example, scratchpad abilities in LaMDA 40M allowed the model to execute algorithms by predicting intermediate states line-by-line. This suggests that internal representations are reorganizing in ways that were not anticipated during initial design.

The Mechanism: Memorization vs. Generalization

So, why does this happen? Recent survey research proposes that emergent abilities result from the competition between memorization and generalization circuits within the neural network. At first, heavy memorization delays generalization. Opposing patterns between task difficulty and model capacity cancel each other out, creating the illusion of stagnant performance. However, once models cross a critical scale, the balance shifts. The reversal of these trends triggers a sudden leap in performance. This finding suggests that emergence is not about acquiring new capabilities from nowhere, but rather about overcoming a hidden trade-off. The model eventually finds a way to generalize beyond rote memorization, unlocking complex pattern recognition.

This mechanism explains why smaller models (defined as fewer than 10 billion parameters) often exhibit random-level performance on diverse tasks. They lack the capacity to break free from memorization. Crossing thresholds around 10^11 effective parameters unlocks proficiency because the architecture finally has enough room to develop robust generalization circuits. This insight is crucial for developers who want to predict when a model might become useful for complex tasks.

The Controversy: Real Phenomenon or Measurement Artifact?

Despite the excitement, severe debate exists regarding whether emergent capabilities represent genuine new competencies or merely measurement artifacts. Stanford's Human-Centered Artificial Intelligence (HAI) institute published an in-depth analysis demonstrating that the highly publicized story of LLMs' emergent abilities springs from the use of harsh metrics. Their research indicates that metric sensitivity and discontinuous evaluations, such as exact match scores, mask gradual sub-task improvements. When using continuous metrics like log-likelihoods, progress appears steady rather than sudden. This distinction has led some researchers to argue that the term "emergence" should be reserved for cases demonstrating clear evidence of new, compressed internal representations and true reorganization inside neural networks.

This debate matters because it affects how we interpret AI development trajectories. If emergence is a mirage created by poor metrics, then scaling laws might be more predictable than we think. If it is real, then we face unpredictable jumps in capability that could complicate safety protocols. The consensus leans toward a hybrid view: while some apparent emergence is due to metric choice, many documented cases show genuine qualitative shifts in behavior that cannot be explained by linear scaling alone.

Implications for AI Safety and Development

The implications of emergent capabilities reshape AI development trajectories, promising breakthroughs while demanding rigorous evaluation. Emergence highlights scaling's dual edge: efficiency gains versus opacity. Policymakers and engineers must prioritize pre-scale predictions to mitigate risks. Past forecasts underestimated the performance leaps seen between 2022 and 2023, and continuous metrics aid prediction but falter on novel tasks like bio-weapon planning. Unforeseen jumps, such as hacking capabilities, complicate safety considerations. Post-training techniques like chain-of-thought boost capabilities unexpectedly, making it harder to anticipate what a model will do in production.

Benchmark limitations mean that holistic real-world impact trumps isolated scores. Red-teaming reveals gaps in absence proofs, showing that models can fail in subtle ways that standard tests miss. Economic drivers accelerate compute scaling, with models like GPT-4 reaching 1.75 trillion parameters, but ethical forecasting lags behind deployment velocity. The phenomenon has garnered attention because of the potential for unpredictable emergence of risky capabilities, fueling extensive AI safety discussions. Proponents catalog over 137 instances spanning few-shot prompting to symbolic reasoning, underscoring LLMs' potential for novel generalization without explicit training.

Future Research Directions

The research community identifies specific challenges in understanding and predicting emergent capabilities. Predictability remains elusive, with past forecasts substantially underestimating capabilities development. Risk amplification through unforeseen capability jumps complicates safety protocols and red-teaming efforts. Benchmark limitations indicate that isolated task performance does not capture real-world impacts adequately. The economic imperative toward compute scaling accelerates capability development while safety research and ethical consideration lag. These factors create what researchers characterize as emergence implying that scaling could unlock unforeseen risks or benefits.

Future research directions address current limitations and advance understanding of emergent phenomena. Key priorities include:

1. Advanced metrics: Developing pre-scale continuous proxies for multi-hop tasks to better track gradual improvements.
2. Mechanistic interpretability: Probing internal activations for causal emergence to understand the neural mechanisms underlying capability emergence.
3. Hybrid scaling approaches: Combining conventional scaling with architectural innovations for controlled capability gains.
4. Safety benchmarks: Emphasizing real-world red-teaming beyond limited benchmark sets like BIG-Bench.
5. Economic modeling: Seeking to predict compute thresholds via scaling laws, enabling better forecasting of capability development.

Interdisciplinary approaches borrow frameworks from physics and complexity science for theoretical understanding. Open catalogs expand Wei's 137+ emergent capability list with standardized evaluations, creating comprehensive repositories of known emergent phenomena. By integrating these methods, researchers hope to move from observing emergence to predicting and controlling it.