#06 · Foundation Models & Inference Infrastructure
Best Long-Context Models (1M+ Tokens)
What is a long-context model?
A long-context model is a large language model engineered to accept and reason over very large amounts of input text in a single request — typically measured in tokens, where one token corresponds roughly to three-quarters of an English word. "Long" is a moving target: a 32K-token window felt expansive in 2023, a 200K window was state-of-the-art in 2024, 1M tokens became standard in 2025, and several models now push toward 2M to 10M tokens. At those sizes, a single request can contain entire codebases, multi-document legal files, full-length books, complete customer histories, or hour-long video transcripts — workloads that previously required complex retrieval orchestration to break into smaller chunks. Long-context capability is enabled by architectural innovations (sparse attention, ring attention, position-encoding schemes like RoPE scaling) and engineering work on KV-cache management, since naive attention scales quadratically with context length.
Why long context matters in enterprise AI.
Long context unlocks genuinely new workload classes. Entire-codebase analysis for refactoring or audit work no longer requires retrieval — the codebase fits. Multi-document legal review (contract families, deposition transcripts, regulatory filings) can be done in a single pass with cross-document reasoning. Full customer conversation histories can be analyzed without chunking. Long-form content generation (technical books, in-depth reports) can maintain coherence across a length that was previously infeasible. The dominant production pattern of 2023–24 — RAG with sliding context windows — becomes optional rather than required for many of these workloads, and in some cases long-context analysis produces meaningfully better results than retrieval because the model sees cross-document relationships RAG would miss.
What to evaluate.
The most important thing buyers should know is that *advertised* context and *effective* context are not the same thing. Independent evaluations consistently find that models lose recall and reasoning quality at the upper end of their advertised windows — sometimes well below the maximum. A model advertising 1M tokens may have effective recall closer to 200K, with degraded reasoning past that point. The "needle-in-a-haystack" test (can the model find a specific fact buried at varying depths?) gives a partial read, but real workloads benefit from task-specific evaluation: how well does the model actually do the thing you need at the length you need. Other evaluation axes: per-token pricing at length (full-context queries can be very expensive), latency (long context adds seconds-to-minutes), and reasoning behavior on long context (some models excel at extraction but degrade on multi-step reasoning over long inputs).
Largest enterprise-grade context window
Meta's Llama 4 Scout pushes the open-weight frontier to 10M tokens — approximately 7,500 pages of text in a single session, enough for entire-codebase analysis or multi-document legal review without chunking. The Scout variant complements the Maverick flagship in the Llama 4 family, with Scout specifically optimized for the long-context use case while Maverick targets reasoning depth. Released under Meta's community license, Scout has been adopted across the inference-provider ecosystem despite the engineering complexity of serving its very long contexts. Best for entire-codebase analysis, multi-document legal review, long-document content workflows, and any workload where retrieval orchestration adds complexity without proportionate benefit. Strengths include open weights, category-leading advertised context, mature ecosystem support, and integration with the broader Llama fine-tuning toolchain. Trade-offs are that effective recall at the upper end of the 10M window varies significantly by task, serving very long contexts requires significant memory and is expensive, and benchmarks at maximum length lag the shorter context window where the model is most reliable.
Best effective long-context performance
Gemini 3.1 Pro offers a native 1M-token context window with notably strong effective recall — Google has invested heavily in the engineering work of making long context actually useful rather than just nominally available. Critically, Gemini's long context handles multimodal content (long videos, mixed text-and-image documents) within the same window, which no competitor matches at this length. Available via Google AI Studio, Vertex AI, and Gemini APIs. Best for long multimodal workloads combining video and documents, scientific and research workflows over large evidence bases, and Google Cloud–standardized enterprises. Strengths include strong effective recall across the advertised length, unique multimodal handling within long context, deep Workspace and GCP integration, and competitive pricing for the capability offered. Trade-offs are premium pricing for full-context use, latency that increases meaningfully with context length, and a smaller fine-tuning ecosystem for long-context customization than Llama.
Largest practical context window
xAI's Grok 4.20 non-reasoning variant currently exposes the largest practical context window in the market at 2M tokens — the largest in any frontier-tier model. The non-reasoning designation reflects xAI's deliberate trade-off: this variant prioritizes context length and throughput over the additional inference-time compute that reasoning modes require, making it the right choice for workloads where deep reasoning is secondary to processing very large inputs. Available via the xAI platform and Grok API. Best for very long document analysis where reasoning depth is secondary, large-scale text extraction and summarization, and applications where Grok's content posture and real-time grounding matter. Strengths include category-leading 2M-token context, integration with X platform real-time signal, and competitive pricing for the length offered. Trade-offs are that the non-reasoning configuration trails the reasoning variant on hard analytical work, the enterprise tooling ecosystem is smaller than established frontier labs, and effective recall at the full 2M length should be benchmarked on actual workloads before commitment.
Strongest effective recall in its window
Claude Opus 4.7 ships a 200K-token context window that's smaller than competitors' headline numbers — but Anthropic has consistently prioritized effective recall over raw advertised length, with the result that Claude often outperforms longer-window competitors on actual long-context tasks at lengths up to 200K. For workloads in the 100K–200K range (long contracts, codebase chunks, multi-document analysis), Claude is frequently the right answer despite the smaller window. Available via Anthropic API, Amazon Bedrock, and Google Vertex AI. Best for workloads valuing reasoning quality and effective recall over raw context length, complex legal and contract analysis up to ~200K tokens, and code analysis over moderately-sized repositories. Strengths include strongest effective recall at advertised length, strong reasoning quality maintained across long context, broad cloud availability, and reliable behavior at the upper end of the window. Trade-offs are that advertised context is smaller than Gemini, Grok, or Llama Scout for workloads that genuinely need 1M+ tokens, and pricing is premium-tier.
Long context within the OpenAI ecosystem
GPT-5 offers a 200K-token context window with strong reasoning performance maintained across the window — comparable in shape to Claude's positioning but with the advantages of the broadest enterprise ecosystem in AI. For OpenAI-standardized stacks, GPT-5 is the natural long-context choice even though its window is smaller than Gemini's or Grok's. Available via OpenAI API and Azure OpenAI Service. Best for reasoning over long documents within OpenAI-standardized stacks, complex analytical work where the OpenAI ecosystem advantages dominate, and Azure-based enterprise deployments. Strengths include strong effective recall, mature ecosystem for long-context applications, broad Azure availability, and Assistants API integration for stateful long-context workflows. Trade-offs are advertised context smaller than Gemini, Grok, or Llama Scout, and pricing at the upper end of the window can be very expensive.
Open-weight long-context with cost advantage
DeepSeek V3.2 provides extended context capability with the family's characteristic cost-performance profile — making it the natural choice for cost-sensitive long-context workloads. While Llama 4 Scout pushes the open-weight context frontier, DeepSeek's mid-tier context (200K-class) combined with very low pricing is often the right trade for high-volume workloads where the absolute longest context isn't needed. Available open-weight via Hugging Face and most inference providers. Best for cost-sensitive long-context workloads, high-volume document analysis at moderate context lengths, and self-hosted long-context deployments. Strengths include open weights with MIT licensing, dramatic cost advantage, and broad inference-provider support. Trade-offs are that absolute context length trails Llama Scout and Grok, sourcing considerations apply, and effective recall benchmarks should be verified on target workloads.
Multilingual long-context option
Alibaba's Qwen 3.5 extended-context variants bring strong long-context capability to multilingual workloads, where many Western frontier models underperform on non-English long-context tasks. Available open-weight via Hugging Face and through Alibaba Cloud Model Studio. Best for multilingual long-context workloads, Asia-Pacific document analysis, and routing diversification across open-weight long-context options. Strengths include open weights, strong multilingual long-context performance, family consistency with Qwen's broader model line, and competitive cost. Trade-offs are sourcing considerations and a smaller English-language long-context community than Llama-derived models.
European long-context option
Mistral's long-context configurations provide credible long-document capability under EU jurisdiction — important for European regulated industries (financial services, healthcare, legal, public sector) that need long-context analytical capability without the data-residency questions of US or China-headquartered providers. Available via Mistral's La Plateforme, Azure, and AWS Bedrock. Best for EU-jurisdiction long-context workloads, regulated European industries, and sovereignty-sensitive long-document analysis. Strengths include EU jurisdiction, both open-weight and proprietary tiers, and clear sovereignty positioning. Trade-offs are that maximum advertised context is smaller than Gemini, Grok, or Llama Scout, and the ecosystem is smaller than US labs.
RAG-optimized long-context model
Cohere's Command R+ family with extended context support is specifically tuned for retrieval-augmented generation over long retrieved contexts — combining long-context capability with the citation-grounded behavior Cohere optimizes for. This is the right architecture for workloads where retrieval is still the primary pattern but the retrieved context is itself long. Best for enterprise RAG over long retrieved documents, citation-grounded long-document Q&A, and regulated industries needing grounded outputs over long contexts. Strengths include retrieval-optimized architecture, grounded citation behavior, and complete RAG stack from one vendor (Embed 4, Rerank 3.5, Command R+). Trade-offs are narrower general-purpose use than Llama or Gemini, and a smaller advertised maximum context.
Long-context for agentic workloads
Zhipu's GLM-5 long-context variants are particularly suited to agentic workloads with extended tool-use chains, where the model needs to maintain context across many steps of reasoning and tool interaction. Available open-weight via Hugging Face and through Z.AI's platform. Best for agentic workloads with very long tool-use chains, complex multi-step agent applications, and routing diversification across open-weight long-context options. Strengths include strong agentic-task performance maintained across long context, open-weight availability, and competitive cost. Trade-offs are a smaller ecosystem than Llama or Qwen, and sourcing considerations for some buyers.