aii.news
Builders21 Nov

Nvidia researchers say huge context windows are often unnecessary and propose cheaper alternatives

woman in green shirt sitting in front of computer
Photo: ThisisEngineering / Unsplash

Nvidia researchers found that periodic summaries and targeted keyword search usually matched full-context results at a fraction of the cost. They propose a new class of models that work with short notes instead of long context.

Why it matters

Developers may cut costs and errors by summarizing and searching rather than stuffing everything into a model's context.

aii.news picks and summarizes stories from trusted news sources with the help of AI. Every story links to the original reporting. How we work.