Context windows size matters!

Whew! What a busy week it’s been, filled with meetings with fantastic people and companies.

There’s a lot of AI news this week. While the new text-to-video models look great, I want to highlight the news about Gemini 1.5 Pro and its extraordinary context window size. This is a game changer.

In our lab, we frequently encountered the token context window and had to address it through software adjustments or by simply changing the use case we had (sigh!). Our main challenge revolved around dealing with GPT-3 and its 2000-token limit.

Gemini introduces 1.5 Pro with a standard 128,000 tokens, similar to GPT-4 Turbo, and a promise of 1 million tokens!

But what does all this token talk really mean? Well, as per Google’s example:

“This means 1.5 Pro can process vast amounts of information in one go — including 1 hour of video, 11 hours of audio, codebases with over 30,000 lines of code, or over 700,000 words.” (link in comments)

In more clear terms, it’s the maximum amount of text the model can consider at any one time when generating a response. Imagine inputting an entire book or movie and letting the LLM work its magic.

More here: https://blog.google/technology/ai/google-gemini-next-generation-model-february-2024/

Picture: Me with… windows!