At the time of writing, Google's Gemini AI, introduced just two months ago, is already set to release its next-generation model, known as Gemini 1.5.
The announcement post delves into the intricacies, providing a detailed explanation of all the AI's improvements. Although quite technical, the key point is that Gemini 1.5 is set to deliver "dramatically enhanced performance." Achieving this involved adopting a "Mixture-of-Experts architecture" (MoE), where multiple AI models collaborate seamlessly. The implementation of this structure not only simplified Gemini's training process but also accelerated its ability to learn complex tasks compared to its previous performance.
The rollout of the upgrade will encompass all three major versions of the AI, with the initial release for early testing being Gemini 1.5 Pro, available today.
A noteworthy feature of this model is its "context window of up to 1 million tokens." In the realm of generative AI, tokens serve as the smallest data units used by LLMs (large language models) for processing and text generation. A larger context window enhances the AI's ability to handle more information simultaneously, and the one-million-token capacity of Gemini 1.5 Pro exceeds the capabilities of GPT-4 Turbo, which has a context window cap of 128,000 tokens, for comparison.
