Introducing Gemini Diffusion

Our state-of-the-art, experimental text diffusion model

output: Gemini Diffusion is an experimental text diffusion model from Google AI, combining the strengths of large-language models with a new generation technique called diffusion.

Traditional language models generate text one token at a time using autoregressive methods. However, in tasks like editing, inserting content in the context of math or code, and merging different ideas, a more flexible and powerful generation paradigm is needed.

Diffusion models offer an alternative approach. Instead of learning to predict text directly, they excel at refining noise to generate outputs step-by-step. This property is leveraged in Gemini Diffusion for efficient generation.

Capabilities

Rapid response

Generates content significantly faster than even our fastest model so far.

More coherent text

Generates entire blocks of tokens at once, meaning it responds more coherently to a user’s prompt than autoregressive models.

Iterative refinement

Corrects errors during generation for more consistent outputs.

Benchmarks

Gemini Diffusion’s external benchmark performance is comparable to much larger models, whilst also being faster.

Methodology

All scores are pass @1 (no majority voting). The Gemini 2.0 Flash-Lite experiments are run with the AI Studio API for the model-id gemini-2.0-flash-lite with the default sampling settings.

* Non-agentic evaluation (single turn edit only), max prompt length of 32K.

Gemini Diffusion speed

Average sampling speed across reported evals.

Try Gemini Diffusion

Gemini Diffusion is currently available as an experimental demo to help develop and refine future models. If you're interested in getting access to the demo, please sign up to the waitlist.