We pioneered context caching in Might of 2024, serving to builders save 75% on repetitive context handed to our fashions with express caching. At present, we’re rolling out the extremely requested characteristic within the Gemini API: implicit caching.
Implicit caching with Gemini API
Implicit caching straight passes cache price financial savings to builders with out the necessity to create an express cache. Now, if you ship a request to one of many Gemini 2.5 fashions, if the request shares a standard prefix as certainly one of earlier requests, then it’s eligible for a cache hit. We are going to dynamically cross price financial savings again to you, offering the identical 75% token low cost.
So as to improve the possibility that your request incorporates a cache hit, you must maintain the content material at the start of the request the identical and add issues like a consumer’s query or different extra context that may change from request to request on the finish of the immediate. You’ll be able to learn extra greatest practices on utilizing implicit caching within the Gemini API docs.
To make extra requests eligible for cache hits, we diminished the minimal request dimension for two.5 Flash to 1024 tokens and a couple of.5 Professional to 2048 tokens.
Understanding token reductions with Gemini 2.5
In instances the place you need to assure price financial savings, you may nonetheless use our express caching API, which helps our Gemini 2.5 and a couple of.0 fashions. In case you are utilizing Gemini 2.5 fashions proper now, you’ll begin to see cached_content_token_count within the utilization metadata which signifies what number of tokens within the request have been cached and subsequently shall be charged on the cheaper price.
Get began
We’re excited to proceed to push the pareto frontier with much more cost-efficiency and stay up for your suggestions on our caching updates!
