Introducing Prompt Caching on SambaCloud: Faster, Cheaper Inference for MiniMax-M2.7

We're excited to announce prompt caching is now live on SambaCloud, starting with support for MiniMax M2.7. If your application sends repeated requests that share a long, stable prefix, such as a system prompt, a reference document, or a set of few-shot examples, prompt caching can meaningfully cut both your latency and your bill, with zero changes required on your end.

TL;DR

• Prompt caching is now live on SambaCloud, starting with MiniMax M2.7, so applications that resend a long, stable prefix (a system prompt, reference document, or set of few-shot examples) get faster, cheaper inference with zero code changes.

• It runs on Automatic Prefix Caching (APC): when a new request shares a common prefix with one recently processed, those tokens are served from cache instead of recomputed, and it turns on automatically once the prefix clears a minimum length threshold of 4,096 tokens.

• Cached tokens are billed 90% below standard input pricing, which for MiniMax M2.7 means $0.06 per million tokens instead of $0.60 on any part of the prompt that hits the cache.

• Caching keeps time-to-first-token (TTFT) nearly flat as context grows, cutting TTFT by 33% at the short end and 91% at the long end for a median 6.1x speedup, and dropping TTFT from 40.6 seconds to 3.7 seconds at 192k tokens.

• Every response now returns a prompt_tokens_details object with cached_tokens and cache_creation_tokens, so you can see exactly how many tokens were served from cache and billed at the discounted rate.

The Problem: Paying Full Price for the Same Tokens, Over and Over

Many real-world LLM workloads don't look like one-off questions. They look like the same long context, asked again and again:

• A coding assistant that resends the same large system prompt on every turn
• A support bot that references the same product documentation for every user query
• A financial analyst tool that loads the same quarterly earnings report and answers a stream of different questions about it

In all of these cases, the model ends up reprocessing the exact same tokens from scratch on every single request, burning compute and adding latency for work it's effectively already done.

The Solution: Automatic Prefix Caching

Prompt caching on SambaCloud is powered by Automatic Prefix Caching (APC). When a new request shares a common prefix with one we've recently processed, we serve those tokens directly from cache instead of recomputing them.

The best part: It's automatic. There's no new parameter to set, no flag to flip, and no special client configuration. As long as your requests share a stable, unchanged prefix above a minimum length threshold, caching kicks in on its own.

To get the most out of it:

• Put your stable content first (system prompts, documents, examples) and keep it identical across requests.
• Avoid changing anything in that prefix; even a small edit invalidates the cache for that prefix.
• Make sure the prefix is long enough to clear the minimum caching threshold. [4096]

What Prompt Caching Means for Latency

Prompt caching changes how your app feels to use, not just what it costs. With a cached prefix, TTFT stays nearly flat as context grows, instead of climbing steeply.

Across 8k to 192k tokens of context, caching cuts TTFT by 33% at the short-end and 91% at the long-end, a median speedup of 6.1×. At 192k tokens, TTFT drops from 40.6 seconds to 3.7 seconds.

The longer your prompt prefix, the more caching pays off, in both dollars and wait time.

Figure 1: Prompt caching vs. no caching — TTFT by context length

What Prompt Caching Means for Your Bill

Prompt caching on SambaCloud cuts your cost on every cached token by 90% compared to standard input pricing. For MiniMax M2.7, that's the difference between paying $0.60 and $0.06 per million tokens on any part of your prompt that hits the cache.

For workloads that reuse a long, stable prefix, a system prompt, a knowledge base, a set of few-shot examples, this adds up fast. The more of your traffic reuses that prefix, and the longer it is, the bigger the share of your bill that drops to the cached rate instead of the standard one.

See Exactly What You're Saving

{
  "usage": {
    "prompt_tokens": 5797,
    "completion_tokens": 100,
    "total_tokens": 5897,
    "prompt_tokens_details": {
      "cache_creation_tokens": 0,
      "cached_tokens": 4096
    }
  }
}

cached_tokens tells you how many tokens were served from cache (and billed at the discounted cached rate), while cache_creation_tokens shows how many were just written to cache on this request. The difference between prompt_tokens and cached_tokens is what was actually processed fresh.

Good to Know

Prompt caching is rolling out with a few current limitations:

• Available on MiniMax M2.7 only for now. Other models will return cached_tokens: 0.
• Maximum cacheable prefix length is 192,000 tokens.
• Cache entries are evicted automatically based on traffic patterns. The time to live of a particular prefix can vary based on traffic patterns.

Get Started

If your workload sends the same long context repeatedly, prompt caching is already working for you. Just check prompt_tokens_details in your next response to see it in action. For full details, see our prompt caching documentation.

We're working on expanding support to more models and improving cache hit rates across the board. Stay tuned.