The ratio is what matters

Every model page in the ModelSite catalog lists the cache-read price next to the input price. The ratio between the two decides what caching is worth. If cache reads cost a few percent of the input price, a stable prefix saves far more than on a model priced at a quarter of it, so look at that ratio instead of the headline input price.

The table below is calculated live from those same prices. It compares the input cost of 1,000 requests that each carry a 100,000-token prompt: once with no cache hits, once with 90% of the prompt served from cache. Output tokens are left out, because caching only changes the input side.

Input cost of 1,000 requests × 100,000-token prompt each, at a 90% cache-hit rate, calculated from live catalog prices on 2026-11-12.

Prices move, so treat any screenshot as dated. The model pages — for example deepseek-v4.1-flash-1017 and glm-5.3 — always show the current numbers.

How to structure a prompt for cache hits

• Put the stable parts first: the system prompt, tool definitions and reference material, before anything that varies.
• Put the volatile parts last: the latest user message, timestamps, request IDs. A timestamp near the top invalidates the whole prefix on every call.
• Keep the prefix byte-identical between calls. Providers generally match on an exact prefix, so even reordering the tool list can cost you the hit.
• Send follow-up requests soon after the first. Caches expire, and the lifetime depends on the provider.
• Stay on one model for a conversation. A cache belongs to a single model.

How to check that it is working

The response reports what was served from cache. In the OpenAI response format the field is usage.prompt_tokens_details.cached_tokens; Anthropic's format uses usage.cache_read_input_tokens; DeepSeek adds prompt_cache_hit_tokens, which passes through unchanged. If the counter stays at zero on a prefix you deliberately repeat, something in that prefix changes between calls.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["MODELSITE_API_KEY"],
    base_url="https://api.modelsite.ai/v1",
)

messages = [
    {"role": "system", "content": LONG_STABLE_PREFIX},  # identical every call
    {"role": "user", "content": "Summarize the third reference document."},  # varies every call
]
resp = client.chat.completions.create(model="glm-5.3", messages=messages)

details = resp.usage.prompt_tokens_details
cached = details.cached_tokens if details else 0
print(f"{cached} of {resp.usage.prompt_tokens} prompt tokens came from cache")

Run it twice with the same long prompt; on most providers the second call reports most of the prefix as cached. DeepSeek builds its disk cache in the background, so the hit can take another call to show up. The example uses the official OpenAI Python SDK against an OpenAI-compatible endpoint, so the same check works on any OpenAI-compatible provider that reports cache hits. Provider fine print, including cache expiry, lives in the docs: OpenAI's prompt caching guide, Anthropic's prompt caching page, and DeepSeek's context caching notes. ModelSite's Anthropic-compatible endpoint reports the same cache_read_input_tokens field.

Worth knowing

• Cached tokens are billed at the cache-read rate instead of the input rate, not on top of it.
• Not every model reports cache hits, and a few price cache writes separately. The model page shows the full breakdown.
• Prefixes below a minimum length never cache at all. The threshold is provider- and model-specific: 1,024 tokens on OpenAI (GPT-5.6 and later; earlier models vary by request settings) and 512 to 4,096 tokens on Anthropic depending on the model. A short test prefix reports zero by design.
• Cache prices and off-peak windows change. The table above was calculated from the live catalog on 2026-11-12; treat it as a snapshot, not the current price.

Originally published on the ModelSite blog on 2026-10-22.

One endpoint, one key

ModelSite exposes an OpenAI-compatible endpoint at https://api.modelsite.ai/v1. You authenticate with a single key (it starts with sk-ms-) in the Authorization: Bearer header and choose the model with the model field. Nothing else about the request changes.

Try it with curl

curl https://api.modelsite.ai/v1/chat/completions \
  -H "Authorization: Bearer $MODELSITE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4.1-flash-1017",
    "messages": [{"role": "user", "content": "Explain TCP slow start in two sentences."}]
  }'

Switch models by changing one string

With the official OpenAI Python SDK, point base_url at ModelSite and loop over model names. The request and response format is the same for every model, so parsing and error handling stay as they are.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["MODELSITE_API_KEY"],
    base_url="https://api.modelsite.ai/v1",
)

for model in ["deepseek-v4.1-flash-1017", "qwen3.7-max", "glm-5.3-flash"]:
    resp = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": "Explain TCP slow start in two sentences."}],
    )
    print(model, "->", resp.choices[0].message.content)

What each of these costs

Prices are per million tokens and change over time, so we don't paste them here. Open a model page for the live price, context length, cache pricing and time-window discounts:

• deepseek-v4.1-flash-1017
• qwen3.7-max
• glm-5.3-flash

Already using the Anthropic SDK?

ModelSite also exposes an Anthropic-compatible endpoint (same keys, different base URL). The SDK guide lists the base URL and the authentication difference.

Calling each vendor directly

Prefer calling each vendor directly? Their docs are the source of truth: DeepSeek, Qwen via Alibaba Cloud, GLM by Zhipu. You trade one key for three, which is the whole trade-off this article is about. For the client side, the OpenAI Python SDK documents base_url and the other options used above.

Originally published on the ModelSite blog, 2026-10-22.