Prompt cache
ProAn animated explainer for prompt caching: the shared prefix of each request is cached once, later requests read it back, and the cost and time saved add up.
Live · keyboard ready
Prompt cache
Claude Sonnet 5.55 minute cache
- Tools3.8K
- System prompt2.4K
- Documents36.2K
- Conversation6K
- New message
- Request 1: not sent yet.
- Request 2: not sent yet.
- Request 3: not sent yet.
- Request 4: not sent yet.
CacheEmpty
0 tokens cached- Input
- $2.00
- Cache write
- $2.50
- Cache read
- $0.20
Per million tokens
- A docs page or blog post that explains why a stable prefix makes repeat requests cheaper and faster.
- A pricing or usage page that shows how cache reads, writes and the time to live affect cost.
- Developer onboarding for an AI product where prompt order and cache breakpoints matter.
- Use context-window to show how much of a model's window a conversation fills and when it compacts.
- Use token-inspector to show how text splits into tokens and what each costs.
- Use model-picker or model-router when the subject is choosing a model rather than reusing a prompt.
- Use a real chart of cache hit rate over time for production analytics; this is a teaching illustration with a simulated clock.
Installation
Pro source and install commands unlock with a Pro plan.
example.tsx
import { PromptCache } from "@/components/arc/prompt-cache/prompt-cache"; export function CachingSection() { return <PromptCache segments={[ { id: "tools", label: "Tools", tokens: 3_800 }, { id: "system", label: "System prompt", tokens: 2_400 }, { id: "documents", label: "Documents", tokens: 36_200 }, { id: "conversation", label: "Conversation", tokens: 6_000 }, ]} requests={[ { message: "Which refunds are still pending this week?", tokens: 180 }, { message: "Draft a reply to Northwind about their late refund", tokens: 240, gap: 42 }, { message: "Summarize ticket 48213 and the attached invoice", tokens: 410, gap: 35 }, { message: "Which customers asked about SSO in October?", tokens: 150, gap: 51 }, ]} onRequest={result => console.log(result.kind, result.cost)} />;}PromptCache
An animated explainer for prompt caching. Four requests stack over time, each a bar of the same prefix segments (tools, system prompt, documents, conversation) plus its own new message, with token counts in the legend. The first request processes every segment in one steady sweep, then the prefix lifts out of the bar and travels into a cache panel, where a time to live ring fills. Later requests light their matching prefix as a cache hit with one quick sweep, only the new message is processed, and their processed tokens, input cost and time to first token roll down from the row above. A toggle edits one segment: the next request matches the cache only up to it, shows a break marker there, and rewrites the rest. When no requests arrive, the ring drains on an accelerated clock and the cache expires. Totals roll into a summary of input cost saved and first token speedup.
PropTypeDefaultDescription
segments{ id: string; label: string; tokens: number; color?: string }[]Tools 3.8K, System prompt 2.4K, Documents 36.2K, Conversation 6KThe stable prefix in prompt order. Prefix matching runs left to right, so put what changes least first. Colors default to the chart series tokens. Bar widths use a square root scale so a short message stays visible beside a long document; the legend shows the real counts.requests{ message?: string; tokens: number; gap?: number }[]Four support questions of 150 to 410 tokens, 35 to 51 seconds apartOne to four requests per round. tokens is the size of the new message (the uncached suffix); gap is the simulated seconds since the previous request. message is used in the accessible description and the onRequest payload.model{ name: string; mark?: ReactNode; inputPrice: number; cacheWritePrice: number; cacheReadPrice: number }Claude Sonnet 5.5: $2.00 input, $2.50 cache write, $0.20 cache read per million tokensThe model line in the header and the prices behind every cost. Prices are dollars per million input tokens. Pass your provider's current prices; the default uses Claude's pattern of reads at 0.1x and five minute writes at 1.25x the input price.ttlnumber300Seconds the cache lives after its last use. Every hit refreshes it. Shown as the ring and the countdown, and in the copy as a duration.changeSegmentstring | null"system"Id of the segment the change toggle edits. The toggle arms one edit for the next request and releases once that request is sent. Pass null to hide it.autoPlaybooleantruePlays on its own while in view. With reduced motion it opens on a finished round instead and waits for play or step.loopbooleantrueAfter a round the cache idles, expires and the story starts over. When false it stops after the cache expires.speednumber1Multiplies the pace of the clock and every animation. 0 holds still.firstToken(tokens: { processed: number; cached: number }) => number0.42 + processed / 40000 + cached / 500000Seconds to the first output token for a prompt. The default is an illustrative model: a fixed overhead, prefill of new tokens, and a much faster load of cached ones.onRequest(result: PromptCacheResult) => void–Called when a request finishes, with index, simulated time, kind (write, hit or partial), breakAt, cachedTokens, writtenTokens, processedTokens, cost, uncachedCost, firstToken, uncachedFirstToken, changed, cold and message.onExpire() => void–Called when the cache expires.messageLabelstring"New message"Legend label for the suffix at the end of each prompt.titlestring"Prompt cache"Heading of the card, also used in its accessible name.classNamestring–Extra class on the root section.styleCSSProperties–Inline styles on the root section.- Tab
- Moves through play or pause, send the next request, reset, and the change toggle.
- Enter, Space
- Plays or pauses, sends the next request, resets, or arms and disarms the edit.
- The card is a section named for the title and marked as a simulation with sample prices.
- Each request is a list item with a sentence that gives its time, message, outcome, tokens read from the cache, tokens processed, input cost and time to first token; the bars and rolling numbers are hidden from assistive technology.
- Requests sent with the step button, the edit toggle, pausing and reset are announced in a polite live region. Autoplay stays silent so it never talks over the page.
- Outcomes are never color alone: every status pairs an icon with a label (Cache write, Cache hit, Partial hit), cached and processed fills differ in fill and outline, and the cache status is written out as Empty, Writing, Reading, Expires in 4:52 or Expired.
- The change toggle is a button with aria-pressed. Icon buttons have names and tooltips.
- Pointer focus draws no ring; keyboard focus shows Arc's shared focus ring.
- One requestAnimationFrame clock drives the story and runs only while playing or stepping, on screen and in a visible tab. The ring and countdown are written straight to the DOM; React renders only on events.
- Processing is one linear sweep at a steady speed across the segments that need it. A cache hit fades the prefix to its cached fill while a single band of each segment's color passes once across the prefix, and the cache panel sweeps in step.
- A write carries the new part of the prompt from the row into the cache on a 460ms ease out, changing from processed to cached on the way; the panel's segments land in place and its token count rolls up. The ring refills on a critically damped follow and drains as simulated time passes.
- A changed segment raises a short warning line in the gap where matching stops; the cache drops everything after it before the rewrite lands.
- Numbers roll from the row above, so cost and first token visibly come down, with tabular numerals in columns sized to their content. Status labels swap with one crossfade. Pausing freezes every CSS animation and the clock together.
- Reduced motion opens on a finished round, drops the sweeps, the flight and the break animation, and changes values at once; play and step still work.
- From 660px of container width the cache panel sits beside the requests; below it the panel moves above them as a strip, so the flight still travels from the row into the cache.
- Below 480px each request takes three lines (label and status, the bar, then its three numbers under shared column headings), the stats stack, and the controls wrap under the title when needed.
- Metric columns share one subgrid, so values align across rows at every width. The card holds at 320px with no horizontal scroll.
- About 120 small elements. Fills, sweeps, the break line and the flight are CSS animations on transform and opacity; the flight also animates the size of one absolutely positioned element.
- Outcomes are computed once per request with plain arithmetic. The loop stops off screen, in hidden tabs and while paused.
Notes for AI
Give your coding assistant the Markdown reference instead of screenshots.
- Use it to explain prompt caching on a pricing page, in API docs, or inside a usage dashboard next to real cache hit numbers.
- Keep segment order realistic: content that rarely changes first (tools, system prompt), then documents, then the conversation. Prefix matching stops at the first difference, which the toggle demonstrates.
- Pass the real prices of the model you show. The defaults are Claude Sonnet 5.5 sample prices; for other providers set inputPrice, cacheWritePrice (equal to inputPrice when writes cost nothing extra) and cacheReadPrice.
- Costs, savings and speedups are computed from the props, never typed in. The first request costs more than an uncached one because of the write, and the summary says so until the hits pay it back.
- Latency numbers are illustrative unless you pass firstToken with your own measurements.
The full library index for assistants is at /llms.txt.