成本优化

这是影响最大的一项。分类、抽取、改写这类任务用旗舰模型，成本高一个数量级，效果差异却很小。

不确定时，先用小模型跑一批样本，人工看准确率是否够用。够就不用换大的。

This is the biggest lever. Using a flagship model for classification, extraction, or rewriting costs an order of magnitude more with little difference in quality.

When unsure, run a batch of samples through a small model and check whether the accuracy is good enough. If it is, don't upgrade.

输入 token 是主要成本来源之一，尤其在多轮对话里。每一轮都重发全部历史，第 10 轮的成本是第 1 轮的十倍。

• 截断过长的历史，只保留最近几轮
• 摘要压缩早期对话，用摘要替代原文
• 能放进系统提示的固定内容，不要每轮重复
• 检索增强时只取最相关的两三条，不是把所有文档都塞进去

Input tokens are a major cost source, especially in multi-turn conversations. Resending the full history every turn makes turn 10 cost ten times what turn 1 did.

• Truncate long histories, keeping only the last few turns
• Summarize earlier conversation and use the summary instead of the original text
• Put fixed content in the system prompt instead of repeating it every turn
• For retrieval-augmented generation, take only the two or three most relevant items, not every document

固定不变的长前缀，重复请求时能命中缓存，输入侧按折扣计价。

• OpenAI 结构自动命中，只要前缀一致
• Claude 需要用 cache_control 显式标记
• Gemini 有隐式与显式两种，长文档适合显式

把固定内容放在最前面，变化的内容放在最后，命中率才高。

A fixed, long prefix that repeats across requests can hit the cache, and the input side is billed at a discount.

• OpenAI-structured requests hit automatically as long as the prefix matches
• Claude requires explicit marking with cache_control
• Gemini offers both implicit and explicit caching; explicit suits long documents

Put fixed content first and variable content last for the best hit rate.

不设 max_tokens 时，偶发的超长输出会显著推高成本。给每类任务设一个合理上限，比如摘要类设 500，问答类设 1000。

Without max_tokens, an occasional overlong response pushes costs up noticeably. Set a sensible ceiling per task type — 500 for summaries, 1000 for Q&A, for example.

并发过高触发 429，重试又产生额外请求。用信号量把并发压在 5 到 10，配合指数退避重试。

Too much concurrency triggers 429s, and retries generate extra requests. Use a semaphore to keep concurrency between 5 and 10, combined with exponential backoff.

视频生成最贵。多数视频模型支持草稿模式：先出 480p 样片确认构图，满意后再出正式片。这样一次失败不会烧掉正式片的钱。

图像也类似，先用低分辨率档位试提示词，效果对了再出高分辨率。

Video generation is the most expensive. Most video models support a draft mode: render a 480p sample to confirm the composition, then render the final version. That way one failure doesn't burn the cost of a finished video.

Images are similar: iterate on the prompt at a low resolution tier, then render at high resolution once it looks right.

图像、视频、音频这类异步任务失败时会自动退款。所以参数校验错误并不可怕，可怕的是「提交成功但结果不可用」。

提交前用审核接口过一遍提示词，能减少无效任务。

Async tasks like image, video, and audio are refunded automatically when they fail. So a parameter validation error isn't the scary part — "submitted successfully but the result is unusable" is.

Run prompts through the moderation endpoint before submitting to cut down on wasted tasks.

缓存有有效期。把同类请求集中在一个时间窗内跑完，命中率最高。分散在一整天里跑，每次都要重新建立缓存。

Caches have a lifetime. Running similar requests together within one time window gives the highest hit rate. Spread across a whole day, each one has to rebuild the cache.

定期用用量接口按模型聚合，看哪类请求花得最多：

curl 'https://api.oairoute.com/v1/dashboard/billing/usage?start_date=2026-10-01&end_date=2026-10-07&group_by=model' \
  -H "Authorization: Bearer YOUR_OAIRoute_KEY"

通常会发现 80% 的花费集中在 20% 的请求上。找到那部分，优化才有意义。

Periodically aggregate usage by model with the usage endpoint to see which requests cost the most:

curl 'https://api.oairoute.com/v1/dashboard/billing/usage?start_date=2026-10-01&end_date=2026-10-07&group_by=model' \
  -H "Authorization: Bearer YOUR_OAIRoute_KEY"

You'll usually find that 80% of the spend comes from 20% of the requests. Finding that 20% is what makes optimization worthwhile.