导入
AI 系统的成本优化策略
探索 AWS 中的关键架构策略，以优化生成式 AI 应用中的每 token 成本，并介绍实用技术，通过高效使用 token、智能模型选择、系统优化和智能缓存，在保持响应质量的同时降低基础模型（FM）成本。
...
Token optimized request lifecycle
...
Model cascading and intelligent routing
Table
| Amazon Bedrock Batch Inference | Amazon Bedrock Provisioned Throughput | |
|---|---|---|
| Primary Use Case | Bulk processing of large datasets where real-time response is not required. | High-volume applications require predictable performance and consistent latency. |
| Performance Goal | Maximizes throughput for large-scale GenAI workloads. | Ensures guaranteed capacity and addresses specific throughput challenges. |
| Timing | Asynchronous processing (ideal for periodic or scheduled tasks). | Synchronous, real-time availability for on-demand application traffic. |
| Resource Strategy | Part of a batch inference strategy to optimize resource utilization. | Part of capacity planning to maximize resource utilization and throughput. |
| Cost Optimization | Reduces costs by grouping requests for efficient background processing. | Optimizes costs through capacity planning and resource allocation for high-traffic patterns. |
| Scaling Mechanism | Managed processing of queued records. | Dedicated model units that provide a fixed amount of processing capacity. |
Table
| Strategy | Key Mechanism | Best Use Case |
|---|---|---|
| Prompt Caching | Reuses the internal state of identical prompt prefixes | Multi-turn chat, static RAG context |
| Semantic Caching | Uses embeddings to match similar query intent | Frequently asked questions (FAQs) |
| Edge Caching | Global distribution via CDN edge locations | Public-facing apps with global traffic |
| Deterministic Hashing | Normalizes and hashes inputs for exact matches | Exact repeat queries and dev iteration |