导入
使用基准测试和指标评估生成式 AI 系统
学习如何使用基准测试和指标来评估生成式 AI 系统，这些指标可诊断质量、安全、性能和成本，并将这些信号应用于基于 AWS 架构的模型选择、优化和故障排查。
...
GenAI evaluation pipeline
Table
| Metric Category | What it Diagnoses | Example AWS Tools | Typical Exam Scenario |
|---|---|---|---|
| Relevance | Alignment with user intent | Bedrock evaluations, LLM judges | The answer is fluent but off-topic |
| Faithfulness | Grounding in source data | Bedrock Knowledge Bases | Hallucinated facts in RAG |
| Retrieval quality | Missing or incomplete context | OpenSearch, vector DB configs | Partially correct answers |
| Toxicity and bias | Unsafe or harmful output | Bedrock Guardrails | Compliance violations |
| Latency | Responsiveness | CloudWatch metrics | Slow interactive responses |
| Cost | Efficiency | Bedrock invocation logs | Excessive token usage |