121 lines
2.4 KiB
Markdown
121 lines
2.4 KiB
Markdown
|
# Eval 指令
|
|||
|
|
|
|||
|
|
管理評估驅動開發 (Eval-driven development) 工作流。
|
|||
|
|
|
|||
|
|
## 使用方式
|
|||
|
|
|
|||
|
|
`/eval [define|check|report|list] [feature-name]`
|
|||
|
|
|
|||
|
|
## 定義評估 (Define Evals)
|
|||
|
|
|
|||
|
|
`/eval define feature-name`
|
|||
|
|
|
|||
|
|
建立新的評估定義：
|
|||
|
|
|
|||
|
|
1. 使用模板建立 `.claude/evals/feature-name.md`：
|
|||
|
|
|
|||
|
|
```markdown
|
|||
|
|
## EVAL: feature-name
|
|||
|
|
建立日期：$(date)
|
|||
|
|
|
|||
|
|
### 能力評估 (Capability Evals)
|
|||
|
|
- [ ] [能力 1 的描述]
|
|||
|
|
- [ ] [能力 2 的描述]
|
|||
|
|
|
|||
|
|
### 迴歸評估 (Regression Evals)
|
|||
|
|
- [ ] [現有行為 1 仍正常運作]
|
|||
|
|
- [ ] [現有行為 2 仍正常運作]
|
|||
|
|
|
|||
|
|
### 成功標準
|
|||
|
|
- 能力評估：pass@3 > 90%
|
|||
|
|
- 迴歸評估：pass^3 = 100%
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
2. 提示使用者填入具體標準
|
|||
|
|
|
|||
|
|
## 檢查評估 (Check Evals)
|
|||
|
|
|
|||
|
|
`/eval check feature-name`
|
|||
|
|
|
|||
|
|
執行功能的評估：
|
|||
|
|
|
|||
|
|
1. 從 `.claude/evals/feature-name.md` 讀取評估定義
|
|||
|
|
2. 針對每項能力評估：
|
|||
|
|
- 嘗試驗證標準
|
|||
|
|
- 記錄 通過/失敗 (PASS/FAIL)
|
|||
|
|
- 在 `.claude/evals/feature-name.log` 中記錄嘗試
|
|||
|
|
3. 針對每項迴歸評估：
|
|||
|
|
- 執行相關測試
|
|||
|
|
- 與基準值進行比較
|
|||
|
|
- 記錄 通過/失敗 (PASS/FAIL)
|
|||
|
|
4. 回報目前狀態：
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
EVAL 檢查：feature-name
|
|||
|
|
========================
|
|||
|
|
能力：X/Y 通過
|
|||
|
|
迴歸：X/Y 通過
|
|||
|
|
狀態：進行中 / 已就緒 (IN PROGRESS / READY)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 評估報告 (Report Evals)
|
|||
|
|
|
|||
|
|
`/eval report feature-name`
|
|||
|
|
|
|||
|
|
產生完整的評估報告：
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
EVAL 報告：feature-name
|
|||
|
|
=========================
|
|||
|
|
產生日期：$(date)
|
|||
|
|
|
|||
|
|
能力評估 (CAPABILITY EVALS)
|
|||
|
|
----------------
|
|||
|
|
[eval-1]: PASS (pass@1)
|
|||
|
|
[eval-2]: PASS (pass@2) - 需重試
|
|||
|
|
[eval-3]: FAIL - 見備註
|
|||
|
|
|
|||
|
|
迴歸評估 (REGRESSION EVALS)
|
|||
|
|
----------------
|
|||
|
|
[test-1]: PASS
|
|||
|
|
[test-2]: PASS
|
|||
|
|
[test-3]: PASS
|
|||
|
|
|
|||
|
|
度量指標 (METRICS)
|
|||
|
|
-------
|
|||
|
|
能力 pass@1: 67%
|
|||
|
|
能力 pass@3: 100%
|
|||
|
|
迴歸 pass^3: 100%
|
|||
|
|
|
|||
|
|
備註 (NOTES)
|
|||
|
|
-----
|
|||
|
|
[任何問題、邊緣情況或觀察結果]
|
|||
|
|
|
|||
|
|
建議 (RECOMMENDATION)
|
|||
|
|
--------------
|
|||
|
|
[可發布 / 需改進 / 已阻斷 (SHIP / NEEDS WORK / BLOCKED)]
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 列出評估 (List Evals)
|
|||
|
|
|
|||
|
|
`/eval list`
|
|||
|
|
|
|||
|
|
顯示所有評估定義：
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
評估定義 (EVAL DEFINITIONS)
|
|||
|
|
================
|
|||
|
|
feature-auth [3/5 通過] 進行中
|
|||
|
|
feature-search [5/5 通過] 已就緒
|
|||
|
|
feature-export [0/4 通過] 未開始
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 參數說明 (Arguments)
|
|||
|
|
|
|||
|
|
$ARGUMENTS：
|
|||
|
|
- `define <name>` - 建立新的評估定義
|
|||
|
|
- `check <name>` - 執行並檢查評估
|
|||
|
|
- `report <name>` - 產生完整報告
|
|||
|
|
- `list` - 顯示所有評估
|
|||
|
|
- `clean` - 移除舊的評估日誌 (保留最近 10 次執行)
|