template-monorepo/docs/e2e-testing.md

425 lines
18 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Gateway E2E 測試指南
兩種測試風格並行build tag: `e2e`code 在 `test/e2e/`
| 風格 | 用途 | 對應指令 |
|------|------|----------|
| **Contract tests** | 單一 endpoint 驗 HTTP contract請求 → 回應),可平行 | `make e2e-full` |
| **Journeys**k6 風格) | 多步驟 user flow**共享狀態**、**任一步 fail 自動 skip 後續** | `make e2e-journey` |
---
## 我現在有哪些測試?
```bash
make e2e-list
```
會分兩節印出來:
```
═══ Contract testsmake e2e-full═══
── Member ──
[M-01 ] GET /api/v1/members/me 讀 profiletenant/uid/status
[M-03/M-04] POST /me/verifications/email/{start,...} 業務 email OTP 申請 → 驗證
── Permission ──
[P-03~P-06] * /api/v1/permissions/roles 租戶角色 CRUD
═══ Journeysmake e2e-journey═══
[J-1] Tenant Owner 入職第一天(已登入後完整 onboarding (12 steps)
▶ [J-1.1] GET /me — 用 seed JWT 確認自己是 tenant_owner 且 status=active
▶ [J-1.2] PATCH /me — 更新 display_name
▶ [J-1.3] POST /me/verifications/email/start — 申請業務 email OTP
...
[J-2] Tenant Admin 從零建立 qa_engineer 角色 → 指派 → 驗證 → 撤銷 (8 steps)
[J-3] Session 生命週期refresh → /me → logout → 舊 token 401 (4 steps)
[J-4] 完整註冊 → 登入 → 看自己(需 ZITADEL目前 stub (5 steps)
⊘ [J-4.1] ...
```
> 新增 contract test → 在 `_test.go` func 開頭加 `e2eStep(t, "ID", "METHOD", "path", "中文")`
> 新增 journey → 在 `journey_xxx_test.go` 用 `NewJourney(t, "J-x", "title")` + `j.Step("id", "desc", fn)``make e2e-list` 都會自動撈到
---
## 一鍵完整測試(推薦)
**全新 Docker volume** 開始,依序:起 Mongo/Redis → 建 index → seed 資料 → 起 Gateway → 跑 E2E → **關閉並刪除 volume**
```bash
cd gateway
make e2e-full
```
等同於:
```bash
bash scripts/e2e-run.sh
```
執行時看到的順序(每個 step 都會印 banner
```
== [1/6] fresh docker composemongo + redis==
== [2/6] wait for healthcheck ==
✔ mongo / redis healthy4s
== [3/6] 建立 Mongo 索引cmd/mongo-index==
== [4/6] seed tenant + member + permission + JWTcmd/e2e-seed==
== [5/6] 啟動 Gateway:18888==
E2E 環境服務
MongoDB 127.0.0.1:27017 database=gateway_e2e
Redis 127.0.0.1:6379 OTP / Casbin policy / blacklist
Gateway http://127.0.0.1:18888 health: /api/v1/health
== [6/6] 跑 E2E每個測試會印 ▶ [ID] METHOD path — 中文情境)==
=== RUN TestMember_GetMe
member_test.go:19: ▶ [M-01] GET /api/v1/members/me — 讀 profiletenant/uid/status
--- PASS: TestMember_GetMe (0.05s)
=== RUN TestMember_EmailVerification_FullFlow
member_test.go:46: ▶ [M-03/M-04] POST /me/verifications/email/{start,confirm} — 業務 email OTP 申請 → 從 Redis 取碼 → 驗證 → email_verified=true
--- PASS: TestMember_EmailVerification_FullFlow (0.21s)
...
✔ E2E OK
```
成功時最後一行:`✔ E2E OK`。失敗會在 banner 之後直接顯示 testify 的 diff能立刻對應到「哪個 ID 哪個情境壞掉」。
### 流程圖
```mermaid
flowchart LR
A[docker compose down -v] --> B[up mongo + redis]
B --> C[cmd/mongo-index]
C --> D[cmd/e2e-seed]
D --> E[gateway :18888]
E --> F[go test -tags=e2e]
F --> G[stop gateway + down -v]
```
### 其他指令
| 指令 | 用途 |
|------|------|
| `make e2e-list` | 列出所有 contract tests + journeys |
| `make e2e-full` | 全新 docker → seed → 跑 contract tests → 關閉 |
| `make e2e-journey` | 全新 docker → seed → 跑 journeysk6 風格) → 關閉 |
| `make e2e-up` | 起環境 + seed + Gateway**不跑測試**(本機除錯) |
| `make test-e2e` | 對已啟動的 Gateway 只跑 contract tests |
| `make test-e2e-journey` | 對已啟動的 Gateway 只跑 journeys |
| `make e2e-casbin` | 以 `Permission.Casbin.Enabled: true` 跑 RBAC reload / deny E2E |
| `make e2e-down` | 停 Gateway + `docker compose down -v`(含 mailhog |
| `E2E_KEEP_DOCKER=1 make e2e-full` | 測完**保留** Docker方便查 Mongo/Redis |
| `E2E_WITH_SMTP=1 make e2e-full` | 額外啟動 MailHoghttp://localhost:8025 |
### E2E 專用設定
| 檔案 | 說明 |
|------|------|
| `test/e2e/fixtures/e2e.yaml` | Port **18888**、DB **gateway_e2e**、Notification mock、Casbin **關閉** |
| `test/e2e/fixtures/e2e.casbin.yaml` | 同上,但 Casbin **開啟**,搭配 `make e2e-casbin` |
| `test/e2e/fixtures/state.json` | seed 產生的 tenant / uid / JWTgitignore執行後生成 |
---
## Journeysk6 風格 user flow
> 「我查看 /me 要先登入;要登入要先註冊」這種**狀態依賴**的測試。每個 journey
> 是一條時間線上一步的結果token / challenge_id / role_id餵下一步**任何
> 一步 fail 都會自動 skip 後續**,避免被噪音 fail 蓋掉真正斷點。
### 目前有的 journeys
| ID | Journey | Steps | Test func | 備註 |
|----|---------|------:|-----------|------|
| **J-1** | Tenant Owner 入職第一天 | 12 | `TestJourney_OwnerOnboarding` | /me → PATCH → email verify → phone verify → TOTP 全鏈路 |
| **J-2** | Tenant Admin 從零建立 qa_engineer 角色 → 指派 → 驗證 → 撤銷 | 8 | `TestJourney_TenantAdminCustomRole` | 含「no-role user 拿到新角色」二人視角驗證 |
| **J-3** | Session 生命週期refresh → /me → logout → 舊 token 401 | 4 | `TestZZZJourney_SessionLifecycle` | 會撤銷 JWT故拆第二輪跑 |
| **J-4** | 完整註冊 → 登入 → 看自己 | 5 (skip) | `TestJourney_FullRegistration` | **需 ZITADEL**,目前 stub接 container 後改 `j.Step()` 就能跑 |
### 執行範例
```
=== RUN TestJourney_OwnerOnboarding
journey_owner_test.go:20: ▶ [J-1] Tenant Owner 入職第一天(已登入後完整 onboarding
=== RUN TestJourney_OwnerOnboarding/J-1.1_GET_/me_...
journey.go:54: ▶ [J-1.1] GET /me — 用 seed JWT 確認自己是 tenant_owner 且 status=active
--- PASS: TestJourney_OwnerOnboarding/J-1.1_... (0.00s)
...
journey.go:85: ✔ [J-1] Tenant Owner 入職第一天(已登入後完整 onboarding — 12/12 steps executed
--- PASS: TestJourney_OwnerOnboarding (0.57s)
```
失敗時:
```
=== RUN TestJourney_OwnerOnboarding/J-1.4_POST_..._confirm_...
journey.go:54: ▶ [J-1.4] POST /me/verifications/email/confirm — 從 Redis 取碼後驗證
require.go:223:
Error Trace: ...
Error: Not equal: expected 102000, got 29202005
journey.go:65: ✗ [J-1.4] FAIL — aborting remaining steps
--- FAIL: TestJourney_OwnerOnboarding/J-1.4_... (0.10s)
=== RUN TestJourney_OwnerOnboarding/J-1.5_GET_/me_...
journey.go:50: ⊘ [J-1.5] skipped — journey aborted at an earlier step
--- SKIP: TestJourney_OwnerOnboarding/J-1.5_... (0.00s)
[J-1.6 ... J-1.12 全 SKIP]
journey.go:85: ✗ [J-1] ... — 4/12 steps executed
```
### 寫新 journey
```go
func TestJourney_PaymentFlow(t *testing.T) {
j := NewJourney(t, "J-5", "下單 → 付款 → 收據")
defer j.Summary()
c := NewClient(t)
var orderID, paymentID string
j.Step("1", "POST /orders — 建立訂單", func(t *testing.T) {
env := c.DoExpectOK(t, "POST", "/api/v1/orders", body, true)
orderID = parseOrderID(t, env.Data)
require.NotEmpty(t, orderID)
})
j.Step("2", "POST /orders/:id/pay — 付款", func(t *testing.T) {
require.NotEmpty(t, orderID)
env := c.DoExpectOK(t, "POST", "/api/v1/orders/"+orderID+"/pay", nil, true)
paymentID = parsePaymentID(t, env.Data)
})
j.Step("3", "GET /payments/:id/receipt — 確認收據", func(t *testing.T) {
require.NotEmpty(t, paymentID)
c.DoExpectOK(t, "GET", "/api/v1/payments/"+paymentID+"/receipt", nil, true)
})
// 需要外部服務的步驟用 SkipStepjourney 不算 fail
j.SkipStep("4", "POST 對接金流 webhook", "需 mock 金流 sandbox")
}
```
`NewJourney` / `j.Step``test/e2e/journey.go`;不要呼叫 `t.Parallel()`journey 需要時間序。
---
## 測試覆蓋矩陣contract tests一目瞭然
圖例:**✅ 自動 E2E** · **⏭ 需 ZITADEL / 手動** · **🔧 基礎設施**
### 基礎設施
| ID | 情境 | 自動 | 測試檔 |
|----|------|:----:|--------|
| INF-01 | Mongo + Redis docker healthy | ✅ | `scripts/e2e-run.sh` |
| INF-02 | 全模組 Mongo index | ✅ | `cmd/mongo-index` |
| INF-03 | E2E tenant + member + permission + JWT seed | ✅ | `cmd/e2e-seed` |
| INF-04 | Gateway 監聽 :18888 | ✅ | `scripts/e2e-run.sh` |
### Normal
| ID | Method | Path | 情境 | 自動 | 測試 |
|----|--------|------|------|:----:|------|
| N-01 | GET | `/api/v1/health` | Ping 200 | ✅ | `TestHealth_Ping` |
| N-02 | GET | `/api/v1/health` | 無需 Bearer | ✅ | `TestHealth_NoAuthRequired` |
### Auth`/api/v1/auth`
| ID | Method | Path | 情境 | 自動 | 測試 / 備註 |
|----|--------|------|------|:----:|-------------|
| A-01 | POST | `/register` | Email 註冊 | ⏭ | 需 **ZITADEL** + invite |
| A-02 | POST | `/register/confirm` | OTP 確認 | ⏭ | 同上 |
| A-03 | POST | `/register/resend` | 重發 OTP | ⏭ | 同上 |
| A-04 | POST | `/register/social/start` | 社交註冊 | ⏭ | 需 ZITADEL OAuth |
| A-05 | GET | `/register/social/callback` | 社交註冊 callback | ⏭ | 需 ZITADEL |
| A-06 | POST | `/login` | 密碼登入 | ⏭ | 需 ZITADEL ROPG |
| A-07 | POST | `/login/social/start` | 社交登入 | ⏭ | 需 ZITADEL |
| A-08 | GET | `/login/social/callback` | 社交登入 callback | ⏭ | 需 ZITADEL |
| A-09 | POST | `/token/exchange` | id_token 換 JWT | ⏭ | 需 ZITADEL |
| A-10 | POST | `/token/refresh` | 刷新 token | ✅ | `TestZZZ_AuthTokenRefreshAndLogout`(最後跑) |
| A-11 | POST | `/logout` | 登出黑名單 jti | ✅ | 同上(同一測試內連續驗證) |
| A-12 | GET | `/members/me`(無 Bearer | 401 | ✅ | `TestAuth_MissingBearer_401` |
| A-13 | POST | `/register`、`/login`、`/token/refresh`、`/login/social/start` | 公開 Auth validation 400 | ✅ | `TestAuth_PublicValidationErrors`(不需 ZITADEL |
> E2E 透過 `cmd/e2e-seed` 直接核發 JWT不走 ZITADEL因此 A-01A-09 列為手動staging 測試。
### Member`/api/v1/members`,需 Bearer
| ID | Method | Path | 情境 | 自動 | 測試 |
|----|--------|------|------|:----:|------|
| M-01 | GET | `/me` | 讀 profile | ✅ | `TestMember_GetMe` |
| M-02 | PATCH | `/me` | 更新 display_name | ✅ | `TestMember_UpdateMe` |
| M-03 | POST | `/me/verifications/email/start` | 發起 email OTP | ✅ | `TestMember_EmailVerification_FullFlow` |
| M-04 | POST | `/me/verifications/email/confirm` | 確認 email OTP | ✅ | 同上(`GATEWAY_E2E=1` 從 Redis 取碼) |
| M-05 | POST | `/me/verifications/phone/start` | 發起 phone OTP | ✅ | `TestMember_PhoneVerification_FullFlow` |
| M-06 | POST | `/me/verifications/phone/confirm` | 確認 phone OTP | ✅ | 同上(`GATEWAY_E2E=1` 從 Redis 取碼) |
| M-07 | GET | `/me/totp` | TOTP 狀態 | ✅ | `TestMember_TOTP_Status` |
| M-08 | POST | `/me/totp/enroll-start` | 開始綁定 | ✅ | `TestMember_TOTP_FullFlow`(解析 `otpauth_url` |
| M-09 | POST | `/me/totp/enroll-confirm` | 確認綁定 | ✅ | 同上 |
| M-10 | POST | `/me/totp/verify` | Step-up 驗碼 + replay 防護 | ✅ | 同上 |
| M-11 | DELETE | `/me/totp` | 解除綁定 | ✅ | 同上 |
| M-12 | POST | `/me/totp/backup-codes` | 重產備援碼 | ✅ | 同上 |
### Permission`/api/v1/permissions`,需 Bearer
| ID | Method | Path | Middleware | 情境 | 自動 | 測試 |
|----|--------|------|------------|------|:----:|------|
| P-01 | GET | `/catalog` | AuthJWT | 權限樹 | ✅ | `TestPermission_Catalog` |
| P-02 | GET | `/me` | AuthJWT | 當前 user 權限 | ✅ | `TestPermission_Me` |
| P-03 | GET | `/roles` | AuthJWT+Casbin* | 列角色 | ✅ | `TestPermission_RoleCRUD` |
| P-04 | POST | `/roles` | AuthJWT+Casbin* | 建角色 | ✅ | 同上 |
| P-05 | PATCH | `/roles/:id` | AuthJWT+Casbin* | 更新角色 | ✅ | 同上 |
| P-06 | DELETE | `/roles/:id` | AuthJWT+Casbin* | 刪角色 | ✅ | 同上 |
| P-07 | GET | `/roles/:id/permissions` | AuthJWT+Casbin* | 讀角色權限 | ✅ | `TestPermission_RolePermissions` |
| P-08 | PUT | `/roles/:id/permissions` | AuthJWT+Casbin* | 取代角色權限 | ✅ | 同上 |
| P-09 | GET | `/users/:uid/roles` | AuthJWT+Casbin* | 列 user 角色 | ✅ | `TestPermission_AssignUserRole` |
| P-10 | POST | `/users/:uid/roles` | AuthJWT+Casbin* | 指派角色 | ✅ | 同上 |
| P-11 | DELETE | `/users/:uid/roles/:role_id` | AuthJWT+Casbin* | 撤銷角色 | ✅ | 同上 |
| P-12 | GET/PUT/DELETE | `/role-mappings` | AuthJWT+Casbin* | 外部映射 CRUD | ✅ | `TestPermission_RoleMappingCRUD` |
| P-13 | POST | `/policy/reload` | AuthJWT+Casbin | 重載 policy | ✅ | `TestPermission_CasbinRBAC``make e2e-casbin` |
| P-14 | GET | `/roles` | AuthJWT+Casbin | no-role user RBAC denied 403 | ✅ | 同上 |
\* 預設 `make e2e-full` 使用 `Permission.Casbin.Enabled: false`Casbin middleware **放行**rbac=nil passthrough。`make e2e-casbin` 會改用 `e2e.casbin.yaml`,並額外驗證 policy reload 與 no-role 403。
### Notification無 HTTP API
| ID | 情境 | 自動 | 備註 |
|----|------|:----:|------|
| NT-01 | 同步 Sendmock email | 🔧 | `make notify-test METHOD=email-send MOCK=1` |
| NT-02 | 異步 Enqueue + Worker | 🔧 | `make notify-test METHOD=email-enqueue MOCK=1` |
| NT-03 | Member email OTP 寄送 | ✅ | 含在 M-03/M-04mock provider |
---
## 統計摘要
| 類別 | 自動 E2E | 待擴充 / 手動 |
|------|:--------:|:-------------:|
| Normal | 2 | 0 |
| Auth | 4 | 9ZITADEL |
| Member | 12 | 0 |
| Permission | 14 | 0 |
| **合計** | **32** | **9** |
> Auth refresh/logout 會撤銷 JWT因此腳本分兩輪跑`member/permission` 先用 seed token最後才跑 `TestZZZ_AuthTokenRefreshAndLogout`。
---
## 手動 / 延伸測試
### Auth 全链路(需 ZITADEL
1. 設定 `etc/gateway.dev.yaml``Zitadel.*`
2. `make deps-up && make run-dev`
3. 依 [`docs/auth-unified-registration.md`](./auth-unified-registration.md) 跑 register → confirm → login
### TOTP 互動測試
```bash
make deps-up
make totp-test STEP=flow
```
### Notification
見 [`docs/notification-testing.md`](notification-testing.md)。
---
## 環境變數
| 變數 | 預設 | 說明 |
|------|------|------|
| `GATEWAY_E2E` | `1`(腳本內) | 開啟 OTP 寫入 Redis `e2e:otp:{challenge_id}` |
| `E2E_STATE_FILE` | `test/e2e/fixtures/state.json` | seed 輸出路徑 |
| `E2E_BASE_URL` | `http://127.0.0.1:18888` | 覆寫 Gateway URL |
| `E2E_KEEP_DOCKER` | — | `1` = 測完不 down -v |
| `E2E_WITH_SMTP` | — | `1` = 額外啟動 MailHogprofile=smtp |
| `GATEWAY_PORT` | `18888` | health check 用 |
| `E2E_ROLE` | `tenant_owner` | 覆寫 seed 指派給主測試使用者的 system role |
| `E2E_TEST_PATTERN` | `Test(Auth_\|Health\|Member\|Permission)` | 覆寫第一輪 `go test -run` pattern |
| `E2E_CASBIN` | — | `1` = 執行 Casbin 專用 assertion`make e2e-casbin` 已設定) |
---
## 目錄結構
```
gateway/
├── cmd/e2e-seed/ # E2E 資料 + JWT seed
├── scripts/
│ ├── e2e-lib.sh # 共用 bash helper顏色 / step banner / gateway lifecycle
│ ├── e2e-run.sh # 一鍵完整流程
│ ├── e2e-up.sh # 只起環境
│ ├── e2e-down.sh # 關閉
│ └── e2e-list.sh # 列出所有測試make e2e-list
├── test/e2e/
│ ├── fixtures/
│ │ ├── e2e.yaml # E2E 設定
│ │ ├── e2e.casbin.yaml # Casbin enabled E2E 設定
│ │ └── state.json # 生成gitignore
│ ├── client.go # HTTP helper
│ ├── setup_test.go # TestMain + e2eStep banner helper
│ ├── journey.go # Journey 框架NewJourney + Step + SkipStep
│ ├── {health,auth,member,permission}_test.go # contract tests
│ └── journey_{owner,rbac,session,registration}_test.go # journeys
└── docs/e2e-testing.md # 本文件
```
---
## CI 建議
```yaml
# 範例 GitHub Actions job
- name: E2E contract
run: make e2e-full
working-directory: gateway
- name: E2E journeys
run: make e2e-journey
working-directory: gateway
- name: E2E Casbin
run: make e2e-casbin
working-directory: gateway
```
PR 門檻:`make check`unit+ `make e2e-full`contract+ `make e2e-journey`user flow+ `make e2e-casbin`RBAC enforcement
---
## 常見問題
**Q: `missing e2e otp for challenge`**
A: Gateway 必須以 `GATEWAY_E2E=1` 啟動(`e2e-run.sh` 已設定)。
**Q: `connection refused :18888`**
A: 先 `make e2e-up` 或確認沒有其他 process 佔用 18888。
**Q: 跑完 `make e2e-full` 後 Gateway 還在 :18888**
A: 舊版用 `go run` 背景跑,`kill $!` 只殺 wrapper、子行程會留著。現已改為編譯 `.cache/e2e-gateway` 再啟動cleanup 會依 **pid 檔 + port + orphan** 三層關閉。若仍有殘留:
```bash
make e2e-down
# 或
lsof -ti tcp:18888 | xargs kill -9
```
**Q: 與 `make run-dev`:8888衝突**
A: E2E 固定用 **18888** + **gateway_e2e** DB互不影響。
**Q: 如何只跑單一測試?**
```bash
make e2e-up
GATEWAY_E2E=1 go test -tags=e2e -v -count=1 ./test/e2e/ -run TestMember_GetMe
make e2e-down
```