feat(db,docs): F1-5b MP-2 prep + 沙箱时光机模块 spec (W1)
MP-2 经 4 轮调研 + Neo 反馈,采纳方案 C(推迟到 F1-6 沙箱时光机阶段 B): - 第 1 轮原方案 D(双口径) → 第 2 轮 D'(单口径) - 第 3 轮 Neo 架构纠正:不读 DWD,走 Core/DWS/app - 第 4 轮 DWS 视图靠谱性审计:dws_assistant_daily_detail 是计费明细 (ledger_amount),不是助教工资(gross_salary 需等级时薪 + 抽成 + 罚分),且缺 effective_hours / work_days - 结论:MP-2 真正实施需要新建 dws_assistant_daily_salary 表(ETL 改造),跟其他 14 个 P1 指标一起做更高效 → 推迟到 F1-6 本次 Wave B 只做 prep:DB schema + 模块 spec + tasks.md 状态调整。 DB 迁移(zqyy_app): - db/zqyy_app/migrations/20260505__add_effective_date_for_excel_adjustments.sql - 3 张 Excel 暂存表(全空,Neo 确认尚无 Excel 上传)ADD COLUMN effective_date DATE NOT NULL(无 DEFAULT,强制未来 Excel 上传必须带): * biz.salary_adjustments(助教薪资扣款/奖励) * biz.stg_finance_expense(月度支出) * biz.stg_platform_income(平台结算收入) - 3 个复合索引 (site_id, effective_date) 支持后续 daily 截断查询 - biz.stg_recharge_commission 已有 recharge_date,无需改造 测试库执行 + 5/5 校验 PASS: - 字段存在(NOT NULL DATE 无 default) - 复合索引存在 + 列序正确 - 字段注释含 'F1-5b MP-2 prep' - INSERT 不带 effective_date 触发 NotNullViolation docs/database/ 同步: - docs/database/changes/2026-05-05__add_effective_date_for_excel_adjustments.md 完整变更说明 + 兼容性 + 回滚 + 5 条校验 SQL + 正式库执行说明 沙箱时光机模块 spec(主干任务排期登记): - docs/_overview/sandbox-replay-engine-spec.md - 22 个相关指标分 P1/P2/P3 优先级: * P1 14 项(daily 视图已有,后端切换) * P2 5 项(算法重算,含 MP-2 完整 daily salary) * P3 3 项(状态算法 + sandbox_audit_log 用户行为) - 4 阶段实施路径: * 阶段 0(本次 prep) * 阶段 A(F1-5a/b 已完成) * 阶段 B(F1-6,2-3 周)— MP-2 真正实施在此 * 阶段 C(F1-7+,1-2 周) - sandbox_replay 模块结构 + runtime_aware decorator 接口契约 - 性能 + 测试 + 前置依赖清单 F1-5b-tasks.md 状态调整: - §4.3 顺序 15:MP-2 从"待开始/C4" → "延期 F1-6" - §6 进度表 MP-2 行同步标"延期 F1-6 + 改方向说明" - 关联到 mp2_prep.md 审计 业务影响: - board-coach sandbox 行为暂遗留(F1-6 解决) - 旧 Excel 模板上传将因 NOT NULL 失败,需 F1-6 同期 ETL UI 改造 + 操作员培训 - 跨页面已 audit:board-finance / customer-records / coach-service-records / customer-service-records 等已合规(F1-5b A1/A3 + MP-1/3/5 收益) 审计:docs/audit/changes/2026-05-05__wave1_f1_5b_mp2_prep.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,173 @@
|
||||
# 2026-05-05 · 3 张 Excel 暂存表加 effective_date 字段(F1-5b MP-2 prep)
|
||||
|
||||
> F1-5b Wave B MP-2 准备工作(后端 board_service 改造延期到 F1-6 沙箱时光机阶段 B)
|
||||
>
|
||||
> migration: `db/zqyy_app/migrations/20260505__add_effective_date_for_excel_adjustments.sql`
|
||||
|
||||
## 背景
|
||||
|
||||
F1-5b MP-2 调研发现:sandbox 模式下需要按业务日上界过滤 Excel 修正(扣款/奖励/支出/收入),但当前 schema 只有 YYYY-MM 月度精度,无法 daily 截断。
|
||||
|
||||
**Neo 决策(2026-05-05)**:
|
||||
1. 三张目标表当前**全为空**(尚无任何 Excel 上传),直接 ADD COLUMN NOT NULL 不带 DEFAULT,强制未来 Excel 上传必须带 effective_date(操作员业务规范)
|
||||
2. 后端 board_service 改造**延期到 F1-6 沙箱时光机阶段 B**(详见 `docs/_overview/sandbox-replay-engine-spec.md`)
|
||||
3. 本次仅做 schema 准备 + 索引,不动 ETL 上传 / 后端业务逻辑
|
||||
|
||||
## 变更说明
|
||||
|
||||
### Schema 变更(3 张表)
|
||||
|
||||
| 表 | 用途 | 改动 |
|
||||
|----|------|------|
|
||||
| `biz.salary_adjustments` | 助教薪资扣款/奖励 | ADD COLUMN effective_date DATE NOT NULL |
|
||||
| `biz.stg_finance_expense` | 月度支出(房租/水电/成本) | ADD COLUMN effective_date DATE NOT NULL |
|
||||
| `biz.stg_platform_income` | 平台结算收入 | ADD COLUMN effective_date DATE NOT NULL |
|
||||
|
||||
`biz.stg_recharge_commission` 已有 `recharge_date DATE` 字段,无需改造。
|
||||
|
||||
### 字段定义
|
||||
|
||||
```sql
|
||||
ALTER TABLE biz.<table_name>
|
||||
ADD COLUMN effective_date DATE NOT NULL;
|
||||
|
||||
COMMENT ON COLUMN biz.<table_name>.effective_date IS
|
||||
'生效日期(F1-5b MP-2 prep): Excel 上传时强制带,'
|
||||
'用于 sandbox 模式下按业务日上界 (effective_date <= business_date) 过滤';
|
||||
```
|
||||
|
||||
### 索引(3 个复合索引)
|
||||
|
||||
```sql
|
||||
CREATE INDEX idx_salary_adj_site_eff_date
|
||||
ON biz.salary_adjustments (site_id, effective_date);
|
||||
CREATE INDEX idx_stg_finance_expense_site_eff
|
||||
ON biz.stg_finance_expense (site_id, effective_date);
|
||||
CREATE INDEX idx_stg_platform_income_site_eff
|
||||
ON biz.stg_platform_income (site_id, effective_date);
|
||||
```
|
||||
|
||||
支持后续 daily 截断查询模式 `WHERE site_id = ? AND effective_date <= ?`。
|
||||
|
||||
## 兼容性
|
||||
|
||||
| 端 | 影响 | 处理 |
|
||||
|----|------|------|
|
||||
| **现有数据** | 三表当前全为空,ADD COLUMN NOT NULL 不引发既有数据迁移失败 | — |
|
||||
| **ETL Excel 上传** | 后续(F1-6)需要在导入流程中解析 effective_date 列 | F1-6 内做 |
|
||||
| **后端 API** | 当前无任何后端 SQL 使用 effective_date,**本次 schema 改动无即时业务影响** | — |
|
||||
| **管理后台 Excel 上传 UI** | tenant-admin 的 ExcelUpload 模块需要在 Excel 模板新增 effective_date 列 | F1-6 内做(配套) |
|
||||
| **既有 Excel 文件模板** | 未来上传必须新模板带日期列;旧模板上传会因 NOT NULL 失败 | 需要操作员培训 + 模板分发 |
|
||||
|
||||
## 回滚策略
|
||||
|
||||
```sql
|
||||
BEGIN;
|
||||
DROP INDEX IF EXISTS biz.idx_salary_adj_site_eff_date;
|
||||
DROP INDEX IF EXISTS biz.idx_stg_finance_expense_site_eff;
|
||||
DROP INDEX IF EXISTS biz.idx_stg_platform_income_site_eff;
|
||||
ALTER TABLE biz.salary_adjustments DROP COLUMN IF EXISTS effective_date;
|
||||
ALTER TABLE biz.stg_finance_expense DROP COLUMN IF EXISTS effective_date;
|
||||
ALTER TABLE biz.stg_platform_income DROP COLUMN IF EXISTS effective_date;
|
||||
COMMIT;
|
||||
```
|
||||
|
||||
幂等,执行多次安全。
|
||||
|
||||
## 验证 SQL(测试库已 PASS)
|
||||
|
||||
### 校验 1:三表都有 effective_date 字段(NOT NULL DATE 无 default)
|
||||
|
||||
```sql
|
||||
SELECT table_name, column_name, data_type, is_nullable, column_default
|
||||
FROM information_schema.columns
|
||||
WHERE table_schema = 'biz'
|
||||
AND table_name IN ('salary_adjustments', 'stg_finance_expense', 'stg_platform_income')
|
||||
AND column_name = 'effective_date'
|
||||
ORDER BY table_name;
|
||||
-- 期望: 3 行,全部 data_type=date / is_nullable='NO' / column_default=NULL
|
||||
```
|
||||
|
||||
### 校验 2:三个复合索引都已创建
|
||||
|
||||
```sql
|
||||
SELECT tablename, indexname FROM pg_indexes
|
||||
WHERE schemaname = 'biz'
|
||||
AND indexname IN (
|
||||
'idx_salary_adj_site_eff_date',
|
||||
'idx_stg_finance_expense_site_eff',
|
||||
'idx_stg_platform_income_site_eff'
|
||||
)
|
||||
ORDER BY indexname;
|
||||
-- 期望: 3 行
|
||||
```
|
||||
|
||||
### 校验 3:索引列序正确 (site_id, effective_date)
|
||||
|
||||
```sql
|
||||
SELECT i.relname AS index_name,
|
||||
array_agg(a.attname ORDER BY array_position(ix.indkey, a.attnum)) AS columns
|
||||
FROM pg_class t
|
||||
JOIN pg_index ix ON t.oid = ix.indrelid
|
||||
JOIN pg_class i ON i.oid = ix.indexrelid
|
||||
JOIN pg_attribute a ON a.attrelid = t.oid AND a.attnum = ANY(ix.indkey)
|
||||
WHERE i.relname IN (
|
||||
'idx_salary_adj_site_eff_date',
|
||||
'idx_stg_finance_expense_site_eff',
|
||||
'idx_stg_platform_income_site_eff'
|
||||
)
|
||||
GROUP BY i.relname
|
||||
ORDER BY i.relname;
|
||||
-- 期望: 全部 ['site_id', 'effective_date']
|
||||
```
|
||||
|
||||
### 校验 4:字段注释存在
|
||||
|
||||
```sql
|
||||
SELECT c.table_name,
|
||||
col_description((c.table_schema||'.'||c.table_name)::regclass::oid, c.ordinal_position) AS comment
|
||||
FROM information_schema.columns c
|
||||
WHERE c.table_schema = 'biz'
|
||||
AND c.table_name IN ('salary_adjustments', 'stg_finance_expense', 'stg_platform_income')
|
||||
AND c.column_name = 'effective_date'
|
||||
ORDER BY c.table_name;
|
||||
-- 期望: 3 行,comment 含 'F1-5b MP-2 prep'
|
||||
```
|
||||
|
||||
### 校验 5:INSERT 不带 effective_date 应失败(NOT NULL 约束生效)
|
||||
|
||||
```sql
|
||||
BEGIN;
|
||||
INSERT INTO biz.salary_adjustments
|
||||
(site_id, assistant_name, assistant_number, salary_month,
|
||||
adjustment_type, amount, reason, upload_batch_id, created_by)
|
||||
VALUES (2790685415443269, 'test', 'T001', '2026-04', 'penalty',
|
||||
100.00, 'test', 1, 1);
|
||||
ROLLBACK;
|
||||
-- 期望: NotNullViolation
|
||||
```
|
||||
|
||||
## 正式库执行说明
|
||||
|
||||
本次 migration **仅在测试库执行**。生产环境同步时:
|
||||
|
||||
```bash
|
||||
psql "$APP_DB_DSN" -f db/zqyy_app/migrations/20260505__add_effective_date_for_excel_adjustments.sql
|
||||
```
|
||||
|
||||
执行前必须确认三张表为空(若已有数据,需先评估回填策略)。执行后跑 5 条校验 SQL。
|
||||
|
||||
## 后续依赖任务(F1-6 沙箱时光机阶段 B)
|
||||
|
||||
详见 `docs/_overview/sandbox-replay-engine-spec.md`:
|
||||
|
||||
1. **ETL Excel 上传 UI 改造**:tenant-admin/ExcelUpload 模块支持 effective_date 列解析
|
||||
2. **Excel 模板分发**:操作员培训 + 新模板分发
|
||||
3. **后端 board_service 改造**:替换 `dws_assistant_salary_calc` 直查为 daily 累计 + adjustments 截断
|
||||
4. **MP-2 实施 + 双口径走查**
|
||||
|
||||
## 关联
|
||||
|
||||
- F1-5b MP-2 调研记录:`docs/audit/changes/2026-05-05__wave1_f1_5b_mp2_prep.md`
|
||||
- 沙箱时光机模块 spec:`docs/_overview/sandbox-replay-engine-spec.md`
|
||||
- F1-5b 任务清单:`docs/_overview/wave1-findings/F1-5b-tasks.md`(MP-2 状态已调整为"延期 F1-6")
|
||||
Reference in New Issue
Block a user