chore(audit): 补追 96 份未入仓审计孤本 — 覆盖 2026-02-26 ~ 2026-04-08
这些审计记录原本堆积在 docs/audit/changes/changes/ 嵌套误产物目录下(由开发机迁移
79d3c2e 前后的不明批量操作产生)。由于同期 .gitignore 屏蔽了 docs/audit/ 全目录,
它们从未入过 git 任何分支 history。删除即永久丢失。
按 docs/specs/audit-gap-recovery/tasks.md 阶段 1 执行,将全部 96 份 D 类孤本
(主目录无同名、git history 亦无记录)复制到 docs/audit/changes/ 主目录入仓。
涵盖主题: P1-P18 全栈集成 / 多模块累积变更 / ETL bug 修复 / 业务日切 /
召回与任务引擎改造 / 租户管理与审批 / 董事会财务 / 客户与助教详情 /
DDL 基线合并 / Kiro 到 Claude Code 迁移
阶段 2(B 类内容漂移 1 份)和阶段 4(嵌套目录删除)独立推进。
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,76 @@
|
||||
# 审计记录:ETL 缺失字段补充 — 第一阶段(DDL + FACT_MAPPINGS)
|
||||
|
||||
- 日期:2026-03-26
|
||||
- 原始原因:field_gap_analysis.md 深度评估识别出 ODS/DWD 层缺失字段
|
||||
- 直接原因:补充缺失字段以支持完整数据流(order_from、活动金额、会员消费统计等)
|
||||
|
||||
## 改动方案
|
||||
|
||||
### 迁移脚本
|
||||
- `db/etl_feiqiu/migrations/2026-03-26_add_missing_fields_from_gap_analysis.sql`
|
||||
- 所有 ALTER TABLE 使用 IF NOT EXISTS,幂等可重复执行
|
||||
- 已在 test_etl_feiqiu 测试库执行成功
|
||||
|
||||
### ODS 层新增列(6 张表)
|
||||
| 表 | 新增列 |
|
||||
|---|---|
|
||||
| member_profiles | other_pay_money_sum, last_consume_time, non_consume_day_num, first_consumption |
|
||||
| assistant_service_records | deduct_leave_seconds, order_from |
|
||||
| store_goods_sales_records | activity_amount, activity_id, order_from |
|
||||
| goods_stock_summary | createtime |
|
||||
| table_fee_transactions | order_from |
|
||||
| settlement_records | orderfrom |
|
||||
|
||||
### DWD 层新增列(6 张表)
|
||||
| 表 | 新增列 |
|
||||
|---|---|
|
||||
| dim_member_ex | other_pay_money_sum, last_consume_time, non_consume_day_num, first_consumption |
|
||||
| dwd_assistant_service_log_ex | deduct_leave_seconds, order_from |
|
||||
| dwd_store_goods_sale_ex | activity_amount, activity_id, order_from |
|
||||
| dwd_goods_stock_summary | create_time |
|
||||
| dwd_table_fee_log_ex | order_from |
|
||||
| dwd_settlement_head_ex | order_from |
|
||||
|
||||
### FACT_MAPPINGS 更新(7 张表)
|
||||
- dim_member_ex: 4 个字段(使用 payload->>'xxx' 从 JSON 提取)
|
||||
- dim_member_card_account_ex: pdassisnatlevel, cxassisnatlevel
|
||||
- dwd_assistant_service_log_ex: deduct_leave_seconds, order_from
|
||||
- dwd_store_goods_sale_ex: activity_amount, activity_id, order_from
|
||||
- dwd_goods_stock_summary: create_time
|
||||
- dwd_table_fee_log_ex: order_from
|
||||
- dwd_settlement_head_ex: order_from
|
||||
|
||||
## 文件清单
|
||||
1. `db/etl_feiqiu/migrations/2026-03-26_add_missing_fields_from_gap_analysis.sql` — 新建
|
||||
2. `apps/etl/connectors/feiqiu/tasks/dwd/dwd_load_task.py` — FACT_MAPPINGS 追加
|
||||
3. `docs/database/ddl/etl_feiqiu__ods.sql` — DDL 基线同步
|
||||
4. `docs/database/ddl/etl_feiqiu__dwd.sql` — DDL 基线同步
|
||||
|
||||
## 风险评估
|
||||
- 低风险:所有 ALTER TABLE 使用 IF NOT EXISTS,幂等安全
|
||||
- ODS 新列为 schema-aware 自动入库,无需修改 Python clean 映射
|
||||
- dim_member_ex 的 4 个字段使用 payload JSON 提取,backfill 时可从历史 payload 获取
|
||||
|
||||
## 回滚策略
|
||||
```sql
|
||||
-- 回滚 ODS
|
||||
ALTER TABLE ods.member_profiles DROP COLUMN IF EXISTS other_pay_money_sum, DROP COLUMN IF EXISTS last_consume_time, DROP COLUMN IF EXISTS non_consume_day_num, DROP COLUMN IF EXISTS first_consumption;
|
||||
ALTER TABLE ods.assistant_service_records DROP COLUMN IF EXISTS deduct_leave_seconds, DROP COLUMN IF EXISTS order_from;
|
||||
ALTER TABLE ods.store_goods_sales_records DROP COLUMN IF EXISTS activity_amount, DROP COLUMN IF EXISTS activity_id, DROP COLUMN IF EXISTS order_from;
|
||||
ALTER TABLE ods.goods_stock_summary DROP COLUMN IF EXISTS createtime;
|
||||
ALTER TABLE ods.table_fee_transactions DROP COLUMN IF EXISTS order_from;
|
||||
ALTER TABLE ods.settlement_records DROP COLUMN IF EXISTS orderfrom;
|
||||
-- 回滚 DWD 同理
|
||||
```
|
||||
|
||||
## 验证 SQL
|
||||
```sql
|
||||
-- 1. 确认 ODS 新列存在
|
||||
SELECT column_name FROM information_schema.columns WHERE table_schema='ods' AND table_name='member_profiles' AND column_name IN ('other_pay_money_sum','last_consume_time','non_consume_day_num','first_consumption');
|
||||
|
||||
-- 2. 确认 DWD 新列存在
|
||||
SELECT column_name FROM information_schema.columns WHERE table_schema='dwd' AND table_name='dwd_settlement_head_ex' AND column_name='order_from';
|
||||
|
||||
-- 3. 确认所有新列默认值正确
|
||||
SELECT column_name, column_default FROM information_schema.columns WHERE table_schema='dwd' AND table_name='dwd_assistant_service_log_ex' AND column_name='deduct_leave_seconds';
|
||||
```
|
||||
Reference in New Issue
Block a user