# 审计记录：ETL 缺失字段补充 — 第一阶段（DDL + FACT_MAPPINGS）

- 日期：2026-03-26
- 原始原因：field_gap_analysis.md 深度评估识别出 ODS/DWD 层缺失字段
- 直接原因：补充缺失字段以支持完整数据流（order_from、活动金额、会员消费统计等）

## 改动方案

### 迁移脚本
- `db/etl_feiqiu/migrations/2026-03-26_add_missing_fields_from_gap_analysis.sql`
- 所有 ALTER TABLE 使用 IF NOT EXISTS，幂等可重复执行
- 已在 test_etl_feiqiu 测试库执行成功

### ODS 层新增列（6 张表）
| 表 | 新增列 |
|---|---|
| member_profiles | other_pay_money_sum, last_consume_time, non_consume_day_num, first_consumption |
| assistant_service_records | deduct_leave_seconds, order_from |
| store_goods_sales_records | activity_amount, activity_id, order_from |
| goods_stock_summary | createtime |
| table_fee_transactions | order_from |
| settlement_records | orderfrom |

### DWD 层新增列（6 张表）
| 表 | 新增列 |
|---|---|
| dim_member_ex | other_pay_money_sum, last_consume_time, non_consume_day_num, first_consumption |
| dwd_assistant_service_log_ex | deduct_leave_seconds, order_from |
| dwd_store_goods_sale_ex | activity_amount, activity_id, order_from |
| dwd_goods_stock_summary | create_time |
| dwd_table_fee_log_ex | order_from |
| dwd_settlement_head_ex | order_from |

### FACT_MAPPINGS 更新（7 张表）
- dim_member_ex: 4 个字段（使用 payload->>'xxx' 从 JSON 提取）
- dim_member_card_account_ex: pdassisnatlevel, cxassisnatlevel
- dwd_assistant_service_log_ex: deduct_leave_seconds, order_from
- dwd_store_goods_sale_ex: activity_amount, activity_id, order_from
- dwd_goods_stock_summary: create_time
- dwd_table_fee_log_ex: order_from
- dwd_settlement_head_ex: order_from

## 文件清单
1. `db/etl_feiqiu/migrations/2026-03-26_add_missing_fields_from_gap_analysis.sql` — 新建
2. `apps/etl/connectors/feiqiu/tasks/dwd/dwd_load_task.py` — FACT_MAPPINGS 追加
3. `docs/database/ddl/etl_feiqiu__ods.sql` — DDL 基线同步
4. `docs/database/ddl/etl_feiqiu__dwd.sql` — DDL 基线同步

## 风险评估
- 低风险：所有 ALTER TABLE 使用 IF NOT EXISTS，幂等安全
- ODS 新列为 schema-aware 自动入库，无需修改 Python clean 映射
- dim_member_ex 的 4 个字段使用 payload JSON 提取，backfill 时可从历史 payload 获取

## 回滚策略
```sql
-- 回滚 ODS
ALTER TABLE ods.member_profiles DROP COLUMN IF EXISTS other_pay_money_sum, DROP COLUMN IF EXISTS last_consume_time, DROP COLUMN IF EXISTS non_consume_day_num, DROP COLUMN IF EXISTS first_consumption;
ALTER TABLE ods.assistant_service_records DROP COLUMN IF EXISTS deduct_leave_seconds, DROP COLUMN IF EXISTS order_from;
ALTER TABLE ods.store_goods_sales_records DROP COLUMN IF EXISTS activity_amount, DROP COLUMN IF EXISTS activity_id, DROP COLUMN IF EXISTS order_from;
ALTER TABLE ods.goods_stock_summary DROP COLUMN IF EXISTS createtime;
ALTER TABLE ods.table_fee_transactions DROP COLUMN IF EXISTS order_from;
ALTER TABLE ods.settlement_records DROP COLUMN IF EXISTS orderfrom;
-- 回滚 DWD 同理
```

## 验证 SQL
```sql
-- 1. 确认 ODS 新列存在
SELECT column_name FROM information_schema.columns WHERE table_schema='ods' AND table_name='member_profiles' AND column_name IN ('other_pay_money_sum','last_consume_time','non_consume_day_num','first_consumption');

-- 2. 确认 DWD 新列存在
SELECT column_name FROM information_schema.columns WHERE table_schema='dwd' AND table_name='dwd_settlement_head_ex' AND column_name='order_from';

-- 3. 确认所有新列默认值正确
SELECT column_name, column_default FROM information_schema.columns WHERE table_schema='dwd' AND table_name='dwd_assistant_service_log_ex' AND column_name='deduct_leave_seconds';
```