chore(audit): 2026-04-20 历史批次预审 + 文档同步 + .gitignore 修正
- 新增 docs/audit/changes/2026-04-20__historical-batch-pre-audit.md 157 文件分批盘点审计(7 条主线 + 10 项高/中风险 + 2 份迁移 SQL DDL 清单) - 补追 docs/audit/changes/2026-04-15__meituan-settle-core-sync.md 原审计产物因 .gitignore 屏蔽长期未入仓,本次一并追回 - 刷新 docs/audit/audit_dashboard.md(33 条审计记录) - .gitignore 白名单放行 docs/audit/changes/*.md 与 audit_dashboard.md 同时屏蔽 changes/changes/ 嵌套误产物目录 - 新增 docs/specs/audit-gap-recovery/tasks.md 扫描嵌套目录发现 96 份 D 类孤本(从未入过 git history), 生成独立 PRD 供单开任务清理与补追 - 文档同步(高风险项): - apps/backend/docs/API-REFERENCE.md (+69) - apps/miniprogram/README.md (+50) - apps/etl/connectors/feiqiu/docs/architecture/data_flow.md (+52/-2) - apps/etl/connectors/feiqiu/docs/architecture/system_overview.md (+5/-3) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
124
docs/audit/changes/2026-04-15__meituan-settle-core-sync.md
Normal file
124
docs/audit/changes/2026-04-15__meituan-settle-core-sync.md
Normal file
@@ -0,0 +1,124 @@
|
||||
# 变更审计记录:美团结算导入 + core 层定位 + CORE_DIM_SYNC 任务
|
||||
|
||||
| 字段 | 值 |
|
||||
|------|-----|
|
||||
| 日期 | 2026-04-15 01:41:01 |
|
||||
|
||||
## 操作摘要
|
||||
|
||||
本次变更包含 5 项关联改动,围绕"美团结算数据入库"和"core 维度层重新启用"两条主线展开:
|
||||
|
||||
1. **美团结算数据导入**:在 dwd.dwd_groupbuy_redemption 新增 6 个 mt_* 字段(mt_settlement_price / mt_gross_income / mt_service_fee / mt_marketing_fee / mt_other_adjust / mt_import_time),通过临时表批量 UPDATE 导入 15,878 条结算价记录(95.4% 覆盖率,未匹配 759 条为非美团券码)。
|
||||
2. **core 层重新定位与数据填充**:明确 core 层为"跨平台统一维度层"(屏蔽 ODS/DWD 多数据源差异),对 core.dim_assistant / dim_member / dim_site / dim_project 执行 TRUNCATE+INSERT 全量填充,修复 app 视图空查询问题。
|
||||
3. **CORE_DIM_SYNC 新 ETL 任务**:新建 core_dim_sync_task.py 实现 DWD->core 全量刷新(TRUNCATE+INSERT 4 个维度表),注册到 ETL task_registry(depends_on=DWD_LOAD_FROM_ODS)和后端 task_registry(admin-web 展示),meta.etl_task 新增 task_id=69。
|
||||
4. **DWS_ASSISTANT_ORDER_CONTRIBUTION 补全**:该任务此前仅在 ETL 侧注册,本次补齐后端 task_registry 注册和 meta.etl_task 配置(task_id=68),并全量回刷 2025-09-01~2026-04-15 数据(2,207 条)。
|
||||
5. **文档同步**:db/README.md 补充 core 层说明,ETL 任务文档新增 Core 层段落,BD 手册补 6 个 mt_* 字段说明。
|
||||
|
||||
## 变更文件
|
||||
|
||||
### 新增
|
||||
|
||||
| 文件 | 说明 |
|
||||
|------|------|
|
||||
| `apps/etl/connectors/feiqiu/tasks/dws/core_dim_sync_task.py` | CORE_DIM_SYNC 任务实现(TRUNCATE+INSERT 4 个维度表) |
|
||||
|
||||
### 修改
|
||||
|
||||
| 文件 | 说明 |
|
||||
|------|------|
|
||||
| `db/etl_feiqiu/schemas/dwd.sql` | DDL 基线同步:dwd_groupbuy_redemption 新增 6 个 mt_* 字段 |
|
||||
| `db/etl_feiqiu/schemas/app.sql` | 视图定义同步(保持指向 core) |
|
||||
| `db/etl_feiqiu/schemas/core.sql` | 注释更新:core 层定位说明 |
|
||||
| `db/README.md` | core 层补充说明 |
|
||||
| `apps/etl/connectors/feiqiu/tasks/dws/__init__.py` | 导出 CoreDimSyncTask |
|
||||
| `apps/etl/connectors/feiqiu/orchestration/task_registry.py` | 注册 CORE_DIM_SYNC(task_id=69, depends_on=DWD_LOAD_FROM_ODS) |
|
||||
| `apps/backend/app/services/task_registry.py` | 注册 CORE_DIM_SYNC + DWS_ASSISTANT_ORDER_CONTRIBUTION 到 admin-web 展示 |
|
||||
| `apps/etl/connectors/feiqiu/docs/etl_tasks/README.md` | 新增 Core 层段落 |
|
||||
| `apps/etl/connectors/feiqiu/docs/database/DWD/main/BD_manual_dwd_groupbuy_redemption.md` | 补充 6 个 mt_* 字段说明 |
|
||||
|
||||
## 改动注解
|
||||
|
||||
### 高风险
|
||||
|
||||
- **db/etl_feiqiu/schemas/dwd.sql**:ALTER TABLE 新增 6 列均为 NULLABLE,无破坏性。DDL 基线已同步数据库现状。数据通过临时表 UPDATE 导入,非 ETL 自动流程。
|
||||
- **db/etl_feiqiu/schemas/app.sql**:app.v_assistant / v_member / v_site 视图保持原始定义(指向 core.dim_*),core 表已填充数据,视图查询恢复正常。
|
||||
- **db/etl_feiqiu/schemas/core.sql**:注释变更,无结构变更。core 层定位从"待废弃"改为"跨平台统一维度层"。
|
||||
- **apps/etl/connectors/feiqiu/orchestration/task_registry.py**:新增 CORE_DIM_SYNC 注册项,depends_on=DWD_LOAD_FROM_ODS 确保在 DWD 加载后执行。
|
||||
- **apps/backend/app/services/task_registry.py**:新增 2 项展示注册(CORE_DIM_SYNC + DWS_ASSISTANT_ORDER_CONTRIBUTION),不影响已有任务。
|
||||
|
||||
### 普通
|
||||
|
||||
- **apps/etl/connectors/feiqiu/tasks/dws/core_dim_sync_task.py**:新建文件,TRUNCATE+INSERT 全量刷新模式,4 个维度表映射关系明确。
|
||||
- **apps/etl/connectors/feiqiu/tasks/dws/__init__.py**:仅添加导出行。
|
||||
- **db/README.md**:文档补充。
|
||||
- **apps/etl/connectors/feiqiu/docs/**:ETL 文档同步,无逻辑影响。
|
||||
|
||||
## 数据库变更
|
||||
|
||||
### etl_feiqiu 库
|
||||
|
||||
| 操作 | 对象 | 详情 | 执行状态 |
|
||||
|------|------|------|----------|
|
||||
| ALTER TABLE | `dwd.dwd_groupbuy_redemption` | 新增 6 列:mt_settlement_price (numeric(10,2)), mt_gross_income (numeric(10,2)), mt_service_fee (numeric(10,2)), mt_marketing_fee (numeric(10,2)), mt_other_adjust (numeric(10,2)), mt_import_time (timestamptz) | 已执行 |
|
||||
| TRUNCATE+INSERT | `core.dim_assistant` | 从 DWD 全量填充 | 已执行 |
|
||||
| TRUNCATE+INSERT | `core.dim_member` | 从 DWD 全量填充 | 已执行 |
|
||||
| TRUNCATE+INSERT | `core.dim_site` | 从 DWD 全量填充 | 已执行 |
|
||||
| TRUNCATE+INSERT | `core.dim_project` | 从 DWD 全量填充 | 已执行 |
|
||||
| INSERT | `meta.etl_task` | task_id=68 (DWS_ASSISTANT_ORDER_CONTRIBUTION) | 已执行 |
|
||||
| INSERT | `meta.etl_task` | task_id=69 (CORE_DIM_SYNC) | 已执行 |
|
||||
| CREATE OR REPLACE VIEW | `app.v_assistant / v_member / v_site` | 确认保持指向 core.dim_*(无变更) | 已验证 |
|
||||
|
||||
## 风险与回滚
|
||||
|
||||
### 风险点
|
||||
|
||||
| 级别 | 风险 | 缓解 |
|
||||
|------|------|------|
|
||||
| 中 | mt_* 字段通过临时表手工导入,非 ETL 自动化流程,后续美团结算单需手动重跑 | 后续可考虑封装为 scripts/ops/ 脚本 |
|
||||
| 中 | CORE_DIM_SYNC 使用 TRUNCATE+INSERT 全量刷新,执行期间 core 表短暂为空 | 任务执行时间极短(4 个维度表数据量小),可接受 |
|
||||
| 低 | DWS_ASSISTANT_ORDER_CONTRIBUTION 回刷数据量大(2,207 条跨 7 个月) | 已验证数据正确性 |
|
||||
|
||||
### 回滚要点
|
||||
|
||||
1. mt_* 字段回滚:`ALTER TABLE dwd.dwd_groupbuy_redemption DROP COLUMN mt_settlement_price, DROP COLUMN mt_gross_income, DROP COLUMN mt_service_fee, DROP COLUMN mt_marketing_fee, DROP COLUMN mt_other_adjust, DROP COLUMN mt_import_time;`
|
||||
2. CORE_DIM_SYNC 回滚:删除 core_dim_sync_task.py,从两个 task_registry 移除注册项,`DELETE FROM meta.etl_task WHERE task_id = 69;`
|
||||
3. DWS_ASSISTANT_ORDER_CONTRIBUTION 回滚:从后端 task_registry 移除,`DELETE FROM meta.etl_task WHERE task_id = 68;`
|
||||
|
||||
## 验证
|
||||
|
||||
```sql
|
||||
-- 1. 验证 mt_* 字段导入覆盖率
|
||||
SELECT
|
||||
COUNT(*) AS total,
|
||||
COUNT(mt_settlement_price) AS has_settle,
|
||||
ROUND(COUNT(mt_settlement_price)::numeric / COUNT(*) * 100, 1) AS pct
|
||||
FROM dwd.dwd_groupbuy_redemption;
|
||||
|
||||
-- 2. 验证 core 维度表非空
|
||||
SELECT 'dim_assistant' AS tbl, COUNT(*) FROM core.dim_assistant
|
||||
UNION ALL SELECT 'dim_member', COUNT(*) FROM core.dim_member
|
||||
UNION ALL SELECT 'dim_site', COUNT(*) FROM core.dim_site
|
||||
UNION ALL SELECT 'dim_project', COUNT(*) FROM core.dim_project;
|
||||
|
||||
-- 3. 验证 app 视图正常返回数据
|
||||
SELECT COUNT(*) FROM app.v_assistant;
|
||||
SELECT COUNT(*) FROM app.v_member;
|
||||
|
||||
-- 4. 验证 meta.etl_task 新增记录
|
||||
SELECT task_id, task_name, is_active FROM meta.etl_task WHERE task_id IN (68, 69);
|
||||
|
||||
-- 5. 验证 DWS_ASSISTANT_ORDER_CONTRIBUTION 回刷数据
|
||||
SELECT COUNT(*), MIN(stat_date), MAX(stat_date)
|
||||
FROM dws.dws_assistant_order_contribution;
|
||||
```
|
||||
|
||||
## 合规检查
|
||||
|
||||
| 项目 | 状态 |
|
||||
|------|------|
|
||||
| DDL 基线同步(db/etl_feiqiu/schemas/) | 已完成:dwd.sql, app.sql, core.sql 已更新 |
|
||||
| db/README.md 文档同步 | 已完成:core 层说明已补充 |
|
||||
| ETL 任务文档同步 | 已完成:docs/etl_tasks/README.md 新增 Core 层段落 |
|
||||
| BD 手册同步 | 已完成:BD_manual_dwd_groupbuy_redemption.md 补 mt_* 字段 |
|
||||
| 后端 task_registry 同步 | 已完成:CORE_DIM_SYNC + DWS_ASSISTANT_ORDER_CONTRIBUTION 已注册 |
|
||||
| RLS 视图双 Schema 规则 | 不涉及:本次无新建视图,现有视图保持不变 |
|
||||
374
docs/audit/changes/2026-04-20__historical-batch-pre-audit.md
Normal file
374
docs/audit/changes/2026-04-20__historical-batch-pre-audit.md
Normal file
@@ -0,0 +1,374 @@
|
||||
# 变更审计记录:2026-04-20 历史累积批次预审
|
||||
|
||||
| 字段 | 值 |
|
||||
|------|-----|
|
||||
| 日期 | 2026-04-20 06:01:35 |
|
||||
| 分类 | 历史批次预审(方案 C:详细) |
|
||||
| 触发时间窗口 | 2026-04-15 ~ 2026-04-20(最后一次 commit 后 5 天累积) |
|
||||
| 覆盖文件数 | 156 |
|
||||
| 高风险文件数 | 54 |
|
||||
| 新迁移 SQL | 2 |
|
||||
| 参与 spec | 7+ |
|
||||
|
||||
## 操作摘要
|
||||
|
||||
本次审计是"历史累积盘点"——156 个文件来自 HEAD (79d3c2e 开发机迁移) 之后 5 天的多次会话改动,用户本次会话无新改动。由于改动跨会话累积,缺失单次 /audit 的即时上下文,本次通过 git diff + 交叉比对 `docs/audit/changes/` 历史审计与 `docs/specs/` 现行规范还原语义上下文。
|
||||
|
||||
改动跨 6 个模块(backend / etl feiqiu / miniprogram / db / docs / 工具脚本)和 7 条主线:开发机迁移(路径基线搬迁)、rns1-customer-coach-api + 04-miniapp-core-business 联合实施、P19 指数回测框架(Phase 1+2)、Core 维度层启用、客户项目标签稳定性优化、美团结算导入收尾、运维工具与分析扩展。
|
||||
|
||||
2 个迁移 SQL 均已在开发库执行,schema 基线与任务代码一致。~95% 路径同步改动(开发机迁移衍生)为低风险;实质性业务变更集中在 16 个 backend + 22 个 miniprogram + 13 个 ETL 文件,全部可追溯到既有 spec。7 项高风险待验证项已在"风险与回滚"章节单独列出。
|
||||
|
||||
## 主线归类
|
||||
|
||||
### 主线 1:开发机迁移(路径同步,~95% 改动量来源)
|
||||
|
||||
- 根目录迁移 `C:\NeoZQYY` → `C:\Project\NeoZQYY`
|
||||
- 涵盖:`.env` / `.env.template` / `.mcp.json` / `apps/backend/.env.local` / `apps/etl/connectors/feiqiu/.env` / `scripts/ops/*.py` / `tools/db/*.py` / `tests/test_property_*.py` / `docs/specs/**/*.md` / `docs/deployment/*.md` / `docs/migrate/*.md` / `docs/mcp/*.md` / `docs/ops/*.md` / `docs/README.md` / `_DEL/**/*.md`
|
||||
- 风险:无(纯路径同步)
|
||||
|
||||
### 主线 2:rns1-customer-coach-api + 04-miniapp-core-business spec 实施
|
||||
|
||||
- **后端 16 文件**:配置 4 / 路由 schema 5 / 服务 9
|
||||
- **小程序 22 文件**:配置 5 / i18n typings 2 / 服务 1 / 工具组件 3 / 页面 11
|
||||
- **关键改动**:
|
||||
- `coach-service-records` 新页面(管理者视角业绩明细,需 `view_board_coach` 权限)
|
||||
- `performance-records` 拆分:去除 coachId 分支,恢复助教自查单一职责
|
||||
- `fdw_queries` 重构:`get_service_records_summary()` 直接 SQL 聚合、`get_last_visit_days()` 改用 `consumption_summary`、`get_consumption_records()` 按结算单粒度聚合
|
||||
- `build_performance_summary()` 公开化,`coach-detail` 和 `task-list` 数据源统一
|
||||
- `isScattered` 散客标记跨端贯通
|
||||
- `task_generator` 回访宽限 48h → 72h,新增 UPSERT 替代策略和 Step 5 清理保底任务
|
||||
- `recall_detector` settle_type=3 限制 + 门店级 resolved 召回解除
|
||||
|
||||
### 主线 3:P19 指数回测框架(Phase 1+2)
|
||||
|
||||
- **DDL**:`dws_member_assistant_relation_index` / `dws_member_winback_index` / `dws_member_newconv_index` 新增 `stat_date DATE NOT NULL DEFAULT CURRENT_DATE`,唯一约束扩展 `(site_id, member_id[, assistant_id], stat_date)`,app schema 视图改 `DISTINCT ON + ORDER BY stat_date DESC`
|
||||
- **任务**:`relation_index` / `winback_index` / `newconv_index` 改日快照模式(按 `stat_date` 删除 + 插入),批量写入优化
|
||||
- **新建**:`DWS_INDEX_BACKFILL` + `DWS_TASK_SIMULATION` 两个工具任务
|
||||
- **task_engine** 升级为双模式(HTTP 实时 / 推演回测)
|
||||
- **前置**:`member_index_base` STOP 分流改为 OLD
|
||||
|
||||
### 主线 4:Core 维度层启用(承接 2026-04-15 美团结算审计后续)
|
||||
|
||||
- 新建 `CORE_DIM_SYNC` 任务(DWD → core 4 维度表 TRUNCATE+INSERT)
|
||||
- `core.sql` 注释更新:定位为"跨平台统一维度层"
|
||||
- 修复 app 视图空查询问题
|
||||
- `task_registry` + `orchestration/task_registry.py` 注册新任务
|
||||
|
||||
### 主线 5:客户项目标签稳定性优化
|
||||
|
||||
- `member_project_tag` 从 `LAST_30/60_DAYS`(固定日期窗口)改为 `LAST_30_VISITS`(最近 30 次消费)
|
||||
- 配套迁移 SQL:`20260411_member_project_tag_visit_based.sql`(DELETE 旧窗口数据 + 新增 `cfg_area_category` 配置行)
|
||||
|
||||
### 主线 6:美团结算数据导入(已有 2026-04-15 审计覆盖)
|
||||
|
||||
- `dwd_groupbuy_redemption` 新增 6 个 `mt_*` 字段(`mt_settlement_price` / `mt_gross_income` / `mt_service_fee` / `mt_marketing_fee` / `mt_other_adjust` / `mt_import_time`)
|
||||
- 本次审计仅作"基线同步"确认:`db/etl_feiqiu/schemas/dwd.sql` + `docs/database/ddl/etl_feiqiu__dwd.sql` 已更新;`apps/etl/connectors/feiqiu/docs/database/DWD/main/BD_manual_dwd_groupbuy_redemption.md` 同步
|
||||
- **主审计记录**:`docs/audit/changes/2026-04-15__meituan-settle-core-sync.md`(本批次不重复审计)
|
||||
|
||||
### 主线 7:运维工具与分析扩展
|
||||
|
||||
- 新增脚本:`scripts/ops/churned_customer_report.py`(流失客户回访报告,~1137 行),`scripts/ops/simulate_historical_tasks.py`(历史任务推演,~524 行)
|
||||
- 新增目录:`tools/task-analysis/`(任务报告工具 + 7 份历史报告)
|
||||
- 新增启动:`start-admin.bat`
|
||||
- 新增报告:`excel_analysis_report.txt` / `sheet_structure.txt`(根目录中间产物)
|
||||
- `demo-miniprogram` SDK 1.6.28 → 1.6.29;项目名归一化
|
||||
|
||||
## 变更文件
|
||||
|
||||
### 后端 backend (16)
|
||||
|
||||
- `apps/backend/app/config.py`:路径注释更新(开发机迁移)
|
||||
- `apps/backend/.env.local`:路径同步
|
||||
- `apps/backend/auth_only.txt`:测试产物刷新
|
||||
- `apps/backend/auth_only_results.txt`:测试产物刷新
|
||||
- `apps/backend/auth_test_results.txt`:测试产物刷新
|
||||
- `apps/backend/app/routers/xcx_coaches.py`:**新增** `GET /xcx/coaches/{id}/banner` 轻量端点(+16)
|
||||
- `apps/backend/app/routers/xcx_performance.py`:**关键**——新增 `coach_id` 可选参数 + 权限分流 `view_board_coach`(+29/-11)
|
||||
- `apps/backend/app/schemas/xcx_coaches.py`:`PerformanceMetrics` 6→25 字段、新增 `CoachBannerResponse` / `CoachTaskStats`(+49/-8)
|
||||
- `apps/backend/app/schemas/xcx_customers.py`:`ConsumptionRecord + foodDetail`,`CustomerNote + creator_name/creator_role`(+3)
|
||||
- `apps/backend/app/schemas/xcx_performance.py`:`DateGroupRecord + is_scattered`(+1)
|
||||
- `apps/backend/app/services/board_service.py`:狭义召回改从 `coach_tasks` 统计(+38/-38)
|
||||
- `apps/backend/app/services/coach_service.py`:**关键**——新增 `get_coach_banner()`、`get_coach_detail` 重构 + `store_name/task_stats`、fdw 查询预容错(+194/-194)
|
||||
- `apps/backend/app/services/customer_service.py`:`notes JOIN auth.users` 获取创建者、结算单粒度消费记录重构(+256/-395)
|
||||
- `apps/backend/app/services/fdw_queries.py`:**关键**——新增 `get_service_records_summary()`、修复 `get_last_visit_days` 改用 `consumption_summary`、`get_consumption_records` 按结算单粒度(+219/-395)
|
||||
- `apps/backend/app/services/performance_service.py`:`assistant_id_override` + 散客标记 + summary 直接 SQL 聚合(+34/-34)
|
||||
- `apps/backend/app/services/recall_detector.py`:**关键**——`settle_type=3` 双重限制 + 门店级 resolved 召回解除 + 72h 宽限期(+109/-109)
|
||||
- `apps/backend/app/services/task_generator.py`:**关键**——回访 48h→72h、UPSERT 替代策略、Step 5 清理保底任务(+194/-194)
|
||||
- `apps/backend/app/services/task_manager.py`:`_build_performance_summary → build_performance_summary` 公开化、`current_tier` 改用数组下标(+18/-18)
|
||||
- `apps/backend/app/services/task_registry.py`:注册 `CORE_DIM_SYNC` / `DWS_ASSISTANT_ORDER_CONTRIBUTION` / `DWS_INDEX_BACKFILL`(+6/-1)
|
||||
|
||||
> 说明:`apps/backend/` 下文件总计 16 个(含 .env.local 和 3 份 auth 测试产物),其余均为实质业务逻辑改动。
|
||||
|
||||
### ETL feiqiu (13)
|
||||
|
||||
- `apps/etl/connectors/feiqiu/.env`:路径迁移 + API_TOKEN 续期(不列值)
|
||||
- `apps/etl/connectors/feiqiu/orchestration/task_registry.py`:注册 `CORE_DIM_SYNC` + `DWS_INDEX_BACKFILL`
|
||||
- `apps/etl/connectors/feiqiu/tasks/dws/__init__.py`:导出 `CoreDimSyncTask`
|
||||
- `apps/etl/connectors/feiqiu/tasks/dws/core_dim_sync_task.py` **[新建]**(+174):DWD → core 4 维度表 TRUNCATE+INSERT
|
||||
- `apps/etl/connectors/feiqiu/tasks/dws/member_project_tag_task.py`:`LAST_30/60_DAYS → LAST_30_VISITS`
|
||||
- `apps/etl/connectors/feiqiu/tasks/dws/task_engine.py`:**关键**——升级双模式(HTTP 实时 / 推演回测)(+767/-96)
|
||||
- `apps/etl/connectors/feiqiu/tasks/dws/index/member_index_base.py`:STOP 分流改 OLD(+2/-1)
|
||||
- `apps/etl/connectors/feiqiu/tasks/dws/index/newconv_index_task.py`:日快照模式(+93/-30)
|
||||
- `apps/etl/connectors/feiqiu/tasks/dws/index/relation_index_task.py`:日快照 + 批量写入优化(+90/-27)
|
||||
- `apps/etl/connectors/feiqiu/tasks/dws/index/winback_index_task.py`:日快照 + 365 天保留策略 + `execute_batch`
|
||||
- `apps/etl/connectors/feiqiu/tasks/utility/index_backfill_task.py` **[新建]**:`DWS_INDEX_BACKFILL` 工具任务
|
||||
- `apps/etl/connectors/feiqiu/tasks/utility/task_simulation_task.py` **[新建]**:`DWS_TASK_SIMULATION` 工具任务
|
||||
- `apps/etl/connectors/feiqiu/scripts/research_coupon_details.py`:路径迁移(+1)
|
||||
- `apps/etl/connectors/feiqiu/docs/database/DWD/main/BD_manual_dwd_groupbuy_redemption.md`:`mt_*` 6 字段说明(+6)
|
||||
- `apps/etl/connectors/feiqiu/docs/database/DWS/main/BD_manual_cfg_area_category.md`:美洲豹赛台 BILLIARD
|
||||
- `apps/etl/connectors/feiqiu/docs/database/DWS/main/BD_manual_dws_member_project_tag.md`:新窗口文档同步
|
||||
- `apps/etl/connectors/feiqiu/docs/etl_tasks/README.md`:Core 层段落新增
|
||||
|
||||
### 小程序 miniprogram (22)
|
||||
|
||||
- `apps/miniprogram/project.config.json`:JSON 格式标准化
|
||||
- `apps/miniprogram/project.miniapp.json`:JSON 格式标准化
|
||||
- `apps/miniprogram/project.private.config.json`:JSON 格式标准化
|
||||
- `apps/miniprogram/miniprogram/app.miniapp.json`:JSON 格式标准化
|
||||
- `apps/miniprogram/miniprogram/app.json`:新增 `coach-service-records` 路由(+1)
|
||||
- `apps/miniprogram/i18n/base.json`:i18n 更新
|
||||
- `apps/miniprogram/typings/types/wx/lib.wx.miniapp.d.ts`:微信 API 类型更新
|
||||
- `apps/miniprogram/miniprogram/services/api.ts`:**关键**——新增 `fetchCoachBanner(coachId)`、`fetchPerformanceRecords` 新增 `coachId` 可选参数(+28/-1)
|
||||
- `apps/miniprogram/miniprogram/utils/format.wxs`:新增 `daysAgo()`(+11)
|
||||
- `apps/miniprogram/miniprogram/utils/perf-progress.ts` **[新建]**(+125):进度条动画参数 + 工具函数单一源
|
||||
- `apps/miniprogram/miniprogram/components/perf-progress-bar/perf-progress-bar.wxss`:CSS 变量化(+1/-1)
|
||||
- `apps/miniprogram/miniprogram/pages/board-coach/board-coach.ts`:`formatCount` 简化(+4/-4)
|
||||
- `apps/miniprogram/miniprogram/pages/coach-detail/coach-detail.ts`:**关键**——进度条去重 + performance 字段扩展(+143/-94)
|
||||
- `apps/miniprogram/miniprogram/pages/coach-detail/coach-detail.wxml`:属性传参 + 散客标记 + Recent 限制 5 条(+18/-4)
|
||||
- `apps/miniprogram/miniprogram/pages/coach-detail/coach-detail.wxss`:散客置灰样式(+4)
|
||||
- `apps/miniprogram/miniprogram/pages/coach-service-records/coach-service-records.json` **[新建]**(+14)
|
||||
- `apps/miniprogram/miniprogram/pages/coach-service-records/coach-service-records.ts` **[新建]**(+200+):权限守卫 `checkPageAccess('view_board_coach')`,`fetchCoachBanner + fetchPerformanceRecords(coach_id)`
|
||||
- `apps/miniprogram/miniprogram/pages/coach-service-records/coach-service-records.wxml` **[新建]**(+150+):助教 banner + 月份切换 + 日期分组
|
||||
- `apps/miniprogram/miniprogram/pages/coach-service-records/coach-service-records.wxss` **[新建]**(+200+):样式同 `performance-records`
|
||||
- `apps/miniprogram/miniprogram/pages/customer-detail/customer-detail.ts`:`foodDetail + phoneFull` + 复制改用 `phoneFull`(+7/-2)
|
||||
- `apps/miniprogram/miniprogram/pages/customer-detail/customer-detail.wxml`:手机号脱敏 + 食品类目动态 + notes 创建者(+8/-4)
|
||||
- `apps/miniprogram/miniprogram/pages/customer-detail/customer-detail.wxss`:食品标签长文本支持(+9)
|
||||
- `apps/miniprogram/miniprogram/pages/customer-records/customer-records.wxml`:食品类目动态同步(+4/-2)
|
||||
- `apps/miniprogram/miniprogram/pages/customer-records/customer-records.wxss`:食品标签样式同步(+9/-1)
|
||||
- `apps/miniprogram/miniprogram/pages/performance-records/performance-records.ts`:变更日志(拆分助教视角)(+1)
|
||||
- `apps/miniprogram/miniprogram/pages/performance-records/performance-records.wxml`:散客标记(+2/-1)
|
||||
- `apps/miniprogram/miniprogram/pages/performance-records/performance-records.wxss`:banner 字体 + 散客置灰(+10/-4)
|
||||
- `apps/miniprogram/miniprogram/pages/task-list/task-list.ts`:进度条逻辑外包到 `perf-progress.ts`(+107/-62,净 -90)
|
||||
- `apps/miniprogram/miniprogram/pages/task-list/task-list.wxml`:`fmt.days → fmt.daysAgo`(+6/-4)
|
||||
|
||||
### 数据库 db (6)
|
||||
|
||||
- `db/etl_feiqiu/migrations/2026-04-12__add_stat_date_to_index_tables.sql` **[新建]**:stat_date 字段 + 唯一约束扩展 + 索引重建 + app 视图 `DISTINCT ON`(详见下方 DDL 清单)
|
||||
- `db/etl_feiqiu/migrations/20260411_member_project_tag_visit_based.sql` **[新建]**:美洲豹赛台配置 + 清理旧窗口数据(详见下方 DDL 清单)
|
||||
- `db/etl_feiqiu/schemas/app.sql`:同步 3 视图 `DISTINCT ON` 改写(+40/-40)
|
||||
- `db/etl_feiqiu/schemas/core.sql`:注释更新,定位为跨平台统一维度层(+6)
|
||||
- `db/etl_feiqiu/schemas/dwd.sql`:`mt_*` 6 字段(对应 2026-04-15 美团审计)
|
||||
- `db/etl_feiqiu/schemas/dws.sql`:3 表 stat_date + 约束 + 索引 + `v_member_recall_priority` 改写(+133/-88)
|
||||
- `db/README.md`:core 层注释补充
|
||||
|
||||
### 文档 docs (40+)
|
||||
|
||||
> 95% 为路径同步(`C:\NeoZQYY → C:\Project\NeoZQYY`);DDL 基线刷新仅 dws / zqyy_app biz 有实质内容;无新 spec。
|
||||
|
||||
**根文档**:
|
||||
|
||||
- `docs/README.md`:路径同步
|
||||
- `docs/assistant-task-detail-report.md`:历史报告(内容保留)
|
||||
|
||||
**DDL 基线(auto-gen,+201/-110 集中在 dws/biz)**:
|
||||
|
||||
- `docs/database/ddl/etl_feiqiu__app.sql`:基线刷新
|
||||
- `docs/database/ddl/etl_feiqiu__core.sql`:基线刷新
|
||||
- `docs/database/ddl/etl_feiqiu__dwd.sql`:基线刷新 + `mt_*` 字段
|
||||
- `docs/database/ddl/etl_feiqiu__dws.sql`:基线刷新 + stat_date + 视图
|
||||
- `docs/database/ddl/etl_feiqiu__meta.sql`:基线刷新
|
||||
- `docs/database/ddl/etl_feiqiu__ods.sql`:基线刷新
|
||||
- `docs/database/ddl/zqyy_app__app.sql`:基线刷新
|
||||
- `docs/database/ddl/zqyy_app__auth.sql`:基线刷新
|
||||
- `docs/database/ddl/zqyy_app__biz.sql`:基线刷新(有内容)
|
||||
- `docs/database/ddl/zqyy_app__public.sql`:基线刷新
|
||||
|
||||
**部署/迁移/MCP/运维**:
|
||||
|
||||
- `docs/deployment/EXPORT-PATHS.md`:路径同步 + LF→CRLF 标准化
|
||||
- `docs/deployment/LAUNCH-CHECKLIST.md`:路径同步 + LF→CRLF 标准化
|
||||
- `docs/mcp/WEIXIN-DEVTOOLS-MCP.md`:路径同步
|
||||
- `docs/migrate/*.md × 2`:路径同步
|
||||
- `docs/ops/init-test-user.md`:路径同步
|
||||
|
||||
**Specs(约 28 份,全部路径同步无进度变更)**:
|
||||
|
||||
- `docs/specs/*/tasks.md × 约 18`:pytest 命令路径同步
|
||||
- `docs/specs/*/design.md × 约 10`:pytest 命令路径同步
|
||||
|
||||
### 工具/脚本/测试/配置 (其余约 43)
|
||||
|
||||
> 95% 路径迁移,实质新增 2 个运维脚本 + 1 个工具目录 + 1 个启动 bat。
|
||||
|
||||
**根配置 5**:
|
||||
|
||||
- `.env`:路径同步
|
||||
- `.env.template`:路径同步
|
||||
- `.mcp.json`:路径同步
|
||||
- `.playwright-mcp/`:新增目录
|
||||
- `uv.lock`:大规模格式重排(不审跨 lockfile 内容)
|
||||
|
||||
**根中间产物 2**:
|
||||
|
||||
- `excel_analysis_report.txt`:Excel 分析中间产物
|
||||
- `sheet_structure.txt`:Sheet 结构中间产物
|
||||
|
||||
**scripts/ops 12**:
|
||||
|
||||
- `scripts/ops/ai_backfill.py`:路径同步
|
||||
- `scripts/ops/backfill_coach_area_hours.py`:路径同步
|
||||
- `scripts/ops/backfill_finance_area_daily.py`:路径同步
|
||||
- `scripts/ops/backfill_index_snapshots.py`:路径同步
|
||||
- `scripts/ops/check_and_refresh_audit.py`:路径同步
|
||||
- `scripts/ops/etl_unified_analysis.py`:路径同步
|
||||
- `scripts/ops/gen_integration_report.py`:路径同步
|
||||
- `scripts/ops/init_test_user.py`:路径同步
|
||||
- `scripts/ops/start-admin.ps1`:路径同步
|
||||
- `scripts/ops/churned_customer_report.py` **[新建]**:流失客户回访报告(~1137 行)
|
||||
- `scripts/ops/simulate_historical_tasks.py` **[新建]**:历史任务推演(~524 行)
|
||||
|
||||
**tools 3 + 新建子目录**:
|
||||
|
||||
- `tools/db/*.py × 3`:路径同步
|
||||
- `tools/task-analysis/` **[新建目录]**:任务报告工具 + 7 份历史报告
|
||||
|
||||
**tests 7**:
|
||||
|
||||
- `tests/test_property_*.py × 7`:常量路径同步
|
||||
|
||||
**apps/demo-miniprogram 4**:
|
||||
|
||||
- SDK 1.6.28 → 1.6.29 + 项目名归一化:
|
||||
- `apps/demo-miniprogram/i18n/base.json`
|
||||
- `apps/demo-miniprogram/miniprogram/app.miniapp.json`
|
||||
- `apps/demo-miniprogram/project.miniapp.json`
|
||||
- `apps/demo-miniprogram/project.private.config.json`
|
||||
|
||||
**_DEL 归档 4**:
|
||||
|
||||
- `_DEL/MIGRATION-PLAYBOOK.md`:路径同步
|
||||
- `_DEL/miniprogram-h5-conversion/steering/action-manual.md`:路径同步
|
||||
- `_DEL/weixin-devtools-mcp.md`:路径同步
|
||||
- `_DEL/weixin-devtools/steering/workflow.md`:路径同步
|
||||
|
||||
**根新增启动**:
|
||||
|
||||
- `start-admin.bat`:根目录启动引导(新建)
|
||||
|
||||
**其他配置**:
|
||||
|
||||
- `.claude/settings.json`:环境配置更新
|
||||
|
||||
## 数据库变更
|
||||
|
||||
### 迁移 1:`db/etl_feiqiu/migrations/2026-04-12__add_stat_date_to_index_tables.sql`
|
||||
|
||||
- **执行状态**:已执行(从 schema 基线一致性 + 任务代码已引用 `stat_date` 推断)
|
||||
- **DDL 清单**:
|
||||
1. `dws.dws_member_assistant_relation_index`:
|
||||
- `ADD COLUMN stat_date DATE NOT NULL DEFAULT CURRENT_DATE`
|
||||
- `DROP CONSTRAINT` 原唯一约束 `(site_id, member_id, assistant_id)`
|
||||
- `ADD CONSTRAINT` 新唯一约束 `(site_id, member_id, assistant_id, stat_date)`
|
||||
- `DROP INDEX` 旧索引(如有)
|
||||
- `CREATE INDEX` 新索引 `(site_id, stat_date)`
|
||||
- `UPDATE` 回填已有行 `stat_date = CURRENT_DATE`
|
||||
2. `dws.dws_member_winback_index`:同上,唯一约束变 `(site_id, member_id, stat_date)`
|
||||
3. `dws.dws_member_newconv_index`:同上,唯一约束变 `(site_id, member_id, stat_date)`
|
||||
4. `app.v_dws_member_assistant_relation_index`:重写 `SELECT DISTINCT ON (site_id, member_id, assistant_id) ... ORDER BY site_id, member_id, assistant_id, stat_date DESC`
|
||||
5. `app.v_dws_member_winback_index`:同上 `DISTINCT ON (site_id, member_id) ... ORDER BY ..., stat_date DESC`
|
||||
6. `app.v_dws_member_newconv_index`:同上 `DISTINCT ON (site_id, member_id) ... ORDER BY ..., stat_date DESC`
|
||||
7. `dws.v_member_recall_priority`:改写以依赖新视图
|
||||
- **影响对象**:3 张 DWS 表 + 4 张视图 + 4 个新索引
|
||||
- **回滚 SQL**:文件末尾提供(TRUNCATE 过期快照 → 恢复原唯一约束 → 删除 `stat_date` 列)
|
||||
- **兼容性**:`stat_date NOT NULL + DEFAULT CURRENT_DATE`,安全;大表 ALTER 因 DATE 列小,锁表风险低
|
||||
|
||||
### 迁移 2:`db/etl_feiqiu/migrations/20260411_member_project_tag_visit_based.sql`
|
||||
|
||||
- **执行状态**:已执行(`member_project_tag_task` 已改为 `LAST_30_VISITS` 模式)
|
||||
- **DDL 清单**:
|
||||
1. `dws.cfg_area_category`:`INSERT ... ON CONFLICT DO NOTHING` 幂等新增 `('美洲豹赛台', 'BILLIARD', '中式/追分')`
|
||||
2. `dws.dws_member_project_tag`:`DELETE WHERE time_window IN ('LAST_30_DAYS', 'LAST_60_DAYS')`
|
||||
- **影响对象**:1 条配置新增 + 旧窗口数据清理
|
||||
- **回滚 SQL**:恢复 DELETE 数据(需从备份) + DELETE 新增配置行
|
||||
- **兼容性**:无破坏性(纯数据操作 + 幂等配置)
|
||||
|
||||
## 风险与回滚
|
||||
|
||||
### 高风险项(需人工验证或测试)
|
||||
|
||||
1. **task_generator UPSERT + Step 5 清理**(`apps/backend/app/services/task_generator.py`):72h 宽限期 + 双任务替代策略 + 保底任务清理——业务逻辑复杂,需回归测试各种任务替代路径
|
||||
2. **recall_detector settle_type=3 双重限制**(`apps/backend/app/services/recall_detector.py`):`settle_type=3` 且 `order_assistant_type=2` 才计入;门店级召回自动解除(新增 resolved 状态)——需验证对历史数据的影响
|
||||
3. **fetchPerformanceRecords coach_id 参数权限**(`apps/backend/app/routers/xcx_performance.py` + `apps/miniprogram/miniprogram/services/api.ts`):管理者跨级查询需要 `view_board_coach` 权限——需 e2e 权限测试防越权
|
||||
4. **coach-service-records 新页面权限守卫**(`apps/miniprogram/miniprogram/pages/coach-service-records/*`):`onShow` 中 `checkPageAccess('view_board_coach')`,URL 参数 `coachId` 需后端二次验证防枚举
|
||||
5. **fdw_queries 消费汇总表口径统一**(`apps/backend/app/services/fdw_queries.py`):`get_last_visit_days` / `batch_query_for_task_list` 改用 `v_dws_member_consumption_summary` 替代 `v_dwd_assistant_service_log`——下游多调用方,需验证聚合无重复
|
||||
6. **isScattered 散客标记端到端**(跨 backend/schemas + 小程序多页):后端 `member_id ≤ 0` 自动标记——需验证所有调用方语义一致
|
||||
7. **task_engine 双模式分支**(`apps/etl/connectors/feiqiu/tasks/dws/task_engine.py`):HTTP 实时 + 推演回测双路径,需独立 `APP_DB_DSN` 连接业务库
|
||||
|
||||
### 中风险项
|
||||
|
||||
- `PerformanceMetrics` schema 扩展(6→25 字段):前端多页依赖,需字段可用性保证
|
||||
- `current_tier` 语义修正(tier_id 主键 → tier_nodes 数组下标 0-based):前端档位展示语义对应
|
||||
- 指数任务日快照模式:DELETE 策略从全量变 `(site_id, stat_date)`,回填任务需防重跑
|
||||
|
||||
### 回滚策略
|
||||
|
||||
- **迁移 SQL**:两份回滚脚本已内嵌(文件末尾注释块)
|
||||
- **后端代码**:git revert 对应文件即可(未 commit,`git checkout HEAD -- <file>` 可立即回退)
|
||||
- **小程序代码**:同上
|
||||
- **ETL 任务**:可禁用 `CORE_DIM_SYNC` / `DWS_INDEX_BACKFILL` / `DWS_TASK_SIMULATION` 任务的编排注册
|
||||
|
||||
## 验证
|
||||
|
||||
### SQL 验证(3 条起步)
|
||||
|
||||
```sql
|
||||
-- 验证 stat_date 已生效
|
||||
SELECT COUNT(DISTINCT stat_date) FROM dws.dws_member_winback_index WHERE site_id = 1;
|
||||
|
||||
-- 验证 member_project_tag 新窗口已写入
|
||||
SELECT DISTINCT time_window FROM dws.dws_member_project_tag;
|
||||
-- 应只返回 LAST_30_VISITS
|
||||
|
||||
-- 验证 core 维度表非空(CORE_DIM_SYNC 已跑过)
|
||||
SELECT COUNT(*) FROM core.dim_assistant;
|
||||
SELECT COUNT(*) FROM core.dim_member;
|
||||
```
|
||||
|
||||
### 功能验证
|
||||
|
||||
- **小程序**:登录 coach 账号打开 `task-list` 查看绩效卡;登录 admin 账号从 `board-coach` 点击助教卡进 `coach-service-records` 页
|
||||
- **后端**:`pytest apps/backend/tests/` 跑现有测试
|
||||
- **ETL**:`python -m cli.main --dry-run --tasks CORE_DIM_SYNC,DWS_MEMBER_PROJECT_TAG,WINBACK_INDEX,RELATION_INDEX,NEWCONV_INDEX`
|
||||
|
||||
## 合规检查
|
||||
|
||||
### 文档同步缺口(预扫描识别的 25 项)
|
||||
|
||||
**已有文档覆盖(免补齐)**:
|
||||
|
||||
- 美团 `mt_*` 字段 → 已在 `docs/audit/changes/2026-04-15__meituan-settle-core-sync.md` 审计覆盖 + `BD_manual_dwd_groupbuy_redemption.md` 已更新
|
||||
- `stat_date` 迁移 → `db/etl_feiqiu/migrations/` 已有完整 SQL,`docs/database/ddl/etl_feiqiu__dws.sql` 已同步
|
||||
- `member_project_tag` 新窗口 → `BD_manual_dws_member_project_tag.md` 已同步
|
||||
|
||||
**待补齐高风险项(建议后续单独任务处理,本次审计仅标注缺口,不强制补齐)**:
|
||||
|
||||
- `apps/backend/docs/API-REFERENCE.md`:需增补 `xcx/coaches/{id}/banner`、`xcx/performance` 的 `coach_id` 参数说明、`xcx/customers` 的 `foodDetail` / `phoneFull` / `notes.creator*` 字段
|
||||
- `apps/miniprogram/README.md`:需增补 `coach-service-records` 新页面说明
|
||||
- `apps/etl/connectors/feiqiu/docs/architecture/`:建议补充 `orchestration/task_registry.py` 新增任务(`CORE_DIM_SYNC` / `DWS_INDEX_BACKFILL` / `DWS_TASK_SIMULATION`)
|
||||
|
||||
### 状态清单
|
||||
|
||||
| 项 | 状态 |
|
||||
|---|---|
|
||||
| 迁移 SQL 回滚脚本 | 已内嵌 |
|
||||
| DDL 基线同步 | 已同步(dws/app 一致) |
|
||||
| 代码→文档映射(美团/stat_date/project_tag) | 已覆盖 |
|
||||
| 代码→文档映射(backend API / miniprogram 新页 / ETL 新任务) | 待补齐(已列入后续任务) |
|
||||
| Spec 与实现对应 | 全部可追溯 |
|
||||
|
||||
## 下一步建议
|
||||
|
||||
1. 提交本基线(本次审计后可作为 commit 起点)
|
||||
2. 后续新改动走标准 `/audit` 流程(单独审计、小颗粒度)
|
||||
3. 高风险 7 项应在 commit 前或上线前完成端到端测试
|
||||
4. 文档补齐列入待办,非阻断本次提交
|
||||
Reference in New Issue
Block a user