feat: P1-P3 全栈集成 — 数据库基础 + DWS 扩展 + 小程序鉴权 + 工程化体系
## P1 数据库基础 - zqyy_app: 创建 auth/biz schema、FDW 连接 etl_feiqiu - etl_feiqiu: 创建 app schema RLS 视图、商品库存预警表 - 清理 assistant_abolish 残留数据 ## P2 ETL/DWS 扩展 - 新增 DWS 助教订单贡献度表 (dws.assistant_order_contribution) - 新增 assistant_order_contribution_task 任务及 RLS 视图 - member_consumption 增加充值字段、assistant_daily 增加处罚字段 - 更新 ODS/DWD/DWS 任务文档及业务规则文档 - 更新 consistency_checker、flow_runner、task_registry 等核心模块 ## P3 小程序鉴权系统 - 新增 xcx_auth 路由/schema（微信登录 + JWT） - 新增 wechat/role/matching/application 服务层 - zqyy_app 鉴权表迁移 + 角色权限种子数据 - auth/dependencies.py 支持小程序 JWT 鉴权 ## 文档与审计 - 新增 DOCUMENTATION-MAP 文档导航 - 新增 7 份 BD_Manual 数据库变更文档 - 更新 DDL 基线快照（etl_feiqiu 6 schema + zqyy_app auth） - 新增全栈集成审计记录、部署检查清单更新 - 新增 BACKLOG 路线图、FDW→Core 迁移计划 ## Kiro 工程化 - 新增 5 个 Spec（P1/P2/P3/全栈集成/核心业务） - 新增审计自动化脚本（agent_on_stop/build_audit_context/compliance_prescan） - 新增 6 个 Hook（合规检查/会话日志/提交审计等） - 新增 doc-map steering 文件 ## 运维与测试 - 新增 ops 脚本：迁移验证/API 健康检查/ETL 监控/集成报告 - 新增属性测试：test_dws_contribution / test_auth_system - 清理过期 export 报告文件 - 更新 .gitignore 排除规则
This commit is contained in:
266
docs/database/BD_Manual_dws_assistant_order_contribution.md
Normal file
266
docs/database/BD_Manual_dws_assistant_order_contribution.md
Normal file
@@ -0,0 +1,266 @@
|
||||
# BD_Manual：dws_assistant_order_contribution（助教订单流水四项统计）
|
||||
|
||||
> DWS 表：`dws.dws_assistant_order_contribution`
|
||||
> DWD 数据源：`dwd.dwd_settlement_head`（结算主表）、`dwd.dwd_table_fee_log`（台费明细）、`dwd.dwd_assistant_service_log`（助教服务记录）
|
||||
> 任务代码：`DWS_ASSISTANT_ORDER_CONTRIBUTION`
|
||||
> 代码位置：`apps/etl/connectors/feiqiu/tasks/dws/assistant_order_contribution_task.py`
|
||||
> DDL 位置：`docs/database/ddl/etl_feiqiu__dws.sql`
|
||||
> 迁移脚本：`db/etl_feiqiu/migrations/2025-02-24__create_dws_assistant_order_contribution.sql`
|
||||
> RLS 视图：`db/etl_feiqiu/migrations/2025-02-24__create_rls_view_assistant_order_contribution.sql`
|
||||
> FDW 映射：`db/zqyy_app/migrations/2025-02-24__add_fdw_dws_extensions.sql`
|
||||
|
||||
---
|
||||
|
||||
## 1. 表结构
|
||||
|
||||
| 列名 | 类型 | 默认值 | 业务含义 | 取值范围/示例 |
|
||||
|------|------|--------|---------|-------------|
|
||||
| `contribution_id` | BIGINT (SERIAL) | nextval 序列 | 自增主键（PK） | 自增 |
|
||||
| `site_id` | INTEGER NOT NULL | — | 门店 ID | 飞球门店 ID |
|
||||
| `tenant_id` | INTEGER NOT NULL | — | 租户 ID | 飞球租户 ID |
|
||||
| `assistant_id` | BIGINT NOT NULL | — | 助教 ID | 飞球助教 ID |
|
||||
| `assistant_nickname` | VARCHAR(100) | NULL | 助教昵称 | 中文昵称 |
|
||||
| `stat_date` | DATE NOT NULL | — | 统计日期 | `2025-01-15` |
|
||||
| `order_gross_revenue` | NUMERIC(14,2) | 0 | 订单总流水 = 台费 + 酒水食品 + 所有助教服务费 | `0.00` ~ 金额值 |
|
||||
| `order_net_revenue` | NUMERIC(14,2) | 0 | 订单净流水 = 订单总流水 - 所有助教服务分成 | `0.00` ~ 金额值 |
|
||||
| `time_weighted_revenue` | NUMERIC(14,2) | 0 | 时效贡献流水 = 台费按时长分摊 + 个人服务费 + 酒水食品按时长比例 | `0.00` ~ 金额值 |
|
||||
| `time_weighted_net_revenue` | NUMERIC(14,2) | 0 | 时效净贡献 = 时效贡献流水 - 个人服务分成 | `0.00` ~ 金额值 |
|
||||
| `order_count` | INTEGER | 0 | 当日参与订单数 | `0` ~ 正整数 |
|
||||
| `total_service_seconds` | INTEGER | 0 | 当日总服务时长（秒） | `0` ~ 正整数 |
|
||||
| `created_at` | TIMESTAMPTZ | NOW() | 记录创建时间 | ISO 时间戳 |
|
||||
| `updated_at` | TIMESTAMPTZ | NOW() | 记录最后更新时间 | ISO 时间戳 |
|
||||
|
||||
---
|
||||
|
||||
## 2. 主键与索引
|
||||
|
||||
| 名称 | 类型 | 列 | 说明 |
|
||||
|------|------|----|------|
|
||||
| `dws_assistant_order_contribution_pkey` | PRIMARY KEY | `contribution_id` | 物理主键（自增序列） |
|
||||
| `idx_aoc_site_assistant_date` | UNIQUE INDEX | `(site_id, assistant_id, stat_date)` | 业务主键：每个门店每个助教每天唯一一条记录 |
|
||||
| `idx_aoc_stat_date` | INDEX | `(site_id, stat_date)` | 按门店+日期查询，支持日度报表 |
|
||||
|
||||
---
|
||||
|
||||
## 3. 数据写入策略
|
||||
|
||||
- **delete-before-insert**：每次执行按 `site_id` + 日期窗口全量刷新
|
||||
1. `DELETE FROM dws.dws_assistant_order_contribution WHERE site_id = %s AND stat_date BETWEEN %s AND %s`
|
||||
2. 批量 `INSERT` 新计算结果
|
||||
- 继承 `BaseDwsTask` 默认 load 实现，幂等可重跑
|
||||
|
||||
---
|
||||
|
||||
## 4. 算法概要
|
||||
|
||||
### 4.1 数据来源
|
||||
|
||||
| 来源表 | 筛选条件 | 提取内容 |
|
||||
|--------|---------|---------|
|
||||
| `dwd.dwd_settlement_head` | 日期窗口内，`settle_type IN (1, 3)` | 结算单信息、酒水食品金额 |
|
||||
| `dwd.dwd_table_fee_log` | 关联结算单 | 台桌使用时长、台费金额、区域 |
|
||||
| `dwd.dwd_assistant_service_log` | 关联结算单 | 助教服务时长、服务流水、分成、课程类型 |
|
||||
|
||||
### 4.2 四项统计公式
|
||||
|
||||
**订单总流水（order_gross_revenue）**
|
||||
```
|
||||
order_gross_revenue = total_table_fee + total_goods_amount + SUM(所有助教 ledger_amount)
|
||||
```
|
||||
每个参与助教获得相同值。
|
||||
|
||||
**订单净流水（order_net_revenue）**
|
||||
```
|
||||
order_net_revenue = order_gross_revenue - SUM(所有助教 commission)
|
||||
```
|
||||
每个参与助教获得相同值。
|
||||
|
||||
**时效贡献流水（time_weighted_revenue）**
|
||||
```
|
||||
对于台桌 t：
|
||||
billable_seconds = MAX(SUM(助教服务时长), 台桌使用时长)
|
||||
台费分摊_a = table_fee_t × (service_seconds_a / billable_seconds)
|
||||
|
||||
酒水食品分摊_a = total_goods_amount × (助教 a 总服务时长 / 所有助教总服务时长)
|
||||
|
||||
time_weighted_revenue_a = SUM(各台桌台费分摊_a) + ledger_amount_a + 酒水食品分摊_a
|
||||
```
|
||||
|
||||
**时效净贡献（time_weighted_net_revenue）**
|
||||
```
|
||||
time_weighted_net_revenue_a = time_weighted_revenue_a - commission_a
|
||||
```
|
||||
|
||||
### 4.3 超休/打赏课特殊处理
|
||||
|
||||
当 `course_type = BONUS` 时，四项统计均等于个人服务流水和分成，不参与订单级分摊。
|
||||
|
||||
---
|
||||
|
||||
## 5. 前置依赖
|
||||
|
||||
- 任务依赖：`DWD_LOAD_FROM_ODS`（需先完成 DWD 层数据加载）
|
||||
- 数据源表：`dwd.dwd_settlement_head`、`dwd.dwd_table_fee_log`、`dwd.dwd_assistant_service_log` 必须已有数据
|
||||
|
||||
---
|
||||
|
||||
## 6. 验证 SQL
|
||||
|
||||
### 6.1 检查表是否存在且有数据
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
COUNT(*) AS total_rows,
|
||||
COUNT(DISTINCT site_id) AS site_count,
|
||||
COUNT(DISTINCT assistant_id) AS assistant_count,
|
||||
MIN(stat_date) AS earliest_date,
|
||||
MAX(stat_date) AS latest_date
|
||||
FROM dws.dws_assistant_order_contribution;
|
||||
```
|
||||
|
||||
### 6.2 检查业务主键唯一性（不应有重复）
|
||||
|
||||
```sql
|
||||
SELECT site_id, assistant_id, stat_date, COUNT(*) AS cnt
|
||||
FROM dws.dws_assistant_order_contribution
|
||||
GROUP BY site_id, assistant_id, stat_date
|
||||
HAVING COUNT(*) > 1;
|
||||
-- 预期：无结果返回
|
||||
```
|
||||
|
||||
### 6.3 检查四项统计数值合理性（非负）
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
COUNT(*) FILTER (WHERE order_gross_revenue < 0) AS neg_gross,
|
||||
COUNT(*) FILTER (WHERE order_net_revenue < 0) AS neg_net,
|
||||
COUNT(*) FILTER (WHERE time_weighted_revenue < 0) AS neg_twr,
|
||||
COUNT(*) FILTER (WHERE time_weighted_net_revenue < 0) AS neg_twnr
|
||||
FROM dws.dws_assistant_order_contribution;
|
||||
-- 预期：所有列均为 0
|
||||
```
|
||||
|
||||
### 6.4 按门店查看统计概况
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
site_id,
|
||||
COUNT(*) AS record_count,
|
||||
SUM(order_count) AS total_orders,
|
||||
ROUND(AVG(order_gross_revenue), 2) AS avg_gross,
|
||||
ROUND(AVG(order_net_revenue), 2) AS avg_net,
|
||||
ROUND(AVG(time_weighted_revenue), 2) AS avg_twr,
|
||||
ROUND(AVG(time_weighted_net_revenue), 2) AS avg_twnr
|
||||
FROM dws.dws_assistant_order_contribution
|
||||
GROUP BY site_id
|
||||
ORDER BY site_id;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. RLS 视图与 FDW 映射
|
||||
|
||||
### 7.1 RLS 视图（ETL 库 app schema）
|
||||
|
||||
```sql
|
||||
-- 视图名：app.v_dws_assistant_order_contribution
|
||||
CREATE OR REPLACE VIEW app.v_dws_assistant_order_contribution AS
|
||||
SELECT * FROM dws.dws_assistant_order_contribution
|
||||
WHERE site_id = current_setting('app.current_site_id')::bigint;
|
||||
```
|
||||
|
||||
### 7.2 FDW 外部表（业务库 fdw_etl schema）
|
||||
|
||||
```sql
|
||||
-- 外部表名：fdw_etl.v_dws_assistant_order_contribution
|
||||
-- 通过 app schema RLS 视图访问，非直接访问 dws schema
|
||||
CREATE FOREIGN TABLE fdw_etl.v_dws_assistant_order_contribution (...)
|
||||
SERVER etl_server
|
||||
OPTIONS (schema_name 'app', table_name 'v_dws_assistant_order_contribution');
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. 兼容性说明
|
||||
|
||||
| 影响范围 | 说明 |
|
||||
|---------|------|
|
||||
| ETL 任务 | 新增任务 `DWS_ASSISTANT_ORDER_CONTRIBUTION`，依赖 `DWD_LOAD_FROM_ODS`。不影响现有 DWS 任务 |
|
||||
| 后端 API | 当前无 API 直接读取此表。后续小程序助教看板需新增接口 |
|
||||
| 管理后台 | 当前无前端页面展示。后续可在助教详情页新增流水统计展示 |
|
||||
| 小程序 | 小程序助教端将通过后端 API 读取此表数据展示四项统计 |
|
||||
| 其他 DWS 表 | 独立于现有 `dws_assistant_daily_detail`，不修改任何已有表或任务逻辑 |
|
||||
|
||||
---
|
||||
|
||||
## 9. 回滚策略
|
||||
|
||||
### 9.1 删除数据（保留表结构）
|
||||
|
||||
```sql
|
||||
DELETE FROM dws.dws_assistant_order_contribution;
|
||||
```
|
||||
|
||||
### 9.2 完整回滚（删除表 + 视图 + FDW）
|
||||
|
||||
```sql
|
||||
-- 1. 删除 FDW 外部表（业务库）
|
||||
DROP FOREIGN TABLE IF EXISTS fdw_etl.v_dws_assistant_order_contribution;
|
||||
|
||||
-- 2. 删除 RLS 视图（ETL 库）
|
||||
DROP VIEW IF EXISTS app.v_dws_assistant_order_contribution;
|
||||
|
||||
-- 3. 删除表和索引（ETL 库）
|
||||
DROP INDEX IF EXISTS dws.idx_aoc_stat_date;
|
||||
DROP INDEX IF EXISTS dws.idx_aoc_site_assistant_date;
|
||||
DROP TABLE IF EXISTS dws.dws_assistant_order_contribution;
|
||||
```
|
||||
|
||||
### 9.3 回滚任务注册
|
||||
|
||||
从 `orchestration/task_registry.py` 中移除 `DWS_ASSISTANT_ORDER_CONTRIBUTION` 注册行，并从 `tasks/dws/__init__.py` 中移除 `AssistantOrderContributionTask` 导出。
|
||||
|
||||
---
|
||||
|
||||
## 10. 代码引用
|
||||
|
||||
- 任务类：`tasks/dws/assistant_order_contribution_task.py` → `AssistantOrderContributionTask`
|
||||
- 数据结构：`TableUsage`、`AssistantService`、`OrderData`（同文件）
|
||||
- 继承：`BaseDwsTask`
|
||||
- 任务注册：`orchestration/task_registry.py` → `DWS_ASSISTANT_ORDER_CONTRIBUTION`
|
||||
- 属性测试：`tests/test_dws_contribution_properties.py`
|
||||
- 迁移脚本：`db/etl_feiqiu/migrations/2025-02-24__create_dws_assistant_order_contribution.sql`
|
||||
- RLS 视图：`db/etl_feiqiu/migrations/2025-02-24__create_rls_view_assistant_order_contribution.sql`
|
||||
- FDW 映射：`db/zqyy_app/migrations/2025-02-24__add_fdw_dws_extensions.sql`
|
||||
- 验证脚本：`apps/etl/connectors/feiqiu/scripts/verify_dws_extensions.py`
|
||||
|
||||
---
|
||||
|
||||
## 11. 关联扩展字段说明
|
||||
|
||||
本次 Spec（02-etl-dws-miniapp-extensions）同时扩展了两张已有表的字段，简要说明如下：
|
||||
|
||||
### 11.1 dws_member_consumption_summary 新增字段
|
||||
|
||||
| 列名 | 类型 | 默认值 | 业务含义 |
|
||||
|------|------|--------|---------|
|
||||
| `recharge_count_30d` | INTEGER | 0 | 近 30 天充值次数 |
|
||||
| `recharge_count_60d` | INTEGER | 0 | 近 60 天充值次数 |
|
||||
| `recharge_count_90d` | INTEGER | 0 | 近 90 天充值次数 |
|
||||
| `recharge_amount_30d` | NUMERIC(14,2) | 0 | 近 30 天充值金额 |
|
||||
| `recharge_amount_60d` | NUMERIC(14,2) | 0 | 近 60 天充值金额 |
|
||||
| `recharge_amount_90d` | NUMERIC(14,2) | 0 | 近 90 天充值金额 |
|
||||
| `avg_ticket_amount` | NUMERIC(14,2) | 0 | 次均消费 = total_consume_amount / MAX(total_visit_count, 1) |
|
||||
|
||||
迁移脚本：`db/etl_feiqiu/migrations/2025-02-24__alter_member_consumption_add_recharge_fields.sql`
|
||||
|
||||
### 11.2 dws_assistant_daily_detail 新增字段
|
||||
|
||||
| 列名 | 类型 | 默认值 | 业务含义 |
|
||||
|------|------|--------|---------|
|
||||
| `penalty_minutes` | NUMERIC(10,2) | 0 | 定档折算惩罚分钟数，无惩罚时为 0 |
|
||||
| `penalty_reason` | TEXT | NULL | 惩罚原因描述，无惩罚时为 NULL |
|
||||
| `is_exempt` | BOOLEAN | FALSE | 是否豁免惩罚 |
|
||||
| `per_hour_contribution` | NUMERIC(14,2) | NULL | 单人每小时贡献流水 = 台费每小时单价 / 助教人数 |
|
||||
|
||||
迁移脚本：`db/etl_feiqiu/migrations/2025-02-24__alter_assistant_daily_add_penalty_fields.sql`
|
||||
Reference in New Issue
Block a user