在前后端开发联调前 的提交20260223
This commit is contained in:
106
docs/database/BD_Manual_assistant_accounts_master.md
Normal file
106
docs/database/BD_Manual_assistant_accounts_master.md
Normal file
@@ -0,0 +1,106 @@
|
||||
# BD_Manual:assistant_accounts_master(助教账号档案)
|
||||
|
||||
> ODS 表:`ods.assistant_accounts_master`
|
||||
> DWD 表:`dwd.dim_assistant`(主表)、`dwd.dim_assistant_ex`(扩展表)
|
||||
> API 接口:助教账号列表
|
||||
> JSON 路径:`assistant_accounts_master.json → data.assistantInfos`
|
||||
> 装载方式:SCD2 维度合并(`DwdLoadTask`)
|
||||
> 代码位置:`apps/etl/connectors/feiqiu/tasks/dwd/dwd_load_task.py`
|
||||
|
||||
---
|
||||
|
||||
## 1. dim_assistant(主表)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `assistant_id` | BIGINT | `id` | FACT_MAPPINGS | 助教唯一标识(PK 之一) | 飞球系统雪花 ID,如 `2947562271297029` |
|
||||
| `user_id` | BIGINT | `user_id` | FACT_MAPPINGS | 关联的系统用户 ID,0 表示未绑定用户账号 | `0` 或飞球用户 ID |
|
||||
| `assistant_no` | TEXT | `assistant_no` | 自动映射 | 助教编号(门店内序号),用于排班和展示 | 如 `31`、`1` |
|
||||
| `real_name` | TEXT | `real_name` | 自动映射 | 助教真实姓名 | 如 `张静然` |
|
||||
| `nickname` | TEXT | `nickname` | 自动映射 | 助教昵称,用于小程序端展示 | 如 `小然` |
|
||||
| `mobile` | TEXT | `mobile` | 自动映射 | 助教手机号 | 11 位手机号 |
|
||||
| `tenant_id` | BIGINT | `tenant_id` | 自动映射 | 所属租户 ID | 飞球租户 ID |
|
||||
| `site_id` | BIGINT | `site_id` | 自动映射 | 所属门店 ID | 飞球门店 ID |
|
||||
| `team_id` | BIGINT | `team_id` | 自动映射 | 所属团队 ID,0 表示未分组 | `0` 或团队 ID |
|
||||
| `team_name` | TEXT | `team_name` | 自动映射 | 团队名称 | 如 `1组`,NULL 表示未分组 |
|
||||
| `level` | INTEGER | `level` | 自动映射 | 助教等级(技能等级编号) | 如 `20`(对应"高级"等) |
|
||||
| `entry_time` | TIMESTAMPTZ | `entry_time` | 自动映射 | 入职时间 | ISO 时间戳 |
|
||||
| `resign_time` | TIMESTAMPTZ | `resign_time` | 自动映射 | 离职时间,NULL 表示在职 | ISO 时间戳或 NULL |
|
||||
| `leave_status` | INTEGER | `leave_status` | 自动映射 | 在职状态:0=在职,1=已离职 | `0` / `1` |
|
||||
| `assistant_status` | INTEGER | `assistant_status` | 自动映射 | 助教状态:1=正常,2=停用 | `1` / `2` |
|
||||
| `scd2_start_time` | TIMESTAMPTZ | — | DWD 元数据 | SCD2 版本生效起点 | — |
|
||||
| `scd2_end_time` | TIMESTAMPTZ | — | DWD 元数据 | SCD2 版本失效时间,`9999-12-31` 表示当前版本 | — |
|
||||
| `scd2_is_current` | INT | — | DWD 元数据 | 当前版本标记:1=当前,0=历史 | `0` / `1` |
|
||||
| `scd2_version` | INT | — | DWD 元数据 | SCD2 版本号(自增) | — |
|
||||
|
||||
---
|
||||
|
||||
## 2. dim_assistant_ex(扩展表)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `assistant_id` | BIGINT | `id` | FACT_MAPPINGS | 助教唯一标识(PK 之一) | 同主表 |
|
||||
| `gender` | INTEGER | `gender` | 自动映射 | 性别:0=未设置 | `0` |
|
||||
| `birth_date` | TIMESTAMPTZ | `birth_date` | 自动映射 | 出生日期 | `0001-01-01` 表示未设置 |
|
||||
| `avatar` | TEXT | `avatar` | 自动映射 | 头像 URL | HTTPS 链接 |
|
||||
| `introduce` | TEXT | `introduce` | FACT_MAPPINGS | 个人简介文本 | 自由文本或 NULL |
|
||||
| `video_introduction_url` | TEXT | `video_introduction_url` | 自动映射 | 视频介绍 URL | HTTPS 链接或 NULL |
|
||||
| `height` | NUMERIC(5,2) | `height` | 自动映射 | 身高(cm),0 表示未填写 | `0.00` 或实际身高 |
|
||||
| `weight` | NUMERIC(5,2) | `weight` | 自动映射 | 体重(kg),0 表示未填写 | `0.00` 或实际体重 |
|
||||
| `shop_name` | TEXT | `shop_name` | 自动映射 | 所属门店名称快照 | 如 `朗朗桌球` |
|
||||
| `group_id` | BIGINT | `group_id` | 自动映射 | 分组 ID,0 表示未分组 | `0` 或分组 ID |
|
||||
| `group_name` | TEXT | `group_name` | FACT_MAPPINGS | 分组名称 | NULL 或分组名 |
|
||||
| `person_org_id` | BIGINT | `person_org_id` | 自动映射 | 人事组织 ID | 飞球组织 ID |
|
||||
| `staff_id` | BIGINT | `staff_id` | 自动映射 | 员工 ID,0 表示未绑定员工档案 | `0` 或员工 ID |
|
||||
| `staff_profile_id` | BIGINT | `staff_profile_id` | 自动映射 | 员工档案 ID,0 表示无档案 | `0` 或档案 ID |
|
||||
| `assistant_grade` | DOUBLE PRECISION | `assistant_grade` | 自动映射 | 助教评分(客户评价均分) | `0.0` ~ `5.0` |
|
||||
| `sum_grade` | DOUBLE PRECISION | `sum_grade` | 自动映射 | 累计评分总和 | `0.0` 或累计值 |
|
||||
| `get_grade_times` | INTEGER | `get_grade_times` | 自动映射 | 被评价次数 | `0` 或正整数 |
|
||||
| `charge_way` | INTEGER | `charge_way` | 自动映射 | 计费方式:2=按时长计费(当前门店全部为 2) | `2` |
|
||||
| `allow_cx` | INTEGER | `allow_cx` | 自动映射 | 是否允许促销服务:1=允许(当前全部为 1) | `1` |
|
||||
| `is_guaranteed` | INTEGER | `is_guaranteed` | 自动映射 | 是否保底:0=不保底,1=保底 | `0` / `1` |
|
||||
| `salary_grant_enabled` | INTEGER | `salary_grant_enabled` | 自动映射 | 工资发放开关:2=启用 | `2` |
|
||||
| `entry_type` | INTEGER | `entry_type` | 自动映射 | 入职类型:1=正常入职 | `1` |
|
||||
| `entry_sign_status` | INTEGER | `entry_sign_status` | 自动映射 | 入职签到状态:0=未签到 | `0` |
|
||||
| `resign_sign_status` | INTEGER | `resign_sign_status` | 自动映射 | 离职签到状态:0=未签到 | `0` |
|
||||
| `work_status` | INTEGER | `work_status` | 自动映射 | 工作状态:1=在岗,2=离岗 | `1` / `2` |
|
||||
| `show_status` | INTEGER | `show_status` | 自动映射 | 展示状态:1=展示 | `1` |
|
||||
| `show_sort` | INTEGER | `show_sort` | 自动映射 | 展示排序序号 | 正整数 |
|
||||
| `online_status` | INTEGER | `online_status` | 自动映射 | 在线状态:1=在线 | `1` |
|
||||
| `is_delete` | INTEGER | `is_delete` | 自动映射 | 软删除标记:0=正常,1=已删除 | `0` / `1` |
|
||||
| `criticism_status` | INTEGER | `criticism_status` | 自动映射 | 差评处理状态:1=正常 | `1` |
|
||||
| `create_time` | TIMESTAMPTZ | `create_time` | 自动映射 | 助教记录创建时间 | ISO 时间戳 |
|
||||
| `update_time` | TIMESTAMPTZ | `update_time` | 自动映射 | 助教记录最后更新时间 | ISO 时间戳 |
|
||||
| `start_time` | TIMESTAMPTZ | `start_time` | 自动映射 | 合同/排班开始时间 | ISO 时间戳 |
|
||||
| `end_time` | TIMESTAMPTZ | `end_time` | 自动映射 | 合同/排班结束时间 | ISO 时间戳 |
|
||||
| `last_table_id` | BIGINT | `last_table_id` | 自动映射 | 最近服务的台桌 ID,0 表示无 | `0` 或台桌 ID |
|
||||
| `last_table_name` | TEXT | `last_table_name` | 自动映射 | 最近服务的台桌名称 | 如 `TV`、`1号台` |
|
||||
| `last_update_name` | TEXT | `last_update_name` | 自动映射 | 最后操作人姓名(带职位前缀) | 如 `管理员:郑丽珊` |
|
||||
| `order_trade_no` | BIGINT | `order_trade_no` | 自动映射 | 当前关联的订单交易号,0 表示空闲 | `0` 或订单号 |
|
||||
| `ding_talk_synced` | INTEGER | `ding_talk_synced` | 自动映射 | 钉钉同步状态:1=已同步 | `1` |
|
||||
| `site_light_cfg_id` | BIGINT | `site_light_cfg_id` | 自动映射 | 门店灯控配置 ID,0 表示未配置 | `0` 或配置 ID |
|
||||
| `light_equipment_id` | TEXT | `light_equipment_id` | FACT_MAPPINGS | 灯控设备 ID | NULL 或设备编号 |
|
||||
| `light_status` | INTEGER | `light_status` | 自动映射 | 台灯状态:2=已开灯 | `1` / `2` |
|
||||
| `is_team_leader` | INTEGER | `is_team_leader` | 自动映射 | 是否组长:0=否,1=是 | `0` / `1` |
|
||||
| `serial_number` | BIGINT | `serial_number` | 自动映射 | 序列号,0 表示未分配 | `0` 或序列号 |
|
||||
| `system_role_id` | BIGINT | `system_role_id` | FACT_MAPPINGS | 系统角色 ID,标识助教在系统中的角色类型。当前门店全部为 10 | `10` |
|
||||
| `job_num` | TEXT | `job_num` | FACT_MAPPINGS | 工号,助教的内部编号标识。当前门店未启用,全部为 NULL | NULL |
|
||||
| `cx_unit_price` | NUMERIC(18,2) | `cx_unit_price` | FACT_MAPPINGS | 促销单价(元),助教提供促销服务时的计费单价。当前门店未在账号表层面设置,全部为 0.00 | `0.00` |
|
||||
| `pd_unit_price` | NUMERIC(18,2) | `pd_unit_price` | FACT_MAPPINGS | 陪打单价(元),助教提供陪打服务时的计费单价。当前门店未在账号表层面设置,全部为 0.00 | `0.00` |
|
||||
| `scd2_*` | — | — | DWD 元数据 | SCD2 慢变维度追踪字段(同主表) | — |
|
||||
|
||||
---
|
||||
|
||||
## 3. 跳过字段说明
|
||||
|
||||
| ODS 字段 | 跳过原因 |
|
||||
|---------|---------|
|
||||
| `siteprofile` | JSONB 嵌套列,已由 `dim_site` / `dim_site_ex` 通过 JSONB 提取映射 |
|
||||
|
||||
---
|
||||
|
||||
## 4. 代码引用
|
||||
|
||||
- FACT_MAPPINGS:`dwd_load_task.py` → `FACT_MAPPINGS["dwd.dim_assistant"]` / `FACT_MAPPINGS["dwd.dim_assistant_ex"]`
|
||||
- TABLE_MAP:`"dwd.dim_assistant" → "ods.assistant_accounts_master"`
|
||||
- DWS 下游:`dws_assistant_daily_task.py`(助教日业绩汇总)、`dws_salary_task.py`(工资计算)
|
||||
113
docs/database/BD_Manual_assistant_service_records.md
Normal file
113
docs/database/BD_Manual_assistant_service_records.md
Normal file
@@ -0,0 +1,113 @@
|
||||
# BD_Manual:assistant_service_records(助教服务流水)
|
||||
|
||||
> ODS 表:`ods.assistant_service_records`
|
||||
> DWD 表:`dwd.dwd_assistant_service_log`(主表)、`dwd.dwd_assistant_service_log_ex`(扩展表)
|
||||
> API 接口:助教服务记录列表
|
||||
> JSON 路径:`assistant_service_records.json → data.orderAssistantLedgers`
|
||||
> 装载方式:事实表增量插入(`DwdLoadTask`)
|
||||
> 代码位置:`apps/etl/connectors/feiqiu/tasks/dwd/dwd_load_task.py`
|
||||
|
||||
---
|
||||
|
||||
## 1. dwd_assistant_service_log(主表,33 列)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `assistant_service_id` | BIGINT | `id` | FACT_MAPPINGS | 助教服务记录唯一标识(PK) | 飞球雪花 ID |
|
||||
| `order_trade_no` | BIGINT | `order_trade_no` | 自动映射 | 关联的订单交易号 | 飞球订单号 |
|
||||
| `order_settle_id` | BIGINT | `order_settle_id` | 自动映射 | 关联的结算单 ID | 飞球结算单 ID |
|
||||
| `order_pay_id` | BIGINT | `order_pay_id` | 自动映射 | 关联的支付单 ID | 飞球支付单 ID |
|
||||
| `order_assistant_id` | BIGINT | `order_assistant_id` | 自动映射 | 订单级助教明细 ID,标识本次服务在订单中的唯一记录 | 飞球雪花 ID |
|
||||
| `order_assistant_type` | INTEGER | `order_assistant_type` | 自动映射 | 助教服务类型:1=陪打,2=促销 | `1` / `2` |
|
||||
| `tenant_id` | BIGINT | `tenant_id` | 自动映射 | 租户 ID | 飞球租户 ID |
|
||||
| `site_id` | BIGINT | `site_id` | 自动映射 | 门店 ID | 飞球门店 ID |
|
||||
| `site_table_id` | BIGINT | `site_table_id` | 自动映射 | 服务台桌 ID,关联 `dim_table.table_id` | 飞球台桌 ID |
|
||||
| `tenant_member_id` | BIGINT | `tenant_member_id` | 自动映射 | 租户维度会员 ID | 飞球会员 ID 或 0 |
|
||||
| `system_member_id` | BIGINT | `system_member_id` | 自动映射 | 系统维度会员 ID(跨租户唯一) | 飞球会员 ID 或 0 |
|
||||
| `assistant_no` | VARCHAR(64) | `assistantno` | FACT_MAPPINGS | 助教编号(门店内序号) | 如 `31` |
|
||||
| `nickname` | VARCHAR(64) | `nickname` | 自动映射 | 助教昵称 | 如 `小张` |
|
||||
| `site_assistant_id` | BIGINT | `site_assistant_id` | FACT_MAPPINGS | 门店维度助教档案 ID,关联 `dim_assistant.assistant_id`。⚠️ 已于 2026-02-20 修正映射源(原错误映射自 `order_assistant_id`) | 飞球助教 ID |
|
||||
| `user_id` | BIGINT | `user_id` | 自动映射 | 助教用户 ID | 飞球用户 ID |
|
||||
| `assistant_team_id` | BIGINT | `assistant_team_id` | 自动映射 | 助教团队 ID | 飞球团队 ID |
|
||||
| `person_org_id` | BIGINT | `person_org_id` | 自动映射 | 人员组织 ID | 飞球组织 ID |
|
||||
| `assistant_level` | INTEGER | `assistant_level` | 自动映射 | 助教等级编码 | 枚举值 |
|
||||
| `level_name` | VARCHAR(64) | `levelname` | FACT_MAPPINGS | 助教等级名称 | 如 `高级`、`中级` |
|
||||
| `skill_id` | BIGINT | `skill_id` | 自动映射 | 服务技能 ID | 飞球技能 ID |
|
||||
| `skill_name` | VARCHAR(64) | `skillname` | FACT_MAPPINGS | 服务技能名称 | 如 `陪打`、`促销` |
|
||||
| `ledger_unit_price` | NUMERIC(10,2) | `ledger_unit_price` | 自动映射 | 分账单价(元/小时) | 金额值 |
|
||||
| `ledger_amount` | NUMERIC(10,2) | `ledger_amount` | 自动映射 | 分账总金额(元) | 金额值 |
|
||||
| `projected_income` | NUMERIC(10,2) | `projected_income` | 自动映射 | 预计收入(元) | 金额值 |
|
||||
| `coupon_deduct_money` | NUMERIC(10,2) | `coupon_deduct_money` | 自动映射 | 优惠券抵扣金额(元) | `0.00` ~ 金额值 |
|
||||
| `income_seconds` | INTEGER | `income_seconds` | 自动映射 | 计费时长(秒) | 正整数 |
|
||||
| `real_use_seconds` | INTEGER | `real_use_seconds` | 自动映射 | 实际使用时长(秒) | 正整数 |
|
||||
| `add_clock` | INTEGER | `add_clock` | 自动映射 | 加钟次数 | `0` ~ 正整数 |
|
||||
| `create_time` | TIMESTAMPTZ | `create_time` | 自动映射 | 记录创建时间 | ISO 时间戳 |
|
||||
| `start_use_time` | TIMESTAMPTZ | `start_use_time` | 自动映射 | 服务开始时间 | ISO 时间戳 |
|
||||
| `last_use_time` | TIMESTAMPTZ | `last_use_time` | 自动映射 | 最后使用时间(服务结束时间) | ISO 时间戳 |
|
||||
| `is_delete` | INTEGER | `is_delete` | 自动映射 | 是否已删除:0=正常,1=已删除 | `0` / `1` |
|
||||
| `real_service_money` | NUMERIC(18,2) | `real_service_money` | FACT_MAPPINGS | 实际服务金额(元),扣除折扣后的实收 | `0.00` ~ 金额值 |
|
||||
|
||||
---
|
||||
|
||||
## 2. dwd_assistant_service_log_ex(扩展表,33 列)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `assistant_service_id` | BIGINT | `id` | FACT_MAPPINGS | 助教服务记录唯一标识(PK) | 同主表 |
|
||||
| `table_name` | VARCHAR(64) | `tablename` | FACT_MAPPINGS | 台桌名称快照 | 如 `1号台` |
|
||||
| `assistant_name` | VARCHAR(64) | `assistantname` | FACT_MAPPINGS | 助教姓名快照 | 如 `张静然` |
|
||||
| `ledger_name` | VARCHAR(128) | `ledger_name` | 自动映射 | 分账项名称 | 如 `陪打费` |
|
||||
| `ledger_group_name` | VARCHAR(128) | `ledger_group_name` | FACT_MAPPINGS | 分账组名称 | 分账组名或 NULL |
|
||||
| `ledger_count` | INTEGER | `ledger_count` | 自动映射 | 分账数量 | 正整数 |
|
||||
| `member_discount_amount` | NUMERIC(10,2) | `member_discount_amount` | 自动映射 | 会员折扣金额(元) | `0.00` ~ 金额值 |
|
||||
| `manual_discount_amount` | NUMERIC(10,2) | `manual_discount_amount` | 自动映射 | 手动折扣金额(元) | `0.00` ~ 金额值 |
|
||||
| `service_money` | NUMERIC(10,2) | `service_money` | 自动映射 | 服务原价金额(元) | 金额值 |
|
||||
| `returns_clock` | INTEGER | `returns_clock` | 自动映射 | 退钟次数 | `0` ~ 正整数 |
|
||||
| `ledger_start_time` | TIMESTAMPTZ | `ledger_start_time` | 自动映射 | 分账开始时间 | ISO 时间戳 |
|
||||
| `ledger_end_time` | TIMESTAMPTZ | `ledger_end_time` | 自动映射 | 分账结束时间 | ISO 时间戳 |
|
||||
| `ledger_status` | INTEGER | `ledger_status` | 自动映射 | 分账状态 | 枚举值 |
|
||||
| `is_confirm` | INTEGER | `is_confirm` | 自动映射 | 是否已确认:0=未确认,1=已确认 | `0` / `1` |
|
||||
| `is_single_order` | INTEGER | `is_single_order` | 自动映射 | 是否单独订单 | `0` / `1` |
|
||||
| `is_not_responding` | INTEGER | `is_not_responding` | 自动映射 | 是否未响应 | `0` / `1` |
|
||||
| `is_trash` | INTEGER | `is_trash` | 自动映射 | 是否已废除:0=正常,1=已废除 | `0` / `1` |
|
||||
| `trash_applicant_id` | BIGINT | `trash_applicant_id` | 自动映射 | 废除申请人 ID | 员工 ID 或 NULL |
|
||||
| `trash_applicant_name` | VARCHAR(64) | `trash_applicant_name` | FACT_MAPPINGS | 废除申请人姓名 | 姓名或 NULL |
|
||||
| `trash_reason` | VARCHAR(255) | `trash_reason` | FACT_MAPPINGS | 废除原因 | 自由文本或 NULL |
|
||||
| `salesman_user_id` | BIGINT | `salesman_user_id` | 自动映射 | 销售员用户 ID | 用户 ID 或 NULL |
|
||||
| `salesman_name` | VARCHAR(64) | `salesman_name` | FACT_MAPPINGS | 销售员姓名 | 姓名或 NULL |
|
||||
| `salesman_org_id` | BIGINT | `salesman_org_id` | 自动映射 | 销售员组织 ID | 组织 ID 或 NULL |
|
||||
| `skill_grade` | INTEGER | `skill_grade` | 自动映射 | 技能评分 | 评分值 |
|
||||
| `service_grade` | INTEGER | `service_grade` | 自动映射 | 服务评分 | 评分值 |
|
||||
| `composite_grade` | NUMERIC(5,2) | `composite_grade` | 自动映射 | 综合评分 | 如 `4.50` |
|
||||
| `sum_grade` | NUMERIC(10,2) | `sum_grade` | 自动映射 | 累计评分 | 累计值 |
|
||||
| `get_grade_times` | INTEGER | `get_grade_times` | 自动映射 | 获评次数 | 正整数 |
|
||||
| `grade_status` | INTEGER | `grade_status` | 自动映射 | 评分状态 | 枚举值 |
|
||||
| `composite_grade_time` | TIMESTAMPTZ | `composite_grade_time` | 自动映射 | 综合评分时间 | ISO 时间戳 |
|
||||
| `assistant_team_name` | TEXT | `assistantteamname` | FACT_MAPPINGS | 助教团队名称快照 | 如 `1组` |
|
||||
| `operator_id` | BIGINT | `operator_id` | FACT_MAPPINGS | 操作员 ID,录入/结算这条助教服务的员工。0 表示系统自动 | `0` 或员工 ID |
|
||||
| `operator_name` | TEXT | `operator_name` | FACT_MAPPINGS | 操作员姓名(带职位前缀),便于直接阅读 | 如 `收银员:郑丽珊`,NULL 表示系统自动 |
|
||||
|
||||
---
|
||||
|
||||
## 3. 映射修正记录
|
||||
|
||||
| 日期 | 字段 | 修正内容 |
|
||||
|------|------|---------|
|
||||
| 2026-02-20 | `site_assistant_id` | ODS 源从 `order_assistant_id`(订单级 ID)修正为 `site_assistant_id`(助教档案 ID) |
|
||||
|
||||
---
|
||||
|
||||
## 4. 跳过字段说明
|
||||
|
||||
| ODS 字段 | 跳过原因 |
|
||||
|---------|---------|
|
||||
| `siteprofile` | JSONB 嵌套列,已由 `dim_site` / `dim_site_ex` 通过 JSONB 提取映射 |
|
||||
|
||||
---
|
||||
|
||||
## 5. 代码引用
|
||||
|
||||
- FACT_MAPPINGS:`dwd_load_task.py` → `FACT_MAPPINGS["dwd.dwd_assistant_service_log"]` / `FACT_MAPPINGS["dwd.dwd_assistant_service_log_ex"]`
|
||||
- TABLE_MAP:`"dwd.dwd_assistant_service_log" → "ods.assistant_service_records"`
|
||||
- DWS 下游:`dws_assistant_daily_task.py`(助教日业绩汇总)
|
||||
- 迁移脚本:`db/_archived/ddl_baseline_2026-02-22/db/etl_feiqiu/migrations/2026-02-20__fix_assistant_service_site_assistant_id.sql`(已归档)、`db/_archived/ddl_baseline_2026-02-22/db/etl_feiqiu/migrations/2026-02-20__add_assistant_service_log_ex_fields.sql`(已归档)
|
||||
80
docs/database/BD_Manual_dws_goods_stock_summary.md
Normal file
80
docs/database/BD_Manual_dws_goods_stock_summary.md
Normal file
@@ -0,0 +1,80 @@
|
||||
# BD_Manual:DWS 库存汇总(日/周/月)
|
||||
|
||||
> DWS 表:`dws.dws_goods_stock_daily_summary`、`dws.dws_goods_stock_weekly_summary`、`dws.dws_goods_stock_monthly_summary`
|
||||
> DWD 数据源:`dwd.dwd_goods_stock_summary`
|
||||
> 任务代码:`DWS_GOODS_STOCK_DAILY`、`DWS_GOODS_STOCK_WEEKLY`、`DWS_GOODS_STOCK_MONTHLY`
|
||||
> 代码位置:`apps/etl/connectors/feiqiu/tasks/dws/goods_stock_daily_task.py`、`goods_stock_weekly_task.py`、`goods_stock_monthly_task.py`
|
||||
> DDL 位置:`docs/database/ddl/etl_feiqiu__dws.sql`
|
||||
> 迁移脚本:`db/_archived/ddl_baseline_2026-02-22/db/etl_feiqiu/migrations/2026-02-20__create_dws_goods_stock_summary.sql`(已归档)
|
||||
|
||||
---
|
||||
|
||||
## 1. 表结构(三张表结构相同)
|
||||
|
||||
| DWS 列名 | 类型 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|-------------|
|
||||
| `site_id` | BIGINT | 门店 ID(PK 之一) | 飞球门店 ID |
|
||||
| `tenant_id` | BIGINT | 租户 ID | 飞球租户 ID |
|
||||
| `stat_date` | DATE | 统计日期(PK 之一)。日度=当天日期,周度=ISO 周一日期,月度=月首日期 | 如 `2026-01-15`、`2026-01-13`(周一)、`2026-01-01`(月首) |
|
||||
| `site_goods_id` | BIGINT | 门店商品 ID(PK 之一),关联 `dim_store_goods.site_goods_id` | 飞球商品 ID |
|
||||
| `goods_name` | TEXT | 商品名称 | 如 `百威啤酒` |
|
||||
| `goods_unit` | TEXT | 计量单位 | 如 `瓶`、`包` |
|
||||
| `goods_category_id` | BIGINT | 一级商品分类 ID | 飞球分类 ID |
|
||||
| `goods_category_second_id` | BIGINT | 二级商品分类 ID | 飞球分类 ID |
|
||||
| `category_name` | TEXT | 一级分类名称 | 如 `酒水` |
|
||||
| `range_start_stock` | NUMERIC | 期初库存(统计周期起点的库存量) | 数值 |
|
||||
| `range_end_stock` | NUMERIC | 期末库存(统计周期终点的库存量) | 数值 |
|
||||
| `range_in` | NUMERIC | 入库数量(统计周期内的采购/调拨入库总量) | 数值 |
|
||||
| `range_out` | NUMERIC | 出库数量(统计周期内的调拨出库/报损总量) | 数值 |
|
||||
| `range_sale` | NUMERIC | 销售数量(统计周期内的销售出库总量) | 数值 |
|
||||
| `range_sale_money` | NUMERIC(12,2) | 销售金额(元),统计周期内的销售总金额 | 金额值 |
|
||||
| `range_inventory` | NUMERIC | 盘点调整量(统计周期内的盘盈/盘亏净量) | 正/负数值 |
|
||||
| `current_stock` | NUMERIC | 当前库存(统计周期末的实时库存量) | 数值 |
|
||||
| `stat_period` | TEXT | 汇总粒度标识 | `'daily'` / `'weekly'` / `'monthly'` |
|
||||
| `created_at` | TIMESTAMPTZ | 记录创建时间 | 自动填充 `now()` |
|
||||
| `updated_at` | TIMESTAMPTZ | 记录最后更新时间 | 自动填充 `now()` |
|
||||
|
||||
---
|
||||
|
||||
## 2. 主键
|
||||
|
||||
`(site_id, stat_date, site_goods_id)` — 按门店、日期、商品维度唯一。
|
||||
|
||||
---
|
||||
|
||||
## 3. 粒度说明
|
||||
|
||||
| 表名 | 粒度 | stat_date 规则 | stat_period |
|
||||
|------|------|---------------|-------------|
|
||||
| `dws_goods_stock_daily_summary` | 日 | 当天日期 | `'daily'` |
|
||||
| `dws_goods_stock_weekly_summary` | 周 | ISO 周一日期 | `'weekly'` |
|
||||
| `dws_goods_stock_monthly_summary` | 月 | 月首日期(如 `2026-01-01`) | `'monthly'` |
|
||||
|
||||
---
|
||||
|
||||
## 4. 聚合逻辑
|
||||
|
||||
- extract:从 `dwd.dwd_goods_stock_summary` 按时间范围查询
|
||||
- transform:按粒度对 `fetched_at` 进行分组聚合
|
||||
- `range_start_stock`:取周期内最早记录的 `range_start_stock`
|
||||
- `range_end_stock`:取周期内最晚记录的 `range_end_stock`
|
||||
- `range_in` / `range_out` / `range_sale` / `range_inventory`:SUM 汇总
|
||||
- `range_sale_money`:SUM 汇总
|
||||
- `current_stock`:取周期内最晚记录的 `current_stock`
|
||||
- `goods_name` / `goods_unit` / `category_name`:取最晚记录的值
|
||||
- load:upsert 写入目标表,主键冲突时更新
|
||||
|
||||
---
|
||||
|
||||
## 5. 前置依赖
|
||||
|
||||
- `dwd.dwd_goods_stock_summary` 表必须已创建并有数据
|
||||
- ODS 任务配置 `requires_window=True` 必须已生效并重新采集
|
||||
|
||||
---
|
||||
|
||||
## 6. 代码引用
|
||||
|
||||
- 任务代码:`tasks/dws/goods_stock_daily_task.py`、`goods_stock_weekly_task.py`、`goods_stock_monthly_task.py`
|
||||
- 继承:`BaseDwsTask`(`tasks/dws/base_dws_task.py`)
|
||||
- 任务注册:`DWS_GOODS_STOCK_DAILY`、`DWS_GOODS_STOCK_WEEKLY`、`DWS_GOODS_STOCK_MONTHLY`
|
||||
62
docs/database/BD_Manual_goods_stock_movements.md
Normal file
62
docs/database/BD_Manual_goods_stock_movements.md
Normal file
@@ -0,0 +1,62 @@
|
||||
# BD_Manual:goods_stock_movements(库存变动流水)
|
||||
|
||||
> ODS 表:`ods.goods_stock_movements`
|
||||
> DWD 表:`dwd.dwd_goods_stock_movement`(事实表,无 ex 扩展表)
|
||||
> API 接口:库存变动记录列表
|
||||
> JSON 路径:`goods_stock_movements.json → data.goodsStockList`
|
||||
> 装载方式:事实表按 `create_time` 增量加载(`DwdLoadTask`)
|
||||
> 代码位置:`apps/etl/connectors/feiqiu/tasks/dwd/dwd_load_task.py`
|
||||
|
||||
---
|
||||
|
||||
## 1. dwd_goods_stock_movement
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `site_goods_stock_id` | BIGINT | `sitegoodsstockid`(ODS 驼峰 → PG 小写) | FACT_MAPPINGS (cast bigint) | 库存变动记录唯一标识(PK) | 飞球雪花 ID |
|
||||
| `tenant_id` | BIGINT | `tenantid` | FACT_MAPPINGS (cast bigint) | 租户 ID | 飞球租户 ID |
|
||||
| `site_id` | BIGINT | `siteid` | FACT_MAPPINGS (cast bigint) | 门店 ID | 飞球门店 ID |
|
||||
| `site_goods_id` | BIGINT | `sitegoodsid` | FACT_MAPPINGS (cast bigint) | 门店商品 ID,关联 `dim_store_goods.site_goods_id` | 飞球商品 ID |
|
||||
| `goods_name` | TEXT | `goodsname` | FACT_MAPPINGS | 商品名称 | 如 `百威啤酒` |
|
||||
| `goods_category_id` | BIGINT | `goodscategoryid` | FACT_MAPPINGS (cast bigint) | 一级商品分类 ID | 飞球分类 ID |
|
||||
| `goods_second_category_id` | BIGINT | `goodssecondcategoryid` | FACT_MAPPINGS (cast bigint) | 二级商品分类 ID | 飞球分类 ID |
|
||||
| `unit` | TEXT | `unit` | FACT_MAPPINGS | 计量单位 | 如 `瓶`、`包`、`张` |
|
||||
| `price` | NUMERIC(18,4) | `price` | FACT_MAPPINGS (cast numeric) | 商品单价(元) | 金额值 |
|
||||
| `stock_type` | INTEGER | `stocktype` | FACT_MAPPINGS (cast integer) | 库存变动类型枚举(详见下方枚举表) | `1`/`2`/`4`/`7`/`8`/`9` |
|
||||
| `change_num` | NUMERIC(18,4) | `changenum` | FACT_MAPPINGS (cast numeric) | 变动数量(正数为增加,负数为减少) | 正/负数值 |
|
||||
| `start_num` | NUMERIC(18,4) | `startnum` | FACT_MAPPINGS (cast numeric) | 变动前库存量 | `0.0000` ~ 正数 |
|
||||
| `end_num` | NUMERIC(18,4) | `endnum` | FACT_MAPPINGS (cast numeric) | 变动后库存量 | `0.0000` ~ 正数 |
|
||||
| `change_num_a` | NUMERIC(18,4) | `changenuma` | FACT_MAPPINGS (cast numeric) | 辅助单位变动量(用于双单位商品,如"箱→瓶"换算) | 数值或 0 |
|
||||
| `start_num_a` | NUMERIC(18,4) | `startnuma` | FACT_MAPPINGS (cast numeric) | 辅助单位变动前库存 | 数值或 0 |
|
||||
| `end_num_a` | NUMERIC(18,4) | `endnuma` | FACT_MAPPINGS (cast numeric) | 辅助单位变动后库存 | 数值或 0 |
|
||||
| `remark` | TEXT | `remark` | FACT_MAPPINGS | 备注说明 | 如 `结账退货`、`采购退货`、`系统自动领用`,或 NULL |
|
||||
| `operator_name` | TEXT | `operatorname` | FACT_MAPPINGS | 操作人姓名 | 姓名或 NULL |
|
||||
| `create_time` | TIMESTAMPTZ | `createtime` | FACT_MAPPINGS (cast timestamptz) | 变动发生时间 | ISO 时间戳 |
|
||||
| `fetched_at` | TIMESTAMPTZ | `fetched_at` | 自动映射 | ETL 采集时间戳 | ISO 时间戳 |
|
||||
|
||||
---
|
||||
|
||||
## 2. stock_type 枚举值详解
|
||||
|
||||
| stock_type | 含义 | 典型 remark | 数据量 |
|
||||
|-----------|------|------------|--------|
|
||||
| 1 | 销售出库 | NULL(系统自动扣减) | 29,573 条 |
|
||||
| 2 | 采购入库 | NULL | 1,033 条 |
|
||||
| 4 | 退货入库 | `结账退货` | 3,300 条 |
|
||||
| 7 | 采购退货(出库) | `采购退货` | 33 条 |
|
||||
| 8 | 领用出库 | `系统自动领用` | 1,033 条 |
|
||||
| 9 | 领用退回(入库) | `系统自动领用退回` | 33 条 |
|
||||
|
||||
---
|
||||
|
||||
## 3. ODS 列名映射说明
|
||||
|
||||
ODS DDL 中列名使用驼峰式(如 `siteGoodsStockId`),PostgreSQL 在无引号时自动小写化。FACT_MAPPINGS 中使用带引号的 ODS 列名以正确引用。部分列(`unit`、`price`、`remark`)在 ODS 中已是小写,无需引号。
|
||||
|
||||
---
|
||||
|
||||
## 4. 代码引用
|
||||
|
||||
- FACT_MAPPINGS:`dwd_load_task.py` → `FACT_MAPPINGS["dwd.dwd_goods_stock_movement"]`
|
||||
- TABLE_MAP:`"dwd.dwd_goods_stock_movement" → "ods.goods_stock_movements"`
|
||||
- DWS 下游:暂无直接 DWS 汇总(库存汇总基于 `dwd_goods_stock_summary`)
|
||||
62
docs/database/BD_Manual_goods_stock_summary.md
Normal file
62
docs/database/BD_Manual_goods_stock_summary.md
Normal file
@@ -0,0 +1,62 @@
|
||||
# BD_Manual:goods_stock_summary(库存汇总)
|
||||
|
||||
> ODS 表:`ods.goods_stock_summary`
|
||||
> DWD 表:`dwd.dwd_goods_stock_summary`(事实表,无 ex 扩展表)
|
||||
> API 接口:库存汇总查询(支持 `startTime`/`endTime` 时间窗口参数)
|
||||
> JSON 路径:`goods_stock_summary.json → data.goodsStockSummaryList`
|
||||
> 装载方式:事实表按时间窗口增量加载(`DwdLoadTask`)
|
||||
> ODS 配置:`requires_window=True`,`time_fields=("startTime", "endTime")`
|
||||
> 代码位置:`apps/etl/connectors/feiqiu/tasks/dwd/dwd_load_task.py`
|
||||
|
||||
---
|
||||
|
||||
## 1. dwd_goods_stock_summary
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `site_goods_id` | BIGINT | `sitegoodsid`(ODS DDL 驼峰 → PG 小写) | FACT_MAPPINGS (cast bigint) | 门店商品 ID(PK 之一),关联 `dim_store_goods.site_goods_id` | 飞球商品 ID,如 `3028609051954117` |
|
||||
| `goods_name` | TEXT | `goodsname` | FACT_MAPPINGS | 商品名称 | 如 `酱香爆珠槟榔`、`百威啤酒` |
|
||||
| `goods_unit` | TEXT | `goodsunit` | FACT_MAPPINGS | 计量单位 | 如 `包`、`瓶`、`张`、`罐` |
|
||||
| `goods_category_id` | BIGINT | `goodscategoryid` | FACT_MAPPINGS (cast bigint) | 一级商品分类 ID | 飞球分类 ID |
|
||||
| `goods_category_second_id` | BIGINT | `goodscategorysecondid` | FACT_MAPPINGS (cast bigint) | 二级商品分类 ID | 飞球分类 ID |
|
||||
| `category_name` | TEXT | `categoryname` | FACT_MAPPINGS | 一级分类名称 | 如 `槟榔`、`酒水`、`其他` |
|
||||
| `range_start_stock` | NUMERIC(18,4) | `rangestartstock` | FACT_MAPPINGS (cast numeric) | 期初库存(查询时间窗口起点的库存量) | 如 `100.0000`、`0.0000` |
|
||||
| `range_end_stock` | NUMERIC(18,4) | `rangeendstock` | FACT_MAPPINGS (cast numeric) | 期末库存(查询时间窗口终点的库存量) | 如 `100.0000`、`0.0000` |
|
||||
| `range_in` | NUMERIC(18,4) | `rangein` | FACT_MAPPINGS (cast numeric) | 入库数量(时间窗口内的采购/调拨入库总量) | `0.0000` ~ 正数 |
|
||||
| `range_out` | NUMERIC(18,4) | `rangeout` | FACT_MAPPINGS (cast numeric) | 出库数量(时间窗口内的调拨出库/报损总量) | `0.0000` ~ 正数 |
|
||||
| `range_sale` | NUMERIC(18,4) | `rangesale` | FACT_MAPPINGS (cast numeric) | 销售数量(时间窗口内的销售出库总量) | `0.0000` ~ 正数 |
|
||||
| `range_sale_money` | NUMERIC(18,2) | `rangesalemoney` | FACT_MAPPINGS (cast numeric) | 销售金额(元),时间窗口内的销售总金额 | `0.00` ~ 金额值 |
|
||||
| `range_inventory` | NUMERIC(18,4) | `rangeinventory` | FACT_MAPPINGS (cast numeric) | 盘点调整量(时间窗口内的盘盈/盘亏净量) | 正数(盘盈)或负数(盘亏) |
|
||||
| `current_stock` | NUMERIC(18,4) | `currentstock` | FACT_MAPPINGS (cast numeric) | 当前库存(API 返回时的实时库存量) | `0.0000` ~ 正数 |
|
||||
| `site_id` | BIGINT | `site_id` | 自动映射 | 门店 ID(ETL 注入) | 飞球门店 ID |
|
||||
| `tenant_id` | BIGINT | `tenant_id` | 自动映射 | 租户 ID(ETL 注入) | 飞球租户 ID |
|
||||
| `fetched_at` | TIMESTAMPTZ | `fetched_at` | 自动映射 | ETL 采集时间戳(PK 之一),标识本次快照的采集时间 | ISO 时间戳 |
|
||||
|
||||
---
|
||||
|
||||
## 2. 主键说明
|
||||
|
||||
主键为 `(site_goods_id, fetched_at)` 复合键。同一商品在不同采集时间窗口会产生多条记录,每条记录代表该时间窗口内的库存汇总快照。
|
||||
|
||||
---
|
||||
|
||||
## 3. ODS 列名映射说明
|
||||
|
||||
ODS DDL 中列名使用驼峰式(如 `siteGoodsId`),但 PostgreSQL 在无引号时自动小写化为 `sitegoodsid`。FACT_MAPPINGS 中使用带引号的 ODS 列名(如 `"siteGoodsId"`)以正确引用。
|
||||
|
||||
---
|
||||
|
||||
## 4. DWS 下游
|
||||
|
||||
- `dws.dws_goods_stock_daily_summary`:日度库存汇总
|
||||
- `dws.dws_goods_stock_weekly_summary`:周度库存汇总(ISO 周)
|
||||
- `dws.dws_goods_stock_monthly_summary`:月度库存汇总
|
||||
|
||||
---
|
||||
|
||||
## 5. 代码引用
|
||||
|
||||
- FACT_MAPPINGS:`dwd_load_task.py` → `FACT_MAPPINGS["dwd.dwd_goods_stock_summary"]`
|
||||
- TABLE_MAP:`"dwd.dwd_goods_stock_summary" → "ods.goods_stock_summary"`
|
||||
- ODS 任务配置:`tasks/ods/ods_tasks.py` → `OdsTaskSpec("ODS_GOODS_STOCK_SUMMARY", requires_window=True, time_fields=("startTime", "endTime"))`
|
||||
- DWS 任务:`tasks/dws/goods_stock_daily_task.py`、`goods_stock_weekly_task.py`、`goods_stock_monthly_task.py`
|
||||
74
docs/database/BD_Manual_member_balance_changes.md
Normal file
74
docs/database/BD_Manual_member_balance_changes.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# BD_Manual:member_balance_changes(会员余额变动)
|
||||
|
||||
> ODS 表:`ods.member_balance_changes`
|
||||
> DWD 表:`dwd.dwd_member_balance_change`(主表)、`dwd.dwd_member_balance_change_ex`(扩展表)
|
||||
> API 接口:会员余额变动记录列表
|
||||
> JSON 路径:`member_balance_changes.json → data.memberAccountChanges`
|
||||
> 装载方式:事实表增量插入(`DwdLoadTask`)
|
||||
> 代码位置:`apps/etl/connectors/feiqiu/tasks/dwd/dwd_load_task.py`
|
||||
|
||||
---
|
||||
|
||||
## 1. dwd_member_balance_change(主表,22 列)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `balance_change_id` | BIGINT | `id` | FACT_MAPPINGS | 余额变动记录唯一标识(PK) | 飞球雪花 ID |
|
||||
| `tenant_id` | BIGINT | `tenant_id` | 自动映射 | 租户 ID | 飞球租户 ID |
|
||||
| `site_id` | BIGINT | `site_id` | 自动映射 | 门店 ID | 飞球门店 ID |
|
||||
| `register_site_id` | BIGINT | `register_site_id` | 自动映射 | 会员注册门店 ID | 飞球门店 ID |
|
||||
| `tenant_member_id` | BIGINT | `tenant_member_id` | 自动映射 | 租户维度会员 ID | 飞球会员 ID |
|
||||
| `system_member_id` | BIGINT | `system_member_id` | 自动映射 | 系统维度会员 ID(跨租户唯一) | 飞球会员 ID |
|
||||
| `tenant_member_card_id` | BIGINT | `tenant_member_card_id` | 自动映射 | 会员卡 ID | 飞球会员卡 ID |
|
||||
| `card_type_id` | BIGINT | `card_type_id` | 自动映射 | 会员卡类型 ID | 飞球卡类型 ID |
|
||||
| `card_type_name` | VARCHAR(32) | `membercardtypename` | FACT_MAPPINGS | 会员卡类型名称快照 | 如 `普通会员卡` |
|
||||
| `member_name` | VARCHAR(64) | `membername` | FACT_MAPPINGS | 会员姓名快照 | 姓名 |
|
||||
| `member_mobile` | VARCHAR(20) | `membermobile` | FACT_MAPPINGS | 会员手机号快照 | 11 位手机号 |
|
||||
| `balance_before` | NUMERIC(18,2) | `before` | FACT_MAPPINGS | 变动前余额(元) | 金额值 |
|
||||
| `change_amount` | NUMERIC(18,2) | `account_data` | FACT_MAPPINGS | 变动金额(元),正数为充入,负数为扣减 | 正/负金额 |
|
||||
| `balance_after` | NUMERIC(18,2) | `after` | FACT_MAPPINGS | 变动后余额(元) | 金额值 |
|
||||
| `from_type` | INTEGER | `from_type` | 自动映射 | 变动来源类型:1=结算扣款,2=充值,3=退款返还,4=系统调整,7=转账,9=其他 | `1`/`2`/`3`/`4`/`7`/`9` |
|
||||
| `payment_method` | INTEGER | `payment_method` | 自动映射 | 支付方式 | 枚举值 |
|
||||
| `change_time` | TIMESTAMPTZ | `create_time` | FACT_MAPPINGS | 变动发生时间 | ISO 时间戳 |
|
||||
| `is_delete` | INTEGER | `is_delete` | 自动映射 | 是否已删除:0=正常,1=已删除 | `0` / `1` |
|
||||
| `remark` | VARCHAR(255) | `remark` | 自动映射 | 备注说明 | 自由文本或 NULL |
|
||||
| `principal_before` | NUMERIC(18,2) | `principal_before` | FACT_MAPPINGS | 变动前本金余额(元),不含赠送金 | 金额值 |
|
||||
| `principal_after` | NUMERIC(18,2) | `principal_after` | FACT_MAPPINGS | 变动后本金余额(元) | 金额值 |
|
||||
| `principal_change_amount` | NUMERIC(18,2) | *计算列* | FACT_MAPPINGS | 本金变动金额(元),= `principal_after - principal_before` | 正/负金额 |
|
||||
|
||||
---
|
||||
|
||||
## 2. dwd_member_balance_change_ex(扩展表,8 列)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `balance_change_id` | BIGINT | `id` | FACT_MAPPINGS | 余额变动记录唯一标识(PK) | 同主表 |
|
||||
| `pay_site_name` | VARCHAR(64) | `paysitename` | FACT_MAPPINGS | 支付门店名称快照 | 如 `朗朗桌球` |
|
||||
| `register_site_name` | VARCHAR(64) | `registersitename` | FACT_MAPPINGS | 注册门店名称快照 | 如 `朗朗桌球` |
|
||||
| `refund_amount` | NUMERIC(18,2) | `refund_amount` | 自动映射 | 退款金额(元) | `0.00` ~ 金额值 |
|
||||
| `operator_id` | BIGINT | `operator_id` | 自动映射 | 操作员 ID,执行本次余额变动的员工。0 表示系统自动 | `0` 或员工 ID |
|
||||
| `operator_name` | VARCHAR(64) | `operator_name` | 自动映射 | 操作员姓名 | 姓名或 NULL |
|
||||
| `principal_data` | TEXT | `principal_data` | FACT_MAPPINGS | 本金变动金额(元),正数为充入本金,负数为扣减本金 | 正/负金额 |
|
||||
| `relate_id` | BIGINT | `relate_id` | FACT_MAPPINGS | 关联业务单据 ID,指向触发本次余额变动的业务记录。按 `from_type` 不同指向不同表:1→结算单 ID,2→充值单 ID,3→退款单 ID,7→转账单 ID。`from_type=4`(系统调整)和 `9`(其他)时为 0 | `0` 或业务单据 ID |
|
||||
|
||||
---
|
||||
|
||||
## 3. from_type 枚举值详解
|
||||
|
||||
| from_type | 含义 | relate_id 指向 | 数据量 |
|
||||
|-----------|------|---------------|--------|
|
||||
| 1 | 结算扣款(消费) | 结算单 ID(`dwd_settlement_head.order_settle_id`) | 5637 条 |
|
||||
| 2 | 充值 | 充值单 ID(`dwd_recharge_order.recharge_order_id`) | 110 条 |
|
||||
| 3 | 退款返还 | 退款单 ID(`dwd_refund.refund_id`) | 650 条 |
|
||||
| 4 | 系统调整 | 0(无关联单据) | 775 条 |
|
||||
| 7 | 转账 | 转账单 ID | 15 条 |
|
||||
| 9 | 其他 | 0(无关联单据) | 179 条 |
|
||||
|
||||
---
|
||||
|
||||
## 4. 代码引用
|
||||
|
||||
- FACT_MAPPINGS:`dwd_load_task.py` → `FACT_MAPPINGS["dwd.dwd_member_balance_change"]` / `FACT_MAPPINGS["dwd.dwd_member_balance_change_ex"]`
|
||||
- TABLE_MAP:`"dwd.dwd_member_balance_change" → "ods.member_balance_changes"`
|
||||
- DWS 下游:`dws_member_analysis_task.py`(会员消费分析)
|
||||
- 迁移脚本:`db/_archived/ddl_baseline_2026-02-22/db/etl_feiqiu/migrations/2026-02-20__add_member_balance_change_ex_relate_id.sql`(已归档)
|
||||
103
docs/database/BD_Manual_recharge_settlements.md
Normal file
103
docs/database/BD_Manual_recharge_settlements.md
Normal file
@@ -0,0 +1,103 @@
|
||||
# BD_Manual:recharge_settlements(充值结算)
|
||||
|
||||
> ODS 表:`ods.recharge_settlements`
|
||||
> DWD 表:`dwd.dwd_recharge_order`(主表)、`dwd.dwd_recharge_order_ex`(扩展表)
|
||||
> API 接口:充值结算记录列表
|
||||
> JSON 路径:`recharge_settlements.json → data.orderSettles`
|
||||
> 装载方式:事实表增量插入(`DwdLoadTask`)
|
||||
> 代码位置:`apps/etl/connectors/feiqiu/tasks/dwd/dwd_load_task.py`
|
||||
|
||||
---
|
||||
|
||||
## 1. dwd_recharge_order(主表)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `recharge_order_id` | BIGINT | `id` | FACT_MAPPINGS | 充值结算记录唯一标识(PK) | 飞球雪花 ID |
|
||||
| `tenant_id` | BIGINT | `tenantid` | FACT_MAPPINGS | 租户 ID | 飞球租户 ID |
|
||||
| `site_id` | BIGINT | `siteid` | FACT_MAPPINGS | 门店 ID | 飞球门店 ID |
|
||||
| `member_id` | BIGINT | `memberid` | FACT_MAPPINGS | 会员 ID | 飞球会员 ID |
|
||||
| `member_name_snapshot` | TEXT | `membername` | FACT_MAPPINGS | 会员姓名快照 | 姓名 |
|
||||
| `member_phone_snapshot` | TEXT | `memberphone` | FACT_MAPPINGS | 会员手机号快照 | 11 位手机号 |
|
||||
| `tenant_member_card_id` | BIGINT | `tenantmembercardid` | FACT_MAPPINGS | 会员卡 ID | 飞球会员卡 ID |
|
||||
| `member_card_type_name` | TEXT | `membercardtypename` | FACT_MAPPINGS | 会员卡类型名称 | 如 `普通会员卡` |
|
||||
| `settle_relate_id` | BIGINT | `settlerelateid` | FACT_MAPPINGS | 关联结算单 ID | 飞球结算 ID |
|
||||
| `settle_type` | INTEGER | `settletype` | FACT_MAPPINGS | 结算类型枚举 | 枚举值 |
|
||||
| `settle_name` | TEXT | `settlename` | FACT_MAPPINGS | 结算类型名称 | 如 `充值` |
|
||||
| `is_first` | INTEGER | `isfirst` | FACT_MAPPINGS | 是否首次充值:0=否,1=是 | `0` / `1` |
|
||||
| `pay_amount` | NUMERIC | `payamount` | FACT_MAPPINGS | 实付金额(元) | 金额值 |
|
||||
| `refund_amount` | NUMERIC | `refundamount` | FACT_MAPPINGS | 退款金额(元) | `0.00` ~ 金额值 |
|
||||
| `point_amount` | NUMERIC | `pointamount` | FACT_MAPPINGS | 积分抵扣金额(元) | `0.00` ~ 金额值 |
|
||||
| `cash_amount` | NUMERIC | `cashamount` | FACT_MAPPINGS | 现金支付金额(元) | `0.00` ~ 金额值 |
|
||||
| `payment_method` | TEXT | `paymentmethod` | FACT_MAPPINGS | 支付方式 | 如 `微信`、`支付宝` |
|
||||
| `create_time` | TIMESTAMPTZ | `createtime` | FACT_MAPPINGS | 充值记录创建时间 | ISO 时间戳 |
|
||||
| `pay_time` | TIMESTAMPTZ | `paytime` | FACT_MAPPINGS | 支付完成时间 | ISO 时间戳 |
|
||||
| `pl_coupon_sale_amount` | NUMERIC | `plcouponsaleamount` | FACT_MAPPINGS | 平台券销售金额(元)。当前门店业务未启用,全部为 0 | `0.00` |
|
||||
| `mervou_sales_amount` | NUMERIC | `mervousalesamount` | FACT_MAPPINGS | 储值券销售金额(元)。当前门店业务未启用,全部为 0 | `0.00` |
|
||||
| `electricity_money` | NUMERIC | `electricitymoney` | FACT_MAPPINGS | 电费金额(元)。当前门店业务未启用,全部为 0 | `0.00` |
|
||||
| `real_electricity_money` | NUMERIC | `realelectricitymoney` | FACT_MAPPINGS | 实际电费金额(元),扣除调整后的电费。当前门店业务未启用,全部为 0 | `0.00` |
|
||||
| `electricity_adjust_money` | NUMERIC | `electricityadjustmoney` | FACT_MAPPINGS | 电费调整金额(元),电费手动调整的差额。当前门店业务未启用,全部为 0 | `0.00` |
|
||||
|
||||
---
|
||||
|
||||
## 2. dwd_recharge_order_ex(扩展表)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `recharge_order_id` | BIGINT | `id` | FACT_MAPPINGS | 充值结算记录唯一标识(PK) | 同主表 |
|
||||
| `site_name_snapshot` | TEXT | `sitename` | FACT_MAPPINGS | 门店名称快照 | 如 `朗朗桌球` |
|
||||
| `settle_status` | INTEGER | `settlestatus` | FACT_MAPPINGS | 结算状态枚举 | 枚举值 |
|
||||
| `is_bind_member` | BOOLEAN | `isbindmember` | FACT_MAPPINGS (cast boolean) | 是否绑定会员 | `true` / `false` |
|
||||
| `is_activity` | BOOLEAN | `isactivity` | FACT_MAPPINGS (cast boolean) | 是否参与活动 | `true` / `false` |
|
||||
| `is_use_coupon` | BOOLEAN | `isusecoupon` | FACT_MAPPINGS (cast boolean) | 是否使用优惠券 | `true` / `false` |
|
||||
| `is_use_discount` | BOOLEAN | `isusediscount` | FACT_MAPPINGS (cast boolean) | 是否使用折扣 | `true` / `false` |
|
||||
| `can_be_revoked` | BOOLEAN | `canberevoked` | FACT_MAPPINGS (cast boolean) | 是否可撤销 | `true` / `false` |
|
||||
| `online_amount` | NUMERIC | `onlineamount` | FACT_MAPPINGS | 线上支付金额(元) | 金额值 |
|
||||
| `balance_amount` | NUMERIC | `balanceamount` | FACT_MAPPINGS | 余额支付金额(元) | 金额值 |
|
||||
| `card_amount` | NUMERIC | `cardamount` | FACT_MAPPINGS | 银行卡支付金额(元) | 金额值 |
|
||||
| `coupon_amount` | NUMERIC | `couponamount` | FACT_MAPPINGS | 优惠券抵扣金额(元) | 金额值 |
|
||||
| `recharge_card_amount` | NUMERIC | `rechargecardamount` | FACT_MAPPINGS | 充值卡支付金额(元) | 金额值 |
|
||||
| `gift_card_amount` | NUMERIC | `giftcardamount` | FACT_MAPPINGS | 赠送卡支付金额(元) | 金额值 |
|
||||
| `prepay_money` | NUMERIC | `prepaymoney` | FACT_MAPPINGS | 预付金额(元) | 金额值 |
|
||||
| `consume_money` | NUMERIC | `consumemoney` | FACT_MAPPINGS | 消费总金额(元) | 金额值 |
|
||||
| `goods_money` | NUMERIC | `goodsmoney` | FACT_MAPPINGS | 商品金额(元) | 金额值 |
|
||||
| `real_goods_money` | NUMERIC | `realgoodsmoney` | FACT_MAPPINGS | 实收商品金额(元) | 金额值 |
|
||||
| `table_charge_money` | NUMERIC | `tablechargemoney` | FACT_MAPPINGS | 台费金额(元) | 金额值 |
|
||||
| `service_money` | NUMERIC | `servicemoney` | FACT_MAPPINGS | 服务费金额(元) | 金额值 |
|
||||
| `activity_discount` | NUMERIC | `activitydiscount` | FACT_MAPPINGS | 活动折扣金额(元) | 金额值 |
|
||||
| `all_coupon_discount` | NUMERIC | `allcoupondiscount` | FACT_MAPPINGS | 全部优惠券折扣金额(元) | 金额值 |
|
||||
| `goods_promotion_money` | NUMERIC | `goodspromotionmoney` | FACT_MAPPINGS | 商品促销金额(元) | 金额值 |
|
||||
| `assistant_promotion_money` | NUMERIC | `assistantpromotionmoney` | FACT_MAPPINGS | 助教促销金额(元) | 金额值 |
|
||||
| `assistant_pd_money` | NUMERIC | `assistantpdmoney` | FACT_MAPPINGS | 助教陪打金额(元) | 金额值 |
|
||||
| `assistant_cx_money` | NUMERIC | `assistantcxmoney` | FACT_MAPPINGS | 助教促销服务金额(元) | 金额值 |
|
||||
| `assistant_manual_discount` | NUMERIC | `assistantmanualdiscount` | FACT_MAPPINGS | 助教手动折扣金额(元) | 金额值 |
|
||||
| `coupon_sale_amount` | NUMERIC | `couponsaleamount` | FACT_MAPPINGS | 券销售金额(元) | 金额值 |
|
||||
| `member_discount_amount` | NUMERIC | `memberdiscountamount` | FACT_MAPPINGS | 会员折扣金额(元) | 金额值 |
|
||||
| `point_discount_price` | NUMERIC | `pointdiscountprice` | FACT_MAPPINGS | 积分折扣价格(元) | 金额值 |
|
||||
| `point_discount_cost` | NUMERIC | `pointdiscountcost` | FACT_MAPPINGS | 积分折扣成本(元) | 金额值 |
|
||||
| `adjust_amount` | NUMERIC | `adjustamount` | FACT_MAPPINGS | 调整金额(元) | 金额值 |
|
||||
| `rounding_amount` | NUMERIC | `roundingamount` | FACT_MAPPINGS | 抹零金额(元) | 金额值 |
|
||||
| `operator_id` | BIGINT | `operatorid` | FACT_MAPPINGS | 操作员 ID | 员工 ID |
|
||||
| `operator_name_snapshot` | TEXT | `operatorname` | FACT_MAPPINGS | 操作员姓名快照 | 如 `郑丽珊` |
|
||||
| `salesman_user_id` | BIGINT | `salesmanuserid` | FACT_MAPPINGS | 销售员用户 ID | 用户 ID |
|
||||
| `salesman_name` | TEXT | `salesmanname` | FACT_MAPPINGS | 销售员姓名 | 姓名或 NULL |
|
||||
| `order_remark` | TEXT | `orderremark` | FACT_MAPPINGS | 订单备注 | 自由文本或 NULL |
|
||||
| `table_id` | BIGINT | `tableid` | FACT_MAPPINGS | 台桌 ID | 飞球台桌 ID |
|
||||
| `serial_number` | INTEGER | `serialnumber` | FACT_MAPPINGS | 流水号 | 正整数 |
|
||||
| `revoke_order_id` | BIGINT | `revokeorderid` | FACT_MAPPINGS | 撤销关联的订单 ID | 订单 ID 或 0 |
|
||||
| `revoke_order_name` | TEXT | `revokeordername` | FACT_MAPPINGS | 撤销关联的订单名称 | 名称或 NULL |
|
||||
| `revoke_time` | TIMESTAMPTZ | `revoketime` | FACT_MAPPINGS | 撤销时间 | ISO 时间戳或 NULL |
|
||||
|
||||
---
|
||||
|
||||
## 3. B 类表说明
|
||||
|
||||
本表属于 B 类(仅补 FACT_MAPPINGS),5 个电费/券字段的 DWD 列在 DDL 中已存在,但之前缺少 FACT_MAPPINGS 条目导致数据未从 ODS 流入 DWD。2026-02-20 补充映射后,这 5 个字段的数据可正常流转。当前门店这 5 个字段的 ODS/DWD 数据全部为 0(业务未启用电费和券销售功能)。
|
||||
|
||||
---
|
||||
|
||||
## 4. 代码引用
|
||||
|
||||
- FACT_MAPPINGS:`dwd_load_task.py` → `FACT_MAPPINGS["dwd.dwd_recharge_order"]` / `FACT_MAPPINGS["dwd.dwd_recharge_order_ex"]`
|
||||
- TABLE_MAP:`"dwd.dwd_recharge_order" → "ods.recharge_settlements"`
|
||||
- DWS 下游:`dws_finance_daily_task.py`(财务日报,充值汇总)
|
||||
61
docs/database/BD_Manual_site_tables_master.md
Normal file
61
docs/database/BD_Manual_site_tables_master.md
Normal file
@@ -0,0 +1,61 @@
|
||||
# BD_Manual:site_tables_master(台桌维表)
|
||||
|
||||
> ODS 表:`ods.site_tables_master`
|
||||
> DWD 表:`dwd.dim_table`(主表)、`dwd.dim_table_ex`(扩展表)
|
||||
> API 接口:门店台桌列表
|
||||
> JSON 路径:`site_tables_master.json → data.siteTables`
|
||||
> 装载方式:SCD2 维度合并(`DwdLoadTask`)
|
||||
> 代码位置:`apps/etl/connectors/feiqiu/tasks/dwd/dwd_load_task.py`
|
||||
|
||||
---
|
||||
|
||||
## 1. dim_table(主表)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `table_id` | BIGINT | `id` | FACT_MAPPINGS | 台桌唯一标识(PK 之一) | 飞球雪花 ID |
|
||||
| `site_id` | BIGINT | `site_id` | 自动映射 | 门店 ID | 飞球门店 ID |
|
||||
| `table_name` | TEXT | `table_name` | 自动映射 | 台桌名称 | 如 `1号台`、`VIP1` |
|
||||
| `site_table_area_id` | BIGINT | `site_table_area_id` | 自动映射 | 门店台桌区域 ID | 飞球区域 ID |
|
||||
| `site_table_area_name` | TEXT | `areaname` | FACT_MAPPINGS | 台桌区域名称 | 如 `大厅`、`VIP区` |
|
||||
| `tenant_table_area_id` | BIGINT | `site_table_area_id` | FACT_MAPPINGS | 租户级台桌区域 ID | 飞球区域 ID |
|
||||
| `table_price` | NUMERIC | `table_price` | 自动映射 | 台费单价(元/小时) | 金额值 |
|
||||
| `order_id` | BIGINT | `order_id` | FACT_MAPPINGS | 当前关联的订单 ID,0 表示空闲 | `0` 或订单 ID |
|
||||
| `scd2_*` | — | — | DWD 元数据 | SCD2 慢变维度追踪字段 | — |
|
||||
|
||||
---
|
||||
|
||||
## 2. dim_table_ex(扩展表)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `table_id` | BIGINT | `id` | FACT_MAPPINGS | 台桌唯一标识(PK 之一) | 同主表 |
|
||||
| `show_status` | INTEGER | `show_status` | 自动映射 | 展示状态 | 枚举值 |
|
||||
| `is_online_reservation` | INTEGER | `is_online_reservation` | 自动映射 | 是否支持在线预约:0=否,1=是 | `0` / `1` |
|
||||
| `table_cloth_use_time` | INTEGER | `table_cloth_use_time` | FACT_MAPPINGS | 台布使用时间(累计使用次数或时长) | 整数 |
|
||||
| `table_cloth_use_cycle` | INTEGER | `table_cloth_use_cycle` | 自动映射 | 台布更换周期 | 整数 |
|
||||
| `table_status` | INTEGER | `table_status` | 自动映射 | 台桌状态枚举 | 枚举值 |
|
||||
| `create_time` | TIMESTAMPTZ | `create_time` | FACT_MAPPINGS | 台桌配置的创建时间或最近一次创建/复制时间 | ISO 时间戳 |
|
||||
| `light_status` | INTEGER | `light_status` | FACT_MAPPINGS | 台灯状态:1=已关灯,2=已开灯 | `1` / `2` |
|
||||
| `tablestatusname` | TEXT | `tablestatusname` | FACT_MAPPINGS | 台桌状态中文名称(如"空闲中""使用中"),仅展示用途。ODS 列 `tableStatusName` 在 PG 中小写化 | 如 `空闲中`、`使用中` |
|
||||
| `sitename` | TEXT | `sitename` | FACT_MAPPINGS | 门店名称快照,冗余字段。ODS 列 `siteName` 在 PG 中小写化 | 如 `朗朗桌球` |
|
||||
| `applet_qr_code_url` | TEXT | `"appletQrCodeUrl"` | FACT_MAPPINGS | 小程序二维码 URL,用于扫码开台等场景。ODS 列用双引号保留驼峰大小写 | HTTPS 链接或 NULL |
|
||||
| `audit_status` | INTEGER | `audit_status` | FACT_MAPPINGS | 审核状态:2=已审核通过(当前全部为 2) | `2` |
|
||||
| `charge_free` | INTEGER | `charge_free` | FACT_MAPPINGS | 是否免费台:0=收费,1=免费(当前全部为 0) | `0` / `1` |
|
||||
| `delay_lights_time` | INTEGER | `delay_lights_time` | FACT_MAPPINGS | 台灯熄灭延迟时间(秒),结账后延时关灯的秒数 | 正整数 |
|
||||
| `is_rest_area` | INTEGER | `is_rest_area` | FACT_MAPPINGS | 是否休息区台桌:0=否,1=是(当前全部为 0) | `0` / `1` |
|
||||
| `only_allow_groupon` | INTEGER | `only_allow_groupon` | FACT_MAPPINGS | 是否仅允许团购开台:0=不限制,1=仅团购,2=不限制(当前全部为 2) | `0` / `1` / `2` |
|
||||
| `order_delay_time` | INTEGER | `order_delay_time` | FACT_MAPPINGS | 订单自动延时时长(秒),超时未结账自动延长的时间 | 正整数 |
|
||||
| `self_table` | INTEGER | `self_table` | FACT_MAPPINGS | 是否自有台桌:1=自有(当前全部为 1) | `1` |
|
||||
| `temporary_light_second` | INTEGER | `temporary_light_second` | FACT_MAPPINGS | 临时开灯秒数,临时开灯持续的时间 | 正整数 |
|
||||
| `virtual_table` | INTEGER | `virtual_table` | FACT_MAPPINGS | 是否虚拟台桌:0=实体台,1=虚拟台(当前全部为 0) | `0` / `1` |
|
||||
| `scd2_*` | — | — | DWD 元数据 | SCD2 慢变维度追踪字段 | — |
|
||||
|
||||
---
|
||||
|
||||
## 3. 代码引用
|
||||
|
||||
- FACT_MAPPINGS:`dwd_load_task.py` → `FACT_MAPPINGS["dwd.dim_table"]` / `FACT_MAPPINGS["dwd.dim_table_ex"]`
|
||||
- TABLE_MAP:`"dwd.dim_table" → "ods.site_tables_master"`
|
||||
- DWS 下游:`dws_finance_daily_task.py`(财务日报,台费汇总按区域分组)
|
||||
- 迁移脚本:`db/_archived/ddl_baseline_2026-02-22/db/etl_feiqiu/migrations/2026-02-20__add_dim_table_ex_fields.sql`(已归档)
|
||||
103
docs/database/BD_Manual_store_goods_master.md
Normal file
103
docs/database/BD_Manual_store_goods_master.md
Normal file
@@ -0,0 +1,103 @@
|
||||
# BD_Manual:store_goods_master(门店商品档案)
|
||||
|
||||
> ODS 表:`ods.store_goods_master`
|
||||
> DWD 表:`dwd.dim_store_goods`(主表)、`dwd.dim_store_goods_ex`(扩展表)
|
||||
> API 接口:门店商品列表
|
||||
> JSON 路径:`store_goods_master.json → data.orderGoodsList`
|
||||
> 装载方式:SCD2 维度合并(`DwdLoadTask`)
|
||||
> 代码位置:`apps/etl/connectors/feiqiu/tasks/dwd/dwd_load_task.py`
|
||||
|
||||
---
|
||||
|
||||
## 1. dim_store_goods(主表)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `site_goods_id` | BIGINT | `id` | FACT_MAPPINGS | 门店商品唯一标识(PK 之一) | 飞球雪花 ID |
|
||||
| `tenant_id` | BIGINT | `tenant_id` | 自动映射 | 租户 ID | 飞球租户 ID |
|
||||
| `site_id` | BIGINT | `site_id` | 自动映射 | 门店 ID | 飞球门店 ID |
|
||||
| `tenant_goods_id` | BIGINT | `tenant_goods_id` | 自动映射 | 租户商品 ID,关联 `dim_tenant_goods` | 飞球商品 ID |
|
||||
| `goods_name` | TEXT | `goods_name` | 自动映射 | 商品名称 | 如 `百威啤酒` |
|
||||
| `goods_category_id` | BIGINT | `goods_category_id` | 自动映射 | 一级分类 ID | 飞球分类 ID |
|
||||
| `goods_second_category_id` | BIGINT | `goods_second_category_id` | 自动映射 | 二级分类 ID | 飞球分类 ID |
|
||||
| `category_level1_name` | TEXT | `onecategoryname` | FACT_MAPPINGS | 一级分类名称 | 如 `酒水` |
|
||||
| `category_level2_name` | TEXT | `twocategoryname` | FACT_MAPPINGS | 二级分类名称 | 如 `啤酒` |
|
||||
| `batch_stock_qty` | INTEGER | `batch_stock_quantity` | FACT_MAPPINGS | 批次库存数量(按批次管理的库存)。⚠️ 2026-02-20 修正映射源(原错误映射自 `stock`,即当前库存) | 数值 |
|
||||
| `sale_qty` | INTEGER | `sale_num` | FACT_MAPPINGS | 累计销售数量 | 数值 |
|
||||
| `total_sales_qty` | INTEGER | `total_sales` | FACT_MAPPINGS | 累计销售总额 | 数值 |
|
||||
| `sale_price` | NUMERIC(18,2) | `sale_price` | 自动映射 | 商品售价(元) | 金额值 |
|
||||
| `created_at` | TIMESTAMPTZ | `create_time` | FACT_MAPPINGS | 商品创建时间 | ISO 时间戳 |
|
||||
| `updated_at` | TIMESTAMPTZ | `update_time` | FACT_MAPPINGS | 商品最后更新时间 | ISO 时间戳 |
|
||||
| `avg_monthly_sales` | NUMERIC(18,4) | `average_monthly_sales` | FACT_MAPPINGS | 月均销量 | 数值 |
|
||||
| `goods_state` | INTEGER | `goods_state` | 自动映射 | 商品状态 | 枚举值 |
|
||||
| `enable_status` | INTEGER | `enable_status` | 自动映射 | 启用状态 | 枚举值 |
|
||||
| `send_state` | INTEGER | `send_state` | 自动映射 | 配送状态 | 枚举值 |
|
||||
| `is_delete` | INTEGER | `is_delete` | 自动映射 | 是否已删除:0=正常,1=已删除 | `0` / `1` |
|
||||
| `commodity_code` | TEXT | `commodity_code` | FACT_MAPPINGS | 商品编码 | 编码字符串 |
|
||||
| `not_sale` | INTEGER | `not_sale` | FACT_MAPPINGS | 是否停售:0=在售,1=停售 | `0` / `1` |
|
||||
| `scd2_*` | — | — | DWD 元数据 | SCD2 慢变维度追踪字段 | — |
|
||||
|
||||
---
|
||||
|
||||
## 2. dim_store_goods_ex(扩展表)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `site_goods_id` | BIGINT | `id` | FACT_MAPPINGS | 门店商品唯一标识(PK 之一) | 同主表 |
|
||||
| `site_name` | TEXT | `sitename` | FACT_MAPPINGS | 门店名称快照 | 如 `朗朗桌球` |
|
||||
| `unit` | TEXT | `unit` | 自动映射 | 计量单位 | 如 `瓶`、`包` |
|
||||
| `goods_barcode` | TEXT | `goods_bar_code` | FACT_MAPPINGS | 商品条码 | 条码字符串或 NULL |
|
||||
| `goods_cover_url` | TEXT | `goods_cover` | FACT_MAPPINGS | 商品封面图 URL | HTTPS 链接或 NULL |
|
||||
| `pinyin_initial` | TEXT | `pinyin_initial` | 自动映射 | 拼音首字母(用于搜索) | 如 `BWPJ` |
|
||||
| `stock_qty` | INTEGER | `stock` | FACT_MAPPINGS | 当前库存数量(实时库存) | 数值 |
|
||||
| `stock_secondary_qty` | INTEGER | `stock_a` | FACT_MAPPINGS | 辅助单位库存数量(双单位商品) | 数值 |
|
||||
| `safety_stock_qty` | INTEGER | `safe_stock` | FACT_MAPPINGS | 安全库存数量(低于此值触发预警) | 数值 |
|
||||
| `cost_price` | NUMERIC(18,4) | `cost_price` | 自动映射 | 成本价(元) | 金额值 |
|
||||
| `cost_price_type` | INTEGER | `cost_price_type` | 自动映射 | 成本价类型 | 枚举值 |
|
||||
| `provisional_total_cost` | NUMERIC(18,2) | `provisional_total_cost` | FACT_MAPPINGS | 暂估总成本(元),按暂估价计算的库存成本。⚠️ 2026-02-20 修正映射源(原错误映射自 `total_purchase_cost`,即实际采购成本) | 金额值 |
|
||||
| `total_purchase_cost` | NUMERIC(18,2) | `total_purchase_cost` | 自动映射 | 实际采购总成本(元) | 金额值 |
|
||||
| `min_discount_price` | NUMERIC(18,2) | `min_discount_price` | 自动映射 | 最低折扣价(元) | 金额值 |
|
||||
| `is_discountable` | INTEGER | `able_discount` | FACT_MAPPINGS | 是否可打折:0=不可,1=可 | `0` / `1` |
|
||||
| `days_on_shelf` | INTEGER | `days_available` | FACT_MAPPINGS | 上架天数 | 正整数 |
|
||||
| `audit_status` | INTEGER | `audit_status` | 自动映射 | 审核状态 | 枚举值 |
|
||||
| `sale_channel` | INTEGER | `sale_channel` | 自动映射 | 销售渠道 | 枚举值 |
|
||||
| `is_warehousing` | INTEGER | `is_warehousing` | 自动映射 | 是否入库管理:0=否,1=是 | `0` / `1` |
|
||||
| `freeze_status` | INTEGER | `freeze` | FACT_MAPPINGS | 冻结状态:0=正常,1=冻结 | `0` / `1` |
|
||||
| `forbid_sell_status` | INTEGER | `forbid_sell_status` | 自动映射 | 禁售状态 | 枚举值 |
|
||||
| `able_site_transfer` | INTEGER | `able_site_transfer` | 自动映射 | 是否允许门店间调拨:0=否,1=是 | `0` / `1` |
|
||||
| `custom_label_type` | INTEGER | `custom_label_type` | 自动映射 | 自定义标签类型 | 枚举值 |
|
||||
| `option_required` | INTEGER | `option_required` | 自动映射 | 是否必选规格:0=否,1=是 | `0` / `1` |
|
||||
| `remark` | TEXT | `remark` | FACT_MAPPINGS | 商品备注 | 自由文本或 NULL |
|
||||
| `sort_order` | INTEGER | `sort` | FACT_MAPPINGS | 排序序号 | 正整数 |
|
||||
| `batch_stock_quantity` | NUMERIC | `batch_stock_quantity` | 自动映射 | 批次库存数量(冗余,与主表 `batch_stock_qty` 同源) | 数值 |
|
||||
| `time_slot_sale` | INTEGER | `time_slot_sale` | FACT_MAPPINGS | 分时段销售标记(当前观测全部为 2)。2026-02-21 新增 | `2` |
|
||||
| `scd2_*` | — | — | DWD 元数据 | SCD2 慢变维度追踪字段 | — |
|
||||
|
||||
---
|
||||
|
||||
## 3. 映射修正记录
|
||||
|
||||
| 日期 | 字段 | 修正内容 |
|
||||
|------|------|---------|
|
||||
| 2026-02-20 | `batch_stock_qty` | ODS 源从 `stock`(当前库存)修正为 `batch_stock_quantity`(批次库存)。验证:仅 7.3% 行两列值相等 |
|
||||
| 2026-02-20 | `provisional_total_cost` | ODS 源从 `total_purchase_cost`(实际采购成本)修正为 `provisional_total_cost`(暂估成本)。验证:93.5% 行相等但 113 行不同 |
|
||||
| 2026-02-21 | `time_slot_sale` | 新增字段。ODS `ods.store_goods_master` + DWD `dwd.dim_store_goods_ex` 同步新增 `time_slot_sale INTEGER`。API 返回值当前全部为 `2` |
|
||||
|
||||
---
|
||||
|
||||
## 4. 跳过字段说明
|
||||
|
||||
| ODS 字段 | 跳过原因 |
|
||||
|---------|---------|
|
||||
| ~~`time_slot_sale`~~ | ~~ODS 列不存在~~ → 2026-02-21 已新增,见映射修正记录 |
|
||||
| `goodsStockWarningInfo` | JSONB 嵌套对象,展开不在本次范围内 |
|
||||
|
||||
---
|
||||
|
||||
## 5. 代码引用
|
||||
|
||||
- FACT_MAPPINGS:`dwd_load_task.py` → `FACT_MAPPINGS["dwd.dim_store_goods"]` / `FACT_MAPPINGS["dwd.dim_store_goods_ex"]`
|
||||
- TABLE_MAP:`"dwd.dim_store_goods" → "ods.store_goods_master"`
|
||||
- DWS 下游:`dws_finance_daily_task.py`(财务日报,商品维度关联)
|
||||
- 迁移脚本:`db/_archived/ddl_baseline_2026-02-22/db/etl_feiqiu/migrations/2026-02-20__fix_store_goods_master_mapping.sql`(已归档)
|
||||
- 迁移脚本:`db/_archived/ddl_baseline_2026-02-22/db/etl_feiqiu/migrations/2026-02-21__add_time_slot_sale_merge_commodity_code.sql`(已归档)
|
||||
93
docs/database/BD_Manual_store_goods_sales_records.md
Normal file
93
docs/database/BD_Manual_store_goods_sales_records.md
Normal file
@@ -0,0 +1,93 @@
|
||||
# BD_Manual:store_goods_sales_records(门店商品销售流水)
|
||||
|
||||
> ODS 表:`ods.store_goods_sales_records`
|
||||
> DWD 表:`dwd.dwd_store_goods_sale`(主表)、`dwd.dwd_store_goods_sale_ex`(扩展表)
|
||||
> API 接口:门店商品销售记录列表
|
||||
> JSON 路径:`store_goods_sales_records.json → data.orderGoodsLedgers`
|
||||
> 装载方式:事实表增量插入(`DwdLoadTask`)
|
||||
> 代码位置:`apps/etl/connectors/feiqiu/tasks/dwd/dwd_load_task.py`
|
||||
|
||||
---
|
||||
|
||||
## 1. dwd_store_goods_sale(主表,25 列)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `store_goods_sale_id` | BIGINT | `id` | FACT_MAPPINGS | 销售记录唯一标识(PK) | 飞球雪花 ID |
|
||||
| `order_trade_no` | BIGINT | `order_trade_no` | 自动映射 | 关联的订单交易号 | 飞球订单号 |
|
||||
| `order_settle_id` | BIGINT | `order_settle_id` | 自动映射 | 关联的结算单 ID | 飞球结算单 ID |
|
||||
| `order_pay_id` | BIGINT | `order_pay_id` | 自动映射 | 关联的支付单 ID | 飞球支付单 ID |
|
||||
| `order_goods_id` | BIGINT | `order_goods_id` | 自动映射 | 订单商品明细 ID | 飞球雪花 ID |
|
||||
| `site_id` | BIGINT | `site_id` | 自动映射 | 门店 ID | 飞球门店 ID |
|
||||
| `tenant_id` | BIGINT | `tenant_id` | 自动映射 | 租户 ID | 飞球租户 ID |
|
||||
| `site_goods_id` | BIGINT | `site_goods_id` | 自动映射 | 门店商品 ID,关联 `dim_store_goods` | 飞球商品 ID |
|
||||
| `tenant_goods_id` | BIGINT | `tenant_goods_id` | 自动映射 | 租户商品 ID,关联 `dim_tenant_goods` | 飞球商品 ID |
|
||||
| `tenant_goods_category_id` | BIGINT | `tenant_goods_category_id` | 自动映射 | 商品分类 ID | 飞球分类 ID |
|
||||
| `tenant_goods_business_id` | BIGINT | `tenant_goods_business_id` | 自动映射 | 商品业务类型 ID | 飞球业务 ID |
|
||||
| `site_table_id` | BIGINT | `site_table_id` | 自动映射 | 消费台桌 ID,关联 `dim_table.table_id` | 飞球台桌 ID |
|
||||
| `ledger_name` | VARCHAR(200) | `ledger_name` | 自动映射 | 分账项名称(商品名称快照) | 如 `百威啤酒` |
|
||||
| `ledger_group_name` | VARCHAR(100) | `ledger_group_name` | 自动映射 | 分账组名称 | 分账组名或 NULL |
|
||||
| `ledger_unit_price` | NUMERIC(18,2) | `ledger_unit_price` | 自动映射 | 分账单价(元/单位) | 金额值 |
|
||||
| `ledger_count` | INTEGER | `ledger_count` | 自动映射 | 分账数量(销售数量) | 正整数 |
|
||||
| `ledger_amount` | NUMERIC(18,2) | `ledger_amount` | 自动映射 | 分账总金额(元),= 单价 × 数量 | 金额值 |
|
||||
| `discount_money` | NUMERIC(18,2) | `discount_money` | FACT_MAPPINGS | 折扣金额(元),会员折扣减免的金额。⚠️ 2026-02-20 由原 `discount_price` 重命名而来,修正列名误导 | `0.00` ~ 金额值 |
|
||||
| `real_goods_money` | NUMERIC(18,2) | `real_goods_money` | 自动映射 | 实收商品金额(元),扣除折扣后 | 金额值 |
|
||||
| `cost_money` | NUMERIC(18,2) | `cost_money` | 自动映射 | 成本金额(元) | 金额值 |
|
||||
| `ledger_status` | INTEGER | `ledger_status` | 自动映射 | 分账状态 | 枚举值 |
|
||||
| `is_delete` | INTEGER | `is_delete` | 自动映射 | 是否已删除:0=正常,1=已删除 | `0` / `1` |
|
||||
| `create_time` | TIMESTAMPTZ | `create_time` | 自动映射 | 销售记录创建时间 | ISO 时间戳 |
|
||||
| `coupon_share_money` | NUMERIC(18,2) | `coupon_share_money` | FACT_MAPPINGS | 优惠券分摊金额(元),该商品分摊的优惠券减免 | `0.00` ~ 金额值 |
|
||||
| `discount_price` | NUMERIC(18,2) | `discount_price` | FACT_MAPPINGS | 折后单价(元),会员折扣后的商品单价。⚠️ 2026-02-20 新增,映射自 ODS 真正的 `discount_price` | `0.00` ~ 金额值 |
|
||||
|
||||
---
|
||||
|
||||
## 2. dwd_store_goods_sale_ex(扩展表,28 列)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `store_goods_sale_id` | BIGINT | `id` | FACT_MAPPINGS | 销售记录唯一标识(PK) | 同主表 |
|
||||
| `legacy_order_goods_id` | BIGINT | `ordergoodsid` | FACT_MAPPINGS | 旧版订单商品 ID(兼容字段) | 飞球 ID |
|
||||
| `site_name` | TEXT | `sitename` | FACT_MAPPINGS | 门店名称快照 | 如 `朗朗桌球` |
|
||||
| `legacy_site_id` | BIGINT | `siteid` | FACT_MAPPINGS | 旧版门店 ID(兼容字段) | 飞球门店 ID |
|
||||
| `goods_remark` | TEXT | `goods_remark` | 自动映射 | 商品备注 | 自由文本或 NULL |
|
||||
| `option_value_name` | TEXT | `option_value_name` | FACT_MAPPINGS | 商品规格选项名称 | 如 `大杯`、NULL |
|
||||
| `operator_name` | TEXT | `operator_name` | 自动映射 | 操作员姓名 | 姓名或 NULL |
|
||||
| `open_salesman_flag` | INTEGER | `opensalesman` | FACT_MAPPINGS (cast integer) | 是否开启销售员提成:0=否,1=是 | `0` / `1` |
|
||||
| `salesman_user_id` | BIGINT | `salesman_user_id` | 自动映射 | 销售员用户 ID | 用户 ID 或 NULL |
|
||||
| `salesman_name` | TEXT | `salesman_name` | FACT_MAPPINGS | 销售员姓名 | 姓名或 NULL |
|
||||
| `salesman_role_id` | BIGINT | `salesman_role_id` | 自动映射 | 销售员角色 ID | 角色 ID 或 NULL |
|
||||
| `salesman_org_id` | BIGINT | `sales_man_org_id` | FACT_MAPPINGS | 销售员组织 ID | 组织 ID 或 NULL |
|
||||
| `discount_money` | NUMERIC(18,2) | `discount_money` | 自动映射 | 折扣金额(元),扩展表中的折扣明细 | `0.00` ~ 金额值 |
|
||||
| `returns_number` | INTEGER | `returns_number` | 自动映射 | 退货数量 | `0` ~ 正整数 |
|
||||
| `coupon_deduct_money` | NUMERIC(18,2) | `coupon_deduct_money` | 自动映射 | 优惠券抵扣金额(元) | `0.00` ~ 金额值 |
|
||||
| `member_discount_amount` | NUMERIC(18,2) | `member_discount_amount` | 自动映射 | 会员折扣金额(元) | `0.00` ~ 金额值 |
|
||||
| `point_discount_money` | NUMERIC(18,2) | `point_discount_money` | 自动映射 | 积分抵扣金额(元) | `0.00` ~ 金额值 |
|
||||
| `point_discount_money_cost` | NUMERIC(18,2) | `point_discount_money_cost` | 自动映射 | 积分抵扣成本(元) | `0.00` ~ 金额值 |
|
||||
| `package_coupon_id` | BIGINT | `package_coupon_id` | 自动映射 | 套餐券 ID | 券 ID 或 NULL |
|
||||
| `order_coupon_id` | BIGINT | `order_coupon_id` | 自动映射 | 订单券 ID | 券 ID 或 NULL |
|
||||
| `member_coupon_id` | BIGINT | `member_coupon_id` | 自动映射 | 会员券 ID | 券 ID 或 NULL |
|
||||
| `option_price` | NUMERIC(18,2) | `option_price` | 自动映射 | 规格选项加价(元) | `0.00` ~ 金额值 |
|
||||
| `option_member_discount_money` | NUMERIC(18,2) | `option_member_discount_money` | 自动映射 | 规格选项会员折扣金额(元) | `0.00` ~ 金额值 |
|
||||
| `option_coupon_deduct_money` | NUMERIC(18,2) | `option_coupon_deduct_money` | 自动映射 | 规格选项优惠券抵扣金额(元) | `0.00` ~ 金额值 |
|
||||
| `push_money` | NUMERIC(18,2) | `push_money` | 自动映射 | 推销提成金额(元) | `0.00` ~ 金额值 |
|
||||
| `is_single_order` | INTEGER | `is_single_order` | 自动映射 | 是否单独订单 | `0` / `1` |
|
||||
| `sales_type` | INTEGER | `sales_type` | 自动映射 | 销售类型 | 枚举值 |
|
||||
| `operator_id` | BIGINT | `operator_id` | 自动映射 | 操作员 ID | 员工 ID 或 NULL |
|
||||
|
||||
---
|
||||
|
||||
## 3. 映射修正记录
|
||||
|
||||
| 日期 | 字段 | 修正内容 |
|
||||
|------|------|---------|
|
||||
| 2026-02-20 | `discount_price` → `discount_money` | 原 DWD `discount_price` 实际映射自 ODS `discount_money`(折扣金额),列名与语义不符,重命名为 `discount_money` |
|
||||
| 2026-02-20 | `discount_price`(新增) | 新增 DWD 列,映射自 ODS 真正的 `discount_price`(折后单价) |
|
||||
|
||||
---
|
||||
|
||||
## 4. 代码引用
|
||||
|
||||
- FACT_MAPPINGS:`dwd_load_task.py` → `FACT_MAPPINGS["dwd.dwd_store_goods_sale"]` / `FACT_MAPPINGS["dwd.dwd_store_goods_sale_ex"]`
|
||||
- TABLE_MAP:`"dwd.dwd_store_goods_sale" → "ods.store_goods_sales_records"`
|
||||
- DWS 下游:`dws_finance_daily_task.py`(财务日报,商品销售汇总)
|
||||
- 迁移脚本:`db/_archived/ddl_baseline_2026-02-22/db/etl_feiqiu/migrations/2026-02-20__fix_store_goods_sale_discount_price.sql`(已归档)
|
||||
80
docs/database/BD_Manual_tenant_goods_master.md
Normal file
80
docs/database/BD_Manual_tenant_goods_master.md
Normal file
@@ -0,0 +1,80 @@
|
||||
# BD_Manual:tenant_goods_master(租户商品档案)
|
||||
|
||||
> ODS 表:`ods.tenant_goods_master`
|
||||
> DWD 表:`dwd.dim_tenant_goods`(主表)、`dwd.dim_tenant_goods_ex`(扩展表)
|
||||
> API 接口:租户商品列表
|
||||
> JSON 路径:`tenant_goods_master.json → data.tenantGoodsList`
|
||||
> 装载方式:SCD2 维度合并(`DwdLoadTask`)
|
||||
> 代码位置:`apps/etl/connectors/feiqiu/tasks/dwd/dwd_load_task.py`
|
||||
|
||||
---
|
||||
|
||||
## 1. dim_tenant_goods(主表)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `tenant_goods_id` | BIGINT | `id` | FACT_MAPPINGS | 租户商品唯一标识(PK 之一) | 飞球雪花 ID |
|
||||
| `tenant_id` | BIGINT | `tenant_id` | 自动映射 | 租户 ID | 飞球租户 ID |
|
||||
| `supplier_id` | BIGINT | `supplier_id` | 自动映射 | 供应商 ID | 飞球供应商 ID 或 0 |
|
||||
| `category_name` | VARCHAR | `categoryname` | FACT_MAPPINGS | 一级分类名称 | 如 `酒水` |
|
||||
| `goods_category_id` | BIGINT | `goods_category_id` | 自动映射 | 一级分类 ID | 飞球分类 ID |
|
||||
| `goods_second_category_id` | BIGINT | `goods_second_category_id` | 自动映射 | 二级分类 ID | 飞球分类 ID |
|
||||
| `goods_name` | VARCHAR | `goods_name` | 自动映射 | 商品名称 | 如 `百威啤酒` |
|
||||
| `goods_number` | VARCHAR | `goods_number` | 自动映射 | 商品编号 | 编号字符串 |
|
||||
| `unit` | VARCHAR | `unit` | 自动映射 | 计量单位 | 如 `瓶`、`包` |
|
||||
| `market_price` | NUMERIC | `market_price` | 自动映射 | 市场价/建议售价(元) | 金额值 |
|
||||
| `goods_state` | INTEGER | `goods_state` | 自动映射 | 商品状态 | 整数枚举 |
|
||||
| `create_time` | TIMESTAMPTZ | `create_time` | 自动映射 | 商品创建时间 | ISO 时间戳 |
|
||||
| `update_time` | TIMESTAMPTZ | `update_time` | 自动映射 | 商品最后更新时间 | ISO 时间戳 |
|
||||
| `is_delete` | INTEGER | `is_delete` | 自动映射 | 软删除标记:0=正常,1=已删除 | `0` / `1` |
|
||||
| `not_sale` | INTEGER | `not_sale` | FACT_MAPPINGS | 是否停售:0=在售,1=停售 | `0` / `1` |
|
||||
| `scd2_*` | — | — | DWD 元数据 | SCD2 慢变维度追踪字段 | — |
|
||||
|
||||
---
|
||||
|
||||
## 2. dim_tenant_goods_ex(扩展表)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `tenant_goods_id` | BIGINT | `id` | FACT_MAPPINGS | 租户商品唯一标识(PK 之一) | 同主表 |
|
||||
| `remark_name` | VARCHAR(128) | `remark_name` | FACT_MAPPINGS | 商品备注名称 | 自由文本或 NULL |
|
||||
| `pinyin_initial` | VARCHAR | `pinyin_initial` | 自动映射 | 商品名称拼音首字母,用于快速检索 | 如 `BWPJ` |
|
||||
| `goods_cover` | VARCHAR | `goods_cover` | 自动映射 | 商品封面图 URL | HTTPS 链接或 NULL |
|
||||
| `goods_bar_code` | VARCHAR | `goods_bar_code` | FACT_MAPPINGS | 商品条码 | 条码字符串或 NULL |
|
||||
| `commodity_code` | VARCHAR | `commodity_code` | 自动映射 | 商品编码(单值,旧字段) | 编码字符串或 NULL |
|
||||
| `commodity_code_list` | TEXT[] | `commoditycode` | FACT_MAPPINGS (cast TEXT[]) | 商品编码数组。ODS `commoditycode` 存储 PG 数组格式 `{CODE1}`,CAST 为 `TEXT[]`。2026-02-21 从 VARCHAR(256) 改为 TEXT[] | `['1234571']` |
|
||||
| `min_discount_price` | NUMERIC | `min_discount_price` | 自动映射 | 最低折扣价(元) | 金额值 |
|
||||
| `cost_price` | NUMERIC | `cost_price` | 自动映射 | 成本价(元) | 金额值 |
|
||||
| `cost_price_type` | INTEGER | `cost_price_type` | 自动映射 | 成本价类型 | 整数枚举 |
|
||||
| `able_discount` | INTEGER | `able_discount` | 自动映射 | 是否允许折扣:0=不允许,1=允许 | `0` / `1` |
|
||||
| `sale_channel` | INTEGER | `sale_channel` | 自动映射 | 销售渠道 | 整数枚举 |
|
||||
| `is_warehousing` | INTEGER | `is_warehousing` | 自动映射 | 是否入库管理:0=否,1=是 | `0` / `1` |
|
||||
| `is_in_site` | BOOLEAN | `isinsite` | FACT_MAPPINGS (cast boolean) | 是否已分配到门店 | `true` / `false` |
|
||||
| `able_site_transfer` | INTEGER | `able_site_transfer` | 自动映射 | 是否允许门店间调拨 | `0` / `1` |
|
||||
| `common_sale_royalty` | INTEGER | `common_sale_royalty` | 自动映射 | 普通销售提成(分) | 整数 |
|
||||
| `point_sale_royalty` | INTEGER | `point_sale_royalty` | 自动映射 | 积分销售提成(分) | 整数 |
|
||||
| `out_goods_id` | BIGINT | `out_goods_id` | 自动映射 | 外部商品 ID(第三方系统关联) | 外部 ID 或 0 |
|
||||
| `scd2_*` | — | — | DWD 元数据 | SCD2 慢变维度追踪字段 | — |
|
||||
|
||||
## 3. 映射修正记录
|
||||
|
||||
| 日期 | 字段 | 修正内容 |
|
||||
|------|------|---------|
|
||||
| 2026-02-21 | `commodity_code_list` | 类型从 `VARCHAR(256)` 改为 `TEXT[]`。映射源从 `commodity_code`(单值)改为 `commoditycode`(PG 数组格式 `{xxx}`),通过 `::TEXT[]` CAST。现有数据通过 `ARRAY[commodity_code_list]` 迁移 |
|
||||
|
||||
---
|
||||
|
||||
## 4. 跳过字段说明
|
||||
|
||||
所有 ODS 业务字段均已映射到 DWD 主表或扩展表,无跳过字段。
|
||||
|
||||
> `commoditycode`(ODS 列)已作为 `commodity_code_list` 的映射源(`commoditycode::TEXT[]`),同时 `commodity_code`(单值旧字段)也保留在扩展表中。
|
||||
|
||||
---
|
||||
|
||||
## 5. 代码引用
|
||||
|
||||
- FACT_MAPPINGS:`dwd_load_task.py` → `FACT_MAPPINGS["dwd.dim_tenant_goods"]` / `FACT_MAPPINGS["dwd.dim_tenant_goods_ex"]`
|
||||
- TABLE_MAP:`"dwd.dim_tenant_goods" → "ods.tenant_goods_master"`
|
||||
- DWS 下游:无直接 DWS 汇总引用
|
||||
- 迁移脚本:`db/_archived/ddl_baseline_2026-02-22/db/etl_feiqiu/migrations/2026-02-21__add_time_slot_sale_merge_commodity_code.sql`(已归档)
|
||||
49
docs/database/README.md
Normal file
49
docs/database/README.md
Normal file
@@ -0,0 +1,49 @@
|
||||
# docs/database/ — 数据库文档中心
|
||||
|
||||
## DDL 基线(`ddl/` 子目录)
|
||||
|
||||
从测试库自动导出的完整 DDL,按 schema 分文件。重新生成:`python scripts/ops/gen_consolidated_ddl.py`
|
||||
|
||||
| 文件 | 数据库 | Schema | 内容 |
|
||||
|------|--------|--------|------|
|
||||
| `etl_feiqiu__meta.sql` | etl_feiqiu | meta | 调度元数据(3 表) |
|
||||
| `etl_feiqiu__ods.sql` | etl_feiqiu | ods | 原始数据层(23 表) |
|
||||
| `etl_feiqiu__dwd.sql` | etl_feiqiu | dwd | 明细数据层(44 表) |
|
||||
| `etl_feiqiu__core.sql` | etl_feiqiu | core | 跨门店标准化(7 表) |
|
||||
| `etl_feiqiu__dws.sql` | etl_feiqiu | dws | 汇总数据层(32 表 + 1 视图 + 8 物化视图) |
|
||||
| `etl_feiqiu__app.sql` | etl_feiqiu | app | RLS 视图层(7 视图,无表) |
|
||||
| `zqyy_app__public.sql` | zqyy_app | public | 小程序业务表(12 表) |
|
||||
| `fdw.sql` | — | — | FDW 跨库映射配置 |
|
||||
|
||||
## 数据字典(BD_Manual — ODS→DWD 字段映射)
|
||||
|
||||
记录每个 ODS 表到 DWD 表的字段映射、装载方式、业务含义。
|
||||
|
||||
| 文件 | ODS 表 | DWD 表 |
|
||||
|------|--------|--------|
|
||||
| `BD_Manual_assistant_accounts_master.md` | ods.assistant_accounts_master | dwd.dim_assistant / dim_assistant_ex |
|
||||
| `BD_Manual_assistant_service_records.md` | ods.assistant_service_records | dwd.dwd_assistant_service_log* |
|
||||
| `BD_Manual_dws_goods_stock_summary.md` | — | dws.dws_goods_stock_*_summary |
|
||||
| `BD_Manual_goods_stock_movements.md` | ods.goods_stock_movements | dwd.dwd_goods_stock_movement |
|
||||
| `BD_Manual_goods_stock_summary.md` | ods.goods_stock_summary | dwd.dwd_goods_stock_summary |
|
||||
| `BD_Manual_member_balance_changes.md` | ods.member_balance_changes | dwd.dwd_member_balance_change* |
|
||||
| `BD_Manual_recharge_settlements.md` | ods.recharge_settlements | dwd.dwd_recharge_order* |
|
||||
| `BD_Manual_site_tables_master.md` | ods.site_tables_master | dwd.dim_table* |
|
||||
| `BD_Manual_store_goods_master.md` | ods.store_goods_master | dwd.dim_store_goods* |
|
||||
| `BD_Manual_store_goods_sales_records.md` | ods.store_goods_sales_records | dwd.dwd_store_goods_sale* |
|
||||
| `BD_Manual_tenant_goods_master.md` | ods.tenant_goods_master | dwd.dim_tenant_goods* |
|
||||
| `BD_Manual_staff_info_master.md` | ods.staff_info_master | dwd.dim_staff / dim_staff_ex |
|
||||
|
||||
## 归档(`_archived/` 子目录)
|
||||
|
||||
已吸收进 DDL 基线的迁移变更记录,仅供历史参考:
|
||||
- 迁移变更类 BD_Manual(加列、改约束、删表、FDW 变更等)
|
||||
- `etl_feiqiu_schema_migration.md`(旧迁移汇总)
|
||||
- `zqyy_app_admin_web_tables.md`(建表记录)
|
||||
|
||||
## 相关资源
|
||||
|
||||
- 种子数据:`db/etl_feiqiu/seeds/`、`db/zqyy_app/seeds/`
|
||||
- FDW 配置:`db/fdw/`
|
||||
- DDL 生成脚本:`scripts/ops/gen_consolidated_ddl.py`
|
||||
- 迁移脚本归档:`db/_archived/ddl_baseline_2026-02-22/`
|
||||
@@ -0,0 +1,52 @@
|
||||
# BD_Manual:assistant_cancellation_records(助教废除记录)
|
||||
|
||||
> ODS 表:`ods.assistant_cancellation_records`
|
||||
> DWD 表:`dwd.dwd_assistant_trash_event`(主表)、`dwd.dwd_assistant_trash_event_ex`(扩展表)
|
||||
> API 接口:助教废除记录列表
|
||||
> JSON 路径:`assistant_cancellation_records.json → data.orderAssistantTrashLedgers`
|
||||
> 装载方式:事实表增量插入(`DwdLoadTask`)
|
||||
> 代码位置:`apps/etl/connectors/feiqiu/tasks/dwd/dwd_load_task.py`
|
||||
|
||||
---
|
||||
|
||||
## 1. dwd_assistant_trash_event(主表)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `assistant_trash_event_id` | BIGINT | `id` | FACT_MAPPINGS | 废除记录唯一标识(PK) | 飞球雪花 ID |
|
||||
| `site_id` | BIGINT | `siteid` | FACT_MAPPINGS | 门店 ID | 飞球门店 ID |
|
||||
| `tenant_id` | BIGINT | `tenant_id` | FACT_MAPPINGS | 租户 ID | 飞球租户 ID |
|
||||
| `assistant_no` | TEXT | `assistanton` | FACT_MAPPINGS | 助教编号(工号/序号)。ODS 列名 `assistantOn` 在 PG 中小写化为 `assistanton`,语义为助教的门店内编号 | 如 `31`、`1` |
|
||||
| `abolish_amount` | NUMERIC | `assistantabolishamount` | FACT_MAPPINGS | 废除金额(元),被废除的服务应收金额 | `0.00` ~ 金额值 |
|
||||
| `charge_minutes_raw` | INTEGER | `pdchargeminutes` | FACT_MAPPINGS | 陪打计费时长(分钟),被废除的服务时长 | 正整数 |
|
||||
| `table_id` | BIGINT | `tableid` | FACT_MAPPINGS | 台桌 ID | 飞球台桌 ID |
|
||||
| `table_area_id` | BIGINT | `tableareaid` | FACT_MAPPINGS | 台桌区域 ID | 飞球区域 ID |
|
||||
| `assistant_name` | TEXT | `assistantname` | FACT_MAPPINGS | 助教姓名快照 | 如 `张静然` |
|
||||
| `trash_reason` | TEXT | `trashreason` | FACT_MAPPINGS | 废除原因 | 自由文本 |
|
||||
| `create_time` | TIMESTAMPTZ | `createtime` | FACT_MAPPINGS | 废除操作时间 | ISO 时间戳 |
|
||||
|
||||
---
|
||||
|
||||
## 2. dwd_assistant_trash_event_ex(扩展表)
|
||||
|
||||
| DWD 列名 | 类型 | ODS 源列 | 映射方式 | 业务含义 | 取值范围/示例 |
|
||||
|----------|------|---------|---------|---------|-------------|
|
||||
| `assistant_trash_event_id` | BIGINT | `id` | FACT_MAPPINGS | 废除记录唯一标识(PK) | 同主表 |
|
||||
| `table_area_name` | TEXT | `tablearea` | FACT_MAPPINGS | 台桌区域名称快照 | 如 `大厅`、`VIP` |
|
||||
| `table_name` | VARCHAR(64) | `tablename` | FACT_MAPPINGS | 台桌名称快照 | 如 `1号台` |
|
||||
|
||||
---
|
||||
|
||||
## 3. 跳过字段说明
|
||||
|
||||
| ODS 字段 | 跳过原因 |
|
||||
|---------|---------|
|
||||
| `siteprofile` | JSONB 嵌套列,已由 `dim_site` / `dim_site_ex` 通过 JSONB 提取映射 |
|
||||
|
||||
---
|
||||
|
||||
## 4. 代码引用
|
||||
|
||||
- FACT_MAPPINGS:`dwd_load_task.py` → `FACT_MAPPINGS["dwd.dwd_assistant_trash_event"]` / `FACT_MAPPINGS["dwd.dwd_assistant_trash_event_ex"]`
|
||||
- TABLE_MAP:`"dwd.dwd_assistant_trash_event" → "ods.assistant_cancellation_records"`
|
||||
- DWS 下游:`dws_assistant_daily_task.py`(助教日业绩汇总,废除金额扣减)
|
||||
68
docs/database/_archived/BD_Manual_dim_member_add_birthday.md
Normal file
68
docs/database/_archived/BD_Manual_dim_member_add_birthday.md
Normal file
@@ -0,0 +1,68 @@
|
||||
# BD 手册:会员生日字段 ETL 链路补齐(C1)
|
||||
|
||||
## 概述
|
||||
|
||||
为支持会员生日信息从上游 API 完整传递到 DWS 层,在 ODS 和 DWD 两层的会员表中新增 `birthday DATE` 列。ODS 层从 API payload 提取生日值落地,DWD 层通过自动列映射和 SCD2 机制同步。
|
||||
|
||||
## 变更说明
|
||||
|
||||
| Schema | 表 | 变更类型 | 字段 | 类型 | 说明 |
|
||||
|--------|---|---------|------|------|------|
|
||||
| ods | member_profiles | 加列 | birthday | DATE | 会员生日,从上游 API payload 提取 |
|
||||
| dwd | dim_member | 加列 | birthday | DATE | 会员生日,ODS → DWD 自动列映射 |
|
||||
|
||||
## 兼容性
|
||||
|
||||
- ETL Connector:DwdLoadTask 通过 `_get_columns()` 自动读取 DWD 表列名,新增列会被自动包含在列映射中;SCD2 变化检测自动覆盖所有非元数据列,`birthday` 无需额外配置
|
||||
- 后端 API / 小程序:无影响(当前未直接读取 `dim_member`)
|
||||
- DWS 层:`member_visit_task` 和 `member_consumption_task` 的 `_extract_member_info()` SQL 中已引用 `birthday` 字段,通过 `COALESCE(fdw_app.member_birthday_manual.birthday_value, dim_member.birthday)` 实现手动补录优先于 API 来源的合并逻辑(详见 `BD_Manual_fdw_reverse_member_birthday.md`)
|
||||
|
||||
## 回滚策略
|
||||
|
||||
```sql
|
||||
BEGIN;
|
||||
ALTER TABLE ods.member_profiles DROP COLUMN IF EXISTS birthday;
|
||||
ALTER TABLE dwd.dim_member DROP COLUMN IF EXISTS birthday;
|
||||
COMMIT;
|
||||
```
|
||||
|
||||
回滚无数据丢失风险:`birthday` 为新增列,删除前所有值均为 NULL 或新写入的生日数据(可从 ODS payload JSONB 重新提取)。
|
||||
|
||||
## 验证步骤
|
||||
|
||||
```sql
|
||||
-- 1. 确认 ods.member_profiles.birthday 列存在且类型正确
|
||||
SELECT column_name, data_type
|
||||
FROM information_schema.columns
|
||||
WHERE table_schema = 'ods'
|
||||
AND table_name = 'member_profiles'
|
||||
AND column_name = 'birthday';
|
||||
-- 预期:1 行,data_type = 'date'
|
||||
|
||||
-- 2. 确认 dwd.dim_member.birthday 列存在且类型正确
|
||||
SELECT column_name, data_type
|
||||
FROM information_schema.columns
|
||||
WHERE table_schema = 'dwd'
|
||||
AND table_name = 'dim_member'
|
||||
AND column_name = 'birthday';
|
||||
-- 预期:1 行,data_type = 'date'
|
||||
|
||||
-- 3. 确认 dwd.dim_member.birthday 列注释已设置
|
||||
SELECT col_description(c.oid, a.attnum) AS column_comment
|
||||
FROM pg_class c
|
||||
JOIN pg_namespace n ON n.oid = c.relnamespace
|
||||
JOIN pg_attribute a ON a.attrelid = c.oid
|
||||
WHERE n.nspname = 'dwd'
|
||||
AND c.relname = 'dim_member'
|
||||
AND a.attname = 'birthday';
|
||||
-- 预期:'会员生日,来源:ODS member_profiles payload 中的 birthday 字段'
|
||||
```
|
||||
|
||||
## 关联文件
|
||||
|
||||
- 迁移脚本:`db/etl_feiqiu/migrations/2026-02-22__C1_dim_member_add_birthday.sql`
|
||||
- 主 DDL:`db/etl_feiqiu/schemas/ods.sql`(`member_profiles` 表)、`db/etl_feiqiu/schemas/dwd.sql`(`dim_member` 表)
|
||||
- 需求文档:`.kiro/specs/etl-aggregation-fix/requirements.md` — 需求 4.1, 4.2, 4.3, 4.4
|
||||
- ODS 入库逻辑:`apps/etl/connectors/feiqiu/tasks/ods/ods_tasks.py` — `member_profiles` 字段列表包含 `birthday`
|
||||
- DWS 生日合并:`BD_Manual_fdw_reverse_member_birthday.md` — FDW 反向映射供 DWS 任务 COALESCE 读取
|
||||
- 手动补录表:`BD_Manual_member_birthday_manual.md` — `zqyy_app.member_birthday_manual` 表文档
|
||||
@@ -0,0 +1,65 @@
|
||||
# BD 手册:删除助教废除(Abolish)独立链路表
|
||||
|
||||
## 迁移脚本
|
||||
|
||||
`db/etl_feiqiu/migrations/2026-02-22__drop_assistant_abolish_tables.sql`
|
||||
|
||||
## 变更说明
|
||||
|
||||
### 删除的表
|
||||
|
||||
| Schema | 表名 | 说明 |
|
||||
|--------|------|------|
|
||||
| `ods` | `assistant_cancellation_records` | 上游废除 API 原始数据(仅 78 条),不再抓取 |
|
||||
| `dwd` | `dwd_assistant_trash_event` | 废除事件主表,无消费者 |
|
||||
| `dwd` | `dwd_assistant_trash_event_ex` | 废除事件扩展表,无消费者 |
|
||||
|
||||
### 删除的索引
|
||||
|
||||
| Schema | 索引名 | 所属表 |
|
||||
|--------|--------|--------|
|
||||
| `ods` | `idx_ods_assistant_cancellation_records_latest` | `assistant_cancellation_records` |
|
||||
|
||||
### 清理的元数据
|
||||
|
||||
- `meta.etl_task` 中 `task_code = 'ODS_ASSISTANT_ABOLISH'` 的注册行
|
||||
|
||||
## 兼容性影响
|
||||
|
||||
- **ETL**:ODS 抓取任务 `ODS_ASSISTANT_ABOLISH` 已在代码层移除(Task 1–3),迁移脚本清理数据库残留
|
||||
- **DWD 加载**:`FACT_MAPPINGS` 和 `TABLE_MAP` 中的废除表映射已在代码层移除
|
||||
- **DWS 聚合**:`assistant_daily_task.py` 已改用 `dwd_assistant_service_log_ex.is_trash` 字段,不受影响
|
||||
- **后端 API**:无直接引用这些表
|
||||
- **小程序**:无直接引用这些表
|
||||
|
||||
## 回滚策略
|
||||
|
||||
1. 从 `db/etl_feiqiu/schemas/ods.sql` 恢复 `assistant_cancellation_records` 的 `CREATE TABLE`
|
||||
2. 从 `db/etl_feiqiu/schemas/dwd.sql` 恢复 `dwd_assistant_trash_event` / `_ex` 的 `CREATE TABLE`
|
||||
3. 重建索引:`CREATE INDEX idx_ods_assistant_cancellation_records_latest ON ods.assistant_cancellation_records (id, fetched_at DESC);`
|
||||
4. 重新注册任务:`INSERT INTO meta.etl_task (task_code, store_id, enabled) VALUES ('ODS_ASSISTANT_ABOLISH', <store_id>, TRUE);`
|
||||
5. ODS 数据可从上游 API 重新抓取(仅 78 条)
|
||||
|
||||
## 验证 SQL
|
||||
|
||||
```sql
|
||||
-- 1. 确认 3 张表已不存在
|
||||
SELECT tablename FROM pg_tables
|
||||
WHERE schemaname IN ('ods', 'dwd')
|
||||
AND tablename IN (
|
||||
'assistant_cancellation_records',
|
||||
'dwd_assistant_trash_event',
|
||||
'dwd_assistant_trash_event_ex'
|
||||
);
|
||||
-- 预期:0 行
|
||||
|
||||
-- 2. 确认索引已不存在
|
||||
SELECT indexname FROM pg_indexes
|
||||
WHERE schemaname = 'ods'
|
||||
AND indexname = 'idx_ods_assistant_cancellation_records_latest';
|
||||
-- 预期:0 行
|
||||
|
||||
-- 3. 确认 meta.etl_task 中无 ODS_ASSISTANT_ABOLISH 注册
|
||||
SELECT * FROM meta.etl_task WHERE task_code = 'ODS_ASSISTANT_ABOLISH';
|
||||
-- 预期:0 行
|
||||
```
|
||||
@@ -0,0 +1,93 @@
|
||||
# BD 手册:助教月度汇总表唯一约束变更(需求 A)
|
||||
|
||||
## 概述
|
||||
|
||||
`dws.dws_assistant_monthly_summary` 的唯一约束从 `(site_id, assistant_id, stat_month)` 变更为 `(site_id, assistant_id, stat_month, assistant_level_code)`,以支持助教月内多档位分段统计。
|
||||
|
||||
## 变更说明
|
||||
|
||||
| Schema | 表 | 变更类型 | 约束名 | 旧定义 | 新定义 |
|
||||
|--------|---|---------|--------|--------|--------|
|
||||
| dws | dws_assistant_monthly_summary | 改约束 | uk_dws_assistant_monthly | `(site_id, assistant_id, stat_month)` | `(site_id, assistant_id, stat_month, assistant_level_code)` |
|
||||
|
||||
### 变更原因
|
||||
|
||||
助教在同一月内可能因升级/降级而存在多个 `assistant_level_code`。旧约束只允许每个助教每月一行记录,导致:
|
||||
- `AssistantMonthlyTask` 按档位分组聚合时,INSERT 违反唯一约束
|
||||
- 临时修复(`MAX()` 聚合)丢失了档位维度信息,无法按档位分段计算工资
|
||||
|
||||
新约束允许同一助教同月按不同档位生成多行记录,支持精确的分段业绩统计和工资计算。
|
||||
|
||||
## 兼容性
|
||||
|
||||
- **ETL Connector**:`AssistantMonthlyTask` 的 `_extract_daily_aggregates()` 将同步修改 GROUP BY 加入 `assistant_level_code`(任务 7.2),INSERT 逻辑适配多行输出
|
||||
- **AssistantSalaryTask**:需适配多行月度汇总结构,按档位分段计算工资(任务 7.4)
|
||||
- **后端 API / 小程序**:无直接影响(当前未直接读取 `dws_assistant_monthly_summary`)
|
||||
- **DWS 下游任务**:`AssistantFinanceTask`、`AssistantCustomerTask` 不依赖此表的唯一约束,无影响
|
||||
|
||||
## 回滚策略
|
||||
|
||||
```sql
|
||||
-- ⚠️ 回滚前须先清理同一 (site_id, assistant_id, stat_month) 的多行数据,
|
||||
-- 否则恢复旧约束会因重复键失败。
|
||||
|
||||
-- 步骤 1:清理多档位数据(保留每组最新一条)
|
||||
DELETE FROM dws.dws_assistant_monthly_summary a
|
||||
USING (
|
||||
SELECT site_id, assistant_id, stat_month,
|
||||
MAX(updated_at) AS keep_updated_at
|
||||
FROM dws.dws_assistant_monthly_summary
|
||||
GROUP BY site_id, assistant_id, stat_month
|
||||
) b
|
||||
WHERE a.site_id = b.site_id
|
||||
AND a.assistant_id = b.assistant_id
|
||||
AND a.stat_month = b.stat_month
|
||||
AND a.updated_at <> b.keep_updated_at;
|
||||
|
||||
-- 步骤 2:恢复旧约束
|
||||
BEGIN;
|
||||
ALTER TABLE dws.dws_assistant_monthly_summary
|
||||
DROP CONSTRAINT IF EXISTS uk_dws_assistant_monthly;
|
||||
ALTER TABLE dws.dws_assistant_monthly_summary
|
||||
ADD CONSTRAINT uk_dws_assistant_monthly
|
||||
UNIQUE (site_id, assistant_id, stat_month);
|
||||
COMMIT;
|
||||
```
|
||||
|
||||
回滚后需重新执行 `DWS_ASSISTANT_MONTHLY` 任务以 MAX() 模式重新聚合数据。
|
||||
|
||||
## 验证步骤
|
||||
|
||||
```sql
|
||||
-- 1. 确认新约束存在且列组合正确
|
||||
SELECT conname, pg_get_constraintdef(oid) AS constraint_def
|
||||
FROM pg_constraint
|
||||
WHERE conrelid = 'dws.dws_assistant_monthly_summary'::regclass
|
||||
AND conname = 'uk_dws_assistant_monthly';
|
||||
-- 预期:1 行,constraint_def 包含 (site_id, assistant_id, stat_month, assistant_level_code)
|
||||
|
||||
-- 2. 确认约束列数为 4
|
||||
SELECT COUNT(*) AS col_count
|
||||
FROM pg_constraint c
|
||||
JOIN LATERAL unnest(c.conkey) AS col_num ON TRUE
|
||||
WHERE c.conrelid = 'dws.dws_assistant_monthly_summary'::regclass
|
||||
AND c.conname = 'uk_dws_assistant_monthly';
|
||||
-- 预期:4
|
||||
|
||||
-- 3. 确认旧的 3 列约束不存在
|
||||
SELECT conname, array_length(conkey, 1) AS col_count
|
||||
FROM pg_constraint
|
||||
WHERE conrelid = 'dws.dws_assistant_monthly_summary'::regclass
|
||||
AND contype = 'u';
|
||||
-- 预期:uk_dws_assistant_monthly 的 col_count = 4(非 3)
|
||||
```
|
||||
|
||||
## 关联文件
|
||||
|
||||
- 迁移脚本:`db/etl_feiqiu/migrations/2026-02-22__A_monthly_summary_uk_change.sql`
|
||||
- 主 DDL:`db/etl_feiqiu/schemas/dws.sql`(已合并)
|
||||
- 工资表约束变更:`BD_Manual_dws_assistant_salary_uk_change.md`
|
||||
- 需求文档:`.kiro/specs/etl-aggregation-fix/requirements.md` — 需求 2.1, 2.2, 2.3
|
||||
- 代码变更:
|
||||
- `apps/etl/connectors/feiqiu/tasks/dws/assistant_monthly_task.py` — GROUP BY 加入 `assistant_level_code`
|
||||
- `apps/etl/connectors/feiqiu/tasks/dws/assistant_salary_task.py` — 按档位分段计算工资
|
||||
@@ -0,0 +1,94 @@
|
||||
# BD 手册:助教工资计算表唯一约束变更(需求 A — 任务 7.4)
|
||||
|
||||
## 概述
|
||||
|
||||
`dws.dws_assistant_salary_calc` 的唯一约束从 `(site_id, assistant_id, salary_month)` 变更为 `(site_id, assistant_id, salary_month, assistant_level_code)`,以支持同一助教同月按不同档位分段计算工资。
|
||||
|
||||
## 变更说明
|
||||
|
||||
| Schema | 表 | 变更类型 | 约束名 | 旧定义 | 新定义 |
|
||||
|--------|---|---------|--------|--------|--------|
|
||||
| dws | dws_assistant_salary_calc | 改约束 | uk_dws_assistant_salary | `(site_id, assistant_id, salary_month)` | `(site_id, assistant_id, salary_month, assistant_level_code)` |
|
||||
|
||||
### 变更原因
|
||||
|
||||
上游 `dws_assistant_monthly_summary` 已按 `(site_id, assistant_id, stat_month, assistant_level_code)` 分行(任务 7.2/7.3)。`AssistantSalaryTask` 逐行读取月度汇总并计算工资,同一助教同月可能产生多条工资记录(对应不同档位)。旧约束 `(site_id, assistant_id, salary_month)` 会导致 INSERT 冲突。
|
||||
|
||||
### 代码变更
|
||||
|
||||
- `assistant_salary_task.py` 的 `get_primary_keys()` 返回值加入 `assistant_level_code`
|
||||
- `_extract_monthly_summary()`、`transform()`、`_calculate_salary()` 无需修改——已天然按行处理
|
||||
|
||||
## 兼容性
|
||||
|
||||
- **ETL Connector**:`AssistantSalaryTask` 的 `get_primary_keys()` 已同步更新,`upsert()` 方法使用该键做 ON CONFLICT
|
||||
- **`_delete_by_month()`**:按 `site_id + salary_month` 整月删除后重新插入,不受约束变更影响
|
||||
- **后端 API / 小程序**:无直接影响(当前未直接读取 `dws_assistant_salary_calc`)
|
||||
- **DWS 下游任务**:`AssistantFinanceDailyTask` 读取 salary_calc 做日度成本分摊,需确认按 assistant_id 聚合时能正确处理多档位行(现有逻辑按 assistant_id + stat_date 聚合,不受影响)
|
||||
|
||||
## 回滚策略
|
||||
|
||||
```sql
|
||||
-- ⚠️ 回滚前须先清理同一 (site_id, assistant_id, salary_month) 的多行数据,
|
||||
-- 否则恢复旧约束会因重复键失败。
|
||||
|
||||
-- 步骤 1:清理多档位数据(保留每组最新一条)
|
||||
DELETE FROM dws.dws_assistant_salary_calc a
|
||||
USING (
|
||||
SELECT site_id, assistant_id, salary_month,
|
||||
MAX(updated_at) AS keep_updated_at
|
||||
FROM dws.dws_assistant_salary_calc
|
||||
GROUP BY site_id, assistant_id, salary_month
|
||||
) b
|
||||
WHERE a.site_id = b.site_id
|
||||
AND a.assistant_id = b.assistant_id
|
||||
AND a.salary_month = b.salary_month
|
||||
AND a.updated_at <> b.keep_updated_at;
|
||||
|
||||
-- 步骤 2:恢复旧约束
|
||||
BEGIN;
|
||||
ALTER TABLE dws.dws_assistant_salary_calc
|
||||
DROP CONSTRAINT IF EXISTS uk_dws_assistant_salary;
|
||||
ALTER TABLE dws.dws_assistant_salary_calc
|
||||
ADD CONSTRAINT uk_dws_assistant_salary
|
||||
UNIQUE (site_id, assistant_id, salary_month);
|
||||
COMMIT;
|
||||
```
|
||||
|
||||
回滚后需重新执行 `DWS_ASSISTANT_SALARY` 任务,此时月度汇总表也应已回滚为单行模式。
|
||||
|
||||
## 验证步骤
|
||||
|
||||
```sql
|
||||
-- 1. 确认新约束存在且列组合正确
|
||||
SELECT conname, pg_get_constraintdef(oid) AS constraint_def
|
||||
FROM pg_constraint
|
||||
WHERE conrelid = 'dws.dws_assistant_salary_calc'::regclass
|
||||
AND conname = 'uk_dws_assistant_salary';
|
||||
-- 预期:1 行,constraint_def 包含 (site_id, assistant_id, salary_month, assistant_level_code)
|
||||
|
||||
-- 2. 确认约束列数为 4
|
||||
SELECT COUNT(*) AS col_count
|
||||
FROM pg_constraint c
|
||||
JOIN LATERAL unnest(c.conkey) AS col_num ON TRUE
|
||||
WHERE c.conrelid = 'dws.dws_assistant_salary_calc'::regclass
|
||||
AND c.conname = 'uk_dws_assistant_salary';
|
||||
-- 预期:4
|
||||
|
||||
-- 3. 测试同一助教同月不同档位可共存
|
||||
INSERT INTO dws.dws_assistant_salary_calc
|
||||
(site_id, tenant_id, assistant_id, salary_month, assistant_level_code)
|
||||
VALUES (99999, 99999, 99999, '2099-01-01', 10),
|
||||
(99999, 99999, 99999, '2099-01-01', 20);
|
||||
-- 预期:成功插入 2 行
|
||||
DELETE FROM dws.dws_assistant_salary_calc
|
||||
WHERE site_id = 99999 AND assistant_id = 99999 AND salary_month = '2099-01-01';
|
||||
```
|
||||
|
||||
## 关联文件
|
||||
|
||||
- 迁移脚本:`db/etl_feiqiu/migrations/2026-02-22__A_salary_calc_uk_change.sql`
|
||||
- 主 DDL:`db/etl_feiqiu/schemas/dws.sql`、`db/etl_feiqiu/schemas/schema_dws.sql`
|
||||
- 代码:`apps/etl/connectors/feiqiu/tasks/dws/assistant_salary_task.py`
|
||||
- 需求文档:`.kiro/specs/etl-aggregation-fix/requirements.md` — 需求 2.4
|
||||
- 前置任务:7.1(monthly_summary UK 变更)、7.2/7.3(月度聚合按档位分行)
|
||||
105
docs/database/_archived/BD_Manual_fdw_reverse_member_birthday.md
Normal file
105
docs/database/_archived/BD_Manual_fdw_reverse_member_birthday.md
Normal file
@@ -0,0 +1,105 @@
|
||||
# BD 手册:FDW 反向映射 — ETL 库读取业务库会员生日
|
||||
|
||||
## 概述
|
||||
|
||||
在 `etl_feiqiu`(生产)/ `test_etl_feiqiu`(测试)数据库中,通过 `postgres_fdw` 创建指向 `zqyy_app` / `test_zqyy_app` 的外部表 `fdw_app.member_birthday_manual`,使 ETL DWS 任务可只读访问助教手动补录的会员生日数据。
|
||||
|
||||
方向:`etl_feiqiu → zqyy_app`(与现有 `setup_fdw.sql` 的 `zqyy_app → etl_feiqiu` 方向相反)。
|
||||
|
||||
## 变更说明
|
||||
|
||||
| 库 | Schema | 对象 | 变更类型 | 说明 |
|
||||
|----|--------|------|---------|------|
|
||||
| etl_feiqiu / test_etl_feiqiu | — | zqyy_app_server / test_zqyy_app_server | 新建外部服务器 | 指向业务库 |
|
||||
| etl_feiqiu / test_etl_feiqiu | — | etl_user → app_reader 映射 | 新建用户映射 | 只读角色映射 |
|
||||
| etl_feiqiu / test_etl_feiqiu | fdw_app | — | 新建 schema | 存放来自业务库的外部表 |
|
||||
| etl_feiqiu / test_etl_feiqiu | fdw_app | member_birthday_manual | 新建外部表 | 映射业务库 public.member_birthday_manual |
|
||||
|
||||
### 外部表列定义
|
||||
|
||||
| 列名 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| id | BIGINT | 自增主键(源表 BIGSERIAL) |
|
||||
| member_id | BIGINT | 会员 ID |
|
||||
| birthday_value | DATE | 补录的生日值 |
|
||||
| recorded_by_assistant_id | BIGINT | 提交助教 ID |
|
||||
| recorded_by_name | VARCHAR(50) | 提交助教姓名 |
|
||||
| recorded_at | TIMESTAMPTZ | 提交时间 |
|
||||
| source | VARCHAR(20) | 数据来源标识 |
|
||||
| site_id | BIGINT | 门店 ID(多门店隔离,Requirements 13.1) |
|
||||
|
||||
### 角色说明
|
||||
|
||||
| 角色 | 所在库 | 用途 |
|
||||
|------|--------|------|
|
||||
| etl_user | etl_feiqiu | ETL 连接角色,通过 FDW 读取业务库数据 |
|
||||
| app_reader | zqyy_app | 业务库只读角色,供 FDW 用户映射使用 |
|
||||
|
||||
## 兼容性
|
||||
|
||||
- **ETL Connector**:DWS 任务(`member_visit_task`、`member_consumption_task`)通过 `fdw_app.member_birthday_manual` 读取手动补录生日,使用 `COALESCE(手动补录值, dim_member.birthday)` 合并
|
||||
- **后端 API**:无影响,后端直接写入 `zqyy_app.member_birthday_manual`
|
||||
- **小程序**:无影响
|
||||
- **现有 FDW**:与现有 `setup_fdw.sql`(zqyy_app → etl_feiqiu 方向)互不干扰,使用不同的 server 名和 schema 名
|
||||
|
||||
## 回滚策略
|
||||
|
||||
在 ETL 库(`etl_feiqiu` 或 `test_etl_feiqiu`)中按逆序执行:
|
||||
|
||||
```sql
|
||||
ALTER DEFAULT PRIVILEGES IN SCHEMA fdw_app REVOKE SELECT ON TABLES FROM etl_user;
|
||||
REVOKE SELECT ON ALL TABLES IN SCHEMA fdw_app FROM etl_user;
|
||||
REVOKE USAGE ON SCHEMA fdw_app FROM etl_user;
|
||||
DROP FOREIGN TABLE IF EXISTS fdw_app.member_birthday_manual;
|
||||
DROP SCHEMA IF EXISTS fdw_app CASCADE;
|
||||
DROP USER MAPPING IF EXISTS FOR etl_user SERVER zqyy_app_server; -- 生产
|
||||
-- DROP USER MAPPING IF EXISTS FOR etl_user SERVER test_zqyy_app_server; -- 测试
|
||||
DROP SERVER IF EXISTS zqyy_app_server CASCADE; -- 生产
|
||||
-- DROP SERVER IF EXISTS test_zqyy_app_server CASCADE; -- 测试
|
||||
```
|
||||
|
||||
回滚后 DWS 任务的生日读取将降级为仅使用 `dim_member.birthday`(API 来源),需确保降级逻辑已实现(任务 9.3)。
|
||||
|
||||
## 验证步骤
|
||||
|
||||
```sql
|
||||
-- 1. 确认外部服务器存在
|
||||
SELECT srvname, srvoptions
|
||||
FROM pg_foreign_server
|
||||
WHERE srvname IN ('zqyy_app_server', 'test_zqyy_app_server');
|
||||
-- 预期:1 行(生产或测试环境对应的 server)
|
||||
|
||||
-- 2. 确认用户映射存在
|
||||
SELECT s.srvname, um.umoptions
|
||||
FROM pg_user_mapping um
|
||||
JOIN pg_foreign_server s ON um.umserver = s.oid
|
||||
WHERE s.srvname IN ('zqyy_app_server', 'test_zqyy_app_server');
|
||||
-- 预期:1 行
|
||||
|
||||
-- 3. 确认 fdw_app schema 存在
|
||||
SELECT schema_name
|
||||
FROM information_schema.schemata
|
||||
WHERE schema_name = 'fdw_app';
|
||||
-- 预期:1 行
|
||||
|
||||
-- 4. 确认外部表列结构完整
|
||||
SELECT column_name, data_type
|
||||
FROM information_schema.columns
|
||||
WHERE table_schema = 'fdw_app'
|
||||
AND table_name = 'member_birthday_manual'
|
||||
ORDER BY ordinal_position;
|
||||
-- 预期:8 列
|
||||
|
||||
-- 5. 确认外部表可读取(需业务库侧表已存在且网络连通)
|
||||
SELECT COUNT(*) FROM fdw_app.member_birthday_manual;
|
||||
-- 预期:返回行数(可能为 0)
|
||||
```
|
||||
|
||||
## 关联文件
|
||||
|
||||
- 生产环境脚本:`db/fdw/setup_fdw_reverse.sql`
|
||||
- 测试环境脚本:`db/fdw/setup_fdw_reverse_test.sql`
|
||||
- 源表迁移脚本:`db/zqyy_app/migrations/2026-02-22__C2_member_birthday_manual.sql`
|
||||
- 源表文档:`docs/database/BD_Manual_member_birthday_manual.md`
|
||||
- 现有正向 FDW:`db/fdw/setup_fdw.sql`(zqyy_app → etl_feiqiu 方向)
|
||||
- 需求文档:`.kiro/specs/etl-aggregation-fix/requirements.md` — 需求 5.3
|
||||
56
docs/database/_archived/BD_Manual_fix_bc_sentinel_dates.md
Normal file
56
docs/database/_archived/BD_Manual_fix_bc_sentinel_dates.md
Normal file
@@ -0,0 +1,56 @@
|
||||
# BD 手册:DWD 层 BC 哨兵日期修复
|
||||
|
||||
## 概述
|
||||
|
||||
上游飞球 API 对"未设置"的日期字段返回哨兵值 `0001-01-01T00:00:00`。该值在 ODS 层以 `timestamp without time zone` 存储,转入 DWD 层 `timestamp with time zone` 列时,PostgreSQL 在 `Asia/Shanghai` 时区下将其转换为 `0001-12-31 23:59:43 BC`(公元前日期)。psycopg2 无法解析 BC 日期,导致 `ValueError: year -1 is out of range`。
|
||||
|
||||
## 受影响对象
|
||||
|
||||
| Schema | 表 | 列 | ODS 源列类型 | DWD 列类型 | 受影响行数 |
|
||||
|--------|---|---|---|---|---|
|
||||
| dwd | dim_assistant_ex | birth_date | timestamp | timestamptz | ~1,107 |
|
||||
| dwd | dim_member_card_account_ex | disable_start_time | timestamp | timestamptz | ~18,172 |
|
||||
| dwd | dim_member_card_account_ex | disable_end_time | timestamp | timestamptz | ~18,172 |
|
||||
| dwd | dwd_assistant_service_log_ex | composite_grade_time | timestamp | timestamptz | ~5,297 |
|
||||
| dwd | dwd_recharge_order_ex | revoke_time | timestamp | timestamptz | ~485 |
|
||||
| dwd | dwd_settlement_head_ex | revoke_time | timestamp | timestamptz | ~26,435 |
|
||||
|
||||
## 修复方案
|
||||
|
||||
### 存量数据
|
||||
|
||||
迁移脚本 `2026-02-22__fix_bc_sentinel_dates_to_null.sql`:将所有 `EXTRACT(year) < 1` 的 BC 日期置为 NULL。
|
||||
|
||||
### 增量防御(代码层)
|
||||
|
||||
在 `dwd_load_task.py` 的 SQL 表达式构造中加入哨兵值过滤(阈值 `0002-01-01`):
|
||||
|
||||
1. `_cast_expr`:对 `timestamptz` CAST 包裹 `CASE WHEN (base)::timestamp >= '0002-01-01'::timestamp THEN ... ELSE NULL END`
|
||||
2. `_build_fact_select_exprs`:事实表 `timestamp → timestamptz` 同类型列加过滤
|
||||
3. `_merge_dim_scd2`:维度表 ODS→DWD SELECT 和 DWD 现有数据读取均加过滤
|
||||
|
||||
### 设计决策
|
||||
|
||||
- 阈值选 `0002-01-01` 而非 `0001-01-02`:留出安全余量,公元 1 年的日期在业务上不可能出现
|
||||
- BC 日期 → NULL 而非保留原值:哨兵值本身无业务含义("未设置"),NULL 语义更准确
|
||||
- `(base)::timestamp` 显式 CAST:因为 base 可能是 JSONB `->>'key'` 提取的 text 类型,不能直接与 timestamp 比较
|
||||
|
||||
## 兼容性
|
||||
|
||||
- ETL Connector:v10 验证 19/19 任务全部成功,0 错误
|
||||
- 后端 API / 小程序:无影响
|
||||
- DWS 层:无影响(不引用这些时间列)
|
||||
|
||||
## 回滚
|
||||
|
||||
存量数据回滚不可行(BC 日期是错误数据)。代码回滚需还原 `_cast_expr`、`_build_fact_select_exprs`、`_merge_dim_scd2` 中的哨兵过滤逻辑。
|
||||
|
||||
## 验证
|
||||
|
||||
见 `docs/database/etl_feiqiu_schema_migration.md` 迁移 11 的验证 SQL。
|
||||
|
||||
## 关联文件
|
||||
|
||||
- 迁移脚本:`db/etl_feiqiu/migrations/2026-02-22__fix_bc_sentinel_dates_to_null.sql`
|
||||
- 代码修改:`apps/etl/connectors/feiqiu/tasks/dwd/dwd_load_task.py`
|
||||
- 存量修复脚本(已废弃):`scripts/ops/fix_bc_dates.py`
|
||||
104
docs/database/_archived/BD_Manual_member_birthday_manual.md
Normal file
104
docs/database/_archived/BD_Manual_member_birthday_manual.md
Normal file
@@ -0,0 +1,104 @@
|
||||
# BD 手册:助教手动补录会员生日表(C2)
|
||||
|
||||
## 概述
|
||||
|
||||
为支持助教手动提交客户生日信息(上游 API 未提供时的补充渠道),在 `zqyy_app` / `test_zqyy_app` 业务库中新建 `member_birthday_manual` 表。该表通过 FDW 只读映射供 ETL DWS 任务读取,与 `dim_member.birthday`(API 来源)配合实现生日数据的双源合并。
|
||||
|
||||
## 变更说明
|
||||
|
||||
| 库 | Schema | 表 | 变更类型 | 说明 |
|
||||
|----|--------|---|---------|------|
|
||||
| zqyy_app / test_zqyy_app | public | member_birthday_manual | 新建表 | 助教手动补录的会员生日信息 |
|
||||
|
||||
### 表结构
|
||||
|
||||
| 列名 | 类型 | 约束 | 说明 |
|
||||
|------|------|------|------|
|
||||
| id | BIGSERIAL | PRIMARY KEY | 自增主键 |
|
||||
| member_id | BIGINT | NOT NULL | 会员 ID |
|
||||
| birthday_value | DATE | NOT NULL | 补录的生日值 |
|
||||
| recorded_by_assistant_id | BIGINT | — | 提交助教 ID |
|
||||
| recorded_by_name | VARCHAR(50) | — | 提交助教姓名 |
|
||||
| recorded_at | TIMESTAMPTZ | NOT NULL DEFAULT NOW() | 提交时间 |
|
||||
| source | VARCHAR(20) | DEFAULT 'assistant' | 数据来源标识 |
|
||||
| site_id | BIGINT | NOT NULL | 门店 ID(多门店隔离,Requirements 13.1) |
|
||||
|
||||
### 约束与索引
|
||||
|
||||
| 名称 | 类型 | 列 | 说明 |
|
||||
|------|------|---|------|
|
||||
| member_birthday_manual_pkey | PRIMARY KEY | id | 主键 |
|
||||
| uk_member_birthday_manual | UNIQUE | (member_id, recorded_by_assistant_id) | 同一助教对同一会员只保留一条记录,支持 UPSERT |
|
||||
| idx_mbd_member | INDEX (btree) | member_id | 加速按会员 ID 查询 |
|
||||
| idx_mbd_site_id | INDEX (btree) | site_id | 加速按门店 ID 查询 |
|
||||
|
||||
## 兼容性
|
||||
|
||||
- **ETL Connector**:后续通过 FDW 反向映射(`fdw_app.member_birthday_manual`)在 ETL 库中只读访问,DWS 任务使用 `COALESCE(手动补录值, API 值)` 合并生日数据
|
||||
- **后端 API**:新增 `POST /member-birthday` 接口执行 UPSERT 写入(任务 9.4)
|
||||
- **小程序**:助教端调用后端 API 提交生日,无直接数据库访问
|
||||
- **现有数据**:新建表,无历史数据影响
|
||||
|
||||
## 回滚策略
|
||||
|
||||
```sql
|
||||
BEGIN;
|
||||
DROP TABLE IF EXISTS member_birthday_manual CASCADE;
|
||||
COMMIT;
|
||||
```
|
||||
|
||||
回滚无数据丢失风险:该表为新建表,回滚前的数据均为手动补录的生日信息,可由助教重新提交。若已建立 FDW 映射,需先在 ETL 库中删除外部表 `fdw_app.member_birthday_manual`。
|
||||
|
||||
## 验证步骤
|
||||
|
||||
```sql
|
||||
-- 1. 确认表存在
|
||||
SELECT table_name
|
||||
FROM information_schema.tables
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'member_birthday_manual';
|
||||
-- 预期:1 行
|
||||
|
||||
-- 2. 确认唯一约束 uk_member_birthday_manual 存在
|
||||
SELECT conname, contype
|
||||
FROM pg_constraint
|
||||
WHERE conrelid = 'member_birthday_manual'::regclass
|
||||
AND conname = 'uk_member_birthday_manual';
|
||||
-- 预期:1 行,contype = 'u'
|
||||
|
||||
-- 3. 确认索引 idx_mbd_member 存在
|
||||
SELECT indexname
|
||||
FROM pg_indexes
|
||||
WHERE tablename = 'member_birthday_manual'
|
||||
AND indexname = 'idx_mbd_member';
|
||||
-- 预期:1 行
|
||||
|
||||
-- 4. 确认表注释已设置
|
||||
SELECT obj_description('member_birthday_manual'::regclass, 'pg_class');
|
||||
-- 预期:'助教手动补录的会员生日信息'
|
||||
|
||||
-- 5. 确认列结构完整
|
||||
SELECT column_name, data_type, is_nullable
|
||||
FROM information_schema.columns
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'member_birthday_manual'
|
||||
ORDER BY ordinal_position;
|
||||
-- 预期:8 列(id, member_id, birthday_value, recorded_by_assistant_id,
|
||||
-- recorded_by_name, recorded_at, source, site_id)
|
||||
|
||||
-- 6. 确认 site_id 索引存在
|
||||
SELECT indexname
|
||||
FROM pg_indexes
|
||||
WHERE tablename = 'member_birthday_manual'
|
||||
AND indexname = 'idx_mbd_site_id';
|
||||
-- 预期:1 行
|
||||
```
|
||||
|
||||
## 关联文件
|
||||
|
||||
- 迁移脚本:`db/zqyy_app/migrations/2026-02-22__C2_member_birthday_manual.sql`
|
||||
- FDW 反向映射(生产):`db/fdw/setup_fdw_reverse.sql` — 在 etl_feiqiu 中创建 `fdw_app.member_birthday_manual` 外部表
|
||||
- FDW 反向映射(测试):`db/fdw/setup_fdw_reverse_test.sql` — 在 test_etl_feiqiu 中创建外部表
|
||||
- FDW 反向映射文档:`docs/database/BD_Manual_fdw_reverse_member_birthday.md`
|
||||
- 需求文档:`.kiro/specs/etl-aggregation-fix/requirements.md` — 需求 5.1, 5.3
|
||||
- 后续任务:9.3(DWS 任务生日读取优先级)、9.4(后端 API 接口)
|
||||
1478
docs/database/_archived/etl_feiqiu_schema_migration.md
Normal file
1478
docs/database/_archived/etl_feiqiu_schema_migration.md
Normal file
File diff suppressed because it is too large
Load Diff
121
docs/database/ddl/etl_feiqiu__app.sql
Normal file
121
docs/database/ddl/etl_feiqiu__app.sql
Normal file
@@ -0,0 +1,121 @@
|
||||
-- =============================================================================
|
||||
-- etl_feiqiu / app(RLS 视图层)
|
||||
-- 生成日期:2026-02-23
|
||||
-- 来源:测试库(通过脚本自动导出)
|
||||
-- =============================================================================
|
||||
|
||||
CREATE SCHEMA IF NOT EXISTS app;
|
||||
|
||||
-- 视图
|
||||
CREATE OR REPLACE VIEW app.v_assistant AS
|
||||
SELECT assistant_id,
|
||||
tenant_id,
|
||||
site_id,
|
||||
real_name,
|
||||
nickname,
|
||||
mobile,
|
||||
level,
|
||||
assistant_status,
|
||||
leave_status
|
||||
FROM core.dim_assistant a;
|
||||
;
|
||||
|
||||
CREATE OR REPLACE VIEW app.v_assistant_daily AS
|
||||
SELECT id,
|
||||
site_id,
|
||||
tenant_id,
|
||||
assistant_id,
|
||||
assistant_nickname,
|
||||
stat_date,
|
||||
total_service_count,
|
||||
total_hours,
|
||||
base_hours,
|
||||
bonus_hours,
|
||||
room_hours,
|
||||
total_ledger_amount,
|
||||
unique_customers,
|
||||
unique_tables,
|
||||
created_at
|
||||
FROM dws.dws_assistant_daily_detail d;
|
||||
;
|
||||
|
||||
CREATE OR REPLACE VIEW app.v_finance_daily AS
|
||||
SELECT id,
|
||||
site_id,
|
||||
tenant_id,
|
||||
stat_date,
|
||||
gross_amount,
|
||||
table_fee_amount,
|
||||
goods_amount,
|
||||
assistant_pd_amount,
|
||||
assistant_cx_amount,
|
||||
discount_total,
|
||||
confirmed_income,
|
||||
cash_inflow_total,
|
||||
recharge_count,
|
||||
recharge_total,
|
||||
order_count,
|
||||
member_order_count,
|
||||
guest_order_count,
|
||||
avg_order_amount,
|
||||
created_at
|
||||
FROM dws.dws_finance_daily_summary f;
|
||||
;
|
||||
|
||||
CREATE OR REPLACE VIEW app.v_member AS
|
||||
SELECT member_id,
|
||||
system_member_id,
|
||||
tenant_id,
|
||||
register_site_id AS site_id,
|
||||
mobile,
|
||||
nickname,
|
||||
member_card_grade_name,
|
||||
status
|
||||
FROM core.dim_member m;
|
||||
;
|
||||
|
||||
CREATE OR REPLACE VIEW app.v_member_consumption AS
|
||||
SELECT id,
|
||||
site_id,
|
||||
tenant_id,
|
||||
member_id,
|
||||
stat_date,
|
||||
member_nickname,
|
||||
card_grade_name,
|
||||
total_visit_count,
|
||||
total_consume_amount,
|
||||
total_recharge_amount,
|
||||
last_consume_date,
|
||||
first_consume_date,
|
||||
days_since_last,
|
||||
customer_tier,
|
||||
created_at
|
||||
FROM dws.dws_member_consumption_summary mc;
|
||||
;
|
||||
|
||||
CREATE OR REPLACE VIEW app.v_order_summary AS
|
||||
SELECT site_id,
|
||||
order_settle_id,
|
||||
order_trade_no,
|
||||
order_date,
|
||||
tenant_id,
|
||||
member_id,
|
||||
member_flag,
|
||||
order_original_amount,
|
||||
order_final_amount,
|
||||
total_paid_amount,
|
||||
refund_amount,
|
||||
net_income,
|
||||
created_at
|
||||
FROM dws.dws_order_summary os;
|
||||
;
|
||||
|
||||
CREATE OR REPLACE VIEW app.v_site AS
|
||||
SELECT site_id,
|
||||
tenant_id,
|
||||
shop_name,
|
||||
site_label,
|
||||
shop_status
|
||||
FROM core.dim_site s;
|
||||
;
|
||||
|
||||
99
docs/database/ddl/etl_feiqiu__core.sql
Normal file
99
docs/database/ddl/etl_feiqiu__core.sql
Normal file
@@ -0,0 +1,99 @@
|
||||
-- =============================================================================
|
||||
-- etl_feiqiu / core(跨门店标准化维度/事实)
|
||||
-- 生成日期:2026-02-23
|
||||
-- 来源:测试库(通过脚本自动导出)
|
||||
-- =============================================================================
|
||||
|
||||
CREATE SCHEMA IF NOT EXISTS core;
|
||||
|
||||
-- 表
|
||||
CREATE TABLE core.dim_assistant (
|
||||
assistant_id bigint NOT NULL,
|
||||
tenant_id bigint NOT NULL,
|
||||
site_id bigint NOT NULL,
|
||||
real_name text NOT NULL,
|
||||
nickname text,
|
||||
mobile text,
|
||||
level integer,
|
||||
assistant_status integer,
|
||||
leave_status integer
|
||||
);
|
||||
|
||||
CREATE TABLE core.dim_goods_category (
|
||||
category_id bigint NOT NULL,
|
||||
tenant_id bigint NOT NULL,
|
||||
category_name text NOT NULL,
|
||||
parent_id bigint,
|
||||
level integer
|
||||
);
|
||||
|
||||
CREATE TABLE core.dim_member (
|
||||
member_id bigint NOT NULL,
|
||||
system_member_id bigint,
|
||||
tenant_id bigint NOT NULL,
|
||||
register_site_id bigint NOT NULL,
|
||||
mobile text,
|
||||
nickname text,
|
||||
member_card_grade_name text,
|
||||
status integer
|
||||
);
|
||||
|
||||
CREATE TABLE core.dim_site (
|
||||
site_id bigint NOT NULL,
|
||||
tenant_id bigint NOT NULL,
|
||||
shop_name text NOT NULL,
|
||||
site_label text,
|
||||
shop_status integer,
|
||||
site_id_alias bigint
|
||||
);
|
||||
|
||||
CREATE TABLE core.dim_table (
|
||||
table_id bigint NOT NULL,
|
||||
site_id bigint NOT NULL,
|
||||
table_name text NOT NULL,
|
||||
site_table_area_name text,
|
||||
table_price numeric(18,2)
|
||||
);
|
||||
|
||||
CREATE TABLE core.fact_payment (
|
||||
payment_id bigint NOT NULL,
|
||||
site_id bigint NOT NULL,
|
||||
order_settle_id bigint,
|
||||
pay_type integer,
|
||||
pay_amount numeric(18,2),
|
||||
pay_time timestamp with time zone,
|
||||
status integer
|
||||
);
|
||||
|
||||
CREATE TABLE core.fact_settlement (
|
||||
order_settle_id bigint NOT NULL,
|
||||
site_id bigint NOT NULL,
|
||||
tenant_id bigint NOT NULL,
|
||||
order_trade_no bigint,
|
||||
member_id bigint,
|
||||
total_amount numeric(18,2),
|
||||
actual_amount numeric(18,2),
|
||||
discount_amount numeric(18,2),
|
||||
pay_status integer,
|
||||
settle_time timestamp with time zone,
|
||||
created_at timestamp with time zone,
|
||||
updated_at timestamp with time zone
|
||||
);
|
||||
|
||||
-- 约束(主键 / 唯一 / 外键)
|
||||
ALTER TABLE core.dim_assistant ADD CONSTRAINT dim_assistant_pkey PRIMARY KEY (assistant_id);
|
||||
ALTER TABLE core.dim_goods_category ADD CONSTRAINT dim_goods_category_pkey PRIMARY KEY (category_id);
|
||||
ALTER TABLE core.dim_member ADD CONSTRAINT dim_member_pkey PRIMARY KEY (member_id);
|
||||
ALTER TABLE core.dim_site ADD CONSTRAINT dim_site_pkey PRIMARY KEY (site_id);
|
||||
ALTER TABLE core.dim_table ADD CONSTRAINT dim_table_pkey PRIMARY KEY (table_id);
|
||||
ALTER TABLE core.fact_payment ADD CONSTRAINT fact_payment_pkey PRIMARY KEY (payment_id);
|
||||
ALTER TABLE core.fact_settlement ADD CONSTRAINT fact_settlement_pkey PRIMARY KEY (order_settle_id);
|
||||
|
||||
-- 索引
|
||||
CREATE INDEX idx_core_assistant_site ON core.dim_assistant USING btree (site_id);
|
||||
CREATE INDEX idx_core_member_site ON core.dim_member USING btree (register_site_id);
|
||||
CREATE INDEX idx_core_table_site ON core.dim_table USING btree (site_id);
|
||||
CREATE INDEX idx_core_payment_site ON core.fact_payment USING btree (site_id);
|
||||
CREATE INDEX idx_core_settlement_site ON core.fact_settlement USING btree (site_id);
|
||||
CREATE INDEX idx_core_settlement_time ON core.fact_settlement USING btree (settle_time);
|
||||
|
||||
1290
docs/database/ddl/etl_feiqiu__dwd.sql
Normal file
1290
docs/database/ddl/etl_feiqiu__dwd.sql
Normal file
File diff suppressed because it is too large
Load Diff
1453
docs/database/ddl/etl_feiqiu__dws.sql
Normal file
1453
docs/database/ddl/etl_feiqiu__dws.sql
Normal file
File diff suppressed because it is too large
Load Diff
77
docs/database/ddl/etl_feiqiu__meta.sql
Normal file
77
docs/database/ddl/etl_feiqiu__meta.sql
Normal file
@@ -0,0 +1,77 @@
|
||||
-- =============================================================================
|
||||
-- etl_feiqiu / meta(ETL 调度元数据)
|
||||
-- 生成日期:2026-02-23
|
||||
-- 来源:测试库(通过脚本自动导出)
|
||||
-- =============================================================================
|
||||
|
||||
CREATE SCHEMA IF NOT EXISTS meta;
|
||||
|
||||
-- 序列
|
||||
CREATE SEQUENCE IF NOT EXISTS meta.etl_cursor_cursor_id_seq AS bigint;
|
||||
CREATE SEQUENCE IF NOT EXISTS meta.etl_run_run_id_seq AS bigint;
|
||||
CREATE SEQUENCE IF NOT EXISTS meta.etl_task_task_id_seq AS bigint;
|
||||
|
||||
-- 表
|
||||
CREATE TABLE meta.etl_cursor (
|
||||
cursor_id bigint DEFAULT nextval('meta.etl_cursor_cursor_id_seq'::regclass) NOT NULL,
|
||||
task_id bigint NOT NULL,
|
||||
store_id bigint NOT NULL,
|
||||
last_start timestamp with time zone,
|
||||
last_end timestamp with time zone,
|
||||
last_id bigint,
|
||||
last_run_id bigint,
|
||||
extra jsonb DEFAULT '{}'::jsonb,
|
||||
created_at timestamp with time zone DEFAULT now(),
|
||||
updated_at timestamp with time zone DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE meta.etl_run (
|
||||
run_id bigint DEFAULT nextval('meta.etl_run_run_id_seq'::regclass) NOT NULL,
|
||||
run_uuid text NOT NULL,
|
||||
task_id bigint NOT NULL,
|
||||
store_id bigint NOT NULL,
|
||||
status text NOT NULL,
|
||||
started_at timestamp with time zone DEFAULT now(),
|
||||
ended_at timestamp with time zone,
|
||||
window_start timestamp with time zone,
|
||||
window_end timestamp with time zone,
|
||||
window_minutes integer,
|
||||
overlap_seconds integer,
|
||||
fetched_count integer DEFAULT 0,
|
||||
loaded_count integer DEFAULT 0,
|
||||
updated_count integer DEFAULT 0,
|
||||
skipped_count integer DEFAULT 0,
|
||||
error_count integer DEFAULT 0,
|
||||
unknown_fields integer DEFAULT 0,
|
||||
export_dir text,
|
||||
log_path text,
|
||||
request_params jsonb DEFAULT '{}'::jsonb,
|
||||
manifest jsonb DEFAULT '{}'::jsonb,
|
||||
error_message text,
|
||||
extra jsonb DEFAULT '{}'::jsonb
|
||||
);
|
||||
|
||||
CREATE TABLE meta.etl_task (
|
||||
task_id bigint DEFAULT nextval('meta.etl_task_task_id_seq'::regclass) NOT NULL,
|
||||
task_code text NOT NULL,
|
||||
store_id bigint NOT NULL,
|
||||
enabled boolean DEFAULT true,
|
||||
cursor_field text,
|
||||
window_minutes_default integer DEFAULT 30,
|
||||
overlap_seconds integer DEFAULT 600,
|
||||
page_size integer DEFAULT 200,
|
||||
retry_max integer DEFAULT 3,
|
||||
params jsonb DEFAULT '{}'::jsonb,
|
||||
created_at timestamp with time zone DEFAULT now(),
|
||||
updated_at timestamp with time zone DEFAULT now()
|
||||
);
|
||||
|
||||
-- 约束(主键 / 唯一 / 外键)
|
||||
ALTER TABLE meta.etl_cursor ADD CONSTRAINT etl_cursor_task_id_fkey FOREIGN KEY (task_id) REFERENCES meta.etl_task(task_id) ON DELETE CASCADE;
|
||||
ALTER TABLE meta.etl_cursor ADD CONSTRAINT etl_cursor_pkey PRIMARY KEY (cursor_id);
|
||||
ALTER TABLE meta.etl_cursor ADD CONSTRAINT etl_cursor_task_id_store_id_key UNIQUE (task_id, store_id);
|
||||
ALTER TABLE meta.etl_run ADD CONSTRAINT etl_run_task_id_fkey FOREIGN KEY (task_id) REFERENCES meta.etl_task(task_id) ON DELETE CASCADE;
|
||||
ALTER TABLE meta.etl_run ADD CONSTRAINT etl_run_pkey PRIMARY KEY (run_id);
|
||||
ALTER TABLE meta.etl_task ADD CONSTRAINT etl_task_pkey PRIMARY KEY (task_id);
|
||||
ALTER TABLE meta.etl_task ADD CONSTRAINT etl_task_task_code_store_id_key UNIQUE (task_code, store_id);
|
||||
|
||||
1133
docs/database/ddl/etl_feiqiu__ods.sql
Normal file
1133
docs/database/ddl/etl_feiqiu__ods.sql
Normal file
File diff suppressed because it is too large
Load Diff
71
docs/database/ddl/fdw.sql
Normal file
71
docs/database/ddl/fdw.sql
Normal file
@@ -0,0 +1,71 @@
|
||||
-- =============================================================================
|
||||
-- FDW 跨库映射(在 zqyy_app 中执行)
|
||||
-- 生成日期:2026-02-23
|
||||
-- 来源:db/fdw/setup_fdw.sql
|
||||
-- =============================================================================
|
||||
|
||||
-- =============================================================================
|
||||
-- FDW 映射配置(生产环境)— 在 zqyy_app 数据库中执行
|
||||
-- 用途:通过 postgres_fdw 将 etl_feiqiu.app schema 只读映射到 zqyy_app,
|
||||
-- 使业务后端无需直接连接 ETL 数据库即可读取汇总/维度数据。
|
||||
-- 前提:etl_feiqiu 数据库已部署 app schema 及 app_reader 角色(见 app.sql)
|
||||
-- 测试环境版本:setup_fdw_test.sql(指向 test_etl_feiqiu / test_zqyy_app)
|
||||
-- Requirements: 8.3, 8.4, 8.5
|
||||
-- =============================================================================
|
||||
|
||||
-- -----------------------------------------------------------------------------
|
||||
-- 1. 安装 postgres_fdw 扩展
|
||||
-- -----------------------------------------------------------------------------
|
||||
CREATE EXTENSION IF NOT EXISTS postgres_fdw;
|
||||
|
||||
-- -----------------------------------------------------------------------------
|
||||
-- 2. 创建外部服务器(指向 etl_feiqiu 数据库)
|
||||
-- 部署时按实际环境替换 host / port
|
||||
-- -----------------------------------------------------------------------------
|
||||
CREATE SERVER IF NOT EXISTS etl_feiqiu_server
|
||||
FOREIGN DATA WRAPPER postgres_fdw
|
||||
OPTIONS (host 'localhost', dbname 'etl_feiqiu', port '5432');
|
||||
|
||||
-- -----------------------------------------------------------------------------
|
||||
-- 3. 创建用户映射(只读角色)
|
||||
-- app_user = zqyy_app 侧的应用连接角色
|
||||
-- app_reader = etl_feiqiu 侧的只读角色(在 app.sql 中已创建)
|
||||
-- 密码占位符 '***',部署时替换为真实凭据
|
||||
-- -----------------------------------------------------------------------------
|
||||
CREATE USER MAPPING IF NOT EXISTS FOR app_user
|
||||
SERVER etl_feiqiu_server
|
||||
OPTIONS (user 'app_reader', password '***');
|
||||
|
||||
-- -----------------------------------------------------------------------------
|
||||
-- 4. 创建目标 schema(存放外部表)
|
||||
-- -----------------------------------------------------------------------------
|
||||
CREATE SCHEMA IF NOT EXISTS fdw_etl;
|
||||
|
||||
-- -----------------------------------------------------------------------------
|
||||
-- 5. 导入 etl_feiqiu.app schema 的所有外部表到 fdw_etl
|
||||
-- 映射为只读,zqyy_app 不存储 ETL 数据副本(Requirements 8.4)
|
||||
-- -----------------------------------------------------------------------------
|
||||
IMPORT FOREIGN SCHEMA app
|
||||
FROM SERVER etl_feiqiu_server
|
||||
INTO fdw_etl;
|
||||
|
||||
-- -----------------------------------------------------------------------------
|
||||
-- 6. 授权:允许 app_user 访问 fdw_etl schema 及其外部表
|
||||
-- -----------------------------------------------------------------------------
|
||||
GRANT USAGE ON SCHEMA fdw_etl TO app_user;
|
||||
GRANT SELECT ON ALL TABLES IN SCHEMA fdw_etl TO app_user;
|
||||
|
||||
-- 未来新导入的外部表自动授权
|
||||
ALTER DEFAULT PRIVILEGES IN SCHEMA fdw_etl GRANT SELECT ON TABLES TO app_user;
|
||||
|
||||
|
||||
-- =============================================================================
|
||||
-- 回滚脚本(按逆序执行)
|
||||
-- =============================================================================
|
||||
-- ALTER DEFAULT PRIVILEGES IN SCHEMA fdw_etl REVOKE SELECT ON TABLES FROM app_user;
|
||||
-- REVOKE SELECT ON ALL TABLES IN SCHEMA fdw_etl FROM app_user;
|
||||
-- REVOKE USAGE ON SCHEMA fdw_etl FROM app_user;
|
||||
-- DROP SCHEMA IF EXISTS fdw_etl CASCADE;
|
||||
-- DROP USER MAPPING IF EXISTS FOR app_user SERVER etl_feiqiu_server;
|
||||
-- DROP SERVER IF EXISTS etl_feiqiu_server CASCADE;
|
||||
-- DROP EXTENSION IF EXISTS postgres_fdw;
|
||||
192
docs/database/ddl/zqyy_app__public.sql
Normal file
192
docs/database/ddl/zqyy_app__public.sql
Normal file
@@ -0,0 +1,192 @@
|
||||
-- =============================================================================
|
||||
-- zqyy_app / public(小程序业务表)
|
||||
-- 生成日期:2026-02-23
|
||||
-- 来源:测试库(通过脚本自动导出)
|
||||
-- =============================================================================
|
||||
|
||||
CREATE SCHEMA IF NOT EXISTS public;
|
||||
|
||||
-- 序列
|
||||
CREATE SEQUENCE IF NOT EXISTS public.admin_users_id_seq AS integer;
|
||||
CREATE SEQUENCE IF NOT EXISTS public.approvals_id_seq AS bigint;
|
||||
CREATE SEQUENCE IF NOT EXISTS public.member_birthday_manual_id_seq AS bigint;
|
||||
CREATE SEQUENCE IF NOT EXISTS public.permissions_id_seq AS integer;
|
||||
CREATE SEQUENCE IF NOT EXISTS public.roles_id_seq AS integer;
|
||||
CREATE SEQUENCE IF NOT EXISTS public.tasks_id_seq AS bigint;
|
||||
CREATE SEQUENCE IF NOT EXISTS public.users_id_seq AS bigint;
|
||||
|
||||
-- 表
|
||||
CREATE TABLE public.admin_users (
|
||||
id integer DEFAULT nextval('admin_users_id_seq'::regclass) NOT NULL,
|
||||
username character varying(64) NOT NULL,
|
||||
password_hash character varying(256) NOT NULL,
|
||||
display_name character varying(128),
|
||||
site_id bigint NOT NULL,
|
||||
is_active boolean DEFAULT true,
|
||||
created_at timestamp with time zone DEFAULT now(),
|
||||
updated_at timestamp with time zone DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE public.approvals (
|
||||
id bigint DEFAULT nextval('approvals_id_seq'::regclass) NOT NULL,
|
||||
task_id bigint,
|
||||
approver_id bigint,
|
||||
status text DEFAULT 'pending'::text,
|
||||
comment text,
|
||||
site_id bigint NOT NULL,
|
||||
created_at timestamp with time zone DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE public.member_birthday_manual (
|
||||
id bigint DEFAULT nextval('member_birthday_manual_id_seq'::regclass) NOT NULL,
|
||||
member_id bigint NOT NULL,
|
||||
birthday_value date NOT NULL,
|
||||
recorded_by_assistant_id bigint,
|
||||
recorded_by_name character varying(50),
|
||||
recorded_at timestamp with time zone DEFAULT now() NOT NULL,
|
||||
source character varying(20) DEFAULT 'assistant'::character varying,
|
||||
site_id bigint NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE public.permissions (
|
||||
id integer DEFAULT nextval('permissions_id_seq'::regclass) NOT NULL,
|
||||
resource text NOT NULL,
|
||||
action text NOT NULL,
|
||||
description text
|
||||
);
|
||||
|
||||
CREATE TABLE public.role_permissions (
|
||||
role_id integer NOT NULL,
|
||||
permission_id integer NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE public.roles (
|
||||
id integer DEFAULT nextval('roles_id_seq'::regclass) NOT NULL,
|
||||
name text NOT NULL,
|
||||
description text,
|
||||
site_id bigint NOT NULL,
|
||||
created_at timestamp with time zone DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE public.scheduled_tasks (
|
||||
id uuid DEFAULT gen_random_uuid() NOT NULL,
|
||||
site_id bigint NOT NULL,
|
||||
name character varying(256) NOT NULL,
|
||||
task_codes _text NOT NULL,
|
||||
task_config jsonb NOT NULL,
|
||||
schedule_config jsonb NOT NULL,
|
||||
enabled boolean DEFAULT true,
|
||||
last_run_at timestamp with time zone,
|
||||
next_run_at timestamp with time zone,
|
||||
run_count integer DEFAULT 0,
|
||||
last_status character varying(20),
|
||||
created_at timestamp with time zone DEFAULT now(),
|
||||
updated_at timestamp with time zone DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE public.task_execution_log (
|
||||
id uuid DEFAULT gen_random_uuid() NOT NULL,
|
||||
queue_id uuid,
|
||||
site_id bigint NOT NULL,
|
||||
task_codes _text NOT NULL,
|
||||
status character varying(20) NOT NULL,
|
||||
started_at timestamp with time zone NOT NULL,
|
||||
finished_at timestamp with time zone,
|
||||
exit_code integer,
|
||||
duration_ms integer,
|
||||
command text,
|
||||
output_log text,
|
||||
error_log text,
|
||||
summary jsonb,
|
||||
created_at timestamp with time zone DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE public.task_queue (
|
||||
id uuid DEFAULT gen_random_uuid() NOT NULL,
|
||||
site_id bigint NOT NULL,
|
||||
config jsonb NOT NULL,
|
||||
status character varying(20) DEFAULT 'pending'::character varying NOT NULL,
|
||||
"position" integer DEFAULT 0 NOT NULL,
|
||||
created_at timestamp with time zone DEFAULT now(),
|
||||
started_at timestamp with time zone,
|
||||
finished_at timestamp with time zone,
|
||||
exit_code integer,
|
||||
error_message text
|
||||
);
|
||||
|
||||
CREATE TABLE public.tasks (
|
||||
id bigint DEFAULT nextval('tasks_id_seq'::regclass) NOT NULL,
|
||||
title text NOT NULL,
|
||||
description text,
|
||||
status text DEFAULT 'pending'::text,
|
||||
assignee_id bigint,
|
||||
creator_id bigint,
|
||||
site_id bigint NOT NULL,
|
||||
created_at timestamp with time zone DEFAULT now(),
|
||||
updated_at timestamp with time zone DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE public.user_roles (
|
||||
user_id bigint NOT NULL,
|
||||
role_id integer NOT NULL,
|
||||
site_id bigint NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE public.users (
|
||||
id bigint DEFAULT nextval('users_id_seq'::regclass) NOT NULL,
|
||||
wx_openid text,
|
||||
mobile text,
|
||||
nickname text,
|
||||
status integer DEFAULT 1,
|
||||
site_id bigint NOT NULL,
|
||||
created_at timestamp with time zone DEFAULT now(),
|
||||
updated_at timestamp with time zone DEFAULT now()
|
||||
);
|
||||
|
||||
-- 约束(主键 / 唯一 / 外键)
|
||||
ALTER TABLE admin_users ADD CONSTRAINT admin_users_pkey PRIMARY KEY (id);
|
||||
ALTER TABLE admin_users ADD CONSTRAINT admin_users_username_key UNIQUE (username);
|
||||
ALTER TABLE approvals ADD CONSTRAINT approvals_approver_id_fkey FOREIGN KEY (approver_id) REFERENCES users(id);
|
||||
ALTER TABLE approvals ADD CONSTRAINT approvals_task_id_fkey FOREIGN KEY (task_id) REFERENCES tasks(id) ON DELETE CASCADE;
|
||||
ALTER TABLE approvals ADD CONSTRAINT approvals_pkey PRIMARY KEY (id);
|
||||
ALTER TABLE member_birthday_manual ADD CONSTRAINT member_birthday_manual_pkey PRIMARY KEY (id);
|
||||
ALTER TABLE member_birthday_manual ADD CONSTRAINT uk_member_birthday_manual UNIQUE (member_id, recorded_by_assistant_id);
|
||||
ALTER TABLE permissions ADD CONSTRAINT permissions_pkey PRIMARY KEY (id);
|
||||
ALTER TABLE permissions ADD CONSTRAINT permissions_resource_action_key UNIQUE (resource, action);
|
||||
ALTER TABLE role_permissions ADD CONSTRAINT role_permissions_permission_id_fkey FOREIGN KEY (permission_id) REFERENCES permissions(id) ON DELETE CASCADE;
|
||||
ALTER TABLE role_permissions ADD CONSTRAINT role_permissions_role_id_fkey FOREIGN KEY (role_id) REFERENCES roles(id) ON DELETE CASCADE;
|
||||
ALTER TABLE role_permissions ADD CONSTRAINT role_permissions_pkey PRIMARY KEY (role_id, permission_id);
|
||||
ALTER TABLE roles ADD CONSTRAINT roles_pkey PRIMARY KEY (id);
|
||||
ALTER TABLE roles ADD CONSTRAINT roles_name_key UNIQUE (name);
|
||||
ALTER TABLE scheduled_tasks ADD CONSTRAINT scheduled_tasks_pkey PRIMARY KEY (id);
|
||||
ALTER TABLE task_execution_log ADD CONSTRAINT task_execution_log_queue_id_fkey FOREIGN KEY (queue_id) REFERENCES task_queue(id);
|
||||
ALTER TABLE task_execution_log ADD CONSTRAINT task_execution_log_pkey PRIMARY KEY (id);
|
||||
ALTER TABLE task_queue ADD CONSTRAINT task_queue_pkey PRIMARY KEY (id);
|
||||
ALTER TABLE tasks ADD CONSTRAINT tasks_assignee_id_fkey FOREIGN KEY (assignee_id) REFERENCES users(id);
|
||||
ALTER TABLE tasks ADD CONSTRAINT tasks_creator_id_fkey FOREIGN KEY (creator_id) REFERENCES users(id);
|
||||
ALTER TABLE tasks ADD CONSTRAINT tasks_pkey PRIMARY KEY (id);
|
||||
ALTER TABLE user_roles ADD CONSTRAINT user_roles_role_id_fkey FOREIGN KEY (role_id) REFERENCES roles(id) ON DELETE CASCADE;
|
||||
ALTER TABLE user_roles ADD CONSTRAINT user_roles_user_id_fkey FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE;
|
||||
ALTER TABLE user_roles ADD CONSTRAINT user_roles_pkey PRIMARY KEY (user_id, role_id);
|
||||
ALTER TABLE users ADD CONSTRAINT users_pkey PRIMARY KEY (id);
|
||||
ALTER TABLE users ADD CONSTRAINT users_wx_openid_key UNIQUE (wx_openid);
|
||||
|
||||
-- 索引
|
||||
CREATE INDEX idx_admin_users_site ON public.admin_users USING btree (site_id);
|
||||
CREATE INDEX idx_approvals_site_id ON public.approvals USING btree (site_id);
|
||||
CREATE INDEX idx_approvals_task_id ON public.approvals USING btree (task_id);
|
||||
CREATE INDEX idx_mbd_member ON public.member_birthday_manual USING btree (member_id);
|
||||
CREATE INDEX idx_mbd_site_id ON public.member_birthday_manual USING btree (site_id);
|
||||
CREATE INDEX idx_roles_site_id ON public.roles USING btree (site_id);
|
||||
CREATE INDEX idx_scheduled_tasks_next_run ON public.scheduled_tasks USING btree (next_run_at) WHERE (enabled = true);
|
||||
CREATE INDEX idx_scheduled_tasks_site ON public.scheduled_tasks USING btree (site_id);
|
||||
CREATE INDEX idx_execution_log_site_started ON public.task_execution_log USING btree (site_id, started_at DESC);
|
||||
CREATE INDEX idx_task_queue_site_position ON public.task_queue USING btree (site_id, "position") WHERE ((status)::text = 'pending'::text);
|
||||
CREATE INDEX idx_task_queue_status ON public.task_queue USING btree (status);
|
||||
CREATE INDEX idx_tasks_assignee_id ON public.tasks USING btree (assignee_id);
|
||||
CREATE INDEX idx_tasks_site_id ON public.tasks USING btree (site_id);
|
||||
CREATE INDEX idx_tasks_status ON public.tasks USING btree (status);
|
||||
CREATE INDEX idx_user_roles_site_id ON public.user_roles USING btree (site_id);
|
||||
CREATE INDEX idx_users_mobile ON public.users USING btree (mobile);
|
||||
CREATE INDEX idx_users_site_id ON public.users USING btree (site_id);
|
||||
|
||||
@@ -1,192 +0,0 @@
|
||||
# etl_feiqiu Schema 迁移文档
|
||||
|
||||
---
|
||||
|
||||
## 迁移 2:ODS "取最新版本"复合索引(2026-02-17)
|
||||
|
||||
### 迁移文件
|
||||
|
||||
`db/etl_feiqiu/migrations/2026-02-17__add_ods_latest_version_indexes.sql`
|
||||
|
||||
### 变更说明
|
||||
|
||||
为全部 23 张 ODS 表添加 `(业务主键, fetched_at DESC)` 复合索引,支持 `DISTINCT ON (pk) ORDER BY pk, fetched_at DESC` 查询模式高效取每条业务记录的最新版本。
|
||||
|
||||
索引命名规范:`idx_ods_{table_name}_latest`
|
||||
|
||||
| # | 表名 | 业务主键列 | 索引名 |
|
||||
|---|------|-----------|--------|
|
||||
| 1 | assistant_accounts_master | id | idx_ods_assistant_accounts_master_latest |
|
||||
| 2 | settlement_records | id | idx_ods_settlement_records_latest |
|
||||
| 3 | table_fee_transactions | id | idx_ods_table_fee_transactions_latest |
|
||||
| 4 | assistant_service_records | id | idx_ods_assistant_service_records_latest |
|
||||
| 5 | assistant_cancellation_records | id | idx_ods_assistant_cancellation_records_latest |
|
||||
| 6 | store_goods_sales_records | id | idx_ods_store_goods_sales_records_latest |
|
||||
| 7 | payment_transactions | id | idx_ods_payment_transactions_latest |
|
||||
| 8 | refund_transactions | id | idx_ods_refund_transactions_latest |
|
||||
| 9 | platform_coupon_redemption_records | id | idx_ods_platform_coupon_redemption_records_latest |
|
||||
| 10 | member_profiles | id | idx_ods_member_profiles_latest |
|
||||
| 11 | member_stored_value_cards | id | idx_ods_member_stored_value_cards_latest |
|
||||
| 12 | member_balance_changes | id | idx_ods_member_balance_changes_latest |
|
||||
| 13 | recharge_settlements | id | idx_ods_recharge_settlements_latest |
|
||||
| 14 | group_buy_packages | id | idx_ods_group_buy_packages_latest |
|
||||
| 15 | group_buy_redemption_records | id | idx_ods_group_buy_redemption_records_latest |
|
||||
| 16 | goods_stock_summary | siteGoodsId | idx_ods_goods_stock_summary_latest |
|
||||
| 17 | goods_stock_movements | siteGoodsStockId | idx_ods_goods_stock_movements_latest |
|
||||
| 18 | site_tables_master | id | idx_ods_site_tables_master_latest |
|
||||
| 19 | stock_goods_category_tree | id | idx_ods_stock_goods_category_tree_latest |
|
||||
| 20 | store_goods_master | id | idx_ods_store_goods_master_latest |
|
||||
| 21 | table_fee_discount_records | id | idx_ods_table_fee_discount_records_latest |
|
||||
| 22 | tenant_goods_master | id | idx_ods_tenant_goods_master_latest |
|
||||
| 23 | settlement_ticket_details | orderSettleId | idx_ods_settlement_ticket_details_latest |
|
||||
|
||||
### 关联需求
|
||||
|
||||
`ods-dedup-standardize` Requirements 6.1, 6.2, 6.3
|
||||
|
||||
### 兼容性
|
||||
|
||||
- **非破坏性变更**:仅新增索引,不修改表结构、不影响现有数据和写入逻辑
|
||||
- **ETL Connector**:无需改动;索引自动加速 `skip_unchanged` 去重查询和 `_mark_missing_as_deleted` 快照对比查询
|
||||
- **后端 API / 小程序**:不受影响(不直接查询 ODS 层)
|
||||
- **`CREATE INDEX CONCURRENTLY`**:在线创建,不阻塞表的读写操作;但不能在事务块内执行,需逐条运行或使用支持单语句模式的工具
|
||||
- **DDL 源文件**:索引定义已同步写入 `db/etl_feiqiu/schemas/ods.sql`,新环境初始化时自动创建
|
||||
|
||||
### 回滚策略
|
||||
|
||||
逐条删除索引即可,不影响数据:
|
||||
|
||||
```sql
|
||||
DROP INDEX IF EXISTS ods.idx_ods_assistant_accounts_master_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_settlement_records_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_table_fee_transactions_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_assistant_service_records_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_assistant_cancellation_records_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_store_goods_sales_records_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_payment_transactions_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_refund_transactions_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_platform_coupon_redemption_records_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_member_profiles_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_member_stored_value_cards_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_member_balance_changes_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_recharge_settlements_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_group_buy_packages_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_group_buy_redemption_records_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_goods_stock_summary_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_goods_stock_movements_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_site_tables_master_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_stock_goods_category_tree_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_store_goods_master_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_table_fee_discount_records_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_tenant_goods_master_latest;
|
||||
DROP INDEX IF EXISTS ods.idx_ods_settlement_ticket_details_latest;
|
||||
```
|
||||
|
||||
### 验证 SQL
|
||||
|
||||
```sql
|
||||
-- 1. 验证 23 个索引均已创建
|
||||
SELECT indexname, tablename
|
||||
FROM pg_indexes
|
||||
WHERE schemaname = 'ods'
|
||||
AND indexname LIKE 'idx_ods_%_latest'
|
||||
ORDER BY indexname;
|
||||
|
||||
-- 2. 验证索引数量为 23
|
||||
SELECT COUNT(*) AS index_count
|
||||
FROM pg_indexes
|
||||
WHERE schemaname = 'ods'
|
||||
AND indexname LIKE 'idx_ods_%_latest';
|
||||
|
||||
-- 3. 验证索引列定义正确(以 member_profiles 为例)
|
||||
SELECT indexdef
|
||||
FROM pg_indexes
|
||||
WHERE schemaname = 'ods'
|
||||
AND indexname = 'idx_ods_member_profiles_latest';
|
||||
|
||||
-- 4. 验证索引可用(非 INVALID 状态)
|
||||
SELECT c.relname AS index_name, i.indisvalid
|
||||
FROM pg_index i
|
||||
JOIN pg_class c ON c.oid = i.indexrelid
|
||||
JOIN pg_namespace n ON n.oid = c.relnamespace
|
||||
WHERE n.nspname = 'ods'
|
||||
AND c.relname LIKE 'idx_ods_%_latest'
|
||||
AND i.indisvalid = false;
|
||||
-- 预期结果:0 行(无无效索引)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 迁移 1:六层 Schema 架构重组
|
||||
|
||||
### 变更说明
|
||||
|
||||
将现有 4 个 schema 重组为 6 层 schema 架构:
|
||||
|
||||
| 原 Schema | 新 Schema | 文件 | 说明 |
|
||||
|-----------|-----------|------|------|
|
||||
| etl_admin | meta | meta.sql | 调度、游标、运行记录(3 表) |
|
||||
| billiards_ods | ods | ods.sql | ODS 原始数据(23 表) |
|
||||
| billiards_dwd | dwd | dwd.sql | DWD 明细,保留 main+EX 拆分(40 表) |
|
||||
| (新增) | core | core.sql | 统一维度/事实最小字段集(7 表) |
|
||||
| billiards_dws | dws | dws.sql | DWS 汇总(29 表) |
|
||||
| (新增) | app | app.sql | 视图+RLS(7 视图,6 策略) |
|
||||
|
||||
### 新增表(core schema)
|
||||
- core.dim_site — 门店维度核心字段
|
||||
- core.dim_member — 会员维度核心字段
|
||||
- core.dim_assistant — 助教维度核心字段
|
||||
- core.dim_table — 台桌维度核心字段
|
||||
- core.dim_goods_category — 商品分类维度核心字段
|
||||
- core.fact_settlement — 结算事实核心字段
|
||||
- core.fact_payment — 支付事实核心字段
|
||||
|
||||
### 新增视图(app schema)
|
||||
- pp.v_site — 门店视图
|
||||
- pp.v_member — 会员视图
|
||||
- pp.v_assistant — 助教视图
|
||||
- pp.v_assistant_daily — 助教日明细视图
|
||||
- pp.v_finance_daily — 财务日报视图
|
||||
- pp.v_member_consumption — 会员消费汇总视图
|
||||
- pp.v_order_summary — 订单汇总视图
|
||||
|
||||
### RLS 策略
|
||||
- 所有 core 表启用 ROW LEVEL SECURITY
|
||||
- 策略基于 current_setting('app.current_site_id')::bigint 过滤
|
||||
- 角色 pp_reader 仅有 SELECT 权限
|
||||
|
||||
## 兼容性
|
||||
|
||||
- **ETL Connector**:所有代码中的 schema 引用已更新完成(etl_admin → meta, billiards_ods → ods, billiards_dwd → dwd, billiards_dws → dws)
|
||||
- **后端 API**:etl_status 路由已更新为 meta.etl_cursor;通过 app schema 视图访问,无需直接引用底层表
|
||||
- **管理后台**:已由 `apps/admin-web/` Web 管理后台完全替代原 PySide6 桌面 GUI,通过后端 API 访问数据
|
||||
- **小程序**:通过 FDW 映射 app schema,不受影响
|
||||
|
||||
## 回滚策略
|
||||
|
||||
1. 删除新 schema:DROP SCHEMA IF EXISTS meta, ods, dwd, core, dws, app CASCADE;
|
||||
2. 重建原 schema:执行原始 schema_etl_admin.sql、schema_ODS_doc.sql、schema_dwd_doc.sql、schema_dws.sql
|
||||
3. 原始 DDL 文件保留在 db/etl_feiqiu/schemas/schema_*.sql 作为参考
|
||||
|
||||
## 验证 SQL
|
||||
|
||||
`sql
|
||||
-- 1. 验证六个 schema 均已创建
|
||||
SELECT schema_name FROM information_schema.schemata
|
||||
WHERE schema_name IN ('meta', 'ods', 'dwd', 'core', 'dws', 'app')
|
||||
ORDER BY schema_name;
|
||||
|
||||
-- 2. 验证各 schema 表数量
|
||||
SELECT table_schema, COUNT(*) AS table_count
|
||||
FROM information_schema.tables
|
||||
WHERE table_schema IN ('meta', 'ods', 'dwd', 'core', 'dws', 'app')
|
||||
GROUP BY table_schema ORDER BY table_schema;
|
||||
|
||||
-- 3. 验证 RLS 策略已启用
|
||||
SELECT schemaname, tablename, rowsecurity
|
||||
FROM pg_tables
|
||||
WHERE schemaname = 'core' AND rowsecurity = true;
|
||||
|
||||
-- 4. 验证 app_reader 角色存在
|
||||
SELECT rolname FROM pg_roles WHERE rolname = 'app_reader';
|
||||
`
|
||||
Reference in New Issue
Block a user