Compare commits

...

13 Commits

Author SHA1 Message Date
Neo
a3f4d04335 Updata2 2026-02-04 21:39:01 +08:00
Neo
ee773a9b52 确认 1 2026-02-04 21:38:22 +08:00
Neo
15948cbd64 同意 2026-02-04 21:19:34 +08:00
Neo
294c6edbc9 更新数据库文档 20260201-2 2026-02-01 23:42:18 +08:00
Neo
9b2c2c5c78 更新20260201-1 2026-02-01 22:04:15 +08:00
Neo
076f5755ca 数据库 数据校验写入等逻辑更新。 2026-02-01 03:46:16 +08:00
Neo
9948000b71 更新1 2026-01-28 20:23:54 +08:00
Neo
4fafc80254 提交 2026-01-28 00:04:30 +08:00
Neo
1a76108209 修复脚本 2026-01-28 00:02:15 +08:00
Neo
c42b516895 同步? 2026-01-28 00:00:22 +08:00
Neo
7e67bc4218 更新 2026-01-27 23:45:36 +08:00
Neo
64a3159f9e 移除依赖 2026-01-27 23:22:21 +08:00
Neo
ba00654ac5 exe 依赖添加 2026-01-27 23:19:54 +08:00
502 changed files with 559911 additions and 17548 deletions

13
.config/dotnet-tools.json Normal file
View File

@@ -0,0 +1,13 @@
{
"version": 1,
"isRoot": true,
"tools": {
"csharpier": {
"version": "1.2.5",
"commands": [
"csharpier"
],
"rollForward": false
}
}
}

157
README.md
View File

@@ -78,6 +78,46 @@ python -m cli.main \
- `FETCH_ONLY`:仅在线抓取落盘,不入库
- `INGEST_ONLY`:仅从本地 JSON 回放入库(适合离线回放/补跑)
## DWS 层(汇总/财务)
### 建表与初始化
- 建表:`INIT_DWS_SCHEMA`
- 配置:`SEED_DWS_CONFIG`
- 订单汇总(可选):`DWS_BUILD_ORDER_SUMMARY`
### 任务与调度建议
- **每小时**`DWS_ASSISTANT_DAILY`、`DWS_FINANCE_DAILY`、`DWS_FINANCE_INCOME_STRUCTURE`
- **每日**`DWS_ASSISTANT_MONTHLY`、`DWS_ASSISTANT_CUSTOMER`、`DWS_MEMBER_CONSUMPTION`、`DWS_MEMBER_VISIT`、`DWS_FINANCE_DISCOUNT_DETAIL`、`DWS_FINANCE_RECHARGE`、`DWS_ASSISTANT_FINANCE`
- **每月(月初)**`DWS_ASSISTANT_SALARY`
- **维护清理(按需)**`DWS_RETENTION_CLEANUP`
调度配置默认保存在 `etl_billiards/scheduled_tasks.json`GUI 调度器会读取该文件。
### 时间口径
- 周起始日:周一
- 月/季度起始:第一天 0 点
- 环比:对比上一个等长区间
- 窗口类型:本周/上周/本月/上月/前3个月不含本月/前3个月含本月/本季度/上季度/最近半年不含本月
### Excel 导入(支出/平台回款/充值提成)
脚本:`etl_billiards/scripts/import_dws_excel.py`
- 支出结构:`--type expense`,按月导入(房租/水电/物业/工资/报销/平台服务费等)
- 平台回款:`--type platform`,按回款日期导入(回款金额、佣金、服务费、订单号等)
- 充值提成:`--type commission`按月份导入助教ID、提成金额、充值订单金额等
### 大客户优惠拆分(可选)
用于将手动调整拆分为“大客户优惠/其他优惠”,可在配置中指定:
- `dws.discount.big_customer_member_ids`会员ID列表逗号分隔
- `dws.discount.big_customer_order_ids`订单ID列表逗号分隔
未配置时,大客户金额为 0手动调整全部计入“其他优惠”。
### 时间分层清理(可选)
任务:`DWS_RETENTION_CLEANUP`,按配置清理历史数据
- `dws.retention.enabled`:是否启用
- `dws.retention.layer`:分层(如 `LAST_3_MONTHS`
- `dws.retention.tables`:需要清理的表列表(逗号分隔)
- `dws.retention.table_layers`表级分层覆盖JSON 字符串)
## 目录结构与关键文件
- 仓库根目录:`etl_billiards/` 主代码;`app/` 示例 runner`开发笔记/` 项目笔记;`tmp/` 草稿/调试归档;`requirements.txt`(仓库根)依赖;`run_etl.sh/.bat` 启动脚本。
- 注意:根目录的 `run_etl.sh/.bat` 运行时要求当前目录为 `etl_billiards/`(因为入口是 `python -m cli.main`)。
@@ -367,6 +407,95 @@ python -m cli.main \
6) 去嵌套:数组展开为子表/子行,重复 profile 提炼为维度。
7) 长期演进:优先加列/加表,减少对已有表结构的破坏。
## DWS 数据层(汇总层)
DWSData Warehouse Service层基于 DWD 明细层数据构建,提供预聚合的数据服务。
### 表结构概览
| 分类 | 表数量 | 说明 |
|------|--------|------|
| 配置表 | 5 | 绩效档位、等级定价、奖金规则、区域分类、技能映射 |
| 助教维度 | 5 | 日度/月度业绩、客户统计、工资计算、充值提成 |
| 客户维度 | 2 | 消费汇总、来店明细 |
| 财务维度 | 7 | 日度汇总、收入结构、优惠明细、充值统计、支出、助教收支、平台结算 |
| 订单汇总 | 1 | 订单级别聚合 |
### 核心特性
- **时间分层**支持近2天/近1月/近3月/全量的时间窗口筛选
- **滚动窗口**支持7/10/15/30/60/90天的滚动统计
- **SCD2 as-of**:维度取值支持按时间点获取历史值(如助教等级)
- **幂等更新**:采用 delete-before-insert 策略,支持重复执行
- **Excel导入**:支出/平台结算/充值提成支持手动导入
### 助教工资计算
**绩效档位6档 + 新入职)**
| 档位 | 业绩阈值 | 专业课抽成 | 打赏课抽成 | 休假 |
|------|----------|-----------|-----------|------|
| T0 | H < 100 | 28元/时 | 50% | 3天 |
| T1 | 100 ≤ H < 130 | 18元/时 | 40% | 4天 |
| T2 | 130 ≤ H < 160 | 15元/时 | 38% | 4天 |
| T3 | 160 ≤ H < 190 | 13元/时 | 35% | 5天 |
| T4 | 190 ≤ H < 220 | 10元/时 | 33% | 6天 |
| T5 | H ≥ 220 | 8元/时 | 30% | 休假自由 |
**工资计算公式**
```
基础课收入 = 基础课小时数 × (客户支付价格 - 专业课抽成)
附加课收入 = 附加课小时数 × 190 × (1 - 打赏课抽成比例)
应发工资 = 课时收入 + 奖金
```
**计算示例中级助教185小时3档**
- 基础课170小时: 170 × (108 - 13) = 16,150元
- 附加课15小时: 15 × 190 × (1 - 0.35) = 1,852.5元
- 课时收入: 18,002.5元
**等级定价(客户支付价格)**
| 等级 | 基础课价格 | 附加课价格 |
|------|-----------|-----------|
| 初级 | 98元/时 | 190元/时 |
| 中级 | 108元/时 | 190元/时 |
| 高级 | 118元/时 | 190元/时 |
| 星级 | 138元/时 | 190元/时 |
### 运行 DWS 任务
```bash
# 初始化 DWS Schema
python -m cli.main --tasks INIT_DWS_SCHEMA
# 执行配置数据初始化
psql -f etl_billiards/database/seed_dws_config.sql
# 执行 DWS 订单汇总构建
python -m cli.main --tasks DWS_BUILD_ORDER_SUMMARY
```
### Excel 数据导入
```bash
# 导入支出数据
python etl_billiards/scripts/import_dws_excel.py --type expense --file expenses.xlsx
# 导入平台结算
python etl_billiards/scripts/import_dws_excel.py --type platform --file platform.xlsx
# 导入充值提成
python etl_billiards/scripts/import_dws_excel.py --type commission --file commission.xlsx
```
### 相关文档
- `etl_billiards/docs/dws_tables_dictionary.md`DWS 数据字典
- `etl_billiards/database/schema_dws.sql`DWS DDL
- `etl_billiards/database/seed_dws_config.sql`:配置初始数据
## 常用 CLI
```bash
cd etl_billiards
@@ -474,11 +603,37 @@ python scripts/test_db_connection.py --dsn "postgresql://user:pwd@host:5432/db"
> 完整字段级映射见 `etl_billiards/docs/` 与 ODS/DWD DDL。
## 当前状态2025-12-09
## 当前状态2026-02-02
- 示例 JSON 已全量灌入DWD 行数与 ODS 对齐。
- 分类维度已展平大类+子类:`dim_goods_category` 26 行category_level/leaf 已赋值)。
- 部分空字段源数据即为空,如需补值请先确认上游。
### 2026-02-02 更新:字段补全
本次更新完成了 API → ODS → DWD 全链路字段补全:
**ODS 新增字段**16 张表,共 50+ 字段):
- `settlement_records`/`recharge_settlements`:电费相关字段(`electricitymoney`、`realelectricitymoney`、`electricityadjustmoney`)、券销售额、结算明细列表
- `table_fee_transactions`:活动折扣金额、订单消费类型、实际服务费
- `assistant_service_records`:助教团队名称、实际服务费
- `group_buy_redemption_records`:会员折扣、各类分摊金额(台费/商品/助教/充值)
- `table_fee_discount_records`:台区信息、台桌名称/价格、免费标记
- `member_stored_value_cards`:本金余额、会员等级、电费相关配置
- `member_profiles`:累计支付/充值金额、注册来源
- `member_balance_changes`:本金变动(前/后/数据)
- `group_buy_packages`排序、首单限制、租户券销售订单项ID
- 其他:商品编码/停售、租户ID等
**DWD 新增字段**
- 主表新增核心业务字段金额、ID、状态
- 扩展表新增配置/明细字段
**数据补全脚本**
- `scripts/backfill_202507_to_now.bat`:从 2025-07-01 重新抓取并装载数据
**文档更新**
- `etl_billiards/docs/bd_manual/` 下所有相关表文档已同步更新
## 可精简/归档
- `tmp/`、`tmp/etl_billiards_misc/` 中草稿、旧备份、调试脚本仅供参考,不影响运行。
- 根级保留必要文件README、requirements、run_etl.*),其余临时文件按需归档至 `tmp/`。

60
collect_env_report.ps1 Normal file
View File

@@ -0,0 +1,60 @@
$ErrorActionPreference = "Stop"
Write-Host "[1/6] Collecting environment info..." -ForegroundColor Cyan
$report = @()
$report += "# ETL Manager Environment Report"
$report += "Timestamp: $(Get-Date -Format 'yyyy-MM-dd HH:mm:ss')"
$report += ""
Write-Host "[2/6] Resolving python/pip..." -ForegroundColor Cyan
$report += "## Executables"
$py = (Get-Command python -ErrorAction SilentlyContinue)
$pip = (Get-Command pip -ErrorAction SilentlyContinue)
if ($py) { $report += "python: $($py.Source)" } else { $report += "python: NOT FOUND" }
if ($pip) { $report += "pip: $($pip.Source)" } else { $report += "pip: NOT FOUND" }
$report += ""
Write-Host "[3/6] Python details..." -ForegroundColor Cyan
$report += "## Python info"
$report += (python -c "import sys, platform; print('version='+sys.version.replace('\n',' ')); print('executable='+sys.executable); print('prefix='+sys.prefix); print('base_prefix='+sys.base_prefix); print('arch='+platform.architecture()[0]); print('platform='+platform.platform())" 2>&1)
$report += ""
Write-Host "[4/6] pip details..." -ForegroundColor Cyan
$report += "## pip"
$report += (python -m pip --version 2>&1)
$report += ""
Write-Host "[5/6] PySide6 details..." -ForegroundColor Cyan
$report += "## PySide6"
$pyside = @'
try:
import PySide6
from PySide6 import QtCore
print('PySide6='+PySide6.__version__)
print('Qt='+QtCore.qVersion())
print('PySide6_path='+PySide6.__file__)
print('Qt_plugins_path='+QtCore.QLibraryInfo.path(QtCore.QLibraryInfo.PluginsPath))
except Exception as e:
print('PySide6_import_error='+repr(e))
'@
$report += ($pyside | python - 2>&1)
$report += ""
Write-Host "[6/6] Installed packages..." -ForegroundColor Cyan
$report += "## Installed packages (freeze)"
$report += (python -m pip list --format=freeze 2>&1)
$reportPath = "D:\env_report_local.txt"
if (-not (Test-Path "D:\")) {
Write-Host "[WARN] D: not found, fallback to current directory." -ForegroundColor Yellow
$reportPath = ".\env_report_local.txt"
}
$reportDir = Split-Path $reportPath
if ($reportDir -and -not (Test-Path $reportDir)) {
New-Item -ItemType Directory -Path $reportDir -Force | Out-Null
}
Write-Host "[WRITE] $reportPath" -ForegroundColor Green
$report -join "`n" | Set-Content -Path $reportPath -Encoding UTF8
Write-Host "[DONE]" -ForegroundColor Green
Write-Output $reportPath

121
env_report_local.txt Normal file
View File

@@ -0,0 +1,121 @@
# ETL Manager Environment Report
Timestamp: 2026-01-28 00:56:34
## Executables
python: C:\ProgramData\miniconda3\python.exe
pip: C:\ProgramData\miniconda3\Scripts\pip.exe
## Python info
version=3.13.9 | packaged by Anaconda, Inc. | (main, Oct 21 2025, 19:09:58) [MSC v.1929 64 bit (AMD64)]
executable=C:\ProgramData\miniconda3\python.exe
prefix=C:\ProgramData\miniconda3
base_prefix=C:\ProgramData\miniconda3
arch=64bit
platform=Windows-10-10.0.17763-SP0
## pip
pip 25.3 from C:\ProgramData\miniconda3\Lib\site-packages\pip (python 3.13)
## PySide6
PySide6=6.10.1
Qt=6.10.1
PySide6_path=C:\ProgramData\miniconda3\Lib\site-packages\PySide6\__init__.py
Qt_plugins_path=C:/ProgramData/miniconda3/Lib/site-packages/PySide6/plugins
## Installed packages (freeze)
altgraph==0.17.5
anaconda-anon-usage==0.7.4
annotated-types==0.6.0
archspec==0.2.5
beautifulsoup4==4.13.4
boltons==25.0.0
brotlicffi==1.1.0.0
certifi==2025.11.12
cffi==2.0.0
chardet==3.0.4
charset-normalizer==3.4.4
colorama==0.4.6
conda==25.9.1
conda-anaconda-telemetry==0.3.0
conda-anaconda-tos==0.2.2
conda-content-trust==0.2.0
conda-libmamba-solver==25.4.0
conda-package-handling==2.4.0
conda_package_streaming==0.12.0
coverage==7.12.0
cryptography==46.0.3
distro==1.9.0
et_xmlfile==2.0.0
frozendict==2.4.6
ftfy==6.3.1
googletrans==4.0.0rc1
h11==0.9.0
h2==3.2.0
hpack==3.0.0
hstspreload==2025.1.1
httpcore==0.9.1
httpx==0.13.3
hyperframe==5.2.0
idna==2.10
iniconfig==2.3.0
jsonpatch==1.33
jsonpointer==3.0.0
libmambapy==2.3.2
lxml==6.0.2
markdown-it-py==4.0.0
mdurl==0.1.2
menuinst==2.4.1
numpy==2.3.2
openpyxl==3.1.5
packaging==25.0
pandas==2.3.1
pefile==2024.8.26
pip==25.2
platformdirs==4.5.0
pluggy==1.5.0
psycopg2==2.9.10
psycopg2-binary==2.9.11
pyaes==1.6.1
pyasn1==0.6.1
pycosat==0.6.6
pycparser==2.23
pydantic==2.12.3
pydantic_core==2.41.4
Pygments==2.19.2
pyinstaller==6.18.0
pyinstaller-hooks-contrib==2025.11
PySide6==6.10.1
PySide6_Addons==6.10.1
PySide6_Essentials==6.10.1
PySocks==1.7.1
pytest==9.0.1
pytest-cov==7.0.0
python-dateutil==2.9.0.post0
python-docx==1.2.0
python-dotenv==1.2.1
pytz==2025.2
pywin32-ctypes==0.2.3
requests==2.32.5
rfc3986==1.5.0
rich==14.2.0
rsa==4.9.1
ruamel.yaml==0.18.16
ruamel.yaml.clib==0.2.14
setuptools==80.9.0
shiboken6==6.10.1
six==1.17.0
sniffio==1.3.1
soupsieve==2.7
Telethon==1.40.0
tqdm==4.67.1
truststore==0.10.1
typing_extensions==4.15.0
typing-inspection==0.4.2
tzdata==2025.2
urllib3==2.5.0
wcwidth==0.2.14
wheel==0.45.1
win_inet_pton==1.1.0
xlsxwriter==3.2.9
zstandard==0.24.0

View File

@@ -32,7 +32,7 @@ SCHEMA_ETL=etl_admin
# API 配置
# ------------------------------------------------------------------------------
API_BASE=https://pc.ficoo.vip/apiprod/admin/v1/
API_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjbGllbnQtdHlwZSI6IjQiLCJ1c2VyLXR5cGUiOiIxIiwiaHR0cDovL3NjaGVtYXMubWljcm9zb2Z0LmNvbS93cy8yMDA4LzA2L2lkZW50aXR5L2NsYWltcy9yb2xlIjoiMTIiLCJyb2xlLWlkIjoiMTIiLCJ0ZW5hbnQtaWQiOiIyNzkwNjgzMTYwNzA5OTU3Iiwibmlja25hbWUiOiLnp5_miLfnrqHnkIblkZjvvJrmganmgakxIiwic2l0ZS1pZCI6IjAiLCJtb2JpbGUiOiIxMzgxMDUwMjMwNCIsInNpZCI6IjI5NTA0ODk2NTgzOTU4NDUiLCJzdGFmZi1pZCI6IjMwMDk5MTg2OTE1NTkwNDUiLCJvcmctaWQiOiIwIiwicm9sZS10eXBlIjoiMyIsInJlZnJlc2hUb2tlbiI6Iks1ZnBhYlRTNkFsR0FpMmN4WGYrMHdJVkk0L2UvTVQrSVBHM3V5VWRrSjg9IiwicmVmcmVzaEV4cGlyeVRpbWUiOiIyMDI2LzEvMzEg5LiL5Y2IMTA6MTQ6NTEiLCJuZWVkQ2hlY2tUb2tlbiI6ImZhbHNlIiwiZXhwIjoxNzY5ODY4ODkxLCJpc3MiOiJ0ZXN0IiwiYXVkIjoiVXNlciJ9.BH3-iwwrBczb8aFfI__6kwe3AIsEPacN9TruaTrQ3nY
API_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjbGllbnQtdHlwZSI6IjQiLCJ1c2VyLXR5cGUiOiIxIiwiaHR0cDovL3NjaGVtYXMubWljcm9zb2Z0LmNvbS93cy8yMDA4LzA2L2lkZW50aXR5L2NsYWltcy9yb2xlIjoiMTIiLCJyb2xlLWlkIjoiMTIiLCJ0ZW5hbnQtaWQiOiIyNzkwNjgzMTYwNzA5OTU3Iiwibmlja25hbWUiOiLnp5_miLfnrqHnkIblkZjvvJrmganmgakxIiwic2l0ZS1pZCI6IjAiLCJtb2JpbGUiOiIxMzgxMDUwMjMwNCIsInNpZCI6IjI5NTA0ODk2NTgzOTU4NDUiLCJzdGFmZi1pZCI6IjMwMDk5MTg2OTE1NTkwNDUiLCJvcmctaWQiOiIwIiwicm9sZS10eXBlIjoiMyIsInJlZnJlc2hUb2tlbiI6IjlES1lWcEVkYWw1bEc5cTMrdFptMkJXeTlyMkVMeEY5MHZuUWRyRnNYVFU9IiwicmVmcmVzaEV4cGlyeVRpbWUiOiIyMDI2LzIvOSDkuIrljYgyOjQzOjU0IiwibmVlZENoZWNrVG9rZW4iOiJmYWxzZSIsImV4cCI6MTc3MDU3NjIzNCwiaXNzIjoidGVzdCIsImF1ZCI6IlVzZXIifQ._1gnWcJHw8O26pcfiT1x8tgQRGn3g56vv2IZP8shgGU
# API 请求超时(秒)
API_TIMEOUT=20

View File

@@ -1,44 +0,0 @@
# -*- mode: python ; coding: utf-8 -*-
a = Analysis(
['C:\\dev\\LLTQ\\ETL\\feiqiu-ETL\\etl_billiards\\gui\\main.py'],
pathex=[],
binaries=[],
datas=[('C:\\dev\\LLTQ\\ETL\\feiqiu-ETL\\etl_billiards\\gui\\resources', 'gui/resources'), ('C:\\dev\\LLTQ\\ETL\\feiqiu-ETL\\etl_billiards\\database\\schema_dwd_doc.sql', 'database'), ('C:\\dev\\LLTQ\\ETL\\feiqiu-ETL\\etl_billiards\\database\\schema_dws.sql', 'database'), ('C:\\dev\\LLTQ\\ETL\\feiqiu-ETL\\etl_billiards\\database\\schema_etl_admin.sql', 'database'), ('C:\\dev\\LLTQ\\ETL\\feiqiu-ETL\\etl_billiards\\database\\schema_ODS_doc.sql', 'database'), ('C:\\dev\\LLTQ\\ETL\\feiqiu-ETL\\etl_billiards\\database\\seed_ods_tasks.sql', 'database'), ('C:\\dev\\LLTQ\\ETL\\feiqiu-ETL\\etl_billiards\\database\\seed_scheduler_tasks.sql', 'database')],
hiddenimports=['PySide6.QtCore', 'PySide6.QtGui', 'PySide6.QtWidgets', 'psycopg2', 'psycopg2.extras', 'psycopg2.extensions', 'gui.models.task_model', 'gui.models.schedule_model', 'gui.utils.cli_builder', 'gui.utils.config_helper', 'gui.utils.app_settings', 'gui.workers.task_worker', 'gui.workers.db_worker', 'gui.widgets.settings_dialog'],
hookspath=[],
hooksconfig={},
runtime_hooks=[],
excludes=['matplotlib', 'numpy', 'pandas', 'scipy', 'PIL', 'cv2', 'tkinter'],
noarchive=False,
optimize=0,
)
pyz = PYZ(a.pure)
exe = EXE(
pyz,
a.scripts,
[],
exclude_binaries=True,
name='ETL_Manager',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
console=False,
disable_windowed_traceback=False,
argv_emulation=False,
target_arch=None,
codesign_identity=None,
entitlements_file=None,
)
coll = COLLECT(
exe,
a.binaries,
a.datas,
strip=False,
upx=True,
upx_exclude=[],
name='ETL_Manager',
)

View File

@@ -4,7 +4,9 @@ from __future__ import annotations
from datetime import datetime
from pathlib import Path
import time
from typing import Any, Iterable, Tuple
from zoneinfo import ZoneInfo
from api.client import APIClient
from api.endpoint_routing import plan_calls
@@ -128,3 +130,57 @@ class RecordingAPIClient:
"pages": len(pages),
"records": total_records,
}
def _cfg_get(cfg, key: str, default=None):
if isinstance(cfg, dict):
cur = cfg
for part in key.split("."):
if not isinstance(cur, dict) or part not in cur:
return default
cur = cur[part]
return cur
getter = getattr(cfg, "get", None)
if callable(getter):
return getter(key, default)
return default
def build_recording_client(
cfg,
*,
task_code: str,
output_dir: Path | str | None = None,
run_id: int | None = None,
write_pretty: bool | None = None,
):
"""Build RecordingAPIClient from AppConfig or dict config."""
base_client = APIClient(
base_url=_cfg_get(cfg, "api.base_url") or "",
token=_cfg_get(cfg, "api.token"),
timeout=int(_cfg_get(cfg, "api.timeout_sec", 20) or 20),
retry_max=int(_cfg_get(cfg, "api.retries.max_attempts", 3) or 3),
headers_extra=_cfg_get(cfg, "api.headers_extra") or {},
)
if write_pretty is None:
write_pretty = bool(_cfg_get(cfg, "io.write_pretty_json", False))
if run_id is None:
run_id = int(time.time())
if output_dir is None:
tz_name = _cfg_get(cfg, "app.timezone", "Asia/Taipei") or "Asia/Taipei"
tz = ZoneInfo(tz_name)
ts = datetime.now(tz).strftime("%Y%m%d-%H%M%S")
fetch_root = _cfg_get(cfg, "pipeline.fetch_root") or _cfg_get(cfg, "io.export_root") or "export/JSON"
task_upper = str(task_code).upper()
output_dir = Path(fetch_root) / task_upper / f"{task_upper}-{run_id}-{ts}"
return RecordingAPIClient(
base_client=base_client,
output_dir=output_dir,
task_code=str(task_code),
run_id=int(run_id),
write_pretty=bool(write_pretty),
)

View File

@@ -90,9 +90,11 @@ def build_exe(onefile: bool = False, console: bool = False):
# 隐式导入
hidden_imports = [
# PySide6 核心模块
"PySide6.QtCore",
"PySide6.QtGui",
"PySide6.QtWidgets",
# 数据库
"psycopg2",
"psycopg2.extras",
"psycopg2.extensions",
@@ -138,6 +140,9 @@ def build_exe(onefile: bool = False, console: bool = False):
result = subprocess.run(cmd, cwd=str(project_root))
if result.returncode == 0:
# 打包后精简:删除不需要的文件
slim_dist(project_root / "dist" / "ETL_Manager" / "_internal")
print()
print("=" * 50)
print("打包成功!")
@@ -149,6 +154,77 @@ def build_exe(onefile: bool = False, console: bool = False):
sys.exit(1)
def slim_dist(internal_dir: Path):
"""精简打包后的文件,删除不需要的内容"""
if not internal_dir.exists():
return
print()
print("精简打包文件...")
removed_size = 0
# 1. 删除不需要的翻译文件(只保留中文和英文)
translations_dir = internal_dir / "PySide6" / "translations"
if translations_dir.exists():
keep_langs = {"zh_CN", "zh_TW", "en"}
for qm_file in translations_dir.glob("*.qm"):
# 检查是否是需要保留的语言
keep = False
for lang in keep_langs:
if lang in qm_file.name:
keep = True
break
if not keep:
size = qm_file.stat().st_size
qm_file.unlink()
removed_size += size
# 2. 删除 opengl32sw.dll软件渲染20MB通常不需要
opengl_sw = internal_dir / "PySide6" / "opengl32sw.dll"
if opengl_sw.exists():
size = opengl_sw.stat().st_size
opengl_sw.unlink()
removed_size += size
print(f" 删除: opengl32sw.dll ({size / 1024 / 1024:.1f} MB)")
# 3. 删除不需要的 Qt 模块 DLL如果存在
unnecessary_dlls = [
"Qt6Pdf.dll", # PDF 支持
"Qt6Qml.dll", # QML 引擎
"Qt6QmlMeta.dll",
"Qt6QmlModels.dll",
"Qt6QmlWorkerScript.dll",
"Qt6Quick.dll", # Quick UI
"Qt6VirtualKeyboard.dll", # 虚拟键盘
]
pyside6_dir = internal_dir / "PySide6"
for dll_name in unnecessary_dlls:
dll_path = pyside6_dir / dll_name
if dll_path.exists():
size = dll_path.stat().st_size
dll_path.unlink()
removed_size += size
print(f" 删除: {dll_name} ({size / 1024 / 1024:.1f} MB)")
# 4. 删除不需要的插件目录
unnecessary_plugins = [
"networkinformation", # 网络信息
"tls", # TLS 支持(数据库已有)
]
plugins_dir = pyside6_dir / "plugins"
if plugins_dir.exists():
for plugin_name in unnecessary_plugins:
plugin_path = plugins_dir / plugin_name
if plugin_path.exists():
size = sum(f.stat().st_size for f in plugin_path.rglob("*") if f.is_file())
shutil.rmtree(plugin_path)
removed_size += size
print(f" 删除插件: {plugin_name}")
print(f"共节省: {removed_size / 1024 / 1024:.1f} MB")
def main():
"""主函数"""
import argparse

View File

@@ -109,9 +109,18 @@ DEFAULTS = {
"mode": "history",
"history_start": "2025-07-01",
"history_end": "",
"include_dimensions": False,
"include_dimensions": True,
"auto_check": False,
"auto_backfill": False,
"compare_content": True,
"content_sample_limit": 50,
"backfill_mismatch": True,
"recheck_after_backfill": True,
"ods_task_codes": "",
"force_monthly_split": True,
},
"dwd": {
"fact_upsert": True,
},
}

View File

@@ -55,7 +55,12 @@ ENV_MAP = {
"INTEGRITY_INCLUDE_DIMENSIONS": ("integrity.include_dimensions",),
"INTEGRITY_AUTO_CHECK": ("integrity.auto_check",),
"INTEGRITY_AUTO_BACKFILL": ("integrity.auto_backfill",),
"INTEGRITY_COMPARE_CONTENT": ("integrity.compare_content",),
"INTEGRITY_CONTENT_SAMPLE_LIMIT": ("integrity.content_sample_limit",),
"INTEGRITY_BACKFILL_MISMATCH": ("integrity.backfill_mismatch",),
"INTEGRITY_RECHECK_AFTER_BACKFILL": ("integrity.recheck_after_backfill",),
"INTEGRITY_ODS_TASK_CODES": ("integrity.ods_task_codes",),
"DWD_FACT_UPSERT": ("dwd.fact_upsert",),
}

View File

@@ -7,7 +7,7 @@ CREATE TABLE IF NOT EXISTS billiards_ods.member_profiles (
tenant_id BIGINT,
register_site_id BIGINT,
site_name TEXT,
id BIGINT PRIMARY KEY,
id BIGINT,
system_member_id BIGINT,
member_card_grade_code BIGINT,
member_card_grade_name TEXT,
@@ -19,10 +19,17 @@ CREATE TABLE IF NOT EXISTS billiards_ods.member_profiles (
status INT,
user_status INT,
create_time TIMESTAMP,
pay_money_sum NUMERIC(18,2),
person_tenant_org_id BIGINT,
person_tenant_org_name TEXT,
recharge_money_sum NUMERIC(18,2),
register_source TEXT,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now(),
payload JSONB NOT NULL
payload JSONB NOT NULL,
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.member_profiles IS 'ODS 原始明细表:会员档案/会员账户信息。来源export/test-json-doc/member_profiles.json分析member_profiles-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -53,7 +60,7 @@ CREATE TABLE IF NOT EXISTS billiards_ods.member_balance_changes (
register_site_id BIGINT,
registerSiteName TEXT,
paySiteName TEXT,
id BIGINT PRIMARY KEY,
id BIGINT,
tenant_member_id BIGINT,
tenant_member_card_id BIGINT,
system_member_id BIGINT,
@@ -73,10 +80,15 @@ CREATE TABLE IF NOT EXISTS billiards_ods.member_balance_changes (
operator_name TEXT,
is_delete INT,
create_time TIMESTAMP,
principal_after NUMERIC(18,2),
principal_before NUMERIC(18,2),
principal_data TEXT,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now(),
payload JSONB NOT NULL
payload JSONB NOT NULL,
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.member_balance_changes IS 'ODS 原始明细表会员余额变更流水。来源export/test-json-doc/member_balance_changes.json分析member_balance_changes-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -117,7 +129,7 @@ CREATE TABLE IF NOT EXISTS billiards_ods.member_stored_value_cards (
system_member_id BIGINT,
register_site_id BIGINT,
site_name TEXT,
id BIGINT PRIMARY KEY,
id BIGINT,
member_card_grade_code BIGINT,
member_card_grade_code_name TEXT,
member_card_type_name TEXT,
@@ -181,10 +193,19 @@ CREATE TABLE IF NOT EXISTS billiards_ods.member_stored_value_cards (
tenantName TEXT,
pdAssisnatLevel TEXT,
cxAssisnatLevel TEXT,
able_share_member_discount BOOLEAN,
electricity_deduct_radio NUMERIC(18,4),
electricity_discount NUMERIC(18,4),
electricitycarddeduct BOOLEAN,
member_grade BIGINT,
principal_balance NUMERIC(18,2),
rechargefreezebalance NUMERIC(18,2),
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now(),
payload JSONB NOT NULL
payload JSONB NOT NULL,
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.member_stored_value_cards IS 'ODS 原始明细表:会员储值/卡券账户列表。来源export/test-json-doc/member_stored_value_cards.json分析member_stored_value_cards-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -264,7 +285,7 @@ COMMENT ON COLUMN billiards_ods.member_stored_value_cards.payload IS '【说明
CREATE TABLE IF NOT EXISTS billiards_ods.recharge_settlements (
id BIGINT PRIMARY KEY,
id BIGINT,
tenantid BIGINT,
siteid BIGINT,
sitename TEXT,
@@ -325,10 +346,18 @@ CREATE TABLE IF NOT EXISTS billiards_ods.recharge_settlements (
isfirst INT,
rechargecardamount NUMERIC(18,2),
giftcardamount NUMERIC(18,2),
electricityadjustmoney NUMERIC(18,2),
electricitymoney NUMERIC(18,2),
mervousalesamount NUMERIC(18,2),
plcouponsaleamount NUMERIC(18,2),
realelectricitymoney NUMERIC(18,2),
settlelist JSONB,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now(),
payload JSONB NOT NULL
payload JSONB NOT NULL,
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.recharge_settlements IS 'ODS 原始明细表充值结算记录。来源export/test-json-doc/recharge_settlements.json分析recharge_settlements-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -400,7 +429,7 @@ COMMENT ON COLUMN billiards_ods.recharge_settlements.payload IS '【说明】完
CREATE TABLE IF NOT EXISTS billiards_ods.settlement_records (
id BIGINT PRIMARY KEY,
id BIGINT,
tenantid BIGINT,
siteid BIGINT,
sitename TEXT,
@@ -461,10 +490,18 @@ CREATE TABLE IF NOT EXISTS billiards_ods.settlement_records (
isfirst INT,
rechargecardamount NUMERIC(18,2),
giftcardamount NUMERIC(18,2),
electricityadjustmoney NUMERIC(18,2),
electricitymoney NUMERIC(18,2),
mervousalesamount NUMERIC(18,2),
plcouponsaleamount NUMERIC(18,2),
realelectricitymoney NUMERIC(18,2),
settlelist JSONB,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now(),
payload JSONB NOT NULL
payload JSONB NOT NULL,
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.settlement_records IS 'ODS 原始明细表:结账/结算记录。来源export/test-json-doc/settlement_records.json分析settlement_records-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -536,7 +573,7 @@ COMMENT ON COLUMN billiards_ods.settlement_records.payload IS '【说明】完
CREATE TABLE IF NOT EXISTS billiards_ods.assistant_cancellation_records (
id BIGINT PRIMARY KEY,
id BIGINT,
siteId BIGINT,
siteProfile JSONB,
assistantName TEXT,
@@ -549,10 +586,13 @@ CREATE TABLE IF NOT EXISTS billiards_ods.assistant_cancellation_records (
tableName TEXT,
trashReason TEXT,
createTime TIMESTAMP,
tenant_id BIGINT,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now(),
payload JSONB NOT NULL
payload JSONB NOT NULL,
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.assistant_cancellation_records IS 'ODS 原始明细表:助教作废/取消记录。来源export/test-json-doc/assistant_cancellation_records.json分析assistant_cancellation_records-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -576,7 +616,7 @@ COMMENT ON COLUMN billiards_ods.assistant_cancellation_records.payload IS '【
CREATE TABLE IF NOT EXISTS billiards_ods.assistant_accounts_master (
id BIGINT PRIMARY KEY,
id BIGINT,
tenant_id BIGINT,
site_id BIGINT,
assistant_no TEXT,
@@ -638,10 +678,12 @@ CREATE TABLE IF NOT EXISTS billiards_ods.assistant_accounts_master (
light_equipment_id TEXT,
entry_sign_status INT,
resign_sign_status INT,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now(),
payload JSONB NOT NULL
payload JSONB NOT NULL,
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.assistant_accounts_master IS 'ODS 原始明细表助教档案主数据。来源export/test-json-doc/assistant_accounts_master.json分析assistant_accounts_master-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -714,7 +756,7 @@ COMMENT ON COLUMN billiards_ods.assistant_accounts_master.payload IS '【说明
CREATE TABLE IF NOT EXISTS billiards_ods.assistant_service_records (
id BIGINT PRIMARY KEY,
id BIGINT,
tenant_id BIGINT,
site_id BIGINT,
siteProfile JSONB,
@@ -778,10 +820,14 @@ CREATE TABLE IF NOT EXISTS billiards_ods.assistant_service_records (
get_grade_times INT,
is_not_responding INT,
is_confirm INT,
assistantteamname TEXT,
real_service_money NUMERIC(18,2),
payload JSONB NOT NULL,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now()
fetched_at TIMESTAMPTZ DEFAULT now(),
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.assistant_service_records IS 'ODS 原始明细表助教服务流水。来源export/test-json-doc/assistant_service_records.json分析assistant_service_records-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -856,7 +902,7 @@ COMMENT ON COLUMN billiards_ods.assistant_service_records.fetched_at IS '【说
CREATE TABLE IF NOT EXISTS billiards_ods.site_tables_master (
id BIGINT PRIMARY KEY,
id BIGINT,
site_id BIGINT,
siteName TEXT,
"appletQrCodeUrl" TEXT,
@@ -881,10 +927,13 @@ CREATE TABLE IF NOT EXISTS billiards_ods.site_tables_master (
table_status INT,
temporary_light_second INT,
virtual_table INT,
order_id BIGINT,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now(),
payload JSONB NOT NULL
payload JSONB NOT NULL,
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.site_tables_master IS 'ODS 原始明细表门店桌台主数据。来源export/test-json-doc/site_tables_master.json分析site_tables_master-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -919,7 +968,7 @@ COMMENT ON COLUMN billiards_ods.site_tables_master.payload IS '【说明】完
CREATE TABLE IF NOT EXISTS billiards_ods.table_fee_discount_records (
id BIGINT PRIMARY KEY,
id BIGINT,
tenant_id BIGINT,
site_id BIGINT,
siteProfile JSONB,
@@ -939,10 +988,20 @@ CREATE TABLE IF NOT EXISTS billiards_ods.table_fee_discount_records (
order_trade_no TEXT,
is_delete INT,
create_time TIMESTAMP,
area_type_id BIGINT,
charge_free BOOLEAN,
site_table_area_id BIGINT,
site_table_area_name TEXT,
sitename TEXT,
table_name TEXT,
table_price NUMERIC(18,2),
tenant_name TEXT,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now(),
payload JSONB NOT NULL
payload JSONB NOT NULL,
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.table_fee_discount_records IS 'ODS 原始明细表台费折扣记录。来源export/test-json-doc/table_fee_discount_records.json分析table_fee_discount_records-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -973,7 +1032,7 @@ COMMENT ON COLUMN billiards_ods.table_fee_discount_records.payload IS '【说明
CREATE TABLE IF NOT EXISTS billiards_ods.table_fee_transactions (
id BIGINT PRIMARY KEY,
id BIGINT,
tenant_id BIGINT,
site_id BIGINT,
siteProfile JSONB,
@@ -1012,10 +1071,15 @@ CREATE TABLE IF NOT EXISTS billiards_ods.table_fee_transactions (
salesman_org_id BIGINT,
salesman_user_id BIGINT,
create_time TIMESTAMP,
activity_discount_amount NUMERIC(18,2),
order_consumption_type INT,
real_service_money NUMERIC(18,2),
payload JSONB NOT NULL,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now()
fetched_at TIMESTAMPTZ DEFAULT now(),
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.table_fee_transactions IS 'ODS 原始明细表台费流水。来源export/test-json-doc/table_fee_transactions.json分析table_fee_transactions-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -1065,7 +1129,7 @@ COMMENT ON COLUMN billiards_ods.table_fee_transactions.fetched_at IS '【说明
CREATE TABLE IF NOT EXISTS billiards_ods.goods_stock_movements (
siteGoodsStockId BIGINT PRIMARY KEY,
siteGoodsStockId BIGINT,
tenantId BIGINT,
siteId BIGINT,
siteGoodsId BIGINT,
@@ -1084,10 +1148,12 @@ CREATE TABLE IF NOT EXISTS billiards_ods.goods_stock_movements (
remark TEXT,
operatorName TEXT,
createTime TIMESTAMP,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now(),
payload JSONB NOT NULL
payload JSONB NOT NULL,
PRIMARY KEY (siteGoodsStockId, content_hash)
);
COMMENT ON TABLE billiards_ods.goods_stock_movements IS 'ODS 原始明细表商品库存变动流水。来源export/test-json-doc/goods_stock_movements.json分析goods_stock_movements-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -1117,7 +1183,7 @@ COMMENT ON COLUMN billiards_ods.goods_stock_movements.payload IS '【说明】
CREATE TABLE IF NOT EXISTS billiards_ods.stock_goods_category_tree (
id BIGINT PRIMARY KEY,
id BIGINT,
tenant_id BIGINT,
category_name TEXT,
alias_name TEXT,
@@ -1128,10 +1194,12 @@ CREATE TABLE IF NOT EXISTS billiards_ods.stock_goods_category_tree (
categoryBoxes JSONB,
sort INT,
is_warehousing INT,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now(),
payload JSONB NOT NULL
payload JSONB NOT NULL,
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.stock_goods_category_tree IS 'ODS 原始明细表商品分类树。来源export/test-json-doc/stock_goods_category_tree.json分析stock_goods_category_tree-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -1153,7 +1221,7 @@ COMMENT ON COLUMN billiards_ods.stock_goods_category_tree.payload IS '【说明
CREATE TABLE IF NOT EXISTS billiards_ods.goods_stock_summary (
siteGoodsId BIGINT PRIMARY KEY,
siteGoodsId BIGINT,
goodsName TEXT,
goodsUnit TEXT,
goodsCategoryId BIGINT,
@@ -1167,10 +1235,12 @@ CREATE TABLE IF NOT EXISTS billiards_ods.goods_stock_summary (
rangeSaleMoney NUMERIC(18,2),
rangeInventory NUMERIC(18,4),
currentStock NUMERIC(18,4),
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now(),
payload JSONB NOT NULL
payload JSONB NOT NULL,
PRIMARY KEY (siteGoodsId, content_hash)
);
COMMENT ON TABLE billiards_ods.goods_stock_summary IS 'ODS 原始明细表商品库存汇总。来源export/test-json-doc/goods_stock_summary.json分析goods_stock_summary-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -1195,7 +1265,7 @@ COMMENT ON COLUMN billiards_ods.goods_stock_summary.payload IS '【说明】完
CREATE TABLE IF NOT EXISTS billiards_ods.payment_transactions (
id BIGINT PRIMARY KEY,
id BIGINT,
site_id BIGINT,
siteProfile JSONB,
relate_type INT,
@@ -1206,10 +1276,13 @@ CREATE TABLE IF NOT EXISTS billiards_ods.payment_transactions (
create_time TIMESTAMP,
payment_method INT,
online_pay_channel INT,
tenant_id BIGINT,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now(),
payload JSONB NOT NULL
payload JSONB NOT NULL,
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.payment_transactions IS 'ODS 原始明细表支付流水。来源export/test-json-doc/payment_transactions.json分析payment_transactions-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -1231,7 +1304,7 @@ COMMENT ON COLUMN billiards_ods.payment_transactions.payload IS '【说明】完
CREATE TABLE IF NOT EXISTS billiards_ods.refund_transactions (
id BIGINT PRIMARY KEY,
id BIGINT,
tenant_id BIGINT,
tenantName TEXT,
site_id BIGINT,
@@ -1263,10 +1336,12 @@ CREATE TABLE IF NOT EXISTS billiards_ods.refund_transactions (
is_delete INT,
balance_frozen_amount NUMERIC(18,2),
card_frozen_amount NUMERIC(18,2),
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now(),
payload JSONB NOT NULL
payload JSONB NOT NULL,
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.refund_transactions IS 'ODS 原始明细表退款流水。来源export/test-json-doc/refund_transactions.json分析refund_transactions-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -1309,7 +1384,7 @@ COMMENT ON COLUMN billiards_ods.refund_transactions.payload IS '【说明】完
CREATE TABLE IF NOT EXISTS billiards_ods.platform_coupon_redemption_records (
id BIGINT PRIMARY KEY,
id BIGINT,
verify_id BIGINT,
certificate_id TEXT,
coupon_code TEXT,
@@ -1335,10 +1410,12 @@ CREATE TABLE IF NOT EXISTS billiards_ods.platform_coupon_redemption_records (
operator_name TEXT,
is_delete INT,
siteProfile JSONB,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now(),
payload JSONB NOT NULL
payload JSONB NOT NULL,
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.platform_coupon_redemption_records IS 'ODS 原始明细表:平台券核销/使用记录。来源export/test-json-doc/platform_coupon_redemption_records.json分析platform_coupon_redemption_records-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -1375,7 +1452,7 @@ COMMENT ON COLUMN billiards_ods.platform_coupon_redemption_records.payload IS '
CREATE TABLE IF NOT EXISTS billiards_ods.tenant_goods_master (
id BIGINT PRIMARY KEY,
id BIGINT,
tenant_id BIGINT,
goods_name TEXT,
goods_bar_code TEXT,
@@ -1406,10 +1483,13 @@ CREATE TABLE IF NOT EXISTS billiards_ods.tenant_goods_master (
remark_name TEXT,
create_time TIMESTAMP,
update_time TIMESTAMP,
not_sale BOOLEAN,
payload JSONB NOT NULL,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now()
fetched_at TIMESTAMPTZ DEFAULT now(),
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.tenant_goods_master IS 'ODS 原始明细表租户商品主数据。来源export/test-json-doc/tenant_goods_master.json分析tenant_goods_master-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -1451,7 +1531,7 @@ COMMENT ON COLUMN billiards_ods.tenant_goods_master.fetched_at IS '【说明】E
CREATE TABLE IF NOT EXISTS billiards_ods.group_buy_packages (
id BIGINT PRIMARY KEY,
id BIGINT,
package_id BIGINT,
package_name TEXT,
selling_price NUMERIC(18,2),
@@ -1486,10 +1566,15 @@ CREATE TABLE IF NOT EXISTS billiards_ods.group_buy_packages (
area_tag_type INT,
creator_name TEXT,
create_time TIMESTAMP,
is_first_limit BOOLEAN,
sort INT,
tenantcouponsaleorderitemid BIGINT,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now(),
payload JSONB NOT NULL
payload JSONB NOT NULL,
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.group_buy_packages IS 'ODS 原始明细表团购套餐主数据。来源export/test-json-doc/group_buy_packages.json分析group_buy_packages-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -1535,7 +1620,7 @@ COMMENT ON COLUMN billiards_ods.group_buy_packages.payload IS '【说明】完
CREATE TABLE IF NOT EXISTS billiards_ods.group_buy_redemption_records (
id BIGINT PRIMARY KEY,
id BIGINT,
tenant_id BIGINT,
site_id BIGINT,
siteName TEXT,
@@ -1578,10 +1663,21 @@ CREATE TABLE IF NOT EXISTS billiards_ods.group_buy_redemption_records (
is_single_order INT,
is_delete INT,
create_time TIMESTAMP,
assistant_service_share_money NUMERIC(18,2),
assistant_share_money NUMERIC(18,2),
coupon_sale_id BIGINT,
good_service_share_money NUMERIC(18,2),
goods_share_money NUMERIC(18,2),
member_discount_money NUMERIC(18,2),
recharge_share_money NUMERIC(18,2),
table_service_share_money NUMERIC(18,2),
table_share_money NUMERIC(18,2),
payload JSONB NOT NULL,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now()
fetched_at TIMESTAMPTZ DEFAULT now(),
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.group_buy_redemption_records IS 'ODS 原始明细表团购核销记录。来源export/test-json-doc/group_buy_redemption_records.json分析group_buy_redemption_records-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -1635,7 +1731,7 @@ COMMENT ON COLUMN billiards_ods.group_buy_redemption_records.fetched_at IS '【
CREATE TABLE IF NOT EXISTS billiards_ods.settlement_ticket_details (
orderSettleId BIGINT PRIMARY KEY,
orderSettleId BIGINT,
actualPayment NUMERIC(18,2),
adjustAmount NUMERIC(18,2),
assistantManualDiscount NUMERIC(18,2),
@@ -1674,9 +1770,11 @@ CREATE TABLE IF NOT EXISTS billiards_ods.settlement_ticket_details (
orderItem JSONB,
tenantMemberCardLogs JSONB,
payload JSONB NOT NULL,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now()
fetched_at TIMESTAMPTZ DEFAULT now(),
PRIMARY KEY (orderSettleId, content_hash)
);
COMMENT ON TABLE billiards_ods.settlement_ticket_details IS 'ODS 原始明细表结算小票明细。来源export/test-json-doc/settlement_ticket_details.json分析settlement_ticket_details-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -1725,7 +1823,7 @@ COMMENT ON COLUMN billiards_ods.settlement_ticket_details.fetched_at IS '【说
CREATE TABLE IF NOT EXISTS billiards_ods.store_goods_master (
id BIGINT PRIMARY KEY,
id BIGINT,
tenant_id BIGINT,
site_id BIGINT,
siteName TEXT,
@@ -1770,10 +1868,14 @@ CREATE TABLE IF NOT EXISTS billiards_ods.store_goods_master (
goods_cover TEXT,
create_time TIMESTAMP,
update_time TIMESTAMP,
commodity_code TEXT,
not_sale INTEGER,
payload JSONB NOT NULL,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now()
fetched_at TIMESTAMPTZ DEFAULT now(),
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.store_goods_master IS 'ODS 原始明细表门店商品主数据。来源export/test-json-doc/store_goods_master.json分析store_goods_master-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';
@@ -1828,7 +1930,7 @@ COMMENT ON COLUMN billiards_ods.store_goods_master.fetched_at IS '【说明】ET
CREATE TABLE IF NOT EXISTS billiards_ods.store_goods_sales_records (
id BIGINT PRIMARY KEY,
id BIGINT,
tenant_id BIGINT,
site_id BIGINT,
siteid BIGINT,
@@ -1879,10 +1981,13 @@ CREATE TABLE IF NOT EXISTS billiards_ods.store_goods_sales_records (
tenant_goods_business_id BIGINT,
tenant_goods_category_id BIGINT,
create_time TIMESTAMP,
coupon_share_money NUMERIC(18,2),
payload JSONB NOT NULL,
content_hash TEXT NOT NULL,
source_file TEXT,
source_endpoint TEXT,
fetched_at TIMESTAMPTZ DEFAULT now()
fetched_at TIMESTAMPTZ DEFAULT now(),
PRIMARY KEY (id, content_hash)
);
COMMENT ON TABLE billiards_ods.store_goods_sales_records IS 'ODS 原始明细表门店商品销售流水。来源export/test-json-doc/store_goods_sales_records.json分析store_goods_sales_records-Analysis.md。字段以导出原样为主ETL 补充 source_file/source_endpoint/fetched_at并保留 payload 为原始记录快照。';

View File

@@ -84,7 +84,7 @@ CREATE TABLE IF NOT EXISTS dim_site (
SCD2_end_time TIMESTAMPTZ DEFAULT '9999-12-31',
SCD2_is_current INT DEFAULT 1,
SCD2_version INT DEFAULT 1,
PRIMARY KEY (site_id)
PRIMARY KEY (site_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_site IS 'DWD 维度表dim_site。ODS 来源表billiards_ods.table_fee_transactions对应 JSONtable_fee_transactions.json分析table_fee_transactions-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -107,7 +107,7 @@ COMMENT ON COLUMN billiards_dwd.dim_site.scd2_is_current IS '【说明】SCD2
COMMENT ON COLUMN billiards_dwd.dim_site.scd2_version IS '【说明】SCD2 版本号(自增),用于与时间段一起避免版本重叠。 【示例】1SCD2 版本号(自增),用于与时间段一起避免版本重叠)。 【ODS来源】table_fee_transactions - 无DWD慢变元数据。 【JSON字段】无 - DWD慢变元数据 - 无。';
CREATE TABLE IF NOT EXISTS dim_site_Ex (
CREATE TABLE IF NOT EXISTS dim_site_ex (
site_id BIGINT,
avatar TEXT,
address TEXT,
@@ -133,7 +133,7 @@ CREATE TABLE IF NOT EXISTS dim_site_Ex (
SCD2_end_time TIMESTAMPTZ DEFAULT '9999-12-31',
SCD2_is_current INT DEFAULT 1,
SCD2_version INT DEFAULT 1,
PRIMARY KEY (site_id)
PRIMARY KEY (site_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_site_ex IS 'DWD 维度表扩展字段表dim_site_ex。ODS 来源表billiards_ods.table_fee_transactions对应 JSONtable_fee_transactions.json分析table_fee_transactions-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -172,11 +172,12 @@ CREATE TABLE IF NOT EXISTS dim_table (
site_table_area_name TEXT,
tenant_table_area_id BIGINT,
table_price NUMERIC(18,2),
order_id BIGINT,
SCD2_start_time TIMESTAMPTZ DEFAULT now(),
SCD2_end_time TIMESTAMPTZ DEFAULT '9999-12-31',
SCD2_is_current INT DEFAULT 1,
SCD2_version INT DEFAULT 1,
PRIMARY KEY (table_id)
PRIMARY KEY (table_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_table IS 'DWD 维度表dim_table。ODS 来源表billiards_ods.site_tables_master对应 JSONsite_tables_master.json分析site_tables_master-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -193,7 +194,7 @@ COMMENT ON COLUMN billiards_dwd.dim_table.scd2_is_current IS '【说明】SCD2
COMMENT ON COLUMN billiards_dwd.dim_table.scd2_version IS '【说明】SCD2 版本号(自增),用于与时间段一起避免版本重叠。 【示例】1SCD2 版本号(自增),用于与时间段一起避免版本重叠)。 【ODS来源】site_tables_master - 无DWD慢变元数据。 【JSON字段】无 - DWD慢变元数据 - 无。';
CREATE TABLE IF NOT EXISTS dim_table_Ex (
CREATE TABLE IF NOT EXISTS dim_table_ex (
table_id BIGINT,
show_status INTEGER,
is_online_reservation INTEGER,
@@ -204,7 +205,7 @@ CREATE TABLE IF NOT EXISTS dim_table_Ex (
SCD2_end_time TIMESTAMPTZ DEFAULT '9999-12-31',
SCD2_is_current INT DEFAULT 1,
SCD2_version INT DEFAULT 1,
PRIMARY KEY (table_id)
PRIMARY KEY (table_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_table_ex IS 'DWD 维度表扩展字段表dim_table_ex。ODS 来源表billiards_ods.site_tables_master对应 JSONsite_tables_master.json分析site_tables_master-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -240,7 +241,7 @@ CREATE TABLE IF NOT EXISTS dim_assistant (
SCD2_end_time TIMESTAMPTZ,
SCD2_is_current INT,
SCD2_version INT,
PRIMARY KEY (assistant_id)
PRIMARY KEY (assistant_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_assistant IS 'DWD 维度表dim_assistant。ODS 来源表billiards_ods.assistant_accounts_master对应 JSONassistant_accounts_master.json分析assistant_accounts_master-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -265,7 +266,7 @@ COMMENT ON COLUMN billiards_dwd.dim_assistant.scd2_is_current IS '【说明】SC
COMMENT ON COLUMN billiards_dwd.dim_assistant.scd2_version IS '【说明】SCD2 版本号(自增),用于与时间段一起避免版本重叠。 【示例】1SCD2 版本号(自增),用于与时间段一起避免版本重叠)。 【ODS来源】assistant_accounts_master - 无DWD慢变元数据。 【JSON字段】无 - DWD慢变元数据 - 无。';
CREATE TABLE IF NOT EXISTS dim_assistant_Ex (
CREATE TABLE IF NOT EXISTS dim_assistant_ex (
assistant_id BIGINT,
gender INTEGER,
birth_date TIMESTAMPTZ,
@@ -314,7 +315,7 @@ CREATE TABLE IF NOT EXISTS dim_assistant_Ex (
SCD2_end_time TIMESTAMPTZ,
SCD2_is_current INT,
SCD2_version INT,
PRIMARY KEY (assistant_id)
PRIMARY KEY (assistant_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_assistant_ex IS 'DWD 维度表扩展字段表dim_assistant_ex。ODS 来源表billiards_ods.assistant_accounts_master对应 JSONassistant_accounts_master.json分析assistant_accounts_master-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -379,11 +380,13 @@ CREATE TABLE IF NOT EXISTS dim_member (
member_card_grade_name TEXT,
create_time TIMESTAMPTZ,
update_time TIMESTAMPTZ,
pay_money_sum NUMERIC(18,2),
recharge_money_sum NUMERIC(18,2),
SCD2_start_time TIMESTAMPTZ,
SCD2_end_time TIMESTAMPTZ,
SCD2_is_current INT,
SCD2_version INT,
PRIMARY KEY (member_id)
PRIMARY KEY (member_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_member IS 'DWD 维度表dim_member。ODS 来源表billiards_ods.member_profiles对应 JSONmember_profiles.json分析member_profiles-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -403,7 +406,7 @@ COMMENT ON COLUMN billiards_dwd.dim_member.scd2_is_current IS '【说明】SCD2
COMMENT ON COLUMN billiards_dwd.dim_member.scd2_version IS '【说明】SCD2 版本号(自增),用于与时间段一起避免版本重叠。 【示例】1SCD2 版本号(自增),用于与时间段一起避免版本重叠)。 【ODS来源】member_profiles - 无DWD慢变元数据。 【JSON字段】无 - DWD慢变元数据 - 无。';
CREATE TABLE IF NOT EXISTS dim_member_Ex (
CREATE TABLE IF NOT EXISTS dim_member_ex (
member_id BIGINT,
referrer_member_id BIGINT,
point NUMERIC(18,2),
@@ -411,11 +414,14 @@ CREATE TABLE IF NOT EXISTS dim_member_Ex (
growth_value NUMERIC(18,2),
user_status INTEGER,
status INTEGER,
person_tenant_org_id BIGINT,
person_tenant_org_name TEXT,
register_source TEXT,
SCD2_start_time TIMESTAMPTZ,
SCD2_end_time TIMESTAMPTZ,
SCD2_is_current INT,
SCD2_version INT,
PRIMARY KEY (member_id)
PRIMARY KEY (member_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_member_ex IS 'DWD 维度表扩展字段表dim_member_ex。ODS 来源表billiards_ods.member_profiles对应 JSONmember_profiles.json分析member_profiles-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -450,11 +456,13 @@ CREATE TABLE IF NOT EXISTS dim_member_card_account (
last_consume_time TIMESTAMPTZ,
status INTEGER,
is_delete INTEGER,
principal_balance NUMERIC(18,2),
member_grade BIGINT,
SCD2_start_time TIMESTAMPTZ,
SCD2_end_time TIMESTAMPTZ,
SCD2_is_current INT,
SCD2_version INT,
PRIMARY KEY (member_card_id)
PRIMARY KEY (member_card_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_member_card_account IS 'DWD 维度表dim_member_card_account。ODS 来源表billiards_ods.member_stored_value_cards对应 JSONmember_stored_value_cards.json分析member_stored_value_cards-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -481,7 +489,7 @@ COMMENT ON COLUMN billiards_dwd.dim_member_card_account.scd2_is_current IS '【
COMMENT ON COLUMN billiards_dwd.dim_member_card_account.scd2_version IS '【说明】SCD2 版本号(自增),用于与时间段一起避免版本重叠。 【示例】1SCD2 版本号(自增),用于与时间段一起避免版本重叠)。 【ODS来源】member_stored_value_cards - 无DWD慢变元数据。 【JSON字段】无 - DWD慢变元数据 - 无。';
CREATE TABLE IF NOT EXISTS dim_member_card_account_Ex (
CREATE TABLE IF NOT EXISTS dim_member_card_account_ex (
member_card_id BIGINT,
site_name TEXT,
tenant_name VARCHAR(64),
@@ -534,11 +542,16 @@ CREATE TABLE IF NOT EXISTS dim_member_card_account_Ex (
goodsCategoryId TEXT,
pdAssisnatLevel TEXT,
cxAssisnatLevel TEXT,
able_share_member_discount BOOLEAN,
electricity_deduct_radio NUMERIC(18,4),
electricity_discount NUMERIC(18,4),
electricity_card_deduct BOOLEAN,
recharge_freeze_balance NUMERIC(18,2),
SCD2_start_time TIMESTAMPTZ,
SCD2_end_time TIMESTAMPTZ,
SCD2_is_current INT,
SCD2_version INT,
PRIMARY KEY (member_card_id)
PRIMARY KEY (member_card_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_member_card_account_ex IS 'DWD 维度表扩展字段表dim_member_card_account_ex。ODS 来源表billiards_ods.member_stored_value_cards对应 JSONmember_stored_value_cards.json分析member_stored_value_cards-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -615,11 +628,12 @@ CREATE TABLE IF NOT EXISTS dim_tenant_goods (
create_time TIMESTAMPTZ,
update_time TIMESTAMPTZ,
is_delete INTEGER,
not_sale INTEGER,
SCD2_start_time TIMESTAMPTZ,
SCD2_end_time TIMESTAMPTZ,
SCD2_is_current INT,
SCD2_version INT,
PRIMARY KEY (tenant_goods_id)
PRIMARY KEY (tenant_goods_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_tenant_goods IS 'DWD 维度表dim_tenant_goods。ODS 来源表billiards_ods.tenant_goods_master对应 JSONtenant_goods_master.json分析tenant_goods_master-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -643,7 +657,7 @@ COMMENT ON COLUMN billiards_dwd.dim_tenant_goods.scd2_is_current IS '【说明
COMMENT ON COLUMN billiards_dwd.dim_tenant_goods.scd2_version IS '【说明】SCD2 版本号(自增),用于与时间段一起避免版本重叠。 【示例】1SCD2 版本号(自增),用于与时间段一起避免版本重叠)。 【ODS来源】tenant_goods_master - 无DWD慢变元数据。 【JSON字段】无 - DWD慢变元数据 - 无。';
CREATE TABLE IF NOT EXISTS dim_tenant_goods_Ex (
CREATE TABLE IF NOT EXISTS dim_tenant_goods_ex (
tenant_goods_id BIGINT,
remark_name VARCHAR(128),
pinyin_initial VARCHAR(128),
@@ -666,7 +680,7 @@ CREATE TABLE IF NOT EXISTS dim_tenant_goods_Ex (
SCD2_end_time TIMESTAMPTZ,
SCD2_is_current INT,
SCD2_version INT,
PRIMARY KEY (tenant_goods_id)
PRIMARY KEY (tenant_goods_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_tenant_goods_ex IS 'DWD 维度表扩展字段表dim_tenant_goods_ex。ODS 来源表billiards_ods.tenant_goods_master对应 JSONtenant_goods_master.json分析tenant_goods_master-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -715,11 +729,13 @@ CREATE TABLE IF NOT EXISTS dim_store_goods (
enable_status INTEGER,
send_state INTEGER,
is_delete INTEGER,
commodity_code TEXT,
not_sale INTEGER,
SCD2_start_time TIMESTAMPTZ,
SCD2_end_time TIMESTAMPTZ,
SCD2_is_current INT,
SCD2_version INT,
PRIMARY KEY (site_goods_id)
PRIMARY KEY (site_goods_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_store_goods IS 'DWD 维度表dim_store_goods。ODS 来源表billiards_ods.store_goods_master对应 JSONstore_goods_master.json分析store_goods_master-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -749,7 +765,7 @@ COMMENT ON COLUMN billiards_dwd.dim_store_goods.scd2_is_current IS '【说明】
COMMENT ON COLUMN billiards_dwd.dim_store_goods.scd2_version IS '【说明】SCD2 版本号(自增),用于与时间段一起避免版本重叠。 【示例】1SCD2 版本号(自增),用于与时间段一起避免版本重叠)。 【ODS来源】store_goods_master - 无DWD慢变元数据。 【JSON字段】无 - DWD慢变元数据 - 无。';
CREATE TABLE IF NOT EXISTS dim_store_goods_Ex (
CREATE TABLE IF NOT EXISTS dim_store_goods_ex (
site_goods_id BIGINT,
site_name TEXT,
unit TEXT,
@@ -780,7 +796,7 @@ CREATE TABLE IF NOT EXISTS dim_store_goods_Ex (
SCD2_end_time TIMESTAMPTZ,
SCD2_is_current INT,
SCD2_version INT,
PRIMARY KEY (site_goods_id)
PRIMARY KEY (site_goods_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_store_goods_ex IS 'DWD 维度表扩展字段表dim_store_goods_ex。ODS 来源表billiards_ods.store_goods_master对应 JSONstore_goods_master.json分析store_goods_master-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -833,7 +849,7 @@ CREATE TABLE IF NOT EXISTS dim_goods_category (
SCD2_end_time TIMESTAMPTZ,
SCD2_is_current INT,
SCD2_version INT,
PRIMARY KEY (category_id)
PRIMARY KEY (category_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_goods_category IS 'DWD 维度表dim_goods_category。ODS 来源表billiards_ods.stock_goods_category_tree对应 JSONstock_goods_category_tree.json分析stock_goods_category_tree-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -872,11 +888,13 @@ CREATE TABLE IF NOT EXISTS dim_groupbuy_package (
create_time TIMESTAMPTZ,
tenant_table_area_id_list VARCHAR(512),
card_type_ids VARCHAR(255),
sort INTEGER,
is_first_limit BOOLEAN,
SCD2_start_time TIMESTAMPTZ,
SCD2_end_time TIMESTAMPTZ,
SCD2_is_current INT,
SCD2_version INT,
PRIMARY KEY (groupbuy_package_id)
PRIMARY KEY (groupbuy_package_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_groupbuy_package IS 'DWD 维度表dim_groupbuy_package。ODS 来源表billiards_ods.group_buy_packages对应 JSONgroup_buy_packages.json分析group_buy_packages-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -902,7 +920,7 @@ COMMENT ON COLUMN billiards_dwd.dim_groupbuy_package.scd2_is_current IS '【说
COMMENT ON COLUMN billiards_dwd.dim_groupbuy_package.scd2_version IS '【说明】SCD2 版本号(自增),用于与时间段一起避免版本重叠。 【示例】1SCD2 版本号(自增),用于与时间段一起避免版本重叠)。 【ODS来源】group_buy_packages - 无DWD慢变元数据。 【JSON字段】无 - DWD慢变元数据 - 无。';
CREATE TABLE IF NOT EXISTS dim_groupbuy_package_Ex (
CREATE TABLE IF NOT EXISTS dim_groupbuy_package_ex (
groupbuy_package_id BIGINT,
site_name VARCHAR(100),
usable_count INTEGER,
@@ -923,11 +941,12 @@ CREATE TABLE IF NOT EXISTS dim_groupbuy_package_Ex (
effective_status INTEGER,
max_selectable_categories INTEGER,
creator_name VARCHAR(100),
tenant_coupon_sale_order_item_id BIGINT,
SCD2_start_time TIMESTAMPTZ,
SCD2_end_time TIMESTAMPTZ,
SCD2_is_current INT,
SCD2_version INT,
PRIMARY KEY (groupbuy_package_id)
PRIMARY KEY (groupbuy_package_id, scd2_start_time)
);
COMMENT ON TABLE billiards_dwd.dim_groupbuy_package_ex IS 'DWD 维度表扩展字段表dim_groupbuy_package_ex。ODS 来源表billiards_ods.group_buy_packages对应 JSONgroup_buy_packages.json分析group_buy_packages-Analysis.md。装载/清洗逻辑参考etl_billiards/tasks/dwd_load_task.pyDwdLoadTask';
@@ -990,6 +1009,11 @@ CREATE TABLE IF NOT EXISTS dwd_settlement_head (
coupon_amount NUMERIC(18,2),
rounding_amount NUMERIC(18,2),
point_amount NUMERIC(18,2),
electricity_money NUMERIC(18,2),
real_electricity_money NUMERIC(18,2),
electricity_adjust_money NUMERIC(18,2),
pl_coupon_sale_amount NUMERIC(18,2),
mervou_sales_amount NUMERIC(18,2),
PRIMARY KEY (order_settle_id)
);
@@ -1028,7 +1052,7 @@ COMMENT ON COLUMN billiards_dwd.dwd_settlement_head.rounding_amount IS '【说
COMMENT ON COLUMN billiards_dwd.dwd_settlement_head.point_amount IS '【说明】金额字段,用于计费/结算/核算等金额计算。 【示例】NULL金额字段用于计费/结算/核算等金额计算)。 【ODS来源】settlement_records - pointamount。 【JSON字段】settlement_records.json - $ - pointamount。';
CREATE TABLE IF NOT EXISTS dwd_settlement_head_Ex (
CREATE TABLE IF NOT EXISTS dwd_settlement_head_ex (
order_settle_id BIGINT,
serial_number INTEGER,
settle_status INTEGER,
@@ -1059,6 +1083,7 @@ CREATE TABLE IF NOT EXISTS dwd_settlement_head_Ex (
order_remark VARCHAR(255),
operator_id BIGINT,
salesman_user_id BIGINT,
settle_list JSONB,
PRIMARY KEY (order_settle_id)
);
@@ -1123,6 +1148,8 @@ CREATE TABLE IF NOT EXISTS dwd_table_fee_log (
ledger_status INTEGER,
is_single_order INTEGER,
is_delete INTEGER,
activity_discount_amount NUMERIC(18,2),
real_service_money NUMERIC(18,2),
PRIMARY KEY (table_fee_log_id)
);
@@ -1156,7 +1183,7 @@ COMMENT ON COLUMN billiards_dwd.dwd_table_fee_log.is_single_order IS '【说明
COMMENT ON COLUMN billiards_dwd.dwd_table_fee_log.is_delete IS '【说明】布尔/开关字段,用于表示是否/可用性等业务开关。 【示例】0布尔/开关字段,用于表示是否/可用性等业务开关)。 【ODS来源】table_fee_transactions - is_delete。 【JSON字段】table_fee_transactions.json - data.siteTableUseDetailsList - is_delete。';
CREATE TABLE IF NOT EXISTS dwd_table_fee_log_Ex (
CREATE TABLE IF NOT EXISTS dwd_table_fee_log_ex (
table_fee_log_id BIGINT,
operator_name VARCHAR(64),
salesman_name VARCHAR(64),
@@ -1169,6 +1196,7 @@ CREATE TABLE IF NOT EXISTS dwd_table_fee_log_Ex (
operator_id BIGINT,
salesman_user_id BIGINT,
salesman_org_id BIGINT,
order_consumption_type INTEGER,
PRIMARY KEY (table_fee_log_id)
);
@@ -1201,6 +1229,9 @@ CREATE TABLE IF NOT EXISTS dwd_table_fee_adjust (
ledger_status INTEGER,
is_delete INTEGER,
adjust_time TIMESTAMPTZ,
table_name TEXT,
table_price NUMERIC(18,2),
charge_free BOOLEAN,
PRIMARY KEY (table_fee_adjust_id)
);
@@ -1220,7 +1251,7 @@ COMMENT ON COLUMN billiards_dwd.dwd_table_fee_adjust.is_delete IS '【说明】
COMMENT ON COLUMN billiards_dwd.dwd_table_fee_adjust.adjust_time IS '【说明】时间/日期字段,用于记录业务时间与统计口径对齐。 【示例】2025-11-09 23:25:11时间/日期字段,用于记录业务时间与统计口径对齐)。 【ODS来源】table_fee_discount_records - create_time。 【JSON字段】table_fee_discount_records.json - data.taiFeeAdjustInfos - create_time。';
CREATE TABLE IF NOT EXISTS dwd_table_fee_adjust_Ex (
CREATE TABLE IF NOT EXISTS dwd_table_fee_adjust_ex (
table_fee_adjust_id BIGINT,
adjust_type INTEGER,
ledger_count INTEGER,
@@ -1229,6 +1260,11 @@ CREATE TABLE IF NOT EXISTS dwd_table_fee_adjust_Ex (
operator_name VARCHAR(64),
applicant_id BIGINT,
operator_id BIGINT,
area_type_id BIGINT,
site_table_area_id BIGINT,
site_table_area_name TEXT,
site_name TEXT,
tenant_name TEXT,
PRIMARY KEY (table_fee_adjust_id)
);
@@ -1267,6 +1303,7 @@ CREATE TABLE IF NOT EXISTS dwd_store_goods_sale (
ledger_status INTEGER,
is_delete INTEGER,
create_time TIMESTAMPTZ,
coupon_share_money NUMERIC(18,2),
PRIMARY KEY (store_goods_sale_id)
);
@@ -1296,7 +1333,7 @@ COMMENT ON COLUMN billiards_dwd.dwd_store_goods_sale.is_delete IS '【说明】
COMMENT ON COLUMN billiards_dwd.dwd_store_goods_sale.create_time IS '【说明】时间/日期字段,用于记录业务时间与统计口径对齐。 【示例】2025-11-09 23:35:57时间/日期字段,用于记录业务时间与统计口径对齐)。 【ODS来源】store_goods_sales_records - create_time。 【JSON字段】store_goods_sales_records.json - data.orderGoodsLedgers - create_time。';
CREATE TABLE IF NOT EXISTS dwd_store_goods_sale_Ex (
CREATE TABLE IF NOT EXISTS dwd_store_goods_sale_ex (
store_goods_sale_id BIGINT,
legacy_order_goods_id BIGINT,
site_name TEXT,
@@ -1392,6 +1429,7 @@ CREATE TABLE IF NOT EXISTS dwd_assistant_service_log (
start_use_time TIMESTAMPTZ,
last_use_time TIMESTAMPTZ,
is_delete INTEGER,
real_service_money NUMERIC(18,2),
PRIMARY KEY (assistant_service_id)
);
@@ -1430,7 +1468,7 @@ COMMENT ON COLUMN billiards_dwd.dwd_assistant_service_log.last_use_time IS '【
COMMENT ON COLUMN billiards_dwd.dwd_assistant_service_log.is_delete IS '【说明】布尔/开关字段,用于表示是否/可用性等业务开关。 【示例】0布尔/开关字段,用于表示是否/可用性等业务开关)。 【ODS来源】assistant_service_records - is_delete。 【JSON字段】assistant_service_records.json - data.orderAssistantDetails - is_delete。';
CREATE TABLE IF NOT EXISTS dwd_assistant_service_log_Ex (
CREATE TABLE IF NOT EXISTS dwd_assistant_service_log_ex (
assistant_service_id BIGINT,
table_name VARCHAR(64),
assistant_name VARCHAR(64),
@@ -1461,6 +1499,7 @@ CREATE TABLE IF NOT EXISTS dwd_assistant_service_log_Ex (
get_grade_times INTEGER,
grade_status INTEGER,
composite_grade_time TIMESTAMPTZ,
assistant_team_name TEXT,
PRIMARY KEY (assistant_service_id)
);
@@ -1508,6 +1547,7 @@ CREATE TABLE IF NOT EXISTS dwd_assistant_trash_event (
abolish_amount NUMERIC(18,2),
trash_reason VARCHAR(255),
create_time TIMESTAMPTZ,
tenant_id BIGINT,
PRIMARY KEY (assistant_trash_event_id)
);
@@ -1524,7 +1564,7 @@ COMMENT ON COLUMN billiards_dwd.dwd_assistant_trash_event.trash_reason IS '【
COMMENT ON COLUMN billiards_dwd.dwd_assistant_trash_event.create_time IS '【说明】时间/日期字段,用于记录业务时间与统计口径对齐。 【示例】2025-11-09 19:23:29时间/日期字段,用于记录业务时间与统计口径对齐)。 【ODS来源】assistant_cancellation_records - createTime。 【JSON字段】assistant_cancellation_records.json - data.abolitionAssistants - createTime。';
CREATE TABLE IF NOT EXISTS dwd_assistant_trash_event_Ex (
CREATE TABLE IF NOT EXISTS dwd_assistant_trash_event_ex (
assistant_trash_event_id BIGINT,
table_name VARCHAR(64),
table_area_name VARCHAR(64),
@@ -1557,6 +1597,8 @@ CREATE TABLE IF NOT EXISTS dwd_member_balance_change (
change_time TIMESTAMPTZ,
is_delete INTEGER,
remark VARCHAR(255),
principal_before NUMERIC(18,2),
principal_after NUMERIC(18,2),
PRIMARY KEY (balance_change_id)
);
@@ -1582,13 +1624,14 @@ COMMENT ON COLUMN billiards_dwd.dwd_member_balance_change.is_delete IS '【说
COMMENT ON COLUMN billiards_dwd.dwd_member_balance_change.remark IS '【说明】明细字段,用于记录事实取值。 【示例】充值退款(明细字段,用于记录事实取值)。 【ODS来源】member_balance_changes - remark。 【JSON字段】member_balance_changes.json - data.tenantMemberCardLogs - remark。';
CREATE TABLE IF NOT EXISTS dwd_member_balance_change_EX (
CREATE TABLE IF NOT EXISTS dwd_member_balance_change_ex (
balance_change_id BIGINT,
pay_site_name VARCHAR(64),
register_site_name VARCHAR(64),
refund_amount NUMERIC(18,2),
operator_id BIGINT,
operator_name VARCHAR(64),
principal_data TEXT,
PRIMARY KEY (balance_change_id)
);
@@ -1625,6 +1668,8 @@ CREATE TABLE IF NOT EXISTS dwd_groupbuy_redemption (
is_delete INTEGER,
ledger_name VARCHAR(128),
create_time TIMESTAMPTZ,
member_discount_money NUMERIC(18,2),
coupon_sale_id BIGINT,
PRIMARY KEY (redemption_id)
);
@@ -1654,7 +1699,7 @@ COMMENT ON COLUMN billiards_dwd.dwd_groupbuy_redemption.ledger_name IS '【说
COMMENT ON COLUMN billiards_dwd.dwd_groupbuy_redemption.create_time IS '【说明】时间/日期字段,用于记录业务时间与统计口径对齐。 【示例】2025-11-09 23:35:57时间/日期字段,用于记录业务时间与统计口径对齐)。 【ODS来源】group_buy_redemption_records - create_time。 【JSON字段】group_buy_redemption_records.json - data.siteTableUseDetailsList - create_time。';
CREATE TABLE IF NOT EXISTS dwd_groupbuy_redemption_Ex (
CREATE TABLE IF NOT EXISTS dwd_groupbuy_redemption_ex (
redemption_id BIGINT,
site_name VARCHAR(64),
table_name VARCHAR(64),
@@ -1676,6 +1721,13 @@ CREATE TABLE IF NOT EXISTS dwd_groupbuy_redemption_Ex (
salesman_role_id BIGINT,
salesman_org_id BIGINT,
ledger_group_name VARCHAR(128),
table_share_money NUMERIC(18,2),
table_service_share_money NUMERIC(18,2),
goods_share_money NUMERIC(18,2),
good_service_share_money NUMERIC(18,2),
assistant_share_money NUMERIC(18,2),
assistant_service_share_money NUMERIC(18,2),
recharge_share_money NUMERIC(18,2),
PRIMARY KEY (redemption_id)
);
@@ -1750,7 +1802,7 @@ COMMENT ON COLUMN billiards_dwd.dwd_platform_coupon_redemption.create_time IS '
COMMENT ON COLUMN billiards_dwd.dwd_platform_coupon_redemption.consume_time IS '【说明】时间/日期字段,用于记录业务时间与统计口径对齐。 【示例】2025-11-09 23:41:04时间/日期字段,用于记录业务时间与统计口径对齐)。 【ODS来源】platform_coupon_redemption_records - consume_time。 【JSON字段】platform_coupon_redemption_records.json - $ - consume_time。';
CREATE TABLE IF NOT EXISTS dwd_platform_coupon_redemption_Ex (
CREATE TABLE IF NOT EXISTS dwd_platform_coupon_redemption_ex (
platform_coupon_redemption_id BIGINT,
coupon_cover VARCHAR(255),
coupon_remark VARCHAR(255),
@@ -1814,7 +1866,7 @@ COMMENT ON COLUMN billiards_dwd.dwd_recharge_order.create_time IS '【说明】
COMMENT ON COLUMN billiards_dwd.dwd_recharge_order.pay_time IS '【说明】时间/日期字段,用于记录业务时间与统计口径对齐。 【示例】NULL时间/日期字段,用于记录业务时间与统计口径对齐)。 【ODS来源】recharge_settlements - paytime。 【JSON字段】recharge_settlements.json - $ - paytime。';
CREATE TABLE IF NOT EXISTS dwd_recharge_order_Ex (
CREATE TABLE IF NOT EXISTS dwd_recharge_order_ex (
recharge_order_id BIGINT,
site_name_snapshot TEXT,
settle_status INTEGER,
@@ -1919,6 +1971,7 @@ CREATE TABLE IF NOT EXISTS dwd_payment (
create_time TIMESTAMPTZ,
pay_time TIMESTAMPTZ,
pay_date DATE,
tenant_id BIGINT,
PRIMARY KEY (payment_id)
);
@@ -1967,7 +2020,7 @@ COMMENT ON COLUMN billiards_dwd.dwd_refund.member_id IS '【说明】标识类 I
COMMENT ON COLUMN billiards_dwd.dwd_refund.member_card_id IS '【说明】标识类 ID 字段,用于关联/定位相关实体。 【示例】0标识类 ID 字段,用于关联/定位相关实体)。 【ODS来源】refund_transactions - member_card_id。 【JSON字段】refund_transactions.json - $ - member_card_id。';
CREATE TABLE IF NOT EXISTS dwd_refund_Ex (
CREATE TABLE IF NOT EXISTS dwd_refund_ex (
refund_id BIGINT,
tenant_name VARCHAR(64),
pay_sn BIGINT,

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,347 @@
-- =============================================================================
-- DWS 配置表初始数据
-- 版本: v3.0
-- 创建日期: 2026-02-01
-- 描述: 初始化配置表数据,包含绩效档位、等级定价、奖金规则、区域分类、技能映射
-- =============================================================================
-- =============================================================================
-- 1. cfg_performance_tier - 绩效档位配置6档 + 新入职)
-- 数据来源DWS 数据库处理需求.md 第35-41行
-- 档位原因考虑 总业绩小时数阈值 专业课抽成(元/小时) 打赏课抽成 次月休假(天)
-- 0档 淘汰压力 H <100 28 50% 3
-- 1档 及格档(重点激励) 100≤ H <130 18 40% 4
-- 2档 良好档(重点激励) 130≤ H <160 15 38% 4
-- 3档 优秀档 160≤ H <190 13 35% 5
-- 4档 卓越加速档(高端人才倾斜) 190≤ H <220 10 33% 6
-- 5档 冠军加速档(高端人才倾斜) H ≥220 8 30% 休假自由
-- =============================================================================
TRUNCATE TABLE billiards_dws.cfg_performance_tier RESTART IDENTITY CASCADE;
INSERT INTO billiards_dws.cfg_performance_tier (
tier_code, tier_name, tier_level,
min_hours, max_hours,
base_deduction, bonus_deduction_ratio, vacation_days, vacation_unlimited,
is_new_hire_tier, effective_from, effective_to, description
) VALUES
-- 0档 淘汰压力: H<100, 专业课抽成28元/小时, 打赏课抽成50%, 休假3天
('T0', '0档-淘汰压力', 0,
0, 100,
28.00, 0.50, 3, FALSE,
FALSE, '2000-01-01', '9999-12-31',
'淘汰压力档H<100专业课抽成28元/小时打赏课抽成50%休假3天'),
-- 1档 及格档: 100≤H<130, 专业课抽成18元/小时, 打赏课抽成40%, 休假4天
('T1', '1档-及格档', 1,
100, 130,
18.00, 0.40, 4, FALSE,
FALSE, '2000-01-01', '9999-12-31',
'及格档重点激励100≤H<130专业课抽成18元/小时打赏课抽成40%休假4天'),
-- 2档 良好档: 130≤H<160, 专业课抽成15元/小时, 打赏课抽成38%, 休假4天
('T2', '2档-良好档', 2,
130, 160,
15.00, 0.38, 4, FALSE,
FALSE, '2000-01-01', '9999-12-31',
'良好档重点激励130≤H<160专业课抽成15元/小时打赏课抽成38%休假4天'),
-- 3档 优秀档: 160≤H<190, 专业课抽成13元/小时, 打赏课抽成35%, 休假5天
('T3', '3档-优秀档', 3,
160, 190,
13.00, 0.35, 5, FALSE,
FALSE, '2000-01-01', '9999-12-31',
'优秀档160≤H<190专业课抽成13元/小时打赏课抽成35%休假5天'),
-- 4档 卓越加速档: 190≤H<220, 专业课抽成10元/小时, 打赏课抽成33%, 休假6天
('T4', '4档-卓越加速档', 4,
190, 220,
10.00, 0.33, 6, FALSE,
FALSE, '2000-01-01', '9999-12-31',
'卓越加速档高端人才倾斜190≤H<220专业课抽成10元/小时打赏课抽成33%休假6天'),
-- 5档 冠军加速档: H≥220, 专业课抽成8元/小时, 打赏课抽成30%, 休假自由
('T5', '5档-冠军加速档', 5,
220, NULL,
8.00, 0.30, 0, TRUE,
FALSE, '2000-01-01', '9999-12-31',
'冠军加速档高端人才倾斜H≥220专业课抽成8元/小时打赏课抽成30%,休假自由'),
-- 新入职档位: 首月特殊处理按1档标准但不参与排名
('NEW', '新入职档位', -1,
0, NULL,
18.00, 0.40, 4, FALSE,
TRUE, '2000-01-01', '9999-12-31',
'新入职月1日0点后入职者首月使用按1档抽成标准不参与排名奖金');
-- =============================================================================
-- 2. cfg_assistant_level_price - 助教等级定价
-- 说明:
-- - level_code 来自 dim_assistant.assistant_level
-- - 8=助教管理, 10=初级, 20=中级, 30=高级, 40=星级
-- - 价格为客户支付价格(对外价格),助教收入=客户支付-档位抽成
-- - 数据来源DWS 数据库处理需求.md
-- =============================================================================
TRUNCATE TABLE billiards_dws.cfg_assistant_level_price RESTART IDENTITY CASCADE;
INSERT INTO billiards_dws.cfg_assistant_level_price (
level_code, level_name,
base_course_price, bonus_course_price,
effective_from, effective_to, description
) VALUES
-- 初级助教基础课对客户收费98元/小时
(10, '初级',
98.00, 190.00,
'2000-01-01', '9999-12-31',
'初级助教基础课98元/时附加课190元/时(客户支付价格)'),
-- 中级助教基础课对客户收费108元/小时
(20, '中级',
108.00, 190.00,
'2000-01-01', '9999-12-31',
'中级助教基础课108元/时附加课190元/时(客户支付价格)'),
-- 高级助教基础课对客户收费118元/小时
(30, '高级',
118.00, 190.00,
'2000-01-01', '9999-12-31',
'高级助教基础课118元/时附加课190元/时(客户支付价格)'),
-- 星级助教基础课对客户收费138元/小时
(40, '星级',
138.00, 190.00,
'2000-01-01', '9999-12-31',
'星级助教基础课138元/时附加课190元/时(客户支付价格)'),
-- 助教管理level_code=8通常不参与客户服务计费此处设置默认值
(8, '助教管理',
98.00, 190.00,
'2000-01-01', '9999-12-31',
'助教管理:不参与客户服务计费,默认按初级价格');
-- =============================================================================
-- 3. cfg_bonus_rules - 奖金规则配置
-- 说明:
-- - SPRINT: 冲刺奖金,按业绩小时数阈值,不累计取最高档
-- - TOP_RANK: Top3排名奖金按有效业绩排名并列都算
-- =============================================================================
TRUNCATE TABLE billiards_dws.cfg_bonus_rules RESTART IDENTITY CASCADE;
INSERT INTO billiards_dws.cfg_bonus_rules (
rule_type, rule_code, rule_name,
threshold_hours, rank_position, bonus_amount,
is_cumulative, priority,
effective_from, effective_to, description
) VALUES
-- 冲刺奖金: H>=190 得300元
('SPRINT', 'SPRINT_190', '冲刺奖金190',
190.00, NULL, 300.00,
FALSE, 1,
'2000-01-01', '9999-12-31',
'业绩≥190小时获得300元冲刺奖金不累计'),
-- 冲刺奖金: H>=220 得800元优先级更高覆盖190档
('SPRINT', 'SPRINT_220', '冲刺奖金220',
220.00, NULL, 800.00,
FALSE, 2,
'2000-01-01', '9999-12-31',
'业绩≥220小时获得800元冲刺奖金覆盖190档'),
-- Top1排名奖金: 1000元
('TOP_RANK', 'TOP_1', 'Top1排名奖金',
NULL, 1, 1000.00,
FALSE, 0,
'2000-01-01', '9999-12-31',
'月度排名第一获得1000元并列都算'),
-- Top2排名奖金: 600元
('TOP_RANK', 'TOP_2', 'Top2排名奖金',
NULL, 2, 600.00,
FALSE, 0,
'2000-01-01', '9999-12-31',
'月度排名第二获得600元并列都算'),
-- Top3排名奖金: 400元
('TOP_RANK', 'TOP_3', 'Top3排名奖金',
NULL, 3, 400.00,
FALSE, 0,
'2000-01-01', '9999-12-31',
'月度排名第三获得400元并列都算');
-- =============================================================================
-- 4. cfg_area_category - 台区分类映射
-- 说明:
-- - 将 dim_table.site_table_area_name 映射到财务报表区域分类
-- - 映射规则: 精确匹配 > 模糊匹配 > 默认兜底
-- - 数据来源: BD_manual_dim_table.md 中的 site_table_area_name 实际分布
-- 分类设计:
-- - BILLIARD: 台球散台A区/B区/C区/TV台
-- - BILLIARD_VIP: 台球VIP包厢
-- - SNOOKER: 斯诺克区
-- - MAHJONG: 麻将区
-- - KTV: K歌/KTV
-- - SPECIAL: 特殊(补时长等)
-- - OTHER: 其他
-- =============================================================================
TRUNCATE TABLE billiards_dws.cfg_area_category RESTART IDENTITY CASCADE;
INSERT INTO billiards_dws.cfg_area_category (
source_area_name, category_code, category_name,
match_type, match_priority, is_active, description
) VALUES
-- ============ 台球散台区(精确匹配)============
('A区', 'BILLIARD', '台球散台',
'EXACT', 10, TRUE, '台球散台A区18台- 中八/追分'),
('B区', 'BILLIARD', '台球散台',
'EXACT', 10, TRUE, '台球散台B区15台- 中八/追分'),
('C区', 'BILLIARD', '台球散台',
'EXACT', 10, TRUE, '台球散台C区6台- 中八/追分'),
('TV台', 'BILLIARD', '台球散台',
'EXACT', 10, TRUE, '台球散台TV台1台- 中八/追分'),
-- ============ 台球VIP包厢精确匹配============
('VIP包厢', 'BILLIARD_VIP', '台球VIP',
'EXACT', 10, TRUE, '台球VIPVIP包厢4台- V1-V4中八, V5斯诺克'),
-- ============ 斯诺克区(精确匹配)============
('斯诺克区', 'SNOOKER', '斯诺克',
'EXACT', 10, TRUE, '斯诺克斯诺克区4台'),
-- ============ 麻将区(精确匹配)============
('麻将房', 'MAHJONG', '麻将棋牌',
'EXACT', 10, TRUE, '麻将棋牌麻将房5台'),
('M7', 'MAHJONG', '麻将棋牌',
'EXACT', 10, TRUE, '麻将棋牌M72台'),
('M8', 'MAHJONG', '麻将棋牌',
'EXACT', 10, TRUE, '麻将棋牌M81台'),
('666', 'MAHJONG', '麻将棋牌',
'EXACT', 10, TRUE, '麻将棋牌6662台'),
('发财', 'MAHJONG', '麻将棋牌',
'EXACT', 10, TRUE, '麻将棋牌发财1台'),
-- ============ KTV/K包精确匹配============
('K包', 'KTV', 'K歌娱乐',
'EXACT', 10, TRUE, 'K歌娱乐K包4台'),
('k包活动区', 'KTV', 'K歌娱乐',
'EXACT', 10, TRUE, 'K歌娱乐k包活动区2台'),
('幸会158', 'KTV', 'K歌娱乐',
'EXACT', 10, TRUE, 'K歌娱乐幸会1582台'),
-- ============ 特殊区域(精确匹配)============
('补时长', 'SPECIAL', '补时长',
'EXACT', 10, TRUE, '特殊补时长7台- 用于时长补录'),
-- ============ 模糊匹配规则(优先级较低)============
('%VIP%', 'BILLIARD_VIP', '台球VIP',
'LIKE', 50, TRUE, '模糊匹配:包含"VIP"的区域'),
('%斯诺克%', 'SNOOKER', '斯诺克',
'LIKE', 50, TRUE, '模糊匹配:包含"斯诺克"的区域'),
('%麻将%', 'MAHJONG', '麻将棋牌',
'LIKE', 50, TRUE, '模糊匹配:包含"麻将"的区域'),
('%K包%', 'KTV', 'K歌娱乐',
'LIKE', 50, TRUE, '模糊匹配:包含"K包"的区域'),
('%KTV%', 'KTV', 'K歌娱乐',
'LIKE', 50, TRUE, '模糊匹配:包含"KTV"的区域'),
-- ============ 默认兜底(优先级最低)============
('DEFAULT', 'OTHER', '其他',
'DEFAULT', 999, TRUE, '兜底规则:无法匹配的区域归入其他');
-- =============================================================================
-- 5. cfg_skill_type - 技能→课程类型映射
-- 说明:
-- - 将 skill_id 映射到课程类型
-- - 基础课/陪打: skill_id = 2791903611396869
-- - 附加课/超休: skill_id = 2807440316432197
-- - 避免依赖 skill_name 文本匹配
-- =============================================================================
TRUNCATE TABLE billiards_dws.cfg_skill_type RESTART IDENTITY CASCADE;
INSERT INTO billiards_dws.cfg_skill_type (
skill_id, skill_name,
course_type_code, course_type_name,
is_active, description
) VALUES
-- 基础课/陪打
(2791903611396869, '台球基础陪打',
'BASE', '基础课',
TRUE, '基础课:陪打服务,按助教等级计价'),
-- 附加课/超休
(2807440316432197, '台球超休服务',
'BONUS', '附加课',
TRUE, '附加课:超休/激励课固定50元/小时'),
-- 包厢课(如有)
(2807440316432198, '包厢服务',
'BASE', '基础课',
TRUE, '包厢服务:归入基础课统计');
-- =============================================================================
-- 6. 优惠类型配置(用于财务优惠明细分析)
-- 说明: 定义各类优惠的代码和名称,便于后续分析
-- =============================================================================
-- 此配置作为代码常量使用,不单独建表
-- GROUPBUY - 团购优惠
-- VIP - 会员折扣
-- GIFT_CARD - 赠送卡抵扣
-- MANUAL - 手动调整
-- ROUNDING - 抹零
-- BIG_CUSTOMER - 大客户优惠(待抽样分析确认)
-- OTHER - 其他优惠
-- =============================================================================
-- 7. 支出类型配置用于Excel导入
-- 说明: 定义各类支出的代码和名称
-- =============================================================================
-- 此配置作为代码常量使用,不单独建表
-- RENT - 房租
-- UTILITY - 水电费
-- PROPERTY - 物业费
-- SALARY - 工资
-- REIMBURSE - 报销
-- PLATFORM_FEE - 平台服务费
-- OTHER - 其他支出
-- =============================================================================
-- 8. 平台类型配置用于Excel导入
-- 说明: 定义各平台的代码和名称
-- =============================================================================
-- 此配置作为代码常量使用,不单独建表
-- MEITUAN - 美团
-- DOUYIN - 抖音
-- DIANPING - 大众点评
-- OTHER - 其他平台
-- =============================================================================
-- 验证数据插入
-- =============================================================================
DO $$
DECLARE
v_tier_count INTEGER;
v_price_count INTEGER;
v_bonus_count INTEGER;
v_area_count INTEGER;
v_skill_count INTEGER;
BEGIN
SELECT COUNT(*) INTO v_tier_count FROM billiards_dws.cfg_performance_tier;
SELECT COUNT(*) INTO v_price_count FROM billiards_dws.cfg_assistant_level_price;
SELECT COUNT(*) INTO v_bonus_count FROM billiards_dws.cfg_bonus_rules;
SELECT COUNT(*) INTO v_area_count FROM billiards_dws.cfg_area_category;
SELECT COUNT(*) INTO v_skill_count FROM billiards_dws.cfg_skill_type;
RAISE NOTICE '配置数据初始化完成:';
RAISE NOTICE ' - cfg_performance_tier: % 条', v_tier_count;
RAISE NOTICE ' - cfg_assistant_level_price: % 条', v_price_count;
RAISE NOTICE ' - cfg_bonus_rules: % 条', v_bonus_count;
RAISE NOTICE ' - cfg_area_category: % 条', v_area_count;
RAISE NOTICE ' - cfg_skill_type: % 条', v_skill_count;
END;
$$;

View File

@@ -0,0 +1,118 @@
-- =============================================================================
-- 指数算法参数初始化脚本
-- 版本: v1.0
-- 创建日期: 2026-02-03
-- 描述: 为客户召回指数和客户-助教亲密指数插入默认参数
-- =============================================================================
-- 清空旧数据(如果需要重新初始化)
-- DELETE FROM billiards_dws.cfg_index_parameters WHERE index_type IN ('RECALL', 'INTIMACY');
-- =============================================================================
-- 客户召回指数RECALL参数
-- =============================================================================
INSERT INTO billiards_dws.cfg_index_parameters
(index_type, param_name, param_value, description, effective_from)
VALUES
-- 基础参数
('RECALL', 'lookback_days', 60, '回溯窗口分析近60天的数据', CURRENT_DATE),
('RECALL', 'sigma_min', 2.0, '波动下限(天):避免σ过小导致超期过敏', CURRENT_DATE),
-- 半衰期参数
('RECALL', 'halflife_new', 7, '新客户半衰期7天后新客户加分衰减到一半', CURRENT_DATE),
('RECALL', 'halflife_recharge', 10, '刚充值半衰期10天后充值加分衰减到一半', CURRENT_DATE),
-- 权重参数
('RECALL', 'weight_overdue', 3.0, '超期紧急性权重主导因子建议3.0', CURRENT_DATE),
('RECALL', 'weight_new', 1.0, '新客户权重建议1.0', CURRENT_DATE),
('RECALL', 'weight_recharge', 1.0, '刚充值权重建议1.0', CURRENT_DATE),
('RECALL', 'weight_hot', 1.0, '热度断档权重建议1.0', CURRENT_DATE),
-- 映射参数
('RECALL', 'percentile_lower', 5, '下锚分位数5分位', CURRENT_DATE),
('RECALL', 'percentile_upper', 95, '上锚分位数95分位', CURRENT_DATE),
('RECALL', 'ewma_alpha', 0.2, 'EWMA平滑系数越小越平滑建议0.2', CURRENT_DATE)
ON CONFLICT (index_type, param_name, effective_from) DO UPDATE SET
param_value = EXCLUDED.param_value,
description = EXCLUDED.description,
updated_at = NOW();
-- =============================================================================
-- 客户-助教亲密指数INTIMACY参数
-- =============================================================================
INSERT INTO billiards_dws.cfg_index_parameters
(index_type, param_name, param_value, description, effective_from)
VALUES
-- 基础参数
('INTIMACY', 'lookback_days', 60, '回溯窗口分析近60天的数据', CURRENT_DATE),
('INTIMACY', 'session_merge_hours', 4, '会话合并间隔小时4小时内的服务算同次', CURRENT_DATE),
('INTIMACY', 'recharge_attribute_hours', 1, '充值归因窗口小时服务结束后1小时内', CURRENT_DATE),
('INTIMACY', 'amount_base', 500, '金额压缩基准(元):选门店常见充值档位', CURRENT_DATE),
('INTIMACY', 'incentive_weight', 1.5, '附加课权重倍数:附加课=基础课的1.5倍', CURRENT_DATE),
-- 半衰期参数
('INTIMACY', 'halflife_session', 14, '会话衰减半衰期14天后权重衰减到一半', CURRENT_DATE),
('INTIMACY', 'halflife_last', 10, '最近一次半衰期10天后温度衰减到一半', CURRENT_DATE),
('INTIMACY', 'halflife_recharge', 21, '充值衰减半衰期21天后充值贡献衰减到一半', CURRENT_DATE),
('INTIMACY', 'halflife_short', 7, '短期激增检测半衰期用于Burst检测', CURRENT_DATE),
('INTIMACY', 'halflife_long', 30, '长期激增检测半衰期用于Burst检测', CURRENT_DATE),
-- 权重参数
('INTIMACY', 'weight_frequency', 2.0, '频次权重建议2.0', CURRENT_DATE),
('INTIMACY', 'weight_recency', 1.5, '最近一次权重建议1.5', CURRENT_DATE),
('INTIMACY', 'weight_recharge', 2.0, '归因充值权重建议2.0', CURRENT_DATE),
('INTIMACY', 'weight_duration', 0.5, '时长权重次要因素建议0.5', CURRENT_DATE),
('INTIMACY', 'burst_gamma', 0.6, '激增放大系数γ建议0.6', CURRENT_DATE),
-- 映射参数
('INTIMACY', 'percentile_lower', 5, '下锚分位数5分位', CURRENT_DATE),
('INTIMACY', 'percentile_upper', 95, '上锚分位数95分位', CURRENT_DATE),
('INTIMACY', 'ewma_alpha', 0.2, 'EWMA平滑系数越小越平滑建议0.2', CURRENT_DATE)
ON CONFLICT (index_type, param_name, effective_from) DO UPDATE SET
param_value = EXCLUDED.param_value,
description = EXCLUDED.description,
updated_at = NOW();
-- =============================================================================
-- 验证
-- =============================================================================
-- 检查参数数量
DO $$
DECLARE
recall_count INTEGER;
intimacy_count INTEGER;
BEGIN
SELECT COUNT(*) INTO recall_count
FROM billiards_dws.cfg_index_parameters
WHERE index_type = 'RECALL';
SELECT COUNT(*) INTO intimacy_count
FROM billiards_dws.cfg_index_parameters
WHERE index_type = 'INTIMACY';
RAISE NOTICE '召回指数参数数量: %', recall_count;
RAISE NOTICE '亲密指数参数数量: %', intimacy_count;
IF recall_count < 10 THEN
RAISE WARNING '召回指数参数不完整期望至少10个';
END IF;
IF intimacy_count < 15 THEN
RAISE WARNING '亲密指数参数不完整期望至少15个';
END IF;
END $$;
-- 显示所有参数
SELECT
index_type,
param_name,
param_value,
description,
effective_from
FROM billiards_dws.cfg_index_parameters
ORDER BY index_type, param_name;

Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More