root/Neo-ZQYY
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat: 2026-04-15~05-02 累积变更基线 — AI 重构 + Runtime Context + DWS 修复

Browse Source 
涵盖（每条对应已存的审计记录）：
- AI 模块拆分：apps/backend/app/ai/apps -> prompts/（8 个 APP + app2a 派生）
  audit: 2026-04-20__ai-module-complete.md
- admin-web AI 管理套件：AIDashboard / AIOperations / AIRunLogs / AITriggers / TriggerManager
  audit: 2026-04-21__admin-web-ai-management-suite.md
- App2 财务洞察 prompt v3 -> v5.1 + 小程序 AI 接入（chat / board-finance）
  audit: 2026-04-22__app2_prompt_v5_1_and_miniprogram_ai_insight.md
- App2 prewarm 全过滤器 + AI 触发器 cron reschedule
  audit: 2026-04-21__app2-finance-prewarm-all-filters.md
  migration: 20260420_ai_trigger_jobs_and_app2_prewarm.sql / 20260421_app2_prewarm_cron_reschedule.sql
- AppType 联合类型对齐 + adminAiAppTypes.test.ts
  audit: 2026-04-30__admin_web_ai_app_type_alignment.md
- DashScope tokens_used 提取修复
  audit: 2026-04-30__backend_dashscope_tokens_used_extraction.md
- App3 线索完整详情 prompt
  audit: 2026-05-01__backend_app3_full_detail_prompt.md
- Runtime Context 沙箱（5-1~5-2 主线）：
  - 后端 schema/service + admin_runtime_context / xcx_runtime_clock 两个 router
  - admin-web RuntimeContext.tsx + miniprogram runtime-clock.ts
  - migration: 20260501__runtime_context_sandbox.sql
  - tools/db/verify_admin_web_sandbox.py + verify_sandbox_end_to_end.py
  - database/changes: 7 份 sandbox_* 验证报告
- 飞球 DWS 修复：finance_area_daily 区域汇总 + task_engine 调整
  + RLS 视图业务日上界（migration 20260502 + scripts/ops/gen_rls_business_date_migration.py）

合规：
- .gitignore 启用 tmp/ 排除
- 不入仓：apps/etl/connectors/feiqiu/.env（API_TOKEN secret，本地修改保留）

待验证清单：
- docs/audit/changes/2026-05-04__cumulative_baseline_pending_verification.md
  每个主题的功能完整性 / 上线验证几乎都未收口，按优先级 P0~P3 逐一处理
This commit is contained in:
Neo
2026-05-04 02:30:19 +08:00
parent 2010034840
commit caf179a5da
130 changed files with 14543 additions and 2717 deletions
Download Patch File Download Diff File Expand all files Collapse all files

219
apps/backend/app/services/fdw_queries.py
View File

@@ -80,7 +80,7 @@ def _get_etl_connection(site_id: int):
@contextmanager
def _fdw_context(conn: Any, site_id: int, *, etl_conn: Any = None):
"""
上下文管理器：直连 ETL 库 + SET LOCAL app.current_site_id。
上下文管理器：直连 ETL 库 + SET LOCAL app.current_site_id + app.current_business_date。
⚠️ 不使用 zqyy_app 的 fdw_etl.* foreign table，而是直连 ETL 库
查询 app.v_* RLS 视图。原因：postgres_fdw 不传递自定义 GUC 参数
@@ -91,7 +91,31 @@ def _fdw_context(conn: Any, site_id: int, *, etl_conn: Any = None):
CHANGE 2026-03-26 | ETL 连接复用：传入 etl_conn 时复用已有连接（不关闭），
不传时新建连接并在 yield 后自动关闭。避免同一请求内多次新建连接（每次 ~2.6s）。
CHANGE 2026-05-02 | 同时设置 app.current_business_date / app.current_runtime_mode，
供 RLS 视图层（C 方案）做日期上界裁剪。conn=None 时降级 live。
"""
from app.services.runtime_context import (
MODE_LIVE,
MODE_SANDBOX,
get_runtime_context,
)
# 业务日：优先从 zqyy_app 业务库的 RuntimeContext 读取；conn 不可用时降级为系统今天
bd_str = ""
rt_mode = MODE_LIVE
try:
if conn is not None:
ctx = get_runtime_context(site_id, conn=conn)
bd_str = ctx.business_date.isoformat()
rt_mode = MODE_SANDBOX if ctx.is_sandbox else MODE_LIVE
else:
from datetime import date as _date
bd_str = _date.today().isoformat()
except Exception:
from datetime import date as _date
bd_str = _date.today().isoformat()
rt_mode = MODE_LIVE
owned = etl_conn is None
if owned:
etl_conn = _get_etl_connection(site_id)
@@ -99,6 +123,8 @@ def _fdw_context(conn: Any, site_id: int, *, etl_conn: Any = None):
with etl_conn.cursor() as cur:
cur.execute("BEGIN")
cur.execute("SET LOCAL app.current_site_id = %s", (str(site_id),))
cur.execute("SET LOCAL app.current_business_date = %s", (bd_str,))
cur.execute("SET LOCAL app.current_runtime_mode = %s", (rt_mode,))
yield cur
etl_conn.commit()
finally:
@@ -180,33 +206,53 @@ def get_last_visit_days(
"""
批量查询客户距上次到店天数。
来源: app.v_dws_member_consumption_summary.days_since_last（基于结算单）。
FIX: 原查 v_dwd_assistant_service_log 导致无助教服务的客户缺失到店记录。
来源: app.v_dws_member_consumption_summary。
consumption_summary 按 stat_date 有多行快照，取最新一行。
CHANGE 2026-05-02 | 修复客户看板「最近到店」数据不准的问题：
- 旧版直接用 days_since_last（ETL 在 stat_date 那天预计算的快照值）。
若 ETL 没跑、跑得迟、或 sandbox_date 与 stat_date 不一致，结果就会严重失真。
- 新版改为实时计算：``business_date - last_consume_date``，
仅取 ``stat_date <= business_date`` 的快照行，沙箱模式下也能拿到一致结果。
返回 {member_id: days_since_visit} 映射，无记录的会员不在结果中。
"""
if not member_ids:
return {}
from app.services.runtime_context import as_runtime_today_param
ref_date = as_runtime_today_param(site_id, conn=conn)
result: dict[int, int | None] = {}
with _fdw_context(conn, site_id, etl_conn=etl_conn) as cur:
cur.execute(
"""
SELECT member_id, days_since_last
SELECT DISTINCT ON (member_id)
member_id,
last_consume_date,
stat_date
FROM app.v_dws_member_consumption_summary
WHERE member_id = ANY(%s)
AND days_since_last IS NOT NULL
AND stat_date <= %s
ORDER BY member_id, stat_date DESC
""",
(member_ids,),
(member_ids, ref_date),
)
seen: set[int] = set()
for row in cur.fetchall():
mid = row[0]
if mid not in seen:
seen.add(mid)
result[mid] = row[1]
last_consume = row[1]
if last_consume is None:
result[mid] = None
continue
try:
# last_consume_date 在 DWS 中是 date；少数实现可能给 timestamp，统一裁剪
if hasattr(last_consume, "date"):
last_consume = last_consume.date()
days = (ref_date - last_consume).days
result[mid] = max(days, 0)
except Exception:
result[mid] = None
return result
@@ -420,22 +466,33 @@ def batch_query_for_task_list(
# 3. 最后到店天数（基于消费汇总表，口径=结算单）
# FIX: 原查 v_dwd_assistant_service_log 导致无助教服务的客户缺失到店记录
# CHANGE 2026-05-02 | 实时按 business_date - last_consume_date 计算，
# 不再依赖 ETL 预计算的 days_since_last（解决看板显示偏差）。
from app.services.runtime_context import as_runtime_today_param
_ref_date = as_runtime_today_param(site_id, conn=conn)
cur.execute(
"""
SELECT member_id, days_since_last
SELECT DISTINCT ON (member_id)
member_id, last_consume_date, stat_date
FROM app.v_dws_member_consumption_summary
WHERE member_id = ANY(%s)
AND days_since_last IS NOT NULL
AND stat_date <= %s
ORDER BY member_id, stat_date DESC
""",
(member_ids,),
(member_ids, _ref_date),
)
seen_members: set[int] = set()
for row in cur.fetchall():
mid = row[0]
if mid not in seen_members:
seen_members.add(mid)
last_visit_map[mid] = row[1]
last_consume = row[1]
if last_consume is None:
last_visit_map[mid] = None
continue
try:
if hasattr(last_consume, "date"):
last_consume = last_consume.date()
last_visit_map[mid] = max((_ref_date - last_consume).days, 0)
except Exception:
last_visit_map[mid] = None
# 4. RS 指数
cur.execute(
@@ -486,10 +543,11 @@ def batch_query_for_task_list(
WHERE sl.site_assistant_id = %s
AND sl.tenant_member_id = ANY(%s)
AND sl.is_delete = 0
AND sl.create_time >= (CURRENT_DATE - INTERVAL '60 days')::timestamptz
AND sl.create_time >= (%s::date - INTERVAL '60 days')::timestamptz
AND sl.create_time < (%s::date + INTERVAL '1 day')::timestamptz
GROUP BY sl.tenant_member_id
""",
(assistant_id, member_ids),
(assistant_id, member_ids, _ref_date, _ref_date),
)
for row in cur.fetchall():
recent60d_map[row[0]] = {
@@ -559,15 +617,19 @@ def batch_query_for_task_list(
# 8. 绩效档位配置（用于构建 tier_nodes + bonus_money 计算）
# CHANGE 2026-03-24 | 增加 bonus_deduction_ratio 用于打赏课抽成差额计算
# CHANGE 2026-05-02 | 用 business_date 替代 CURRENT_DATE，沙箱按当时生效档位
from app.services.runtime_context import as_runtime_today_param as _rt_today
_ref_date = _rt_today(site_id, conn=conn)
cur.execute(
"""
SELECT tier_id, tier_code, tier_name, tier_level,
min_hours, max_hours, base_deduction, bonus_deduction_ratio
FROM app.v_cfg_performance_tier
WHERE effective_from <= CURRENT_DATE
AND effective_to >= CURRENT_DATE
WHERE effective_from <= %s::date
AND effective_to >= %s::date
ORDER BY tier_level
"""
""",
(_ref_date, _ref_date),
)
tier_rows = cur.fetchall()
performance_tiers = [
@@ -640,17 +702,21 @@ def get_performance_tiers(
返回 [{tier_id, tier_code, tier_name, tier_level, min_hours, max_hours,
base_deduction, bonus_deduction_ratio}, ...]。
CHANGE 2026-05-02 | 用 business_date 替代 CURRENT_DATE，沙箱按当时生效档位
"""
from app.services.runtime_context import as_runtime_today_param as _rt_today
ref_date = _rt_today(site_id, conn=conn)
with _fdw_context(conn, site_id) as cur:
cur.execute(
"""
SELECT tier_id, tier_code, tier_name, tier_level,
min_hours, max_hours, base_deduction, bonus_deduction_ratio
FROM app.v_cfg_performance_tier
WHERE effective_from <= CURRENT_DATE
AND effective_to >= CURRENT_DATE
WHERE effective_from <= %s::date
AND effective_to >= %s::date
ORDER BY tier_level
"""
""",
(ref_date, ref_date),
)
rows = cur.fetchall()
@@ -680,15 +746,18 @@ def get_level_map(conn: Any, site_id: int) -> dict[int, str]:
查询失败时返回空 dict（调用方应优雅降级）。
"""
try:
from app.services.runtime_context import as_runtime_today_param as _rt_today
ref_date = _rt_today(site_id, conn=conn)
with _fdw_context(conn, site_id) as cur:
cur.execute(
"""
SELECT DISTINCT level_code, level_name
FROM app.v_cfg_assistant_level_price
WHERE effective_from <= CURRENT_DATE
AND effective_to >= CURRENT_DATE
WHERE effective_from <= %s::date
AND effective_to >= %s::date
ORDER BY level_code
"""
""",
(ref_date, ref_date),
)
return {row[0]: row[1] for row in cur.fetchall()}
except Exception:
@@ -1198,8 +1267,11 @@ def get_coach_60d_stats(
来源: app.v_dwd_assistant_service_log。
⚠️ 废单排除: is_delete = 0。
CHANGE 2026-05-02 | 用 business_date 替代 CURRENT_DATE，沙箱不读「未来」60 天。
返回 {service_count, total_hours, avg_hours}。
"""
from app.services.runtime_context import as_runtime_today_param
ref_date = as_runtime_today_param(site_id, conn=conn)
with _fdw_context(conn, site_id) as cur:
cur.execute(
"""
@@ -1212,9 +1284,10 @@ def get_coach_60d_stats(
WHERE site_assistant_id = %s
AND tenant_member_id = %s
AND is_delete = 0
AND create_time >= (CURRENT_DATE - INTERVAL '60 days')::timestamptz
AND create_time >= (%s::date - INTERVAL '60 days')::timestamptz
AND create_time < (%s::date + INTERVAL '1 day')::timestamptz
""",
(assistant_id, member_id),
(assistant_id, member_id, ref_date, ref_date),
)
row = cur.fetchone()
if not row:
@@ -1917,14 +1990,17 @@ def get_customer_board_recall(
total = cur.fetchone()[0]
# 分页数据
# CHANGE 2026-05-02 | elapsed_days/overdue_days 用 business_date 替代 CURRENT_DATE
from app.services.runtime_context import as_runtime_today_param as _rt_today
ref_date = _rt_today(site_id, conn=conn)
offset = (page - 1) * page_size
cur.execute(
f"""
SELECT wi.member_id,
dm.nickname,
wi.ideal_interval_days,
CURRENT_DATE - wi.last_visit_time::date AS elapsed_days,
(CURRENT_DATE - wi.last_visit_time::date) - COALESCE(wi.ideal_interval_days, 0) AS overdue_days,
%s::date - wi.last_visit_time::date AS elapsed_days,
(%s::date - wi.last_visit_time::date) - COALESCE(wi.ideal_interval_days, 0) AS overdue_days,
wi.visits_30d,
wi.display_score,
COALESCE(ca.balance, 0) AS balance
@@ -1937,11 +2013,11 @@ def get_customer_board_recall(
WHERE scd2_is_current = 1
GROUP BY tenant_member_id
) ca ON wi.member_id = ca.tenant_member_id
WHERE 1=1 {proj_clause}
WHERE wi.last_visit_time <= %s::date + INTERVAL '1 day' {proj_clause}
ORDER BY wi.display_score DESC, wi.member_id
LIMIT %s OFFSET %s
""",
(*proj_params, page_size, offset),
(ref_date, ref_date, ref_date, *proj_params, page_size, offset),
)
items = []
for row in cur.fetchall():
@@ -2165,6 +2241,10 @@ def get_customer_board_recharge(
)
total = cur.fetchone()[0]
# CHANGE 2026-05-02 | 60 天充值窗口、stat_date、pay_time 全部按 business_date 截断
from app.services.runtime_context import as_runtime_today_param
ref_date = as_runtime_today_param(site_id, conn=conn)
offset = (page - 1) * page_size
cur.execute(
f"""
@@ -2173,7 +2253,8 @@ def get_customer_board_recharge(
MAX(ro.pay_time::date) AS last_recharge_date,
SUM(ro.pay_amount) AS recharge_amount,
COUNT(*) FILTER (
WHERE ro.pay_time >= CURRENT_DATE - INTERVAL '60 days'
WHERE ro.pay_time >= %s::date - INTERVAL '60 days'
AND ro.pay_time < %s::date + INTERVAL '1 day'
) AS recharges_60d,
COALESCE(ca_agg.balance, 0) AS current_balance,
cs.days_since_last
@@ -2190,15 +2271,16 @@ def get_customer_board_recharge(
SELECT cs2.days_since_last
FROM app.v_dws_member_consumption_summary cs2
WHERE cs2.member_id = ro.member_id
AND cs2.stat_date <= %s
ORDER BY cs2.stat_date DESC
LIMIT 1
) cs ON true
WHERE 1=1 {proj_clause}
WHERE ro.pay_time <= %s::date + INTERVAL '1 day' {proj_clause}
GROUP BY ro.member_id, dm.nickname, ca_agg.balance, cs.days_since_last
ORDER BY MAX(ro.pay_time::date) DESC, ro.member_id
LIMIT %s OFFSET %s
""",
(*proj_params, page_size, offset),
(ref_date, ref_date, ref_date, ref_date, *proj_params, page_size, offset),
)
items = []
for row in cur.fetchall():
@@ -2228,6 +2310,13 @@ def get_customer_board_recent(
不再硬编码为 0。来源: v_dws_member_visit_detail + v_dim_member + v_dws_member_winback_index。
按 last_visit_date 降序。
"""
# CHANGE 2026-05-02 | 客户看板「最近到店」修复：
# 1) WHERE/COUNT 中的 30/60 天窗口按 business_date 计算，沙箱不读「未来」到店；
# 2) days_ago 用 business_date - last_visit_date，与窗口对齐。
from app.services.runtime_context import as_runtime_today_param
ref_date = as_runtime_today_param(site_id, conn=conn)
proj_clause, proj_params = _project_filter_clause(project, "vd.member_id")
with _fdw_context(conn, site_id) as cur:
@@ -2235,9 +2324,9 @@ def get_customer_board_recent(
f"""
SELECT COUNT(DISTINCT vd.member_id)
FROM app.v_dws_member_visit_detail vd
WHERE 1=1 {proj_clause}
WHERE vd.visit_date <= %s {proj_clause}
""",
proj_params,
(ref_date, *proj_params),
)
total = cur.fetchone()[0]
@@ -2248,11 +2337,11 @@ def get_customer_board_recent(
SELECT vd.member_id,
MAX(vd.visit_date) AS last_visit_date,
COUNT(*) AS total_visits,
COUNT(*) FILTER (WHERE vd.visit_date >= CURRENT_DATE - INTERVAL '30 days') AS visits_30d,
COUNT(*) FILTER (WHERE vd.visit_date >= CURRENT_DATE - INTERVAL '60 days') AS visits_60d,
COUNT(*) FILTER (WHERE vd.visit_date >= %s::date - INTERVAL '30 days') AS visits_30d,
COUNT(*) FILTER (WHERE vd.visit_date >= %s::date - INTERVAL '60 days') AS visits_60d,
AVG(vd.total_consume) AS avg_spend
FROM app.v_dws_member_visit_detail vd
WHERE 1=1 {proj_clause}
WHERE vd.visit_date <= %s {proj_clause}
GROUP BY vd.member_id
)
SELECT ma.member_id,
@@ -2271,14 +2360,13 @@ def get_customer_board_recent(
ORDER BY ma.last_visit_date DESC, ma.member_id
LIMIT %s OFFSET %s
""",
(*proj_params, page_size, offset),
(ref_date, ref_date, ref_date, *proj_params, page_size, offset),
)
items = []
for row in cur.fetchall():
last_visit = row[2]
# CHANGE 2026-03-29 | 补充 days_ago（距今天数）和 visits_60d
from datetime import date as _date
days_ago = (_date.today() - last_visit).days if last_visit else None
# CHANGE 2026-05-02 | days_ago 按 business_date 计算，沙箱与窗口对齐
days_ago = (ref_date - last_visit).days if last_visit else None
items.append({
"member_id": row[0],
"name": row[1] or "",
@@ -2378,6 +2466,10 @@ def get_customer_board_freq60(
按 visit_count_60d 降序。
CHANGE 2026-04-08 | Fix：同 spend60，DISTINCT ON 取最新快照。
"""
# CHANGE 2026-05-02 | freq60 全链路按 business_date 截断（stat_date <= ref_date + 8 周窗口）
from app.services.runtime_context import as_runtime_today_param
ref_date = as_runtime_today_param(site_id, conn=conn)
proj_clause, proj_params = _project_filter_clause(project, "cs.member_id")
with _fdw_context(conn, site_id) as cur:
@@ -2387,11 +2479,11 @@ def get_customer_board_freq60(
FROM (
SELECT DISTINCT ON (cs.member_id) cs.member_id
FROM app.v_dws_member_consumption_summary cs
WHERE 1=1 {proj_clause}
WHERE cs.stat_date <= %s {proj_clause}
ORDER BY cs.member_id, cs.stat_date DESC
) sub
""",
proj_params,
(ref_date, *proj_params),
)
total = cur.fetchone()[0]
@@ -2402,7 +2494,7 @@ def get_customer_board_freq60(
SELECT DISTINCT ON (cs.member_id)
cs.member_id, cs.visit_count_60d, cs.consume_amount_60d
FROM app.v_dws_member_consumption_summary cs
WHERE 1=1 {proj_clause}
WHERE cs.stat_date <= %s {proj_clause}
ORDER BY cs.member_id, cs.stat_date DESC
)
SELECT cs.member_id,
@@ -2415,7 +2507,7 @@ def get_customer_board_freq60(
ORDER BY cs.visit_count_60d DESC, cs.member_id
LIMIT %s OFFSET %s
""",
(*proj_params, page_size, offset),
(ref_date, *proj_params, page_size, offset),
)
items = []
member_ids = []
@@ -2436,21 +2528,31 @@ def get_customer_board_freq60(
# 批量查询 8 周到店数据
if member_ids:
weekly_map = _get_weekly_visits_batch(cur, member_ids)
weekly_map = _get_weekly_visits_batch(cur, member_ids, ref_date=ref_date)
for item in items:
item["weekly_visits"] = weekly_map.get(item["member_id"], _empty_weekly())
return {"items": items, "total": total, "page": page, "page_size": page_size}
def _get_weekly_visits_batch(cur: Any, member_ids: list[int]) -> dict[int, list[dict]]:
def _get_weekly_visits_batch(
cur: Any, member_ids: list[int], *, ref_date: Any = None,
) -> dict[int, list[dict]]:
"""
批量查询客户最近 8 周的到店次数（用于 freq60 维度柱状图）。
CHANGE 2026-04-07 | Fix-5：数据源从 v_dwd_assistant_service_log 改为
v_dwd_settlement_head（settle_type IN (1,3)），与汇总维度口径一致。
CHANGE 2026-05-02 | 8 周窗口锚定 ref_date（业务日），沙箱不读「未来」。
返回 {member_id: [{val: int, pct: int}, ...]}，固定 8 个元素。
"""
from datetime import date as _date, timedelta as _timedelta
if ref_date is None:
ref_date = _date.today()
elif hasattr(ref_date, "date") and not isinstance(ref_date, _date):
ref_date = ref_date.date()
cur.execute(
"""
WITH weekly AS (
@@ -2460,14 +2562,15 @@ def _get_weekly_visits_batch(cur: Any, member_ids: list[int]) -> dict[int, list[
FROM app.v_dwd_settlement_head
WHERE member_id = ANY(%s)
AND settle_type IN (1, 3)
AND pay_time >= CURRENT_DATE - INTERVAL '56 days'
AND pay_time >= %s::date - INTERVAL '56 days'
AND pay_time < %s::date + INTERVAL '1 day'
GROUP BY member_id, DATE_TRUNC('week', pay_time::date)
)
SELECT member_id, week_start, cnt
FROM weekly
ORDER BY member_id, week_start
""",
(member_ids,),
(member_ids, ref_date, ref_date),
)
from collections import defaultdict
@@ -2477,11 +2580,9 @@ def _get_weekly_visits_batch(cur: Any, member_ids: list[int]) -> dict[int, list[
week_key = row[1].date() if hasattr(row[1], 'date') else row[1]
raw[row[0]][str(week_key)] = row[2]
# 生成最近 8 周的周一日期
from datetime import date, timedelta
today = date.today()
this_monday = today - timedelta(days=today.weekday())
weeks = [this_monday - timedelta(weeks=i) for i in range(7, -1, -1)]
# 生成最近 8 周的周一日期，以业务日为锚
this_monday = ref_date - _timedelta(days=ref_date.weekday())
weeks = [this_monday - _timedelta(weeks=i) for i in range(7, -1, -1)]
result: dict[int, list[dict]] = {}
for mid in member_ids: