在前后端开发联调前的提交20260223

2026-02-23 23:02:20 +08:00
parent 254ccb1e77
commit fafc95e64c
1142 changed files with 10366960 additions and 36957 deletions
--- a/docs/audit/prompt_logs/prompt_log_20260222_223703.md
+++ b/docs/audit/prompt_logs/prompt_log_20260222_223703.md
@@ -0,0 +1,284 @@
+- [P20260222-223703] 2026-02-22 22:37:03 +0800
+  - summary: CONTEXT TRANSFER: We are continuing a conversation that had gotten too long. Here is a summary: --- ## TASK 1: Database …
+  - prompt:
+```text
+CONTEXT TRANSFER: We are continuing a conversation that had gotten too long. Here is a summary:
+
+---
+## TASK 1: Database DDL Consolidation — Per-Schema Split
+
+**STATUS**: done
+
+**USER QUERIES**: 1 ("审查任务：查库现在数据库，以数据库现状为准，对db目录下的各个sql文件整合，输出一套最全的DDL"), 2 ("再单独输出各Schema的文件，替换当前DDL")
+
+**DETAILS**:
+- Originally generated a single `docs/database/consolidated_ddl.sql` (176 KB) from test databases
+- Then split into 8 per-schema files in `docs/database/ddl/`:
+  - `etl_feiqiu__meta.sql` (3.2 KB, 3 tables)
+  - `etl_feiqiu__ods.sql` (37.8 KB, 22 tables)
+  - `etl_feiqiu__dwd.sql` (50.3 KB, 42 tables)
+  - `etl_feiqiu__core.sql` (3.3 KB, 7 tables)
+  - `etl_feiqiu__dws.sql` (65.8 KB, 32 tables)
+  - `etl_feiqiu__app.sql` (2.5 KB, views only)
+  - `zqyy_app__public.sql` (8.8 KB, 12 tables)
+  - `fdw.sql` (3.8 KB)
+- Old `consolidated_ddl.sql` was deleted by the script
+- Script uses `TEST_DB_DSN` / `TEST_APP_DB_DSN` from `.env`
+
+**FILEPATHS**:
+- `scripts/ops/gen_consolidated_ddl.py` — generation script (updated to per-schema output)
+- `docs/database/ddl/` — 8 output DDL files
+
+---
+
+## TASK 2: Archive Old DDL Baseline Files
+
+**STATUS**: done
+
+**USER QUERIES**: 3 ("把原有的，可以被这几个覆盖掉的DDL，全部归档打包吧")
+
+**DETAILS**:
+- Phase 1: Archived 12 old baseline DDL files from `db/etl_feiqiu/schemas/` and `db/zqyy_app/schemas/` to `db/_archived/ddl_baseline_2026-02-22/`
+  - 6 current baseline files: `meta.sql`, `ods.sql`, `dwd.sql`, `core.sql`, `dws.sql`, `app.sql`
+  - 5 legacy files: `schema_dwd_doc.sql`, `schema_dws.sql`, `schema_etl_admin.sql`, `schema_ODS_doc.sql`, `schema_verify_perf_indexes.sql`
+  - 1 zqyy_app file: `init.sql`
+- `.gitkeep` files preserved in emptied directories
+
+**FILEPATHS**:
+- `scripts/ops/_archive_old_ddl.py` — archive script (phase 1)
+- `db/_archived/ddl_baseline_2026-02-22/` — archive destination
+
+---
+
+## TASK 3: Archive Migration Scripts + Outdated Change Docs
+
+**STATUS**: done
+
+**USER QUERIES**: 4 ("db\etl_feiqiu\migrations这个目录下的sql文件是干啥的？不归档么"), 5 ("归档，并db docs两个路径的数据库相关的文档和sql文件进行遍历，该归档的统一归档，该增补修订的都进行处理")
+
+**DETAILS**:
+- Phase 2: Archived 29 migration SQL files + 9 migration change docs
+  - 26 migration SQL from `db/etl_feiqiu/migrations/`
+  - 2 migration SQL from `db/zqyy_app/migrations/`
+  - 1 data migration script from `db/scripts/migrate_test_data.sql`
+  - 9 migration-type BD_Manual docs from `docs/database/` → `docs/database/_archived/`
+    - `BD_Manual_dim_member_add_birthday.md`, `BD_Manual_drop_assistant_abolish_tables.md`, `BD_Manual_dws_assistant_monthly_uk_change.md`, `BD_Manual_dws_assistant_salary_uk_change.md`, `BD_Manual_fix_bc_sentinel_dates.md`, `BD_Manual_fdw_reverse_member_birthday.md`, `BD_Manual_member_birthday_manual.md`, `etl_feiqiu_schema_migration.md`, `zqyy_app_admin_web_tables.md`
+- Preserved: seeds, FDW configs, create_test_db scripts, data dictionary BD_Manuals
+- Created/updated README files:
+  - `db/README.md` — updated with new structure
+  - `db/etl_feiqiu/README.md` — updated (was previously ETL module's database/ README, now correct)
+  - `docs/database/README.md` — new index document
+
+**FILEPATHS**:
+- `scripts/ops/_archive_phase2.py` — archive script (phase 2)
+- `db/README.md`, `db/etl_feiqiu/README.md`, `docs/database/README.md` — updated READMEs
+
+---
+
+## TASK 4: Archive ETL Module's Outdated Database Docs
+
+**STATUS**: done
+
+**USER QUERIES**: 6 ("apps 文件夹中，也有数据库有关的文档和DDL，也要相同的方式，进行整理"), 7 ("这里文档有过期的，或者更新后遗漏的内容么？apps\etl\connectors\feiqiu\docs\database")
+
+**DETAILS**:
+- Searched `apps/` — no `.sql` files found, no DDL in markdown
+- Found `apps/etl/connectors/feiqiu/docs/database/` with extensive BD_manual docs
+- Archived 15 outdated files to `apps/etl/connectors/feiqiu/docs/database/_archived/`:
+  - 7 change records from `DWD/changes/`, `DWS/changes/`, `ODS/changes/`
+  - 1 outdated DDL compare report (`ddl_compare_results.md`, from 2025-07-25)
+  - 3 outdated overview dictionaries (`overview/dwd_main_tables_dictionary.md`, `dws_tables_dictionary.md`, `ods_tables_dictionary.md`) — referenced old DDL paths
+  - 3 deleted table docs (`dwd_assistant_trash_event`, `dwd_assistant_trash_event_ex`, `assistant_cancellation_records`)
+  - 1 deleted table mapping (`mapping_GetAbolitionAssistant_assistant_cancellation_records.md`)
+- Also archived `docs/database/BD_Manual_assistant_cancellation_records.md` (deleted table)
+- Updated `docs/database/README.md` to remove archived file from index
+- Updated `apps/etl/connectors/feiqiu/docs/database/README.md` with new structure
+- NOTE: `dwd_assistant_trash_event` tables still exist in test DB (migration not yet executed there), but docs archived per user's abolish cleanup spec
+
+**FILEPATHS**:
+- `scripts/ops/_archive_etl_db_docs.py` — archive script
+- `apps/etl/connectors/feiqiu/docs/database/README.md` — updated
+- `apps/etl/connectors/feiqiu/docs/database/_archived/` — archive destination
+
+---
+
+## TASK 5: Update Retained BD_Manual Docs to Current State
+
+**STATUS**: in-progress
+
+**USER QUERIES**: 8 ("保留的这些内容，也有过时的信息，帮我更新到现状版本。")
+
+**DETAILS**:
+- User wants all retained BD_Manual docs updated to reflect current database state
+- Created `scripts/ops/_fix_bd_manual_refs.py` — batch regex replacement script for outdated path references
+- Script handles:
+  - Migration script paths → marked as archived with new path
+  - Old DDL schema file references → updated to `docs/database/ddl/` paths
+  - Old `database/schema_*.sql` references → updated
+- Script was created but **NOT YET EXECUTED**
+- The script was the last file written before context limit hit
+
+**NEXT STEPS**:
+1. Run `python scripts/ops/_fix_bd_manual_refs.py` to batch-fix path references
+2. Verify the output — check a few files to confirm replacements are correct
+3. The 11 retained BD_Manual files in `docs/database/` are:
+   - `BD_Manual_assistant_accounts_master.md`
+   - `BD_Manual_assistant_service_records.md`
+   - `BD_Manual_dws_goods_stock_summary.md`
+   - `BD_Manual_goods_stock_movements.md`
+   - `BD_Manual_goods_stock_summary.md`
+   - `BD_Manual_member_balance_changes.md`
+   - `BD_Manual_recharge_settlements.md`
+   - `BD_Manual_site_tables_master.md`
+   - `BD_Manual_store_goods_master.md`
+   - `BD_Manual_store_goods_sales_records.md`
+   - `BD_Manual_tenant_goods_master.md`
+4. Also need to check/fix docs in `apps/etl/connectors/feiqiu/docs/database/` (ODS/main/, DWD/main/, DWD/Ex/, DWS/main/, ETL_Admin/main/, ODS/mappings/) — the script already covers these directories
+5. Beyond path fixes, may need to verify field-level accuracy against database (DB query showed column counts; e.g., `dim_assistant_ex` has 52 cols in DB vs ~45 documented)
+
+**FILEPATHS**:
+- `scripts/ops/_fix_bd_manual_refs.py` — **created but not yet run**
+- `docs/database/BD_Manual_*.md` — 11 files to update
+- `apps/etl/connectors/feiqiu/docs/database/` — ~80+ BD_manual files to update
+
+---
+
+## USER CORRECTIONS AND INSTRUCTIONS:
+- 以数据库现状为准 (database reality takes precedence over SQL files)
+- Language: Chinese for documentation/comments, English for code identifiers
+- `docs/database/` files are project documentation (DDL baseline), NOT runtime output — lives in `docs/` not `export/`
+- `export-paths.md` applies to runtime outputs only
+- Database connections must use test databases via `TEST_DB_DSN` / `TEST_APP_DB_DSN`
+- Prefer Python scripts over complex PowerShell (per tech.md)
+- Scripts go in `scripts/ops/` (one-off ops scripts)
+- MCP tools: `mcp_pg_etl_test_execute_sql` (for test_etl_feiqiu) and `mcp_pg_app_test_execute_sql` (for test_zqyy_app)
+- Audit reminder is pending — user should trigger `/audit` when ready
+- `dwd_assistant_trash_event` tables still exist in test DB despite abolish migration being written
+
+## Files to read:
+- `scripts/ops/_fix_bd_manual_refs.py`
+- `docs/database/README.md`
+- `docs/database/BD_Manual_tenant_goods_master.md`
+- `docs/database/BD_Manual_assistant_accounts_master.md`
+- `apps/etl/connectors/feiqiu/docs/database/README.md`
+- `scripts/ops/gen_consolidated_ddl.py`
+- `db/README.md`
+
+USER QUERIES(most recent first):
+1. 保留的这些内容，也有过时的信息，帮我更新到现状版本。
+2. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+3. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+4. 保留的这些内容，也有过时的信息，帮我更新到现状版本。
+5. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+6. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+7. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+8. 这里文档有过期的，或者更新后遗漏的内容么？apps\etl\connectors\feiqiu\docs\database
+9. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+10. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+11. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+12. apps 文件夹中，也有数据库有关的文档和DDL，也要相同的方式，进行整理。
+13. Output:
+Hook execution failed with exit code 1.
+
+Error output:
+[AUDIT REMINDER] Pending audit detected (root-file, dir:admin-web, dir:backend, dir:db, db-schema-change). Run /audit (Manual: Run /audit hook) to sync docs & write audit artifacts. (rate limit: 15min)
+
+
+
+Exit Code: 1
+14. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+15. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+16. 归档，并db docs两个路径的数据库相关的文档和sql文件进行遍历，该归档的统一归档，该增补修订的都进行处理。保持数据库方面的信息规整，全面，可控，方便维护。
+17. Output:
+Hook execution failed with exit code 1.
+
+Error output:
+[AUDIT REMINDER] Pending audit detected (root-file, dir:admin-web, dir:backend, dir:db, db-schema-change). Run /audit (Manual: Run /audit hook) to sync docs & write audit artifacts. (rate limit: 15min)
+
+
+
+Exit Code: 1
+18. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+19. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+20. db\etl_feiqiu\migrations这个目录下的sql文件是干啥的？不归档么
+21. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+22. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+23. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+24. 把原有的，可以被这几个覆盖掉的DDL，全部归档打包吧
+25. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+26. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+27. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+28. 好的，再单独输出各Schema的文件，替换当前DDL
+29. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+30. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+31. Output:
+Command executed successfully with no output.
+
+Exit Code: 0
+---
+
+METADATA:
+The previous conversation had 16 messages.
+
+INSTRUCTIONS:
+Continue working until the user query has been fully addressed. Do not ask for clarification - proceed with the work based on the context provided.