#322 P2 方案A 全线: 10 张 public 干净视图 + analytics 全改读, C2 基线清零
Some checks failed
backend-test / commit-message-lint (push) Has been cancelled
backend-test / analytics-ods-isolation (push) Has been cancelled
backend-test / backend-lint-test (push) Has been cancelled
backend-test / frontend-lint-test (push) Has been cancelled
gitleaks / gitleaks (push) Has been cancelled

- 新增迁移 c3d4e5f6a7b8: 批量建 bms_order_clean/device_combo_clean/card_complaint_clean/
  external_complaint_clean/cs_record_clean/wallet_withdrawal_clean/device_flow_usage_clean/
  manual_refund_clean/refund_record_report_clean/shanmao_refund_clean 共 10 张薄视图,
  逐列取自 ods 源表(口径逐字不变, 不做任何加工), 仅排除内部溯源列 synced_at/raw_extra/updated_by_raw
- 新增 app/models/ods_clean.py: CLEAN_VIEWS 作为单一列清单来源 + 11 个只读模型
  (RefundClean 从 refund.py 迁入); 迁移与 conftest 视图重建共用同一来源, 消除双份定义漂移
- analytics 14 文件: ORM 模型类与裸 SQL 表名全部由 ods 改读 *_clean 视图
- 手动退款质检写路径移出 analytics → app/services/quality/manual_refund_review.py
  (读走 ManualRefundClean; 确认/修正写走 ods 基表 ManualRefund, 视图不可写);
  analytics 门面继续导出原 4 个函数, 路由零改动
- tests/conftest.py: refund_clean 专用拆表重建 → 按 CLEAN_VIEWS 通用循环
- C2 棘轮基线 64 → 0(scripts/analytics_ods_isolation_baseline.txt 仅剩表头)
This commit is contained in:
2026-10-14 17:30:50 +08:00
parent cc37a68c37
commit 89d1057b1b
23 changed files with 1016 additions and 652 deletions

View File

@@ -0,0 +1,49 @@
"""#322 方案A: 批量新建 public 干净层只读视图(10 张, over ods 源表)
背景(#319/#322 数据分层):
- #319 已把 21 张信源表移入 ods schema; C2 契约要求 services/analytics/** 不得直读 ods 源表。
- 本迁移为剩余各域落地「方案A 薄视图」: 一表一视图, **逐列取自 ods 源表, 不做任何加工**
(不改变任何口径/过滤/时间换算), 仅把 analytics 的「读法」从 ods 换成 public 视图。
- refund 域的 public.refund_clean 已由上一迁移 a9b0c1d2e3f4 建好, **本迁移不重建**。
单一列清单来源: app.models.ods_clean.CLEAN_VIEWS(键=视图名, 值=(源表全限定名, 列清单))。
列清单由对应只读模型的 __table__.columns 生成, 与 conftest 视图重建共用同一来源, 保证不漂移。
暴露列口径: 除内部溯源列 synced_at / raw_extra / updated_by_raw 外全部保留;
quality_status / quality_note 必须保留(analytics 质检清单/统计排除会读)。
写入不受影响: upsert 入库 / actual_refund_time 回填等仍直写 ods 源表, 视图仅给 analytics 读。
Revision ID: c3d4e5f6a7b8
Revises: a9b0c1d2e3f4
Create Date: 2026-10-14 01:00:00.000000
"""
from collections.abc import Sequence
from alembic import op
from app.models.ods_clean import CLEAN_VIEWS
# revision identifiers, used by Alembic.
revision: str = "c3d4e5f6a7b8"
down_revision: str = "a9b0c1d2e3f4"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
# refund_clean 已由 a9b0c1d2e3f4 建好, 本迁移只建其余 10 张
_NEW_VIEWS = [v for v in CLEAN_VIEWS if v != "refund_clean"]
def upgrade() -> None:
for view, (src_table, cols) in CLEAN_VIEWS.items():
if view == "refund_clean":
continue # 已存在, 不重建
op.execute(
f"CREATE VIEW public.{view} AS SELECT {cols} FROM {src_table}"
)
def downgrade() -> None:
for view in _NEW_VIEWS:
op.execute(f"DROP VIEW IF EXISTS public.{view}")

View File

@@ -86,6 +86,21 @@ from app.models.metric import (
) # noqa: F401 ) # noqa: F401
from app.models.notification import Notification # noqa: F401 from app.models.notification import Notification # noqa: F401
from app.models.notification_channel import NotificationChannel # noqa: F401 from app.models.notification_channel import NotificationChannel # noqa: F401
# #322 方案A: public 干净层视图的只读模型集中登记(analytics 改读视图, 不直读 ods)
from app.models.ods_clean import ( # noqa: F401
BmsOrderClean,
CardComplaintClean,
CsRecordClean,
DeviceComboClean,
DeviceFlowUsageClean,
ExternalComplaintClean,
ManualRefundClean,
RefundClean,
RefundRecordReportClean,
ShanmaoRefundClean,
WalletWithdrawalClean,
)
from app.models.onboarding_prompt import OnboardingPrompt # noqa: F401 from app.models.onboarding_prompt import OnboardingPrompt # noqa: F401
from app.models.operation_log import OperationLog # noqa: F401 from app.models.operation_log import OperationLog # noqa: F401
from app.models.person import Person, PersonAlias, PersonSkillTag # noqa: F401 from app.models.person import Person, PersonAlias, PersonSkillTag # noqa: F401
@@ -94,7 +109,7 @@ from app.models.prompt_config import PromptConfig # noqa: F401
from app.models.quality_disposition_rule import QualityDispositionRule # noqa: F401 from app.models.quality_disposition_rule import QualityDispositionRule # noqa: F401
from app.models.reconcile_check import ReconcileCheck # noqa: F401 from app.models.reconcile_check import ReconcileCheck # noqa: F401
from app.models.reconcile_match import ReconcileMatch # noqa: F401 from app.models.reconcile_match import ReconcileMatch # noqa: F401
from app.models.refund import Refund, RefundClean # noqa: F401 from app.models.refund import Refund # noqa: F401
from app.models.refund_record_report import RefundRecordReport # noqa: F401 from app.models.refund_record_report import RefundRecordReport # noqa: F401
from app.models.refund_wide import RefundWide # noqa: F401 from app.models.refund_wide import RefundWide # noqa: F401
from app.models.report import Report, ReportStatus, ReportType # noqa: F401 from app.models.report import Report, ReportStatus, ReportType # noqa: F401
@@ -220,6 +235,16 @@ __all__ = [
"ReconcileMatch", "ReconcileMatch",
"Refund", "Refund",
"RefundClean", "RefundClean",
"BmsOrderClean",
"DeviceComboClean",
"CardComplaintClean",
"ExternalComplaintClean",
"CsRecordClean",
"WalletWithdrawalClean",
"DeviceFlowUsageClean",
"ManualRefundClean",
"RefundRecordReportClean",
"ShanmaoRefundClean",
"RefundWide", "RefundWide",
"Report", "Report",
"ReportStatus", "ReportStatus",

View File

@@ -0,0 +1,379 @@
"""public 干净层视图的**只读** ORM 映射 (#322 方案A, 仅供 analytics 读)。
背景(#319/#322 数据分层):
- #319 把 21 张信源表移入 `ods` schema 后, C2 契约要求 `services/analytics/**`
**不得直读 ods 源表**, 须改读 public 干净层(视图/宽表/字典)。
- 本模块集中登记 analytics 需要的「方案A 薄视图」: 一表一视图, **逐列取自 ods 源表,
不做任何加工**(不改变任何口径/过滤/时间换算), 仅把「读法」换成 public 视图。
单一列清单来源: `CLEAN_VIEWS` —— 键=视图名, 值=(源表全限定名, 列清单)。
列清单由**对应 Clean 模型的 `__table__.columns`** 生成(见 `_cols`), 保证不漏列,
且与迁移 / conftest 视图重建逐字一致(改模型即自动同步)。
暴露列口径: 除「内部溯源列」`synced_at` / `raw_extra` / `updated_by_raw` 外全部保留;
`quality_status` / `quality_note` **必须保留**(analytics 质检清单/统计排除会读)。
⚠️ 只读: 视图不可写。analytics 侧的任何**写入**(如手动退款质检回写)必须落在
ods 源表(经对应写入模型, 见 `app.services.quality.manual_refund_review`),
切勿在这些 Clean 模型上 INSERT/UPDATE/DELETE。
⚠️ 不继承 QualityMixin / TimestampMixin: 避免引入未暴露字段; created_at / updated_at
按视图列显式声明为普通列。不声明 ForeignKey / relationship(视图无约束)。
⚠️ `__table_args__` 一律留空 → 落 public schema(视图位于 public.<view>)。
"""
from __future__ import annotations
from datetime import datetime
from decimal import Decimal
from sqlalchemy import BigInteger, Boolean, DateTime, Integer, Numeric, String, Text
from sqlalchemy.orm import Mapped, mapped_column
from app.core.database import Base
class RefundClean(Base):
"""`public.refund_clean` 视图 (#322 方案A, over ods.refunds)。
只暴露 analytics 实际消费的 21 列(其余 5 列 package_price / raw_extra /
synced_at / updated_by_raw / quality_note 不对外), 口径与 refunds 逐字一致。
"""
__tablename__ = "refund_clean"
# 无 __table_args__: 默认 public schema
refund_no: Mapped[str] = mapped_column(String(64), primary_key=True, comment="退费单号 RF...")
platform_order_no: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="平台订单号 SP/WS/WX...")
third_order_no: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="三方订单号")
device_sn: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="设备SN")
refund_reason: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="退费原因")
package_name: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="套餐名称")
merchant_name: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="商户名称")
refund_amount: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="退费金额")
actual_amount: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="实退金额")
remark: Mapped[str | None] = mapped_column(Text, nullable=True, comment="备注")
status: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="状态:申请中/已退款/无需退款")
refund_method: Mapped[str | None] = mapped_column(
String(255), nullable=True, comment="退款方式:线上/线下(导出数据可能含描述性文本)"
)
audit_remark: Mapped[str | None] = mapped_column(Text, nullable=True, comment="审核备注")
auditor: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="审核人")
operator: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="操作人:企微-XXX/sys/system")
created_at_raw: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="原始创建时间")
updated_at_raw: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="原始更新时间")
actual_refund_time: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True, comment="实际扣款时间(E路refund_time回填, 对账用)"
)
quality_status: Mapped[str | None] = mapped_column(
String(16), nullable=True,
comment="数据质量状态: None=正常 / ok / auto_fixed / pending / dup_residual(排除) / ignored(排除)",
)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
class BmsOrderClean(Base):
"""`public.bms_order_clean` 视图 (#322 方案A, over ods.bms_orders)。
排除内部溯源列 synced_at / raw_extra(其余业务列全保留)。
"""
__tablename__ = "bms_order_clean"
order_no: Mapped[str] = mapped_column(String(64), primary_key=True, comment="订单号")
order_type: Mapped[str | None] = mapped_column(String(32), nullable=True, comment="订单类型(预存/储值卡充值等)")
device_sn: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="设备SN")
combo_name: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="套餐名称")
combo_price: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="套餐价格(元)")
payable_price: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="应付价格(元)")
actual_pay_price: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="实际支付价格(元)")
balance_pay_price: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="余额支付(元)")
wx_order_no: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="微信订单号")
pay_date: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="支付时间")
pay_type: Mapped[str | None] = mapped_column(String(32), nullable=True, comment="支付类型(1微信支付等)")
order_status: Mapped[str | None] = mapped_column(String(32), nullable=True, comment="订单状态(0未支付/1已支付)")
order_result: Mapped[str | None] = mapped_column(Text, nullable=True, comment="订单结果/下单返回")
order_status_raw: Mapped[int | None] = mapped_column(Integer, nullable=True, comment="订单状态原始码")
create_user: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="创建人(WeChat/Wallet/SYS)")
create_date: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="创建时间(同步锚点)")
business_name: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="商户名称")
business_id: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="内部商户ID")
device_type_name: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="设备类型")
refund_status: Mapped[str | None] = mapped_column(String(32), nullable=True, comment="退款状态")
active_date: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="激活时间")
last_stored_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="最后储值时间")
mch_id: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="微信商户Id")
merchant_name: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="微信商户名称")
before_serve_valid_date: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="续费前服务有效期")
after_serve_valid_date: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="续费后服务有效期")
days: Mapped[int | None] = mapped_column(Integer, nullable=True, comment="有效期天数")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
class DeviceComboClean(Base):
"""`public.device_combo_clean` 视图 (#322 方案A, over ods.device_combos)。"""
__tablename__ = "device_combo_clean"
recharge_order_no: Mapped[str] = mapped_column(String(64), primary_key=True, comment="充值订单号")
device_sn: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="设备SN")
package_name: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="套餐名称")
package_type: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="套餐类型:续费套餐")
traffic_type: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="流量类型:无限/固定流量")
merchant_name: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="商户名称")
actual_amount: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="实际支付金额")
start_date: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="开始日期")
end_date: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="结束日期")
updated_at_raw: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="原始更新时间")
created_at_raw: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="原始创建时间")
valid_days: Mapped[int | None] = mapped_column(Integer, nullable=True, comment="套餐有效天数")
creator: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="创建人:WeChat/SYS/Wallet")
available_traffic: Mapped[int | None] = mapped_column(BigInteger, nullable=True, comment="可用流量(字节,-1=无限)")
used_traffic: Mapped[int | None] = mapped_column(BigInteger, nullable=True, comment="已用流量(字节)")
remaining_traffic: Mapped[int | None] = mapped_column(BigInteger, nullable=True, comment="剩余流量(字节)")
status: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="当前状态:流量正常")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
class CardComplaintClean(Base):
"""`public.card_complaint_clean` 视图 (#322 方案A, over ods.card_complaints)。"""
__tablename__ = "card_complaint_clean"
id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True, comment="自增主键")
complaint_date: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="投诉日期(主时间字段)")
complaint_group: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="投诉群")
complainant: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="投诉人")
complainant_phone: Mapped[str | None] = mapped_column(String(32), nullable=True, comment="投诉手机号")
device_sn: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="设备号 SN")
problem: Mapped[str | None] = mapped_column(Text, nullable=True, comment="投诉问题")
problem_category: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="投诉问题分类(二级)")
result: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="结果")
solution: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="解决方案")
handler: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="处理人(人员别名来源)")
handle_result: Mapped[str | None] = mapped_column(Text, nullable=True, comment="处理结果")
matched_ticket_no: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="关联服务工单号")
matched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="关联工单时间")
dedup_hash: Mapped[str | None] = mapped_column(String(32), nullable=True, comment="业务去重指纹(md5)")
quality_status: Mapped[str | None] = mapped_column(String(16), nullable=True, comment="数据质量状态")
quality_note: Mapped[str | None] = mapped_column(String(512), nullable=True, comment="质量说明")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
class ExternalComplaintClean(Base):
"""`public.external_complaint_clean` 视图 (#322 方案A, over ods.external_complaints)。"""
__tablename__ = "external_complaint_clean"
id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True, comment="自增主键")
complaint_no: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="投诉编号")
complaint_date: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="投诉日期(主时间字段)")
platform: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="平台")
complaint_account: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="投诉账号")
company_entity: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="公司主体")
complaint_shop: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="投诉店铺")
purchase_order_no: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="购买订单编号")
complainant: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="投诉人")
complainant_phone: Mapped[str | None] = mapped_column(String(32), nullable=True, comment="投诉手机号")
device_sn: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="投诉设备号 SN")
wechat_id: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="微信账户id")
problem: Mapped[str | None] = mapped_column(Text, nullable=True, comment="投诉问题")
keyword: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="投诉关键词")
complaint_category: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="投诉分类")
solution: Mapped[str | None] = mapped_column(String(512), nullable=True, comment="解决方案")
result: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="结果")
follower: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="跟进人")
handle_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="处理时间")
handle_detail: Mapped[str | None] = mapped_column(Text, nullable=True, comment="处理明细")
matched_ticket_no: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="关联服务工单号")
matched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="关联工单时间")
dedup_hash: Mapped[str | None] = mapped_column(String(32), nullable=True, comment="业务去重指纹(md5)")
quality_status: Mapped[str | None] = mapped_column(String(16), nullable=True, comment="数据质量状态")
quality_note: Mapped[str | None] = mapped_column(String(512), nullable=True, comment="质量说明")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
class CsRecordClean(Base):
"""`public.cs_record_clean` 视图 (#322 方案A, over ods.cs_records)。"""
__tablename__ = "cs_record_clean"
id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True, comment="自增主键")
problem_category: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="问题大类")
problem_type: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="问题类型")
device_sn: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="设备号 SN")
solution: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="方案")
operation_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="操作时间(主时间字段)")
operator: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="操作人:企微-XXX")
send_content: Mapped[str | None] = mapped_column(Text, nullable=True, comment="发送内容")
description: Mapped[str | None] = mapped_column(Text, nullable=True, comment="说明")
execution_result: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="执行结果说明")
user_accepted: Mapped[str | None] = mapped_column(String(32), nullable=True, comment="用户是否接受")
consult_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="咨询时间")
problem_create_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="问题类型创建时间")
last_consult_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="最后咨询时间")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
class WalletWithdrawalClean(Base):
"""`public.wallet_withdrawal_clean` 视图 (#322 方案A, over ods.wallet_withdrawals)。"""
__tablename__ = "wallet_withdrawal_clean"
id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True, comment="自增主键")
account_id: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="账户ID")
wallet_balance: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="钱包余额")
device_sn: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="SN")
op_type: Mapped[str | None] = mapped_column(String(32), nullable=True, comment="操作类型:用户提现/订单退费转入")
op_amount: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="操作金额")
remark: Mapped[str | None] = mapped_column(Text, nullable=True, comment="备注(含退费单号 RF...)")
op_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="操作时间")
operator: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="操作人员:SYS/用户提现")
actual_refund_time: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True, comment="实际扣款时间(E路refund_time回填, 对账用)"
)
quality_status: Mapped[str | None] = mapped_column(String(16), nullable=True, comment="数据质量状态")
quality_note: Mapped[str | None] = mapped_column(String(512), nullable=True, comment="质量说明")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
class DeviceFlowUsageClean(Base):
"""`public.device_flow_usage_clean` 视图 (#322 方案A, over ods.device_flow_usage)。"""
__tablename__ = "device_flow_usage_clean"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True, comment="自增主键")
device_sn: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="设备SN")
merchant_id: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="商户ID(businessId)")
merchant_name: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="商户名称")
device_type_name: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="设备类型")
device_group_name: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="设备组")
device_combo_name: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="当前套餐")
stat_month: Mapped[str | None] = mapped_column(String(16), nullable=True, comment="统计月份 YYYY-MM")
flow_gb: Mapped[float | None] = mapped_column(Numeric(12, 4), nullable=True, comment="该月流量(GB)")
month_complete: Mapped[bool | None] = mapped_column(Boolean, nullable=True, comment="是否完整月")
fetched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="本次抓取时间")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
class ManualRefundClean(Base):
"""`public.manual_refund_clean` 视图 (#322 方案A, over ods.manual_refunds)。
⚠️ 只读: 手动退款**质检回写**(quality_status/quality_note 及字段修正)不可写视图,
必须经 ods 源表(见 `app.services.quality.manual_refund_review`)。
"""
__tablename__ = "manual_refund_clean"
id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True, comment="自增主键")
register_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="登记日期")
device_sn: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="设备号")
order_no: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="订单号")
merchant_id: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="商户ID(微信商户号)")
order_amount: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="订单金额")
registrant: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="登记人")
is_refunded: Mapped[str | None] = mapped_column(String(8), nullable=True, comment="是否已退款(是/否)")
refund_amount: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="退款金额")
refund_operator: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="退款操作人")
op_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="操作时间")
remark: Mapped[str | None] = mapped_column(Text, nullable=True, comment="备注")
actual_refund_time: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True, comment="实际扣款时间(E路refund_time回填, 对账用)"
)
quality_status: Mapped[str | None] = mapped_column(String(16), nullable=True, comment="数据质量状态")
quality_note: Mapped[str | None] = mapped_column(String(512), nullable=True, comment="质量说明")
register_time_raw: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="登记日期原始值")
backfill_refund_record_report_id: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="op_time 回填来源 refund_record_reports.id")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
class RefundRecordReportClean(Base):
"""`public.refund_record_report_clean` 视图 (#322 方案A, over ods.refund_record_reports)。
排除内部溯源列 raw_extra / synced_at。
"""
__tablename__ = "refund_record_report_clean"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True, comment="自增主键")
platform_order_no: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="平台订单号")
wx_order_no: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="微信订单号")
merchant_id: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="商户号(对账锚点)")
merchant_name: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="商户名")
device_sn: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="设备号(对账锚点)")
product_name: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="商品名称")
refund_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="退款时间")
apply_refund_amount: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="申请退款金额(元)")
refund_amount: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="退款金额(元)")
refund_status: Mapped[str | None] = mapped_column(String(32), nullable=True, comment="退款状态")
backfill_op_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="回填到 D 路 op_time 的时间")
backfill_manual_refund_id: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="已回填的 manual_refunds.id")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
class ShanmaoRefundClean(Base):
"""`public.shanmao_refund_clean` 视图 (#322 方案A, over ods.shanmao_refunds)。
排除内部溯源列 raw_extra。
"""
__tablename__ = "shanmao_refund_clean"
refund_no: Mapped[str] = mapped_column(String(64), primary_key=True, comment="闪猫退款单号(TF开头)")
order_no: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="关联订单号")
device_sn: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="设备SN")
phone: Mapped[str | None] = mapped_column(String(32), nullable=True, comment="手机号码")
paid_amount: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="实际支付金额")
refund_amount: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="申请退款金额")
actual_amount: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="实际退款金额")
package_name: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="套餐/商品名称")
merchant_name: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="商户名称")
refund_reason: Mapped[str | None] = mapped_column(String(255), nullable=True, comment="退款原因")
refund_method: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="退款方式")
status: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="状态:已退款/审核中/已驳回等")
applicant: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="申请人")
auditor: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="审核人")
operator: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="操作人(打款人)")
apply_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="申请时间")
audit_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="审核时间")
refund_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="实际退款完成时间")
remark: Mapped[str | None] = mapped_column(Text, nullable=True, comment="备注")
audit_remark: Mapped[str | None] = mapped_column(Text, nullable=True, comment="审核备注")
quality_status: Mapped[str | None] = mapped_column(String(16), nullable=True, comment="数据质量状态")
quality_note: Mapped[str | None] = mapped_column(String(512), nullable=True, comment="质量说明")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
def _cols(model: type[Base]) -> str:
"""由只读模型 `__table__.columns` 生成列名单(逗号分隔, 保证不漏列)。
视图 DDL 与 conftest 视图重建共用此清单, 与迁移逐字一致。
"""
return ", ".join(c.name for c in model.__table__.columns)
# 视图名 → (源表全限定名, 列清单)。refund_clean 亦纳入, 便于 conftest 统一处理
# (其迁移/模型本次不改, 列清单由 RefundClean 模型自动得出, 与旧迁移逐字一致)。
CLEAN_VIEWS: dict[str, tuple[str, str]] = {
"refund_clean": ("ods.refunds", _cols(RefundClean)),
"bms_order_clean": ("ods.bms_orders", _cols(BmsOrderClean)),
"device_combo_clean": ("ods.device_combos", _cols(DeviceComboClean)),
"card_complaint_clean": ("ods.card_complaints", _cols(CardComplaintClean)),
"external_complaint_clean": ("ods.external_complaints", _cols(ExternalComplaintClean)),
"cs_record_clean": ("ods.cs_records", _cols(CsRecordClean)),
"wallet_withdrawal_clean": ("ods.wallet_withdrawals", _cols(WalletWithdrawalClean)),
"device_flow_usage_clean": ("ods.device_flow_usage", _cols(DeviceFlowUsageClean)),
"manual_refund_clean": ("ods.manual_refunds", _cols(ManualRefundClean)),
"refund_record_report_clean": ("ods.refund_record_reports", _cols(RefundRecordReportClean)),
"shanmao_refund_clean": ("ods.shanmao_refunds", _cols(ShanmaoRefundClean)),
}

View File

@@ -64,49 +64,6 @@ class Refund(QualityMixin, TimestampMixin, Base):
# 覆盖 TimestampMixin 的 created_at/updated_at 含义:这里指系统导入时间 # 覆盖 TimestampMixin 的 created_at/updated_at 含义:这里指系统导入时间
# created_at/updated_at 来自 TimestampMixin(系统导入时间) # created_at/updated_at 来自 TimestampMixin(系统导入时间)
# 注意: `public.refund_clean` 的**只读**映射 `RefundClean` 已迁至
class RefundClean(Base): # `app/models/ods_clean.py`(#322 方案A: 全部干净层视图模型集中一处, 便于
"""`public.refund_clean` 视图的**只读**映射 (#322 方案A, 仅供 analytics 读)。 # 迁移/conftest 复用同一列清单 `CLEAN_VIEWS`)。本文件仅保留可写的 ods `Refund`。
背景: #319 把信源表 refunds 移入 ods schema 后, C2 契约要求
`services/analytics/**` 不得直读 ods 源表。本视图 over `ods.refunds`,
只暴露 analytics 实际消费的 21 列(其余 5 列 package_price / raw_extra /
synced_at / updated_by_raw / quality_note 不对外), 口径与 refunds 逐字一致。
⚠️ 只读: 写入(upsert 入库 / actual_refund_time 回填)仍走 `Refund` → `ods.refunds`,
切勿在本模型上做 INSERT/UPDATE/DELETE(视图不可写)。
⚠️ 不继承 QualityMixin / TimestampMixin: 视图仅暴露 quality_status(不含
quality_note), created_at / updated_at 亦显式声明为普通列, 避免引入未暴露字段。
"""
__tablename__ = "refund_clean"
# 无 __table_args__: 默认 public schema(视图位于 public.refund_clean)
refund_no: Mapped[str] = mapped_column(String(64), primary_key=True, comment="退费单号 RF...")
platform_order_no: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="平台订单号 SP/WS/WX...")
third_order_no: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="三方订单号")
device_sn: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="设备SN")
refund_reason: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="退费原因")
package_name: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="套餐名称")
merchant_name: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="商户名称")
refund_amount: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="退费金额")
actual_amount: Mapped[Decimal | None] = mapped_column(Numeric(12, 2), nullable=True, comment="实退金额")
remark: Mapped[str | None] = mapped_column(Text, nullable=True, comment="备注")
status: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="状态:申请中/已退款/无需退款")
refund_method: Mapped[str | None] = mapped_column(
String(255), nullable=True, comment="退款方式:线上/线下(导出数据可能含描述性文本)"
)
audit_remark: Mapped[str | None] = mapped_column(Text, nullable=True, comment="审核备注")
auditor: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="审核人")
operator: Mapped[str | None] = mapped_column(String(64), nullable=True, comment="操作人:企微-XXX/sys/system")
created_at_raw: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="原始创建时间")
updated_at_raw: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, comment="原始更新时间")
actual_refund_time: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True, comment="实际扣款时间(E路refund_time回填, 对账用)"
)
quality_status: Mapped[str | None] = mapped_column(
String(16), nullable=True,
comment="数据质量状态: None=正常 / ok / auto_fixed / pending / dup_residual(排除) / ignored(排除)",
)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, comment="系统导入时间")

View File

@@ -150,11 +150,7 @@ from app.services.analytics.refund import (
_RR_CS_FAIL_KW, _RR_CS_FAIL_KW,
_RR_SYS_KW, _RR_SYS_KW,
_empty_kpi, _empty_kpi,
_mr_quality_out,
_refund_recovery_core, _refund_recovery_core,
manual_refund_quality_confirm,
manual_refund_quality_fix,
manual_refund_quality_list,
refund_decision_overview, refund_decision_overview,
refund_device_analysis, refund_device_analysis,
refund_device_detail, refund_device_detail,
@@ -213,6 +209,12 @@ from app.services.analytics.today import (
today_refund_breakdown, today_refund_breakdown,
today_staff_efficiency, today_staff_efficiency,
) )
from app.services.quality.manual_refund_review import (
_mr_quality_out,
manual_refund_quality_confirm,
manual_refund_quality_fix,
manual_refund_quality_list,
)
__all__ = [ __all__ = [
'COMPLAINT_SPECIALIST_POSITION', 'COMPLAINT_SPECIALIST_POSITION',

View File

@@ -19,13 +19,11 @@ from sqlalchemy import and_, case, exists, func, literal, or_, select, text
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from app.models.group import PersonGroupMembership from app.models.group import PersonGroupMembership
from app.models.ods_clean import RefundClean, RefundRecordReportClean, WalletWithdrawalClean
from app.models.person import Person, PersonAlias from app.models.person import Person, PersonAlias
from app.models.refund import RefundClean
from app.models.refund_record_report import RefundRecordReport
from app.models.specialty import PersonSpecialtyAssignment, Specialty from app.models.specialty import PersonSpecialtyAssignment, Specialty
from app.models.staff_assignment import StaffAssignment from app.models.staff_assignment import StaffAssignment
from app.models.ticket import ServiceTicket from app.models.ticket import ServiceTicket
from app.models.wallet import WalletWithdrawal
from app.services.complaint_scope import ( from app.services.complaint_scope import (
BUCKET_OTHER, # noqa: F401 BUCKET_OTHER, # noqa: F401
complaint_bucket_case, complaint_bucket_case,
@@ -38,7 +36,7 @@ from app.services.quality_check import stats_quality_include
# ═══════════════════════════════════════════════════════════════════ # ═══════════════════════════════════════════════════════════════════
# 2026-08-14: 投诉登记节点下的外部投诉/卡商群投诉两个文件曾"暂不纳入" # 2026-08-14: 投诉登记节点下的外部投诉/卡商群投诉两个文件曾"暂不纳入"
# 系统汇总分析。设置为 False 后, 部分 analytics 聚合将跳过 # 系统汇总分析。设置为 False 后, 部分 analytics 聚合将跳过
# ExternalComplaint / CardComplaint 两张表, 仅使用 service_tickets # 外部投诉 / 卡商群投诉两封信源表, 仅使用 service_tickets
# 中的投诉分类作为统计口径。 # 中的投诉分类作为统计口径。
# #
# ⚠️⚠️ 2026-09-10 (#47) 改为 True —— 用户拍板 ⚠️⚠️ # ⚠️⚠️ 2026-09-10 (#47) 改为 True —— 用户拍板 ⚠️⚠️
@@ -323,12 +321,12 @@ def _route_b_withdraw_filter():
避免被冻结/拒绝的提现被误计入成功退款统计。 避免被冻结/拒绝的提现被误计入成功退款统计。
备注为 NULL 视为非失败(COALESCE 保护), 正常计入。 备注为 NULL 视为非失败(COALESCE 保护), 正常计入。
""" """
conds = [WalletWithdrawal.op_type == "用户提现"] conds = [WalletWithdrawalClean.op_type == "用户提现"]
for _k in _WITHDRAW_FAIL_REMARK_KEYWORDS: for _k in _WITHDRAW_FAIL_REMARK_KEYWORDS:
conds.append(func.coalesce(WalletWithdrawal.remark, "").notlike(f"%{_k}%")) conds.append(func.coalesce(WalletWithdrawalClean.remark, "").notlike(f"%{_k}%"))
# #201 S-P0-1:排除 dup_residual/ignored(wallet 在 QUALITY_TABLES 内可被打标, # #201 S-P0-1:排除 dup_residual/ignored(wallet 在 QUALITY_TABLES 内可被打标,
# 与 cash_flow_wide B 路口径一致) # 与 cash_flow_wide B 路口径一致)
conds.append(stats_quality_include(WalletWithdrawal.quality_status)) conds.append(stats_quality_include(WalletWithdrawalClean.quality_status))
return tuple(conds) return tuple(conds)
@@ -343,7 +341,7 @@ _E_ROUTE_SUCCESS_STATUSES = ("退款成功", "成功", "SUCCESS")
def _route_e_cashflow_filter(): def _route_e_cashflow_filter():
"""【实际现金流 E】refund_record_reports: refund_status 为退款成功 全部""" """【实际现金流 E】refund_record_reports: refund_status 为退款成功 全部"""
return (RefundRecordReport.refund_status.in_(_E_ROUTE_SUCCESS_STATUSES),) return (RefundRecordReportClean.refund_status.in_(_E_ROUTE_SUCCESS_STATUSES),)
@@ -1148,8 +1146,8 @@ def _device_dim_matched_sns(
激活时间改查 bms_orders MIN(active_date) 或 devices.active_time 兜底(并集)。 激活时间改查 bms_orders MIN(active_date) 或 devices.active_time 兜底(并集)。
不同条件查不同源, 各自取 SN 集合后取交集(AND 语义)。 不同条件查不同源, 各自取 SN 集合后取交集(AND 语义)。
""" """
from app.models.bms_order import BmsOrder
from app.models.device import Device from app.models.device import Device
from app.models.ods_clean import BmsOrderClean
sn_sets: list[set[str]] = [] sn_sets: list[set[str]] = []
@@ -1164,10 +1162,10 @@ def _device_dim_matched_sns(
# 否则图表桶(回落值)点击下钻查不到。 # 否则图表桶(回落值)点击下钻查不到。
if device_type or combo: if device_type or combo:
latest_sub = ( latest_sub = (
select(BmsOrder.device_sn, BmsOrder.device_type_name, BmsOrder.combo_name) select(BmsOrderClean.device_sn, BmsOrderClean.device_type_name, BmsOrderClean.combo_name)
.where(BmsOrder.device_sn.isnot(None)) .where(BmsOrderClean.device_sn.isnot(None))
.distinct(BmsOrder.device_sn) .distinct(BmsOrderClean.device_sn)
.order_by(BmsOrder.device_sn, BmsOrder.create_date.desc().nullslast()) .order_by(BmsOrderClean.device_sn, BmsOrderClean.create_date.desc().nullslast())
).subquery() ).subquery()
latest_sq = select(latest_sub.c.device_sn) latest_sq = select(latest_sub.c.device_sn)
if device_type: if device_type:
@@ -1185,7 +1183,7 @@ def _device_dim_matched_sns(
dev_sq = ( dev_sq = (
select(Device.sn) select(Device.sn)
.where(Device.sn.isnot(None), or_(*dev_conds)) .where(Device.sn.isnot(None), or_(*dev_conds))
.where(~exists(select(1).where(BmsOrder.device_sn == Device.sn))) .where(~exists(select(1).where(BmsOrderClean.device_sn == Device.sn)))
) )
for s in db.scalars(dev_sq).all(): for s in db.scalars(dev_sq).all():
if s: if s:
@@ -1217,14 +1215,14 @@ def _device_dim_matched_sns(
hi = datetime.combine(d1, _dt.time.max, tzinfo=TZ).astimezone(_dt.timezone.utc).replace(tzinfo=None) if d1 else None hi = datetime.combine(d1, _dt.time.max, tzinfo=TZ).astimezone(_dt.timezone.utc).replace(tzinfo=None) if d1 else None
active_matched: set[str] = set() active_matched: set[str] = set()
# 1) bms_orders 有 MIN(active_date) 落在区间 # 1) bms_orders 有 MIN(active_date) 落在区间
bms_sq = select(BmsOrder.device_sn).where(BmsOrder.active_date.isnot(None)).group_by(BmsOrder.device_sn) bms_sq = select(BmsOrderClean.device_sn).where(BmsOrderClean.active_date.isnot(None)).group_by(BmsOrderClean.device_sn)
if lo: if lo:
bms_sq = bms_sq.having(func.min(BmsOrder.active_date) >= lo) bms_sq = bms_sq.having(func.min(BmsOrderClean.active_date) >= lo)
if hi: if hi:
bms_sq = bms_sq.having(func.min(BmsOrder.active_date) <= hi) bms_sq = bms_sq.having(func.min(BmsOrderClean.active_date) <= hi)
active_matched.update(s for s in db.scalars(bms_sq).all() if s) active_matched.update(s for s in db.scalars(bms_sq).all() if s)
# 2) devices.active_time 落在区间 且 该 SN 无 bms_orders 激活记录(兜底, 不重复) # 2) devices.active_time 落在区间 且 该 SN 无 bms_orders 激活记录(兜底, 不重复)
has_bms_active = select(BmsOrder.device_sn).where(BmsOrder.active_date.isnot(None)) has_bms_active = select(BmsOrderClean.device_sn).where(BmsOrderClean.active_date.isnot(None))
dev_conds = [Device.active_time.isnot(None), ~Device.sn.in_(has_bms_active)] dev_conds = [Device.active_time.isnot(None), ~Device.sn.in_(has_bms_active)]
if lo: if lo:
dev_conds.append(Device.active_time >= lo) dev_conds.append(Device.active_time >= lo)
@@ -1248,14 +1246,14 @@ def _device_dim_options(db: Session) -> dict[str, list[str]]:
· 套餐 = bms_orders.combo_name distinct(排除工厂质检 3 变体) · 套餐 = bms_orders.combo_name distinct(排除工厂质检 3 变体)
· 商户号 = devices.merchant_name(设备归属) · 商户号 = devices.merchant_name(设备归属)
""" """
from app.models.bms_order import BmsOrder
from app.models.device import Device from app.models.device import Device
from app.models.ods_clean import BmsOrderClean
# 设备类型: bms_orders.device_type_name(原文), distinct 取后排序 # 设备类型: bms_orders.device_type_name(原文), distinct 取后排序
type_names = sorted({ type_names = sorted({
t for t in db.scalars( t for t in db.scalars(
select(BmsOrder.device_type_name) select(BmsOrderClean.device_type_name)
.where(BmsOrder.device_type_name.isnot(None)) .where(BmsOrderClean.device_type_name.isnot(None))
.distinct() .distinct()
).all() if t ).all() if t
}) })
@@ -1263,8 +1261,8 @@ def _device_dim_options(db: Session) -> dict[str, list[str]]:
# 套餐: bms_orders.combo_name, 排除工厂质检测试套餐(3 变体不计入分析) # 套餐: bms_orders.combo_name, 排除工厂质检测试套餐(3 变体不计入分析)
combos = sorted({ combos = sorted({
c for c in db.scalars( c for c in db.scalars(
select(BmsOrder.combo_name) select(BmsOrderClean.combo_name)
.where(BmsOrder.combo_name.isnot(None)) .where(BmsOrderClean.combo_name.isnot(None))
.distinct() .distinct()
).all() if c and "工厂质检" not in c ).all() if c and "工厂质检" not in c
}) })
@@ -1339,9 +1337,8 @@ def _device_dim_attrs(db: Session, sns: set[str]) -> dict[str, dict[str, Any]]:
""" """
if not sns: if not sns:
return {} return {}
from app.models.bms_order import BmsOrder
from app.models.device import Device from app.models.device import Device
from app.models.device_combo import DeviceCombo from app.models.ods_clean import BmsOrderClean, DeviceComboClean
# 1) devices 表: merchant_name + active_time(激活兜底) + device_type/current_combo_name(#293 回落) # 1) devices 表: merchant_name + active_time(激活兜底) + device_type/current_combo_name(#293 回落)
dev_rows = db.execute( dev_rows = db.execute(
@@ -1353,10 +1350,10 @@ def _device_dim_attrs(db: Session, sns: set[str]) -> dict[str, dict[str, Any]]:
# 2) bms_orders: 最新订单(create_date DESC 取首条)的 device_type_name; # 2) bms_orders: 最新订单(create_date DESC 取首条)的 device_type_name;
# combo 取最新**非质检**订单(#312: 跳过质检单, 防历史正常套餐被顶掉) # combo 取最新**非质检**订单(#312: 跳过质检单, 防历史正常套餐被顶掉)
order_rows = db.execute( order_rows = db.execute(
select(BmsOrder.device_sn, BmsOrder.device_type_name, select(BmsOrderClean.device_sn, BmsOrderClean.device_type_name,
BmsOrder.combo_name, BmsOrder.create_date) BmsOrderClean.combo_name, BmsOrderClean.create_date)
.where(BmsOrder.device_sn.in_(sns)) .where(BmsOrderClean.device_sn.in_(sns))
.order_by(BmsOrder.device_sn, BmsOrder.create_date.desc().nullslast()) .order_by(BmsOrderClean.device_sn, BmsOrderClean.create_date.desc().nullslast())
).all() ).all()
latest: dict[str, tuple[str | None, str | None]] = {} latest: dict[str, tuple[str | None, str | None]] = {}
latest_nonqc_combo: dict[str, str] = {} latest_nonqc_combo: dict[str, str] = {}
@@ -1371,9 +1368,9 @@ def _device_dim_attrs(db: Session, sns: set[str]) -> dict[str, dict[str, Any]]:
# 2b) device_combos 最新充值流水 package_name(#312 第三回落源, 非质检) # 2b) device_combos 最新充值流水 package_name(#312 第三回落源, 非质检)
dc_combo: dict[str, str] = {} dc_combo: dict[str, str] = {}
for sn, pkg, _c in db.execute( for sn, pkg, _c in db.execute(
select(DeviceCombo.device_sn, DeviceCombo.package_name, DeviceCombo.created_at_raw) select(DeviceComboClean.device_sn, DeviceComboClean.package_name, DeviceComboClean.created_at_raw)
.where(DeviceCombo.device_sn.in_(sns), DeviceCombo.package_name.isnot(None)) .where(DeviceComboClean.device_sn.in_(sns), DeviceComboClean.package_name.isnot(None))
.order_by(DeviceCombo.device_sn, DeviceCombo.created_at_raw.desc().nullslast()) .order_by(DeviceComboClean.device_sn, DeviceComboClean.created_at_raw.desc().nullslast())
).all(): ).all():
sn_s = str(sn) sn_s = str(sn)
if sn_s not in dc_combo and pkg and "工厂质检" not in pkg: if sn_s not in dc_combo and pkg and "工厂质检" not in pkg:
@@ -1382,9 +1379,9 @@ def _device_dim_attrs(db: Session, sns: set[str]) -> dict[str, dict[str, Any]]:
# 3) 最早激活日: bms_orders MIN(active_date)(#251 同源) # 3) 最早激活日: bms_orders MIN(active_date)(#251 同源)
first_active: dict[str, Any] = { first_active: dict[str, Any] = {
sn: fa for sn, fa in db.execute( sn: fa for sn, fa in db.execute(
select(BmsOrder.device_sn, func.min(BmsOrder.active_date)) select(BmsOrderClean.device_sn, func.min(BmsOrderClean.active_date))
.where(BmsOrder.device_sn.in_(sns), BmsOrder.active_date.isnot(None)) .where(BmsOrderClean.device_sn.in_(sns), BmsOrderClean.active_date.isnot(None))
.group_by(BmsOrder.device_sn) .group_by(BmsOrderClean.device_sn)
).all() ).all()
} }

View File

@@ -8,10 +8,12 @@ from typing import Any
from sqlalchemy import and_, func, select, text from sqlalchemy import and_, func, select, text
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from app.models.manual_refund import ManualRefund from app.models.ods_clean import (
from app.models.refund import RefundClean ManualRefundClean,
from app.models.refund_record_report import RefundRecordReport RefundClean,
from app.models.wallet import WalletWithdrawal RefundRecordReportClean,
WalletWithdrawalClean,
)
# 跨子模块引用(拆分自 app.services.analytics, 原同文件内直接调用) # 跨子模块引用(拆分自 app.services.analytics, 原同文件内直接调用)
from app.services.analytics._core import ( from app.services.analytics._core import (
@@ -90,7 +92,8 @@ def operation_method_dist(db: Session) -> list[dict]:
# 金额: actual_amount(实际退款金额) 而非 refund_amount(申请退款金额) # 金额: actual_amount(实际退款金额) 而非 refund_amount(申请退款金额)
# 时间: audit_time(审核通过时间 ≈ 退款完成时间) Excel 无独立 refund_time 列 # 时间: audit_time(审核通过时间 ≈ 退款完成时间) Excel 无独立 refund_time 列
# 状态列: status 成功值: '已退款'(中文), 其他状态(申请中/已驳回/退款失败等)不计入经营出款 # 状态列: status 成功值: '已退款'(中文), 其他状态(申请中/已驳回/退款失败等)不计入经营出款
_C_SHANMAO_TABLE = "shanmao_refunds" # #322 方案A: 读路径切到 public 薄视图 shanmao_refund_clean(over 闪猫退款信源表), 口径逐字不变
_C_SHANMAO_TABLE = "shanmao_refund_clean"
_C_SHANMAO_AMOUNT_COL = "actual_amount" _C_SHANMAO_AMOUNT_COL = "actual_amount"
@@ -168,7 +171,8 @@ def _route_c_shanmao_agg(db: Session, start, end) -> tuple[int, float]:
# 数据源: 商户号手机退款处理登记.xlsx (客服在商户号后台手动操作退款台账) # 数据源: 商户号手机退款处理登记.xlsx (客服在商户号后台手动操作退款台账)
# 金额: refund_amount(退款金额); 状态: is_refunded='是' 计入经营出款, '否' = 登记蓄水(水位) # 金额: refund_amount(退款金额); 状态: is_refunded='是' 计入经营出款, '否' = 登记蓄水(水位)
# 时间: op_time(实际退款操作时间) 为主, 缺失回退 register_time(登记日期) # 时间: op_time(实际退款操作时间) 为主, 缺失回退 register_time(登记日期)
_D_OTHER_TABLE = "manual_refunds" # #322 方案A: 读路径切到 public 薄视图 manual_refund_clean(over 手动退款信源表), 口径逐字不变
_D_OTHER_TABLE = "manual_refund_clean"
_D_OTHER_AMOUNT_COL = "refund_amount" _D_OTHER_AMOUNT_COL = "refund_amount"
@@ -228,7 +232,7 @@ def _quality_include_cond() -> list:
(同一份报表里两条路径混用时, 曾出现一条排除 dup_residual、另一条没排除)。 (同一份报表里两条路径混用时, 曾出现一条排除 dup_residual、另一条没排除)。
#201 S-P0-1:真身收敛到 quality_check.stats_quality_include。 #201 S-P0-1:真身收敛到 quality_check.stats_quality_include。
""" """
return [stats_quality_include(ManualRefund.quality_status)] return [stats_quality_include(ManualRefundClean.quality_status)]
def _route_d_other_agg(db: Session, start, end) -> tuple[int, float]: def _route_d_other_agg(db: Session, start, end) -> tuple[int, float]:
"""【路线D】手动退款(manual_refunds)独立表聚合 (表不存在则返回 0,0,不报错). """【路线D】手动退款(manual_refunds)独立表聚合 (表不存在则返回 0,0,不报错).
@@ -353,8 +357,8 @@ def _route_d_other_trend(db: Session, start, end, grain: str, slot_idx_fn) -> tu
def _cash_routes_total(db: Session, def _cash_routes_total(db: Session,
col_a: Any = RefundClean.created_at_raw, col_a: Any = RefundClean.created_at_raw,
col_b: Any = WalletWithdrawal.op_time, col_b: Any = WalletWithdrawalClean.op_time,
col_e: Any = RefundRecordReport.refund_time, col_e: Any = RefundRecordReportClean.refund_time,
start: _dt.datetime | None = None, start: _dt.datetime | None = None,
end: _dt.datetime | None = None) -> dict[str, Any]: end: _dt.datetime | None = None) -> dict[str, Any]:
"""【统一核算聚合器 v3】四路经营A/B/C/D + 实际现金流E + 对账偏差. """【统一核算聚合器 v3】四路经营A/B/C/D + 实际现金流E + 对账偏差.
@@ -395,7 +399,7 @@ def _cash_routes_total(db: Session,
# ── B 钱包用户取现 ── # ── B 钱包用户取现 ──
b_row = db.execute(select( b_row = db.execute(select(
func.count(), func.count(),
func.coalesce(func.sum(WalletWithdrawal.op_amount), 0), func.coalesce(func.sum(WalletWithdrawalClean.op_amount), 0),
).where( ).where(
*_route_b_withdraw_filter(), *_route_b_withdraw_filter(),
_between(col_b, start, end), _between(col_b, start, end),
@@ -410,7 +414,7 @@ def _cash_routes_total(db: Session,
# ── E 实际现金流: refund_record_reports 退款成功合计 (对账基准) ── # ── E 实际现金流: refund_record_reports 退款成功合计 (对账基准) ──
e_row = db.execute(select( e_row = db.execute(select(
func.count(), func.count(),
func.coalesce(func.sum(RefundRecordReport.refund_amount), 0), func.coalesce(func.sum(RefundRecordReportClean.refund_amount), 0),
).where( ).where(
*_route_e_cashflow_filter(), *_route_e_cashflow_filter(),
_between(col_e, start, end), _between(col_e, start, end),
@@ -475,11 +479,11 @@ def cash_out_trend(db: Session, start: datetime | None = None, end: datetime | N
# ── B 用户提现按日 (路线B wallet) ── # ── B 用户提现按日 (路线B wallet) ──
withdraw_rows = db.execute( withdraw_rows = db.execute(
select( select(
func.to_char(func.timezone("Asia/Shanghai", WalletWithdrawal.op_time), "YYYY-MM-DD").label("date"), func.to_char(func.timezone("Asia/Shanghai", WalletWithdrawalClean.op_time), "YYYY-MM-DD").label("date"),
func.coalesce(func.sum(WalletWithdrawal.op_amount), 0).label("amount"), func.coalesce(func.sum(WalletWithdrawalClean.op_amount), 0).label("amount"),
func.count().label("count"), func.count().label("count"),
) )
.where(WalletWithdrawal.op_time.between(s, e), *_route_b_withdraw_filter()) .where(WalletWithdrawalClean.op_time.between(s, e), *_route_b_withdraw_filter())
.group_by("date") .group_by("date")
).all() ).all()
withdraw_map = {r.date: {"amount": D(r.amount), "count": r.count} for r in withdraw_rows} withdraw_map = {r.date: {"amount": D(r.amount), "count": r.count} for r in withdraw_rows}
@@ -531,11 +535,11 @@ def cash_out_trend(db: Session, start: datetime | None = None, end: datetime | N
# 日分桶统一按北京日 (与决策概览 _day AT TIME ZONE 口径一致) # 日分桶统一按北京日 (与决策概览 _day AT TIME ZONE 口径一致)
actual_rows = db.execute( actual_rows = db.execute(
select( select(
func.to_char(func.timezone("Asia/Shanghai", RefundRecordReport.refund_time), "YYYY-MM-DD").label("date"), func.to_char(func.timezone("Asia/Shanghai", RefundRecordReportClean.refund_time), "YYYY-MM-DD").label("date"),
func.coalesce(func.sum(RefundRecordReport.refund_amount), 0).label("amount"), func.coalesce(func.sum(RefundRecordReportClean.refund_amount), 0).label("amount"),
func.count().label("count"), func.count().label("count"),
) )
.where(RefundRecordReport.refund_time.between(s, e), *_route_e_cashflow_filter()) .where(RefundRecordReportClean.refund_time.between(s, e), *_route_e_cashflow_filter())
.group_by("date") .group_by("date")
).all() ).all()
actual_map = {r.date: {"amount": D(r.amount), "count": r.count} for r in actual_rows} actual_map = {r.date: {"amount": D(r.amount), "count": r.count} for r in actual_rows}
@@ -592,20 +596,20 @@ def wallet_vs_withdraw(db: Session, start: datetime | None = None, end: datetime
# 退回钱包: wallet 中 op_type='订单退费转入' # 退回钱包: wallet 中 op_type='订单退费转入'
wallet_in = db.execute( wallet_in = db.execute(
select( select(
func.to_char(func.timezone("Asia/Shanghai", WalletWithdrawal.op_time), "YYYY-MM-DD").label("date"), func.to_char(func.timezone("Asia/Shanghai", WalletWithdrawalClean.op_time), "YYYY-MM-DD").label("date"),
func.coalesce(func.sum(WalletWithdrawal.op_amount), 0).label("amount"), func.coalesce(func.sum(WalletWithdrawalClean.op_amount), 0).label("amount"),
) )
.where(WalletWithdrawal.op_time.between(start, end), WalletWithdrawal.op_type == "订单退费转入") .where(WalletWithdrawalClean.op_time.between(start, end), WalletWithdrawalClean.op_type == "订单退费转入")
.group_by("date") .group_by("date")
).all() ).all()
wallet_in_map = {r.date: D(r.amount) for r in wallet_in} wallet_in_map = {r.date: D(r.amount) for r in wallet_in}
# 实际提现: op_type='用户提现' 且 备注非失败/冻结 (2026-08-28 排除冻结记录) # 实际提现: op_type='用户提现' 且 备注非失败/冻结 (2026-08-28 排除冻结记录)
withdraw = db.execute( withdraw = db.execute(
select( select(
func.to_char(func.timezone("Asia/Shanghai", WalletWithdrawal.op_time), "YYYY-MM-DD").label("date"), func.to_char(func.timezone("Asia/Shanghai", WalletWithdrawalClean.op_time), "YYYY-MM-DD").label("date"),
func.coalesce(func.sum(WalletWithdrawal.op_amount), 0).label("amount"), func.coalesce(func.sum(WalletWithdrawalClean.op_amount), 0).label("amount"),
) )
.where(WalletWithdrawal.op_time.between(start, end), *_route_b_withdraw_filter()) .where(WalletWithdrawalClean.op_time.between(start, end), *_route_b_withdraw_filter())
.group_by("date") .group_by("date")
).all() ).all()
withdraw_map = {r.date: D(r.amount) for r in withdraw} withdraw_map = {r.date: D(r.amount) for r in withdraw}
@@ -666,13 +670,13 @@ def merchant_cash_summary(db: Session) -> list[dict]:
""" """
rows = db.execute( rows = db.execute(
select( select(
RefundRecordReport.merchant_id, RefundRecordReportClean.merchant_id,
func.coalesce(func.sum(RefundRecordReport.refund_amount), 0).label("amount"), func.coalesce(func.sum(RefundRecordReportClean.refund_amount), 0).label("amount"),
func.count().label("count"), func.count().label("count"),
) )
.where(*_route_e_cashflow_filter()) .where(*_route_e_cashflow_filter())
.group_by(RefundRecordReport.merchant_id) .group_by(RefundRecordReportClean.merchant_id)
.order_by(func.sum(RefundRecordReport.refund_amount).desc()) .order_by(func.sum(RefundRecordReportClean.refund_amount).desc())
.limit(50) .limit(50)
).all() ).all()
return [ return [
@@ -690,11 +694,11 @@ def wallet_balance_top(db: Session, limit: int = 20) -> list[dict]:
# 用窗口函数取每个账户最新一条记录的余额 # 用窗口函数取每个账户最新一条记录的余额
rows = db.execute( rows = db.execute(
select( select(
WalletWithdrawal.account_id, WalletWithdrawalClean.account_id,
WalletWithdrawal.wallet_balance, WalletWithdrawalClean.wallet_balance,
WalletWithdrawal.op_time, WalletWithdrawalClean.op_time,
) )
.order_by(WalletWithdrawal.account_id, WalletWithdrawal.op_time.desc()) .order_by(WalletWithdrawalClean.account_id, WalletWithdrawalClean.op_time.desc())
.limit(limit * 5) .limit(limit * 5)
).all() ).all()
# 去重保留每个账户最新 # 去重保留每个账户最新

View File

@@ -35,11 +35,9 @@ from sqlalchemy.orm import Session
from app.models.combo_dict import ComboDict, ComboNameAlias from app.models.combo_dict import ComboDict, ComboNameAlias
from app.models.device import Device from app.models.device import Device
from app.models.device_combo import DeviceCombo from app.models.ods_clean import DeviceComboClean, DeviceFlowUsageClean, RefundClean
from app.models.device_flow_usage import DeviceFlowUsage
# from app.models.merchant import DeviceMerchantLink, Merchant # #271: 暂用 devices.merchant_name,links/merchants 表全空 # from app.models.merchant import DeviceMerchantLink, Merchant # #271: 暂用 devices.merchant_name,links/merchants 表全空
from app.models.refund import RefundClean
from app.models.ticket import ServiceTicket from app.models.ticket import ServiceTicket
from app.services import complaint_scope from app.services import complaint_scope
from app.services.analytics._core import D from app.services.analytics._core import D
@@ -92,15 +90,15 @@ def _fetch_active_devices(db: Session, since: datetime, end: datetime) -> set[st
环比全线假降 60-85%。 环比全线假降 60-85%。
""" """
# 质检套餐排除(#293): 出厂质检流水非真实充值, 不应计入充值活跃 # 质检套餐排除(#293): 出厂质检流水非真实充值, 不应计入充值活跃
not_qc = DeviceCombo.package_name.notilike("%质检%") not_qc = DeviceComboClean.package_name.notilike("%质检%")
# 充值活跃 # 充值活跃
rc = { rc = {
r[0] r[0]
for r in db.execute( for r in db.execute(
select(distinct(DeviceCombo.device_sn)) select(distinct(DeviceComboClean.device_sn))
.where(DeviceCombo.device_sn.isnot(None)) .where(DeviceComboClean.device_sn.isnot(None))
.where(DeviceCombo.created_at_raw >= since) .where(DeviceComboClean.created_at_raw >= since)
.where(DeviceCombo.created_at_raw <= end) .where(DeviceComboClean.created_at_raw <= end)
.where(not_qc) .where(not_qc)
) )
} }
@@ -108,11 +106,11 @@ def _fetch_active_devices(db: Session, since: datetime, end: datetime) -> set[st
fl = { fl = {
r[0] r[0]
for r in db.execute( for r in db.execute(
select(distinct(DeviceFlowUsage.device_sn)) select(distinct(DeviceFlowUsageClean.device_sn))
.where(DeviceFlowUsage.device_sn.isnot(None)) .where(DeviceFlowUsageClean.device_sn.isnot(None))
.where(DeviceFlowUsage.month_complete.is_(True)) .where(DeviceFlowUsageClean.month_complete.is_(True))
.where(DeviceFlowUsage.stat_month >= since.strftime("%Y-%m")) .where(DeviceFlowUsageClean.stat_month >= since.strftime("%Y-%m"))
.where(DeviceFlowUsage.stat_month <= end.strftime("%Y-%m")) .where(DeviceFlowUsageClean.stat_month <= end.strftime("%Y-%m"))
) )
} }
return rc | fl return rc | fl
@@ -126,32 +124,32 @@ def _fetch_recharge_by_dim(db: Session, dim: str, since: datetime, end: datetime
修复前 CategoryBoard 充值金额虚高 ~2.4 倍。 修复前 CategoryBoard 充值金额虚高 ~2.4 倍。
""" """
# 质检套餐排除条件(出厂质检, 非用户付费) # 质检套餐排除条件(出厂质检, 非用户付费)
not_qc = DeviceCombo.package_name.notilike("%质检%") not_qc = DeviceComboClean.package_name.notilike("%质检%")
if dim == "device_type": if dim == "device_type":
stmt = ( stmt = (
select( select(
Device.device_type.label("key"), Device.device_type.label("key"),
func.count().label("cnt"), func.count().label("cnt"),
func.coalesce(func.sum(DeviceCombo.actual_amount), 0).label("amount"), func.coalesce(func.sum(DeviceComboClean.actual_amount), 0).label("amount"),
) )
.select_from(DeviceCombo) .select_from(DeviceComboClean)
.join(Device, Device.sn == DeviceCombo.device_sn) .join(Device, Device.sn == DeviceComboClean.device_sn)
.where(DeviceCombo.created_at_raw >= since) .where(DeviceComboClean.created_at_raw >= since)
.where(DeviceCombo.created_at_raw <= end) .where(DeviceComboClean.created_at_raw <= end)
.where(not_qc) .where(not_qc)
.group_by(Device.device_type) .group_by(Device.device_type)
) )
elif dim == "combo": elif dim == "combo":
stmt = ( stmt = (
select( select(
DeviceCombo.package_name.label("key"), DeviceComboClean.package_name.label("key"),
func.count().label("cnt"), func.count().label("cnt"),
func.coalesce(func.sum(DeviceCombo.actual_amount), 0).label("amount"), func.coalesce(func.sum(DeviceComboClean.actual_amount), 0).label("amount"),
) )
.where(DeviceCombo.created_at_raw >= since) .where(DeviceComboClean.created_at_raw >= since)
.where(DeviceCombo.created_at_raw <= end) .where(DeviceComboClean.created_at_raw <= end)
.where(not_qc) .where(not_qc)
.group_by(DeviceCombo.package_name) .group_by(DeviceComboClean.package_name)
) )
elif dim == "merchant": elif dim == "merchant":
# #271: 走 devices.merchant_name(links/merchants 表全空) # #271: 走 devices.merchant_name(links/merchants 表全空)
@@ -159,12 +157,12 @@ def _fetch_recharge_by_dim(db: Session, dim: str, since: datetime, end: datetime
select( select(
Device.merchant_name.label("key"), Device.merchant_name.label("key"),
func.count().label("cnt"), func.count().label("cnt"),
func.coalesce(func.sum(DeviceCombo.actual_amount), 0).label("amount"), func.coalesce(func.sum(DeviceComboClean.actual_amount), 0).label("amount"),
) )
.select_from(DeviceCombo) .select_from(DeviceComboClean)
.join(Device, Device.sn == DeviceCombo.device_sn) .join(Device, Device.sn == DeviceComboClean.device_sn)
.where(DeviceCombo.created_at_raw >= since) .where(DeviceComboClean.created_at_raw >= since)
.where(DeviceCombo.created_at_raw <= end) .where(DeviceComboClean.created_at_raw <= end)
.where(not_qc) .where(not_qc)
.where(Device.merchant_name.isnot(None)) .where(Device.merchant_name.isnot(None))
.group_by(Device.merchant_name) .group_by(Device.merchant_name)
@@ -200,7 +198,7 @@ def _fetch_refund_by_dim(db: Session, dim: str, since: datetime, end: datetime)
.group_by(Device.device_type) .group_by(Device.device_type)
) )
elif dim == "combo": elif dim == "combo":
# #270: 去掉 JOIN device_combos(充值流水一对多导致退款重复计数 2.5x), # #270: 去掉对充值流水表的 JOIN(充值流水一对多导致退款重复计数 2.5x),
# 改按 Device.current_combo_name 分组(与设备数/活跃同口径) # 改按 Device.current_combo_name 分组(与设备数/活跃同口径)
stmt = ( stmt = (
select(Device.current_combo_name.label("key"), func.count().label("cnt")) select(Device.current_combo_name.label("key"), func.count().label("cnt"))
@@ -243,7 +241,7 @@ def _fetch_complaint_by_dim(db: Session, dim: str, since: datetime, end: datetim
.group_by(Device.device_type) .group_by(Device.device_type)
) )
elif dim == "combo": elif dim == "combo":
# #270: 同退款修复,去掉 JOIN device_combos 放大,按 current_combo_name 分组 # #270: 同退款修复,去掉对充值流水表的 JOIN 放大,按 current_combo_name 分组
stmt = ( stmt = (
select(Device.current_combo_name.label("key"), func.count().label("cnt")) select(Device.current_combo_name.label("key"), func.count().label("cnt"))
.select_from(ServiceTicket) .select_from(ServiceTicket)

View File

@@ -7,12 +7,10 @@ from typing import Any
from sqlalchemy import and_, case, func, or_, select, text from sqlalchemy import and_, case, func, or_, select, text
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from app.models.bms_order import BmsOrder
from app.models.combo_dict import ComboDict, ComboNameAlias from app.models.combo_dict import ComboDict, ComboNameAlias
from app.models.cs_record import CsRecord
from app.models.device import Device, DimDeviceType from app.models.device import Device, DimDeviceType
from app.models.merchant import DeviceMerchantLink, Merchant from app.models.merchant import DeviceMerchantLink, Merchant
from app.models.refund import RefundClean from app.models.ods_clean import BmsOrderClean, CsRecordClean, RefundClean
from app.models.shop import DeviceShopLink, Shop from app.models.shop import DeviceShopLink, Shop
from app.models.ticket import ServiceTicket from app.models.ticket import ServiceTicket
from app.services.complaint_scope import ( from app.services.complaint_scope import (
@@ -368,10 +366,10 @@ def complaint_health_metrics(
# 口径: 窗口内 `cs_records.operation_time` 命中的去重 `device_sn` # 口径: 窗口内 `cs_records.operation_time` 命中的去重 `device_sn`
active_sn = int( active_sn = int(
db.scalar( db.scalar(
select(func.count(func.distinct(CsRecord.device_sn))).where( select(func.count(func.distinct(CsRecordClean.device_sn))).where(
CsRecord.operation_time >= ts, CsRecordClean.operation_time >= ts,
CsRecord.operation_time <= te, CsRecordClean.operation_time <= te,
CsRecord.device_sn.isnot(None), CsRecordClean.device_sn.isnot(None),
) )
) )
or 0 or 0
@@ -449,13 +447,13 @@ def complaint_health_metrics(
# 客服接待数(按操作人,取接待会话数) # 客服接待数(按操作人,取接待会话数)
reception_rows = db.execute( reception_rows = db.execute(
select(CsRecord.operator, func.count()) select(CsRecordClean.operator, func.count())
.where( .where(
CsRecord.operation_time >= ts, CsRecordClean.operation_time >= ts,
CsRecord.operation_time <= te, CsRecordClean.operation_time <= te,
CsRecord.operator.isnot(None), CsRecordClean.operator.isnot(None),
) )
.group_by(CsRecord.operator) .group_by(CsRecordClean.operator)
).all() ).all()
# 在岗人数(按 bucket)—— 只统计**有别名**的人,避免重复 # 在岗人数(按 bucket)—— 只统计**有别名**的人,避免重复
@@ -894,11 +892,11 @@ def complaint_extended_dims(
# 无激活记录优先判定, NULL 天数不会误落首桶(与 _active_age_bucket Python 侧同序)。 # 无激活记录优先判定, NULL 天数不会误落首桶(与 _active_age_bucket Python 侧同序)。
fa_cte = ( fa_cte = (
select( select(
BmsOrder.device_sn.label("dsn"), BmsOrderClean.device_sn.label("dsn"),
func.min(BmsOrder.active_date).label("fa"), func.min(BmsOrderClean.active_date).label("fa"),
) )
.where(BmsOrder.device_sn.isnot(None), BmsOrder.active_date.isnot(None)) .where(BmsOrderClean.device_sn.isnot(None), BmsOrderClean.active_date.isnot(None))
.group_by(BmsOrder.device_sn) .group_by(BmsOrderClean.device_sn)
).cte("first_active") ).cte("first_active")
days_expr = ( days_expr = (
func.date(ServiceTicket.register_time.op("AT TIME ZONE")("Asia/Shanghai")) func.date(ServiceTicket.register_time.op("AT TIME ZONE")("Asia/Shanghai"))

View File

@@ -6,10 +6,9 @@ from typing import Any
from sqlalchemy import and_, case, func, select, text from sqlalchemy import and_, case, func, select, text
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from app.models.cs_record import CsRecord
from app.models.group import Group, PersonGroupMembership from app.models.group import Group, PersonGroupMembership
from app.models.ods_clean import CsRecordClean, RefundClean
from app.models.person import Person, PersonAlias from app.models.person import Person, PersonAlias
from app.models.refund import RefundClean
from app.models.ticket import ServiceTicket from app.models.ticket import ServiceTicket
# 跨子模块引用(拆分自 app.services.analytics, 原同文件内直接调用) # 跨子模块引用(拆分自 app.services.analytics, 原同文件内直接调用)
@@ -228,7 +227,7 @@ def _category_case():
for bucket, keywords in _CATEGORY_BUCKETS[:-1]: # 除最后一项"其他问题"(兜底) for bucket, keywords in _CATEGORY_BUCKETS[:-1]: # 除最后一项"其他问题"(兜底)
if keywords: if keywords:
for kw in keywords: for kw in keywords:
whens.append((CsRecord.problem_category.ilike(f"%{kw}%"), bucket)) whens.append((CsRecordClean.problem_category.ilike(f"%{kw}%"), bucket))
fallback = _CATEGORY_BUCKETS[-1][0] # "其他问题" fallback = _CATEGORY_BUCKETS[-1][0] # "其他问题"
return case(*whens, else_=fallback).label("category_bucket") return case(*whens, else_=fallback).label("category_bucket")
@@ -289,16 +288,16 @@ def reception_overview_trend(
- per_capita_daily: 人均接待量/天 = reception_total / staff_count / day_count - per_capita_daily: 人均接待量/天 = reception_total / staff_count / day_count
""" """
start, end = _date_range(start, end) start, end = _date_range(start, end)
week_trunc = func.date_trunc("week", CsRecord.operation_time).label("week_start") week_trunc = func.date_trunc("week", CsRecordClean.operation_time).label("week_start")
rows = db.execute( rows = db.execute(
select( select(
week_trunc, week_trunc,
func.to_char(func.date_trunc("week", CsRecord.operation_time), "IW").label("week_no"), func.to_char(func.date_trunc("week", CsRecordClean.operation_time), "IW").label("week_no"),
func.count().label("total"), func.count().label("total"),
func.count(func.distinct(func.to_char(CsRecord.operation_time, "YYYY-MM-DD"))).label("day_count"), func.count(func.distinct(func.to_char(CsRecordClean.operation_time, "YYYY-MM-DD"))).label("day_count"),
func.count(func.distinct(CsRecord.operator)).filter(CsRecord.operator.isnot(None)).label("staff_count"), func.count(func.distinct(CsRecordClean.operator)).filter(CsRecordClean.operator.isnot(None)).label("staff_count"),
) )
.where(CsRecord.operation_time.between(start, end)) .where(CsRecordClean.operation_time.between(start, end))
.group_by("week_start") .group_by("week_start")
.order_by("week_start") .order_by("week_start")
).all() ).all()
@@ -327,16 +326,16 @@ def reception_category_summary(
输出: [{week_start, week_no, 投诉, 退款, 其他问题, 套餐问题, 咨询, 综合问题}] 输出: [{week_start, week_no, 投诉, 退款, 其他问题, 套餐问题, 咨询, 综合问题}]
""" """
start, end = _date_range(start, end) start, end = _date_range(start, end)
week_trunc = func.date_trunc("week", CsRecord.operation_time).label("week_start") week_trunc = func.date_trunc("week", CsRecordClean.operation_time).label("week_start")
bucket = _category_case() bucket = _category_case()
rows = db.execute( rows = db.execute(
select( select(
week_trunc, week_trunc,
func.to_char(func.date_trunc("week", CsRecord.operation_time), "IW").label("week_no"), func.to_char(func.date_trunc("week", CsRecordClean.operation_time), "IW").label("week_no"),
bucket, bucket,
func.count().label("cnt"), func.count().label("cnt"),
) )
.where(CsRecord.operation_time.between(start, end)) .where(CsRecordClean.operation_time.between(start, end))
.group_by("week_start", "category_bucket") .group_by("week_start", "category_bucket")
.order_by("week_start", "category_bucket") .order_by("week_start", "category_bucket")
).all() ).all()
@@ -376,13 +375,13 @@ def reception_per_capita(
# 步骤1: 按周 × operator × bucket 聚合每人每类接待数 # 步骤1: 按周 × operator × bucket 聚合每人每类接待数
op_cat_rows = db.execute( op_cat_rows = db.execute(
select( select(
func.date_trunc("week", CsRecord.operation_time).label("week_start"), func.date_trunc("week", CsRecordClean.operation_time).label("week_start"),
func.to_char(func.date_trunc("week", CsRecord.operation_time), "IW").label("week_no"), func.to_char(func.date_trunc("week", CsRecordClean.operation_time), "IW").label("week_no"),
CsRecord.operator.label("operator"), CsRecordClean.operator.label("operator"),
bucket, bucket,
func.count().label("cnt"), func.count().label("cnt"),
) )
.where(CsRecord.operation_time.between(start, end), CsRecord.operator.isnot(None)) .where(CsRecordClean.operation_time.between(start, end), CsRecordClean.operator.isnot(None))
.group_by("week_start", "week_no", "operator", "category_bucket") .group_by("week_start", "week_no", "operator", "category_bucket")
).all() ).all()
@@ -510,18 +509,18 @@ def reception_group_detail(
bucket = _category_case() bucket = _category_case()
rows = db.execute( rows = db.execute(
select( select(
CsRecord.operator.label("operator"), CsRecordClean.operator.label("operator"),
op_person.c.display_name.label("display_name"), op_person.c.display_name.label("display_name"),
op_person.c.group_name.label("group_name"), op_person.c.group_name.label("group_name"),
op_person.c.specialty.label("specialty"), op_person.c.specialty.label("specialty"),
func.to_char(CsRecord.operation_time, "YYYY-MM-DD").label("day"), func.to_char(CsRecordClean.operation_time, "YYYY-MM-DD").label("day"),
bucket, bucket,
func.count().label("cnt"), func.count().label("cnt"),
) )
.outerjoin(op_person, CsRecord.operator == op_person.c.operator) .outerjoin(op_person, CsRecordClean.operator == op_person.c.operator)
.where(CsRecord.operation_time.between(start, end), CsRecord.operator.isnot(None)) .where(CsRecordClean.operation_time.between(start, end), CsRecordClean.operator.isnot(None))
.group_by( .group_by(
CsRecord.operator, CsRecordClean.operator,
op_person.c.display_name, op_person.c.display_name,
op_person.c.group_name, op_person.c.group_name,
op_person.c.specialty, op_person.c.specialty,

View File

@@ -22,7 +22,6 @@ from typing import Any
from sqlalchemy import and_, func, select, text from sqlalchemy import and_, func, select, text
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from app.models.manual_refund import ManualRefund
from app.models.person import Person, PersonAlias from app.models.person import Person, PersonAlias
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -157,13 +156,13 @@ def _refund_recovery_core(db: Session, ts, te) -> dict:
if info["is_recovery_staff"]: if info["is_recovery_staff"]:
recovery_ops[op] = info recovery_ops[op] = info
# ---- 3. 分母SN (cs_records problem_category='退款') ---- # ---- 3. 分母SN (cs_record_clean problem_category='退款') ----
cs_refund_sn_sql = text( cs_refund_sn_sql = text(
""" """
SELECT c.operator AS op, c.device_sn AS sn, SELECT c.operator AS op, c.device_sn AS sn,
c.operation_time AS op_time, c.operation_time AS op_time,
c.problem_type AS problem_type c.problem_type AS problem_type
FROM cs_records c FROM cs_record_clean c
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.operation_time >= :ts AND c.operation_time <= :te AND c.operation_time >= :ts AND c.operation_time <= :te
AND NOT (c.operator = ANY(:sys_bl)) AND NOT (c.operator = ANY(:sys_bl))
@@ -265,7 +264,7 @@ def _refund_recovery_core(db: Session, ts, te) -> dict:
'cs_kw' AS src, LEFT(COALESCE(c.description,''),100) AS reason, 'cs_kw' AS src, LEFT(COALESCE(c.description,''),100) AS reason,
0 AS amount, '退款流程' AS status, 'cs_record关键词' AS method, 0 AS amount, '退款流程' AS status, 'cs_record关键词' AS method,
c.operation_time AS event_time c.operation_time AS event_time
FROM cs_records c FROM cs_record_clean c
JOIN host_sns hs ON c.device_sn = hs.sn JOIN host_sns hs ON c.device_sn = hs.sn
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.device_sn IS NOT NULL AND c.device_sn IS NOT NULL
@@ -455,7 +454,7 @@ def refund_personnel_analysis(
for r in db.execute(text( for r in db.execute(text(
""" """
SELECT c.operator AS op, MIN(c.operation_time) AS first_t, MAX(c.operation_time) AS last_t SELECT c.operator AS op, MIN(c.operation_time) AS first_t, MAX(c.operation_time) AS last_t
FROM cs_records c FROM cs_record_clean c
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.operation_time >= :ts AND c.operation_time <= :te AND c.operation_time >= :ts AND c.operation_time <= :te
AND NOT (c.operator = ANY(:sys_bl)) AND NOT (c.operator = ANY(:sys_bl))
@@ -536,7 +535,7 @@ def refund_person_devices(
SELECT c.device_sn AS sn, SELECT c.device_sn AS sn,
MIN(c.operation_time) AS first_t, MAX(c.operation_time) AS last_t, MIN(c.operation_time) AS first_t, MAX(c.operation_time) AS last_t,
COUNT(*) AS cnt COUNT(*) AS cnt
FROM cs_records c FROM cs_record_clean c
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.operation_time >= :ts AND c.operation_time <= :te AND c.operation_time >= :ts AND c.operation_time <= :te
AND c.operator = :op AND c.operator = :op
@@ -637,7 +636,7 @@ def refund_personnel_trend(
""" """
SELECT c.operator AS op, c.device_sn AS sn, SELECT c.operator AS op, c.device_sn AS sn,
DATE_TRUNC('week', c.operation_time AT TIME ZONE 'Asia/Shanghai')::date AS wk DATE_TRUNC('week', c.operation_time AT TIME ZONE 'Asia/Shanghai')::date AS wk
FROM cs_records c FROM cs_record_clean c
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.operation_time >= :ts AND c.operation_time <= :te AND c.operation_time >= :ts AND c.operation_time <= :te
AND NOT (c.operator = ANY(:sys_bl)) AND NOT (c.operator = ANY(:sys_bl))
@@ -655,7 +654,7 @@ def refund_personnel_trend(
""" """
SELECT c.operator AS op, c.device_sn AS sn, SELECT c.operator AS op, c.device_sn AS sn,
DATE(c.operation_time AT TIME ZONE 'Asia/Shanghai') AS d DATE(c.operation_time AT TIME ZONE 'Asia/Shanghai') AS d
FROM cs_records c FROM cs_record_clean c
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.operation_time >= :ts AND c.operation_time <= :te AND c.operation_time >= :ts AND c.operation_time <= :te
AND NOT (c.operator = ANY(:sys_bl)) AND NOT (c.operator = ANY(:sys_bl))
@@ -785,7 +784,7 @@ def refund_device_analysis(
for r in db.execute(text( for r in db.execute(text(
""" """
SELECT c.device_sn AS sn, COUNT(*) AS cnt SELECT c.device_sn AS sn, COUNT(*) AS cnt
FROM cs_records c FROM cs_record_clean c
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.operation_time >= :ts AND c.operation_time <= :te AND c.operation_time >= :ts AND c.operation_time <= :te
AND c.problem_category = '退款' AND c.problem_category = '退款'
@@ -834,7 +833,7 @@ def refund_device_analysis(
for r in db.execute(text( for r in db.execute(text(
""" """
SELECT c.device_sn AS sn, COUNT(DISTINCT c.operator) AS op_cnt SELECT c.device_sn AS sn, COUNT(DISTINCT c.operator) AS op_cnt
FROM cs_records c FROM cs_record_clean c
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.operation_time >= :ts AND c.operation_time <= :te AND c.operation_time >= :ts AND c.operation_time <= :te
AND c.problem_category = '退款' AND c.problem_category = '退款'
@@ -895,7 +894,7 @@ def refund_device_detail(
""" """
SELECT c.operation_time, c.operator, c.problem_type, c.description, SELECT c.operation_time, c.operator, c.problem_type, c.description,
c.execution_result, c.solution, c.consult_time c.execution_result, c.solution, c.consult_time
FROM cs_records c FROM cs_record_clean c
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.operation_time >= :ts AND c.operation_time <= :te AND c.operation_time >= :ts AND c.operation_time <= :te
AND c.device_sn = :sn AND c.device_sn = :sn
@@ -959,7 +958,7 @@ def refund_device_detail(
for r in db.execute(text( for r in db.execute(text(
""" """
SELECT w.id, w.op_time, w.op_amount, w.op_type SELECT w.id, w.op_time, w.op_amount, w.op_type
FROM wallet_withdrawals w FROM wallet_withdrawal_clean w
WHERE w.device_sn = :sn AND w.op_time IS NOT NULL WHERE w.device_sn = :sn AND w.op_time IS NOT NULL
ORDER BY w.op_time DESC LIMIT 10 ORDER BY w.op_time DESC LIMIT 10
""" """
@@ -1033,7 +1032,7 @@ def refund_type_trend(
f""" f"""
SELECT {date_col} AS d, c.device_sn AS sn, SELECT {date_col} AS d, c.device_sn AS sn,
c.problem_type, c.operator c.problem_type, c.operator
FROM cs_records c FROM cs_record_clean c
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.operation_time >= :ts AND c.operation_time <= :te AND c.operation_time >= :ts AND c.operation_time <= :te
AND NOT (c.operator = ANY(:sys_bl)) AND NOT (c.operator = ANY(:sys_bl))
@@ -1110,7 +1109,7 @@ def refund_decision_overview(
wallet_rows = db.execute(text(f""" wallet_rows = db.execute(text(f"""
SELECT {_day("w.op_time")} AS d, w.op_type, SELECT {_day("w.op_time")} AS d, w.op_type,
count(*) AS cnt, COALESCE(sum(w.op_amount), 0) AS amt count(*) AS cnt, COALESCE(sum(w.op_amount), 0) AS amt
FROM wallet_withdrawals w FROM wallet_withdrawal_clean w
WHERE w.op_time >= :ts AND w.op_time <= :te WHERE w.op_time >= :ts AND w.op_time <= :te
AND w.op_type IN ('订单退费转入', '用户提现', '系统扣费') AND w.op_type IN ('订单退费转入', '用户提现', '系统扣费')
AND {_withdraw_fail_remark_sql_cond()} AND {_withdraw_fail_remark_sql_cond()}
@@ -1323,7 +1322,7 @@ def refund_decision_overview(
wd_rows = db.execute(text(f""" wd_rows = db.execute(text(f"""
SELECT {_day("COALESCE(w.actual_refund_time, w.op_time)")} AS d, SELECT {_day("COALESCE(w.actual_refund_time, w.op_time)")} AS d,
count(*) AS cnt, COALESCE(sum(w.op_amount), 0) AS amt count(*) AS cnt, COALESCE(sum(w.op_amount), 0) AS amt
FROM wallet_withdrawals w FROM wallet_withdrawal_clean w
WHERE COALESCE(w.actual_refund_time, w.op_time) >= :ts AND COALESCE(w.actual_refund_time, w.op_time) <= :te AND w.op_type = '用户提现' WHERE COALESCE(w.actual_refund_time, w.op_time) >= :ts AND COALESCE(w.actual_refund_time, w.op_time) <= :te AND w.op_type = '用户提现'
AND {_withdraw_fail_remark_sql_cond()} AND {_withdraw_fail_remark_sql_cond()}
AND {stats_quality_excluded_sql('w.quality_status')} AND {stats_quality_excluded_sql('w.quality_status')}
@@ -1334,31 +1333,31 @@ def refund_decision_overview(
sm_rows = db.execute(text(f""" sm_rows = db.execute(text(f"""
SELECT {_day("COALESCE(s.refund_time, s.audit_time, s.created_at)")} AS d, SELECT {_day("COALESCE(s.refund_time, s.audit_time, s.created_at)")} AS d,
count(*) AS cnt, COALESCE(sum(COALESCE(s.actual_amount, s.refund_amount, 0)), 0) AS amt count(*) AS cnt, COALESCE(sum(COALESCE(s.actual_amount, s.refund_amount, 0)), 0) AS amt
FROM shanmao_refunds s FROM shanmao_refund_clean s
WHERE COALESCE(s.refund_time, s.audit_time, s.created_at) >= :ts WHERE COALESCE(s.refund_time, s.audit_time, s.created_at) >= :ts
AND COALESCE(s.refund_time, s.audit_time, s.created_at) <= :te AND COALESCE(s.refund_time, s.audit_time, s.created_at) <= :te
AND s.status = '已退款' AND s.status = '已退款'
AND {stats_quality_excluded_sql('s.quality_status')} AND {stats_quality_excluded_sql('s.quality_status')}
GROUP BY 1 GROUP BY 1
"""), {"ts": cur_s, "te": cur_e}).fetchall() """), {"ts": cur_s, "te": cur_e}).fetchall()
# D 手动退款 (manual_refunds, is_refunded='是'; 对账归日用实际扣款时间 actual_refund_time(2026-09-09 E回填), # D 手动退款 (manual_refund_clean, is_refunded='是'; 对账归日用实际扣款时间 actual_refund_time(2026-09-09 E回填),
# 缺失回退 op_time/register_time; 排除 dup_residual/ignored → 经营口径不与重复/忽略行虚高; A-20260909-001) # 缺失回退 op_time/register_time; 排除 dup_residual/ignored → 经营口径不与重复/忽略行虚高; A-20260909-001)
_d_excl_q = _quality_exclude_sql(db) _d_excl_q = _quality_exclude_sql(db)
d_rows = db.execute(text(f""" d_rows = db.execute(text(f"""
SELECT {_day("COALESCE(m.actual_refund_time, m.op_time, m.register_time)")} AS d, SELECT {_day("COALESCE(m.actual_refund_time, m.op_time, m.register_time)")} AS d,
count(*) AS cnt, COALESCE(sum(m.refund_amount), 0) AS amt count(*) AS cnt, COALESCE(sum(m.refund_amount), 0) AS amt
FROM manual_refunds m FROM manual_refund_clean m
WHERE COALESCE(m.actual_refund_time, m.op_time, m.register_time) >= :ts WHERE COALESCE(m.actual_refund_time, m.op_time, m.register_time) >= :ts
AND COALESCE(m.actual_refund_time, m.op_time, m.register_time) <= :te AND COALESCE(m.actual_refund_time, m.op_time, m.register_time) <= :te
AND m.is_refunded = '是' AND m.is_refunded = '是'
{"AND " + _EXCL_QSTATUS_SQL if _d_excl_q else ''} {"AND " + _EXCL_QSTATUS_SQL if _d_excl_q else ''}
GROUP BY 1 GROUP BY 1
"""), {"ts": cur_s, "te": cur_e}).fetchall() """), {"ts": cur_s, "te": cur_e}).fetchall()
# E 实际现金流 (refund_record_reports 退款成功, 对账基准, 独立展示不与四路累加) # E 实际现金流 (refund_record_report_clean 退款成功, 对账基准, 独立展示不与四路累加)
e_rows = db.execute(text(f""" e_rows = db.execute(text(f"""
SELECT {_day("e.refund_time")} AS d, SELECT {_day("e.refund_time")} AS d,
count(*) AS cnt, COALESCE(sum(e.refund_amount), 0) AS amt count(*) AS cnt, COALESCE(sum(e.refund_amount), 0) AS amt
FROM refund_record_reports e FROM refund_record_report_clean e
WHERE e.refund_time >= :ts AND e.refund_time <= :te WHERE e.refund_time >= :ts AND e.refund_time <= :te
AND e.refund_status IN ('退款成功', '成功', 'SUCCESS') AND e.refund_status IN ('退款成功', '成功', 'SUCCESS')
GROUP BY 1 GROUP BY 1
@@ -1415,7 +1414,7 @@ def refund_decision_overview(
# #201 S-P0-1:B 路窗口汇总补 dup_residual/ignored 排除 # #201 S-P0-1:B 路窗口汇总补 dup_residual/ignored 排除
for r in db.execute(text(f""" for r in db.execute(text(f"""
SELECT count(*) AS cnt, COALESCE(sum(op_amount), 0) AS amt SELECT count(*) AS cnt, COALESCE(sum(op_amount), 0) AS amt
FROM wallet_withdrawals w WHERE COALESCE(w.actual_refund_time, w.op_time) >= :ts FROM wallet_withdrawal_clean w WHERE COALESCE(w.actual_refund_time, w.op_time) >= :ts
AND COALESCE(w.actual_refund_time, w.op_time) <= :te AND w.op_type = '用户提现' AND COALESCE(w.actual_refund_time, w.op_time) <= :te AND w.op_type = '用户提现'
AND {_withdraw_fail_remark_sql_cond()} AND {_withdraw_fail_remark_sql_cond()}
AND {stats_quality_excluded_sql('w.quality_status')} AND {stats_quality_excluded_sql('w.quality_status')}
@@ -1424,7 +1423,7 @@ def refund_decision_overview(
# #201 S-P0-1:C 路窗口汇总补 dup_residual/ignored 排除 # #201 S-P0-1:C 路窗口汇总补 dup_residual/ignored 排除
for r in db.execute(text(f""" for r in db.execute(text(f"""
SELECT count(*) AS cnt, COALESCE(sum(COALESCE(actual_amount, refund_amount, 0)), 0) AS amt SELECT count(*) AS cnt, COALESCE(sum(COALESCE(actual_amount, refund_amount, 0)), 0) AS amt
FROM shanmao_refunds s FROM shanmao_refund_clean s
WHERE COALESCE(s.refund_time, s.audit_time, s.created_at) >= :ts WHERE COALESCE(s.refund_time, s.audit_time, s.created_at) >= :ts
AND COALESCE(s.refund_time, s.audit_time, s.created_at) <= :te AND COALESCE(s.refund_time, s.audit_time, s.created_at) <= :te
AND s.status = '已退款' AND s.status = '已退款'
@@ -1434,17 +1433,17 @@ def refund_decision_overview(
# D 手动退款 (is_refunded='是'; 归日=actual_refund_time 优先(实际扣款); 排除 dup_residual/ignored 已治理行) # D 手动退款 (is_refunded='是'; 归日=actual_refund_time 优先(实际扣款); 排除 dup_residual/ignored 已治理行)
for r in db.execute(text(f""" for r in db.execute(text(f"""
SELECT count(*) AS cnt, COALESCE(sum(refund_amount), 0) AS amt SELECT count(*) AS cnt, COALESCE(sum(refund_amount), 0) AS amt
FROM manual_refunds m FROM manual_refund_clean m
WHERE COALESCE(m.actual_refund_time, m.op_time, m.register_time) >= :ts WHERE COALESCE(m.actual_refund_time, m.op_time, m.register_time) >= :ts
AND COALESCE(m.actual_refund_time, m.op_time, m.register_time) <= :te AND COALESCE(m.actual_refund_time, m.op_time, m.register_time) <= :te
AND m.is_refunded = '是' AND m.is_refunded = '是'
{"AND " + _EXCL_QSTATUS_SQL if _d_excl_q else ''} {"AND " + _EXCL_QSTATUS_SQL if _d_excl_q else ''}
"""), {"ts": s, "te": e}).fetchall(): """), {"ts": s, "te": e}).fetchall():
tot["d_cnt"] += int(r.cnt or 0); tot["d_amt"] += D(r.amt) tot["d_cnt"] += int(r.cnt or 0); tot["d_amt"] += D(r.amt)
# E 实际现金流 (refund_record_reports 退款成功, 对账基准) # E 实际现金流 (refund_record_report_clean 退款成功, 对账基准)
for r in db.execute(text(""" for r in db.execute(text("""
SELECT count(*) AS cnt, COALESCE(sum(refund_amount), 0) AS amt SELECT count(*) AS cnt, COALESCE(sum(refund_amount), 0) AS amt
FROM refund_record_reports e FROM refund_record_report_clean e
WHERE e.refund_time >= :ts AND e.refund_time <= :te WHERE e.refund_time >= :ts AND e.refund_time <= :te
AND e.refund_status IN ('退款成功', '成功', 'SUCCESS') AND e.refund_status IN ('退款成功', '成功', 'SUCCESS')
"""), {"ts": s, "te": e}).fetchall(): """), {"ts": s, "te": e}).fetchall():
@@ -1475,14 +1474,14 @@ def refund_decision_overview(
mreg_rows = db.execute(text(f""" mreg_rows = db.execute(text(f"""
SELECT {_day("m.register_time")} AS d, SELECT {_day("m.register_time")} AS d,
count(*) AS cnt, COALESCE(sum(m.refund_amount), 0) AS amt count(*) AS cnt, COALESCE(sum(m.refund_amount), 0) AS amt
FROM manual_refunds m FROM manual_refund_clean m
WHERE m.register_time >= :ts AND m.register_time <= :te WHERE m.register_time >= :ts AND m.register_time <= :te
GROUP BY 1 ORDER BY 1 GROUP BY 1 ORDER BY 1
"""), {"ts": cur_s, "te": cur_e}).fetchall() """), {"ts": cur_s, "te": cur_e}).fetchall()
mdone_rows = db.execute(text(f""" mdone_rows = db.execute(text(f"""
SELECT {_day("COALESCE(m.op_time, m.register_time)")} AS d, SELECT {_day("COALESCE(m.op_time, m.register_time)")} AS d,
count(*) AS cnt, COALESCE(sum(m.refund_amount), 0) AS amt count(*) AS cnt, COALESCE(sum(m.refund_amount), 0) AS amt
FROM manual_refunds m FROM manual_refund_clean m
WHERE COALESCE(m.op_time, m.register_time) >= :ts AND COALESCE(m.op_time, m.register_time) <= :te WHERE COALESCE(m.op_time, m.register_time) >= :ts AND COALESCE(m.op_time, m.register_time) <= :te
AND m.is_refunded = '是' AND m.is_refunded = '是'
GROUP BY 1 ORDER BY 1 GROUP BY 1 ORDER BY 1
@@ -1504,10 +1503,10 @@ def refund_decision_overview(
manual_water["net"].append(float(round(net, 2))) manual_water["net"].append(float(round(net, 2)))
manual_water["cum_net"].append(float(round(m_cum, 2))) manual_water["cum_net"].append(float(round(m_cum, 2)))
# E 数据最新日(refund_record_reports 持续采集, 显示其最新退款日作参考) # E 数据最新日(refund_record_report_clean 持续采集, 显示其最新退款日作参考)
e_data_until = None e_data_until = None
e_max = db.execute(text( e_max = db.execute(text(
"SELECT max(refund_time) FROM refund_record_reports " "SELECT max(refund_time) FROM refund_record_report_clean "
"WHERE refund_status IN ('退款成功', '成功', 'SUCCESS')" "WHERE refund_status IN ('退款成功', '成功', 'SUCCESS')"
)).scalar() )).scalar()
if e_max is not None: if e_max is not None:
@@ -1585,12 +1584,12 @@ def refund_decision_overview(
+ m_note + m_note
) )
# ── #309 窗口充值(真实现金流充入): device_combos 按 created_at_raw 落窗, # ── #309 窗口充值(真实现金流充入): device_combo_clean 按 created_at_raw 落窗,
# 剔除工厂质检专用套餐流水(package_name 含「质检」), 与今日看板充值卡/ # 剔除工厂质检专用套餐流水(package_name 含「质检」), 与今日看板充值卡/
# 分品类运营看板 #293 同一判据。用于「退款 vs 充值」资金对照。 # 分品类运营看板 #293 同一判据。用于「退款 vs 充值」资金对照。
_rc = db.execute(text(""" _rc = db.execute(text("""
SELECT count(*) AS cnt, COALESCE(sum(dc.actual_amount), 0) AS amt SELECT count(*) AS cnt, COALESCE(sum(dc.actual_amount), 0) AS amt
FROM device_combos dc FROM device_combo_clean dc
WHERE dc.created_at_raw >= :ts AND dc.created_at_raw <= :te WHERE dc.created_at_raw >= :ts AND dc.created_at_raw <= :te
AND dc.package_name NOT ILIKE '%质检%' AND dc.package_name NOT ILIKE '%质检%'
"""), {"ts": cur_s, "te": cur_e}).first() """), {"ts": cur_s, "te": cur_e}).first()
@@ -1661,7 +1660,7 @@ def refund_decision_overview(
SELECT w.device_sn AS sn, w.op_amount AS amt, SELECT w.device_sn AS sn, w.op_amount AS amt,
COALESCE(w.actual_refund_time, w.op_time) AS ev, COALESCE(w.actual_refund_time, w.op_time) AS ev,
NULL::text AS pkg NULL::text AS pkg
FROM wallet_withdrawals w FROM wallet_withdrawal_clean w
WHERE COALESCE(w.actual_refund_time, w.op_time) >= :ts WHERE COALESCE(w.actual_refund_time, w.op_time) >= :ts
AND COALESCE(w.actual_refund_time, w.op_time) <= :te AND COALESCE(w.actual_refund_time, w.op_time) <= :te
AND w.op_type = '用户提现' AND w.op_type = '用户提现'
@@ -1670,12 +1669,12 @@ def refund_decision_overview(
"""), {"ts": cur_s, "te": cur_e}).fetchall(): """), {"ts": cur_s, "te": cur_e}).fetchall():
route_item_rows.append(("B 钱包提现", r.sn, D(r.amt), r.ev, r.pkg)) route_item_rows.append(("B 钱包提现", r.sn, D(r.amt), r.ev, r.pkg))
# C 闪猫退款: 已退款+排除已治理行; 金额=实际→申请; 归日=退款→审核→创建 # C 闪猫退款: 已退款+排除已治理行; 金额=实际→申请; 归日=退款→审核→创建
# #311 方案A: pkg=shanmao_refunds.package_name(闪猫侧套餐名) # #311 方案A: pkg=shanmao_refund_clean.package_name(闪猫侧套餐名)
for r in db.execute(text(f""" for r in db.execute(text(f"""
SELECT s.device_sn AS sn, COALESCE(s.actual_amount, s.refund_amount, 0) AS amt, SELECT s.device_sn AS sn, COALESCE(s.actual_amount, s.refund_amount, 0) AS amt,
COALESCE(s.refund_time, s.audit_time, s.created_at) AS ev, COALESCE(s.refund_time, s.audit_time, s.created_at) AS ev,
s.package_name AS pkg s.package_name AS pkg
FROM shanmao_refunds s FROM shanmao_refund_clean s
WHERE COALESCE(s.refund_time, s.audit_time, s.created_at) >= :ts WHERE COALESCE(s.refund_time, s.audit_time, s.created_at) >= :ts
AND COALESCE(s.refund_time, s.audit_time, s.created_at) <= :te AND COALESCE(s.refund_time, s.audit_time, s.created_at) <= :te
AND s.status = '已退款' AND s.status = '已退款'
@@ -1683,12 +1682,12 @@ def refund_decision_overview(
"""), {"ts": cur_s, "te": cur_e}).fetchall(): """), {"ts": cur_s, "te": cur_e}).fetchall():
route_item_rows.append(("C 闪猫退款", r.sn, D(r.amt), r.ev, r.pkg)) route_item_rows.append(("C 闪猫退款", r.sn, D(r.amt), r.ev, r.pkg))
# D 手动退款: is_refunded='是'+排除已治理行(_d_excl_q 见区块1); 归日=实际扣款→操作→登记 # D 手动退款: is_refunded='是'+排除已治理行(_d_excl_q 见区块1); 归日=实际扣款→操作→登记
# D 路 manual_refunds 无 package_name 字段(商户号后台退款台账无套餐), pkg=NULL, 走 dd_attrs.combo 回落 # D 路 manual_refund_clean 无 package_name 字段(商户号后台退款台账无套餐), pkg=NULL, 走 dd_attrs.combo 回落
for r in db.execute(text(f""" for r in db.execute(text(f"""
SELECT m.device_sn AS sn, m.refund_amount AS amt, SELECT m.device_sn AS sn, m.refund_amount AS amt,
COALESCE(m.actual_refund_time, m.op_time, m.register_time) AS ev, COALESCE(m.actual_refund_time, m.op_time, m.register_time) AS ev,
NULL::text AS pkg NULL::text AS pkg
FROM manual_refunds m FROM manual_refund_clean m
WHERE COALESCE(m.actual_refund_time, m.op_time, m.register_time) >= :ts WHERE COALESCE(m.actual_refund_time, m.op_time, m.register_time) >= :ts
AND COALESCE(m.actual_refund_time, m.op_time, m.register_time) <= :te AND COALESCE(m.actual_refund_time, m.op_time, m.register_time) <= :te
AND m.is_refunded = '是' AND m.is_refunded = '是'
@@ -1724,12 +1723,12 @@ def refund_decision_overview(
# devices 表外的少量 SN(dd_attrs 无行): 仍按 bms MIN(active_date) 并集兜底, 不误归「无激活记录」 # devices 表外的少量 SN(dd_attrs 无行): 仍按 bms MIN(active_date) 并集兜底, 不误归「无激活记录」
_no_dev = {sn for (_rk, sn, _a, _e, _p) in route_item_rows if sn and sn not in dd_attrs} _no_dev = {sn for (_rk, sn, _a, _e, _p) in route_item_rows if sn and sn not in dd_attrs}
if _no_dev: if _no_dev:
from app.models.bms_order import BmsOrder # 局部导入(与 _core 同法, 防循环依赖) from app.models.ods_clean import BmsOrderClean # 局部导入(与 _core 同法, 防循环依赖)
for sn, fa in db.execute( for sn, fa in db.execute(
select(BmsOrder.device_sn, func.min(BmsOrder.active_date)) select(BmsOrderClean.device_sn, func.min(BmsOrderClean.active_date))
.where(BmsOrder.device_sn.in_(_no_dev), BmsOrder.active_date.isnot(None)) .where(BmsOrderClean.device_sn.in_(_no_dev), BmsOrderClean.active_date.isnot(None))
.group_by(BmsOrder.device_sn) .group_by(BmsOrderClean.device_sn)
).all(): ).all():
if fa is not None: if fa is not None:
# active_date 为 timestamptz: naive 按 UTC 读入转上海; 与 _device_dim_attrs 同法 # active_date 为 timestamptz: naive 按 UTC 读入转上海; 与 _device_dim_attrs 同法
@@ -1862,90 +1861,6 @@ def refund_decision_overview(
} }
# ---------- 手动退款登记 数据质量审核 (L3: 自动纠错记录 + 待人工确认) ----------
def _mr_quality_out(r: ManualRefund) -> dict[str, Any]:
"""手动退款登记行 → 质量审核输出"""
def _sh(dt):
return dt.astimezone(TZ).strftime("%Y-%m-%d %H:%M:%S") if dt else None
return {
"id": r.id,
"register_time": _sh(r.register_time),
"op_time": _sh(r.op_time),
"is_refunded": r.is_refunded,
"refund_amount": float(r.refund_amount or 0),
"order_amount": float(r.order_amount or 0),
"device_sn": r.device_sn,
"order_no": r.order_no,
"merchant_id": r.merchant_id,
"registrant": r.registrant,
"refund_operator": r.refund_operator,
"remark": r.remark,
"quality_status": r.quality_status,
"quality_note": r.quality_note,
}
def manual_refund_quality_list(db: Session, status: str = "all", page: int = 1, page_size: int = 50) -> dict[str, Any]:
"""手动退款登记质量审核列表: status = all/pending/auto_fixed/ok
统计摘要: 各状态计数(供前端徽标)。列表按登记日期倒序。
"""
summary = {}
for st in ("pending", "auto_fixed", "ok"):
summary[st] = db.scalar(
select(func.count()).select_from(ManualRefund).where(ManualRefund.quality_status == st)
) or 0
q = select(ManualRefund)
if status in ("pending", "auto_fixed", "ok"):
q = q.where(ManualRefund.quality_status == status)
total = db.scalar(select(func.count()).select_from(q.subquery())) or 0
rows = db.execute(
q.order_by(ManualRefund.register_time.desc())
.offset((page - 1) * page_size).limit(page_size)
).scalars().all()
return {"total": total, "summary": summary,
"list": [_mr_quality_out(r) for r in rows]}
def manual_refund_quality_confirm(db: Session, rid: int) -> dict[str, Any]:
"""人工确认通过: quality_status → ok (保留纠错说明, 追加确认标记)"""
r = db.get(ManualRefund, rid)
if not r:
raise ValueError(f"手动退款记录不存在: id={rid}")
r.quality_status = "ok"
r.quality_note = ((r.quality_note or "") + " | 人工确认通过") if r.quality_note else "人工确认通过"
db.commit()
return {"id": rid, "quality_status": "ok"}
def manual_refund_quality_fix(db: Session, rid: int, payload: dict[str, Any]) -> dict[str, Any]:
"""人工修正: 更新指定字段(register_time/op_time/is_refunded/refund_amount/order_no/device_sn等),
修正后置 ok。仅接受白名单字段, 时间字段支持 'YYYY-MM-DD' / 'YYYY-MM-DD HH:MM:SS' / ISO 格式。
"""
r = db.get(ManualRefund, rid)
if not r:
raise ValueError(f"手动退款记录不存在: id={rid}")
allowed = {"register_time", "op_time", "is_refunded", "refund_amount", "order_amount",
"order_no", "device_sn", "merchant_id", "registrant", "refund_operator", "remark"}
changes = []
for k, v in (payload or {}).items():
if k not in allowed or v is None:
continue
if k in ("register_time", "op_time"):
setattr(r, k, _dt.datetime.fromisoformat(str(v)) if isinstance(v, str) and "T" in str(v)
else _dt.datetime.strptime(str(v), "%Y-%m-%d %H:%M:%S")
if isinstance(v, str) and len(str(v)) > 10
else _dt.datetime.strptime(str(v), "%Y-%m-%d"))
changes.append(k)
elif k in ("refund_amount", "order_amount"):
setattr(r, k, D(v))
changes.append(k)
else:
setattr(r, k, str(v))
changes.append(k)
r.quality_status = "ok"
r.quality_note = ((r.quality_note or "") + f" | 人工修正字段: {','.join(changes)}") if changes else (r.quality_note or "")
db.commit()
return {"id": rid, "quality_status": "ok", "changed": changes}
def refund_export_original( def refund_export_original(
db: Session, start: str | None = None, end: str | None = None, db: Session, start: str | None = None, end: str | None = None,
online_refunded_only: bool = False, online_refunded_only: bool = False,
@@ -2068,10 +1983,10 @@ def refund_export_all(
明细 sheet(2026-09-09 放宽为台账原始, 人工分析用, 不砍状态/类型行, 附状态列自筛): 明细 sheet(2026-09-09 放宽为台账原始, 人工分析用, 不砍状态/类型行, 附状态列自筛):
- A退费单明细(全量): refunds 全部(线上直退/线下/转入提现账户/未填写 + 驳回/无需退款/空白), 时间=created_at_raw - A退费单明细(全量): refunds 全部(线上直退/线下/转入提现账户/未填写 + 驳回/无需退款/空白), 时间=created_at_raw
- B钱包流水明细(全量): wallet_withdrawals 全部 op_type(用户提现/订单退费转入/系统扣费), 时间=op_time - B钱包流水明细(全量): wallet_withdrawal_clean 全部 op_type(用户提现/订单退费转入/系统扣费), 时间=op_time
- C闪猫退款明细(全量): shanmao_refunds 全部状态, 时间=COALESCE(audit_time, created_at) - C闪猫退款明细(全量): shanmao_refund_clean 全部状态, 时间=COALESCE(audit_time, created_at)
- D手动退款明细(全量): manual_refunds 全部(is_refunded 是/否), 时间=COALESCE(op_time, register_time) - D手动退款明细(全量): manual_refund_clean 全部(is_refunded 是/否), 时间=COALESCE(op_time, register_time)
- E退款记录报表(全量): refund_record_reports 全部 refund_status, 时间=refund_time - E退款记录报表(全量): refund_record_report_clean 全部 refund_status, 时间=refund_time
汇总 sheet(五路汇总(A-E))保持成功口径, 与 refund_decision_overview.routes_daily 一致: 汇总 sheet(五路汇总(A-E))保持成功口径, 与 refund_decision_overview.routes_daily 一致:
- A 线上直退: refund_method='线上' AND status='已退款' - A 线上直退: refund_method='线上' AND status='已退款'
@@ -2098,7 +2013,7 @@ def refund_export_all(
# 含失败/冻结备注; 附"操作类型/备注"列供人工筛出真实出款; 2026-09-09 放宽) # 含失败/冻结备注; 附"操作类型/备注"列供人工筛出真实出款; 2026-09-09 放宽)
b_rows = db.execute(text(""" b_rows = db.execute(text("""
SELECT account_id, device_sn, op_type, op_amount, wallet_balance, operator, op_time, remark SELECT account_id, device_sn, op_type, op_amount, wallet_balance, operator, op_time, remark
FROM wallet_withdrawals w FROM wallet_withdrawal_clean w
WHERE w.op_time >= :ts AND w.op_time <= :te WHERE w.op_time >= :ts AND w.op_time <= :te
ORDER BY w.op_time ORDER BY w.op_time
"""), {"ts": cur_s, "te": cur_e}).fetchall() """), {"ts": cur_s, "te": cur_e}).fetchall()
@@ -2118,7 +2033,7 @@ def refund_export_all(
SELECT refund_no, order_no, device_sn, phone, paid_amount, refund_amount, actual_amount, SELECT refund_no, order_no, device_sn, phone, paid_amount, refund_amount, actual_amount,
package_name, merchant_name, refund_reason, refund_method, status, package_name, merchant_name, refund_reason, refund_method, status,
applicant, auditor, operator, apply_time, audit_time, refund_time, remark, audit_remark applicant, auditor, operator, apply_time, audit_time, refund_time, remark, audit_remark
FROM shanmao_refunds FROM shanmao_refund_clean
WHERE COALESCE(audit_time, created_at) >= :ts AND COALESCE(audit_time, created_at) <= :te WHERE COALESCE(audit_time, created_at) >= :ts AND COALESCE(audit_time, created_at) <= :te
ORDER BY COALESCE(audit_time, created_at) ORDER BY COALESCE(audit_time, created_at)
"""), {"ts": cur_s, "te": cur_e}).fetchall() """), {"ts": cur_s, "te": cur_e}).fetchall()
@@ -2150,7 +2065,7 @@ def refund_export_all(
d_rows = db.execute(text(""" d_rows = db.execute(text("""
SELECT register_time, device_sn, order_no, merchant_id, order_amount, refund_amount, SELECT register_time, device_sn, order_no, merchant_id, order_amount, refund_amount,
registrant, is_refunded, refund_operator, op_time, remark, quality_status registrant, is_refunded, refund_operator, op_time, remark, quality_status
FROM manual_refunds FROM manual_refund_clean
WHERE COALESCE(op_time, register_time) >= :ts AND COALESCE(op_time, register_time) <= :te WHERE COALESCE(op_time, register_time) >= :ts AND COALESCE(op_time, register_time) <= :te
ORDER BY COALESCE(op_time, register_time) ORDER BY COALESCE(op_time, register_time)
"""), {"ts": cur_s, "te": cur_e}).fetchall() """), {"ts": cur_s, "te": cur_e}).fetchall()
@@ -2173,7 +2088,7 @@ def refund_export_all(
e_rows = db.execute(text(""" e_rows = db.execute(text("""
SELECT platform_order_no, wx_order_no, merchant_id, merchant_name, device_sn, SELECT platform_order_no, wx_order_no, merchant_id, merchant_name, device_sn,
product_name, refund_time, apply_refund_amount, refund_amount, refund_status product_name, refund_time, apply_refund_amount, refund_amount, refund_status
FROM refund_record_reports FROM refund_record_report_clean
WHERE refund_time >= :ts AND refund_time <= :te WHERE refund_time >= :ts AND refund_time <= :te
ORDER BY refund_time ORDER BY refund_time
"""), {"ts": cur_s, "te": cur_e}).fetchall() """), {"ts": cur_s, "te": cur_e}).fetchall()
@@ -2205,7 +2120,7 @@ def refund_export_all(
GROUP BY 1"""), GROUP BY 1"""),
("B 钱包提现", f""" ("B 钱包提现", f"""
SELECT {_day("COALESCE(w.actual_refund_time, w.op_time)")} AS d, count(*) AS cnt, COALESCE(sum(w.op_amount), 0) AS amt SELECT {_day("COALESCE(w.actual_refund_time, w.op_time)")} AS d, count(*) AS cnt, COALESCE(sum(w.op_amount), 0) AS amt
FROM wallet_withdrawals w WHERE COALESCE(w.actual_refund_time, w.op_time) >= :ts FROM wallet_withdrawal_clean w WHERE COALESCE(w.actual_refund_time, w.op_time) >= :ts
AND COALESCE(w.actual_refund_time, w.op_time) <= :te AND w.op_type = '用户提现' AND COALESCE(w.actual_refund_time, w.op_time) <= :te AND w.op_type = '用户提现'
AND {_withdraw_fail_remark_sql_cond()} AND {_withdraw_fail_remark_sql_cond()}
AND {stats_quality_excluded_sql('w.quality_status')} AND {stats_quality_excluded_sql('w.quality_status')}
@@ -2213,7 +2128,7 @@ def refund_export_all(
("C 闪猫退款", f""" ("C 闪猫退款", f"""
SELECT {_day("COALESCE(s.refund_time, s.audit_time, s.created_at)")} AS d, count(*) AS cnt, SELECT {_day("COALESCE(s.refund_time, s.audit_time, s.created_at)")} AS d, count(*) AS cnt,
COALESCE(sum(COALESCE(s.actual_amount, s.refund_amount, 0)), 0) AS amt COALESCE(sum(COALESCE(s.actual_amount, s.refund_amount, 0)), 0) AS amt
FROM shanmao_refunds s FROM shanmao_refund_clean s
WHERE COALESCE(s.refund_time, s.audit_time, s.created_at) >= :ts WHERE COALESCE(s.refund_time, s.audit_time, s.created_at) >= :ts
AND COALESCE(s.refund_time, s.audit_time, s.created_at) <= :te AND COALESCE(s.refund_time, s.audit_time, s.created_at) <= :te
AND s.status = '已退款' AND s.status = '已退款'
@@ -2222,7 +2137,7 @@ def refund_export_all(
("D 手动退款", f""" ("D 手动退款", f"""
SELECT {_day("COALESCE(m.actual_refund_time, m.op_time, m.register_time)")} AS d, count(*) AS cnt, SELECT {_day("COALESCE(m.actual_refund_time, m.op_time, m.register_time)")} AS d, count(*) AS cnt,
COALESCE(sum(m.refund_amount), 0) AS amt COALESCE(sum(m.refund_amount), 0) AS amt
FROM manual_refunds m FROM manual_refund_clean m
WHERE COALESCE(m.actual_refund_time, m.op_time, m.register_time) >= :ts WHERE COALESCE(m.actual_refund_time, m.op_time, m.register_time) >= :ts
AND COALESCE(m.actual_refund_time, m.op_time, m.register_time) <= :te AND COALESCE(m.actual_refund_time, m.op_time, m.register_time) <= :te
AND m.is_refunded = '是' AND m.is_refunded = '是'
@@ -2231,7 +2146,7 @@ def refund_export_all(
("E退款记录报表", f""" ("E退款记录报表", f"""
SELECT {_day("r.refund_time")} AS d, count(*) AS cnt, SELECT {_day("r.refund_time")} AS d, count(*) AS cnt,
COALESCE(sum(r.refund_amount), 0) AS amt COALESCE(sum(r.refund_amount), 0) AS amt
FROM refund_record_reports r FROM refund_record_report_clean r
WHERE r.refund_status IN ('退款成功','成功','SUCCESS') WHERE r.refund_status IN ('退款成功','成功','SUCCESS')
AND r.refund_time >= :ts AND r.refund_time <= :te AND r.refund_time >= :ts AND r.refund_time <= :te
GROUP BY 1"""), GROUP BY 1"""),
@@ -2280,7 +2195,7 @@ def refund_problem_type_detail(
for r in db.execute(text( for r in db.execute(text(
""" """
SELECT c.device_sn AS sn, c.problem_type SELECT c.device_sn AS sn, c.problem_type
FROM cs_records c FROM cs_record_clean c
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.operation_time >= :ts AND c.operation_time <= :te AND c.operation_time >= :ts AND c.operation_time <= :te
AND NOT (c.operator = ANY(:sys_bl)) AND NOT (c.operator = ANY(:sys_bl))
@@ -2501,7 +2416,7 @@ def refund_recovery_analysis(
"_note": "当前时间窗口未识别到人员数据, 请检查 Person 维表配置.", "_note": "当前时间窗口未识别到人员数据, 请检查 Person 维表配置.",
} }
# ---- 3. 分母: 所有人员 在周期内 cs_records 接待的【真退款类】 SN 集合 (按 op 分组) ---- # ---- 3. 分母: 所有人员 在周期内 cs_record_clean 接待的【真退款类】 SN 集合 (按 op 分组) ----
# 口径修正(2026-08-11): # 口径修正(2026-08-11):
# - problem_category = '退款' (大类是退款诉求) # - problem_category = '退款' (大类是退款诉求)
# - 不再限制operator必须是退款挽回专员: 任何人员(含组长/主管/退款处理员)接待的退款类SN都算入分母 # - 不再限制operator必须是退款挽回专员: 任何人员(含组长/主管/退款处理员)接待的退款类SN都算入分母
@@ -2512,7 +2427,7 @@ def refund_recovery_analysis(
SELECT c.operator AS op, c.device_sn AS sn, SELECT c.operator AS op, c.device_sn AS sn,
c.operation_time AS op_time, c.operation_time AS op_time,
c.problem_type AS problem_type c.problem_type AS problem_type
FROM cs_records c FROM cs_record_clean c
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.operation_time >= :ts AND c.operation_time <= :te AND c.operation_time >= :ts AND c.operation_time <= :te
AND NOT (c.operator = ANY(:sys_bl)) AND NOT (c.operator = ANY(:sys_bl))
@@ -2530,13 +2445,13 @@ def refund_recovery_analysis(
{"ts": ts, "te": te, "sys_bl": list(SYS_OP_BLACKLIST_RR), **kw_params}, {"ts": ts, "te": te, "sys_bl": list(SYS_OP_BLACKLIST_RR), **kw_params},
).fetchall() ).fetchall()
# 收集所有在cs_records中出现的operator,然后补全他们的meta信息 # 收集所有在cs_record_clean中出现的operator,然后补全他们的meta信息
all_cs_ops: set[str] = set() all_cs_ops: set[str] = set()
for r in cs_sn_rows: for r in cs_sn_rows:
if r.op: if r.op:
all_cs_ops.add(r.op) all_cs_ops.add(r.op)
# 将cs_records中出现但不在person_rows中的operator也加入all_ops_meta # 将cs_record_clean中出现但不在person_rows中的operator也加入all_ops_meta
person_op_map = {p.operator: p for p in person_rows} person_op_map = {p.operator: p for p in person_rows}
for op in all_cs_ops: for op in all_cs_ops:
if op not in all_ops_meta: if op not in all_ops_meta:
@@ -2596,7 +2511,7 @@ def refund_recovery_analysis(
# 修正说明: 之前只判断了全局时间窗口,没有校验"退款时间必须在接待时间之后",导致接待前已退款的SN被误判为失败 # 修正说明: 之前只判断了全局时间窗口,没有校验"退款时间必须在接待时间之后",导致接待前已退款的SN被误判为失败
# 说明: 允许专员接待后最长7天走到退款审批(到账/工单结案)才判定为失败, 避免"假成功" # 说明: 允许专员接待后最长7天走到退款审批(到账/工单结案)才判定为失败, 避免"假成功"
# 2026-08-11 修正: 增加第3个失败来源——cs_record描述关键词匹配 # 2026-08-11 修正: 增加第3个失败来源——cs_record描述关键词匹配
# 大量退款操作通过企微等渠道执行,记录在cs_records.description中(如"退款说明"/"退款后重新充值说明"等) # 大量退款操作通过企微等渠道执行,记录在cs_record_clean.description中(如"退款说明"/"退款后重新充值说明"等)
# 但没有同步到refunds表或service_tickets表,导致这些已进入退款流程的SN被误判为"挽回成功" # 但没有同步到refunds表或service_tickets表,导致这些已进入退款流程的SN被误判为"挽回成功"
# 新增cs_kw_sn分支: cs_record描述含退款流程关键词的SN直接判定为失败 # 新增cs_kw_sn分支: cs_record描述含退款流程关键词的SN直接判定为失败
import datetime as _dt2 import datetime as _dt2
@@ -2653,7 +2568,7 @@ def refund_recovery_analysis(
'退款流程' AS status, '退款流程' AS status,
'cs_record关键词' AS method, 'cs_record关键词' AS method,
c.operation_time AS event_time c.operation_time AS event_time
FROM cs_records c FROM cs_record_clean c
JOIN host_sns hs ON c.device_sn = hs.sn JOIN host_sns hs ON c.device_sn = hs.sn
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.device_sn IS NOT NULL AND c.device_sn IS NOT NULL
@@ -3064,7 +2979,7 @@ def refund_recovery_analysis(
""" """
SELECT DISTINCT c.device_sn AS sn, SELECT DISTINCT c.device_sn AS sn,
MIN(c.operation_time) AS first_op_time MIN(c.operation_time) AS first_op_time
FROM cs_records c FROM cs_record_clean c
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.operation_time >= :ts AND c.operation_time <= :te AND c.operation_time >= :ts AND c.operation_time <= :te
AND NOT (c.operator = ANY(:sys_bl)) AND NOT (c.operator = ANY(:sys_bl))
@@ -3235,7 +3150,7 @@ def refund_recovery_analysis(
""" """
SELECT DATE(c.operation_time AT TIME ZONE 'Asia/Shanghai') AS d, SELECT DATE(c.operation_time AT TIME ZONE 'Asia/Shanghai') AS d,
c.device_sn AS sn c.device_sn AS sn
FROM cs_records c FROM cs_record_clean c
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.operation_time >= :ts AND c.operation_time <= :te AND c.operation_time >= :ts AND c.operation_time <= :te
AND NOT (c.operator = ANY(:sys_bl)) AND NOT (c.operator = ANY(:sys_bl))
@@ -3274,7 +3189,7 @@ def refund_recovery_analysis(
AND st.device_sn IS NOT NULL AND st.device_sn IS NOT NULL
UNION UNION
SELECT c.device_sn AS sn SELECT c.device_sn AS sn
FROM cs_records c FROM cs_record_clean c
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.operation_time >= :ts AND c.operation_time <= :te7 AND c.operation_time >= :ts AND c.operation_time <= :te7
AND c.problem_category = '退款' AND c.problem_category = '退款'
@@ -3318,7 +3233,7 @@ def refund_recovery_analysis(
""" """
SELECT EXTRACT(HOUR FROM c.operation_time AT TIME ZONE 'Asia/Shanghai') AS hh, SELECT EXTRACT(HOUR FROM c.operation_time AT TIME ZONE 'Asia/Shanghai') AS hh,
c.device_sn AS sn c.device_sn AS sn
FROM cs_records c FROM cs_record_clean c
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.operation_time >= :ts AND c.operation_time <= :te AND c.operation_time >= :ts AND c.operation_time <= :te
AND NOT (c.operator = ANY(:sys_bl)) AND NOT (c.operator = ANY(:sys_bl))
@@ -3354,7 +3269,7 @@ def refund_recovery_analysis(
WHERE COALESCE(st.service_category,'') = '退款' AND COALESCE(st.status, '') != '终止' WHERE COALESCE(st.service_category,'') = '退款' AND COALESCE(st.status, '') != '终止'
UNION UNION
SELECT DISTINCT c.device_sn AS sn SELECT DISTINCT c.device_sn AS sn
FROM cs_records c JOIN arr a ON c.device_sn = a.sn FROM cs_record_clean c JOIN arr a ON c.device_sn = a.sn
WHERE c.problem_category = '退款' WHERE c.problem_category = '退款'
AND NOT (c.operator = ANY(:sys_bl)) AND NOT (c.operator = ANY(:sys_bl))
AND (COALESCE(c.description,'') ILIKE ANY(ARRAY[:ck1,:ck2,:ck3,:ck4]) AND (COALESCE(c.description,'') ILIKE ANY(ARRAY[:ck1,:ck2,:ck3,:ck4])

View File

@@ -7,9 +7,7 @@ from typing import Any
from sqlalchemy import func, select, text from sqlalchemy import func, select, text
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from app.models.manual_refund import ManualRefund from app.models.ods_clean import ManualRefundClean, RefundClean, WalletWithdrawalClean
from app.models.refund import RefundClean
from app.models.wallet import WalletWithdrawal
# 跨子模块引用(拆分自 app.services.analytics, 原同文件内直接调用) # 跨子模块引用(拆分自 app.services.analytics, 原同文件内直接调用)
from app.services.analytics._core import ( from app.services.analytics._core import (
@@ -48,9 +46,9 @@ def _build_chain_rows(db: Session, start: datetime | None = None, end: datetime
all_rf = list(db.execute(select(RefundClean)).scalars().all()) all_rf = list(db.execute(select(RefundClean)).scalars().all())
# 2026-08-28: 只取"成功提现"(排除 提现申请-冻结 等失败/冻结备注), 避免冻结记录被当作真实出款 # 2026-08-28: 只取"成功提现"(排除 提现申请-冻结 等失败/冻结备注), 避免冻结记录被当作真实出款
all_wd = list(db.execute( all_wd = list(db.execute(
select(WalletWithdrawal).where(*_route_b_withdraw_filter()) select(WalletWithdrawalClean).where(*_route_b_withdraw_filter())
).scalars().all()) ).scalars().all())
all_wi = list(db.execute(select(WalletWithdrawal).where(WalletWithdrawal.op_type == "订单退费转入")).scalars().all()) all_wi = list(db.execute(select(WalletWithdrawalClean).where(WalletWithdrawalClean.op_type == "订单退费转入")).scalars().all())
# 所有 datetime 先统一 naive, 避免 tz 比较报错 # 所有 datetime 先统一 naive, 避免 tz 比较报错
for r in all_rf: for r in all_rf:
@@ -206,8 +204,8 @@ def _build_chain_rows(db: Session, start: datetime | None = None, end: datetime
# 2026-09-09 (A-20260909-001): 扩展排除 ignored(人工彻底忽略) # 2026-09-09 (A-20260909-001): 扩展排除 ignored(人工彻底忽略)
# abc-a: 改用统一入口 _quality_include_cond(), 与 raw SQL 路径同口径 # abc-a: 改用统一入口 _quality_include_cond(), 与 raw SQL 路径同口径
all_mr = list(db.execute( all_mr = list(db.execute(
select(ManualRefund).where( select(ManualRefundClean).where(
ManualRefund.is_refunded == "是", ManualRefundClean.is_refunded == "是",
*_quality_include_cond(), *_quality_include_cond(),
) )
).scalars().all()) ).scalars().all())
@@ -629,7 +627,7 @@ def refund_withdrawal_sleeping(
ORDER BY created_at_raw ORDER BY created_at_raw
"""), {"ts": s, "te": e}).fetchall() """), {"ts": s, "te": e}).fetchall()
wd_rows = db.execute(text(f""" wd_rows = db.execute(text(f"""
SELECT remark, device_sn, op_amount, op_time FROM wallet_withdrawals w SELECT remark, device_sn, op_amount, op_time FROM wallet_withdrawal_clean w
WHERE w.op_type = '用户提现' AND {_withdraw_fail_remark_sql_cond()} WHERE w.op_type = '用户提现' AND {_withdraw_fail_remark_sql_cond()}
""")).fetchall() """)).fetchall()

View File

@@ -7,8 +7,7 @@ from typing import Callable as _Callable
from sqlalchemy import and_, case, func, literal, select, text, union from sqlalchemy import and_, case, func, literal, select, text, union
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from app.models.card_complaint import CardComplaint from app.models.ods_clean import CardComplaintClean, ExternalComplaintClean
from app.models.external_complaint import ExternalComplaint
from app.models.ticket import ServiceTicket from app.models.ticket import ServiceTicket
# 跨子模块引用(拆分自 app.services.analytics, 原同文件内直接调用) # 跨子模块引用(拆分自 app.services.analytics, 原同文件内直接调用)
@@ -506,29 +505,29 @@ def _view_complaint_root_heatmap(db: Session, params: _PARAMS_T) -> _RET_T:
_ticket_category_case() == "投诉", _ticket_category_case() == "投诉",
), ),
select( select(
ExternalComplaint.device_sn.label("sn"), ExternalComplaintClean.device_sn.label("sn"),
ExternalComplaint.complainant_phone.label("phone"), ExternalComplaintClean.complainant_phone.label("phone"),
func.coalesce(ExternalComplaint.complaint_category, literal("外部投诉")).label("sub"), func.coalesce(ExternalComplaintClean.complaint_category, literal("外部投诉")).label("sub"),
ExternalComplaint.complaint_date.label("rt"), ExternalComplaintClean.complaint_date.label("rt"),
).where( ).where(
ExternalComplaint.complaint_date >= ts - timedelta(days=1), ExternalComplaintClean.complaint_date >= ts - timedelta(days=1),
ExternalComplaint.complaint_date <= te + timedelta(days=1), ExternalComplaintClean.complaint_date <= te + timedelta(days=1),
), ),
select( select(
CardComplaint.device_sn.label("sn"), CardComplaintClean.device_sn.label("sn"),
CardComplaint.complainant_phone.label("phone"), CardComplaintClean.complainant_phone.label("phone"),
func.coalesce(CardComplaint.problem_category, literal("卡商投诉")).label("sub"), func.coalesce(CardComplaintClean.problem_category, literal("卡商投诉")).label("sub"),
CardComplaint.complaint_date.label("rt"), CardComplaintClean.complaint_date.label("rt"),
).where( ).where(
CardComplaint.complaint_date >= ts - timedelta(days=1), CardComplaintClean.complaint_date >= ts - timedelta(days=1),
CardComplaint.complaint_date <= te + timedelta(days=1), CardComplaintClean.complaint_date <= te + timedelta(days=1),
), ),
) )
# 再建一级 CTE 便于 JOIN # 再建一级 CTE 便于 JOIN
u = select(union_subq.c.sn, union_subq.c.phone, union_subq.c.sub, union_subq.c.rt).cte("u") u = select(union_subq.c.sn, union_subq.c.phone, union_subq.c.sub, union_subq.c.rt).cte("u")
# ±24h 窗口 JOIN cs_record, 拿根因分类 # ±24h 窗口 JOIN cs_record, 拿根因分类
from app.models.cs_record import CsRecord as _CsR from app.models.ods_clean import CsRecordClean as _CsR
stmt = ( stmt = (
select( select(
u.c.sub.label("投诉子类"), u.c.sub.label("投诉子类"),
@@ -668,20 +667,20 @@ def _view_today_phone_dup(db: Session, params: _PARAMS_T) -> _RET_T:
_ticket_category_case() == "投诉", _ticket_category_case() == "投诉",
) )
comp_ec = select( comp_ec = select(
_clean_col_expr(ExternalComplaint.complainant_phone).label("phone_c"), _clean_col_expr(ExternalComplaintClean.complainant_phone).label("phone_c"),
func.coalesce(ExternalComplaint.complaint_category, literal("外部投诉")).label("sub"), func.coalesce(ExternalComplaintClean.complaint_category, literal("外部投诉")).label("sub"),
ExternalComplaint.device_sn.label("sn_c"), ExternalComplaintClean.device_sn.label("sn_c"),
literal("EC").label("src"), literal("EC").label("src"),
).where( ).where(
ExternalComplaint.complaint_date >= ts, ExternalComplaint.complaint_date <= te, ExternalComplaintClean.complaint_date >= ts, ExternalComplaintClean.complaint_date <= te,
) )
comp_cc = select( comp_cc = select(
_clean_col_expr(CardComplaint.complainant_phone).label("phone_c"), _clean_col_expr(CardComplaintClean.complainant_phone).label("phone_c"),
func.coalesce(CardComplaint.problem_category, literal("卡商投诉")).label("sub"), func.coalesce(CardComplaintClean.problem_category, literal("卡商投诉")).label("sub"),
CardComplaint.device_sn.label("sn_c"), CardComplaintClean.device_sn.label("sn_c"),
literal("CC").label("src"), literal("CC").label("src"),
).where( ).where(
CardComplaint.complaint_date >= ts, CardComplaint.complaint_date <= te, CardComplaintClean.complaint_date >= ts, CardComplaintClean.complaint_date <= te,
) )
comp_u = union(comp_st, comp_ec, comp_cc).cte("u") comp_u = union(comp_st, comp_ec, comp_cc).cte("u")
# 全部工单 (非投诉也统计) 用于总进线次数 # 全部工单 (非投诉也统计) 用于总进线次数

View File

@@ -15,20 +15,20 @@ from typing import Any
from sqlalchemy import and_, case, func, literal, or_, select, text, union from sqlalchemy import and_, case, func, literal, or_, select, text, union
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from app.models.card_complaint import CardComplaint from app.models.ods_clean import CardComplaintClean
from app.models.cs_record import CsRecord from app.models.ods_clean import CsRecordClean
from app.models.device_combo import DeviceCombo from app.models.ods_clean import DeviceComboClean
from app.models.external_complaint import ExternalComplaint from app.models.ods_clean import ExternalComplaintClean
from app.models.group import Group, PersonGroupMembership from app.models.group import Group, PersonGroupMembership
from app.models.incident import Incident from app.models.incident import Incident
from app.models.person import Person, PersonAlias from app.models.person import Person, PersonAlias
from app.models.refund import RefundClean from app.models.ods_clean import RefundClean
from app.models.refund_record_report import RefundRecordReport from app.models.ods_clean import RefundRecordReportClean
from app.models.specialty import PersonSpecialtyAssignment, Specialty from app.models.specialty import PersonSpecialtyAssignment, Specialty
from app.models.task import SpecialTask from app.models.task import SpecialTask
from app.models.ticket import ServiceTicket from app.models.ticket import ServiceTicket
from app.models.todo import TodoItem from app.models.todo import TodoItem
from app.models.wallet import WalletWithdrawal from app.models.ods_clean import WalletWithdrawalClean
from app.services.complaint_scope import ( from app.services.complaint_scope import (
ALL_BUCKETS, ALL_BUCKETS,
BUCKET_CARD, BUCKET_CARD,
@@ -103,15 +103,15 @@ def today_kpi(db: Session, date: str | None = None, start: str | None = None, en
return and_(col.isnot(None), col >= s, col <= e) return and_(col.isnot(None), col >= s, col <= e)
# 1. 今日接待量 (cs_records.operation_time) # 1. 今日接待量 (cs_records.operation_time)
cs_today = db.execute(select(func.count()).where(_between(CsRecord.operation_time, ts, te))).scalar() or 0 cs_today = db.execute(select(func.count()).where(_between(CsRecordClean.operation_time, ts, te))).scalar() or 0
cs_yoy = db.execute(select(func.count()).where(_between(CsRecord.operation_time, ys, ye))).scalar() or 0 cs_yoy = db.execute(select(func.count()).where(_between(CsRecordClean.operation_time, ys, ye))).scalar() or 0
# 1b. 今日服务设备数 (cs_records.device_sn 去重, 排除 NULL/空串) — 补充口径:1台设备多次进线算1次,体现覆盖面 # 1b. 今日服务设备数 (cs_records.device_sn 去重, 排除 NULL/空串) — 补充口径:1台设备多次进线算1次,体现覆盖面
cs_device_today = db.execute(select( cs_device_today = db.execute(select(
func.count(func.distinct(func.nullif(func.trim(CsRecord.device_sn), ""))) func.count(func.distinct(func.nullif(func.trim(CsRecordClean.device_sn), "")))
).where(_between(CsRecord.operation_time, ts, te))).scalar() or 0 ).where(_between(CsRecordClean.operation_time, ts, te))).scalar() or 0
cs_device_yoy = db.execute(select( cs_device_yoy = db.execute(select(
func.count(func.distinct(func.nullif(func.trim(CsRecord.device_sn), ""))) func.count(func.distinct(func.nullif(func.trim(CsRecordClean.device_sn), "")))
).where(_between(CsRecord.operation_time, ys, ye))).scalar() or 0 ).where(_between(CsRecordClean.operation_time, ys, ye))).scalar() or 0
# 1c. ★ #78 接待来源分层(真人 / AI全自动 / 系统 / 客户自助) # 1c. ★ #78 接待来源分层(真人 / AI全自动 / 系统 / 客户自助)
# 设计说明(Design Rationale): # 设计说明(Design Rationale):
@@ -129,22 +129,22 @@ def today_kpi(db: Session, date: str | None = None, start: str | None = None, en
# 有操作人且非系统账户 → 真人(含人工点 AI 助手, 场景 1/2/4) # 有操作人且非系统账户 → 真人(含人工点 AI 助手, 场景 1/2/4)
( (
and_( and_(
CsRecord.operator.isnot(None), CsRecordClean.operator.isnot(None),
func.trim(CsRecord.operator) != "", func.trim(CsRecordClean.operator) != "",
~func.lower(func.trim(CsRecord.operator)).in_( ~func.lower(func.trim(CsRecordClean.operator)).in_(
["wechat", "sys", "system", "用户自助", "客户自助", "客户", "系统"] ["wechat", "sys", "system", "用户自助", "客户自助", "客户", "系统"]
), ),
), ),
SourceKind.HUMAN.value, SourceKind.HUMAN.value,
), ),
(func.lower(func.trim(func.coalesce(CsRecord.operator, ""))) == "wechat", SourceKind.AI_AUTO.value), (func.lower(func.trim(func.coalesce(CsRecordClean.operator, ""))) == "wechat", SourceKind.AI_AUTO.value),
# 客户自助: 空操作人 + description 以 openid 开头(≥20 位字母数字) # 客户自助: 空操作人 + description 以 openid 开头(≥20 位字母数字)
# 注: PostgreSQL 正则匹配符 `~` 在 SQLAlchemy 需用 .op("~")(pattern) 表达 # 注: PostgreSQL 正则匹配符 `~` 在 SQLAlchemy 需用 .op("~")(pattern) 表达
# (Python 的 `~` 是一元按位取反, 直接写 a ~ b 会语法错误) # (Python 的 `~` 是一元按位取反, 直接写 a ~ b 会语法错误)
( (
and_( and_(
func.coalesce(func.trim(CsRecord.operator), "") == "", func.coalesce(func.trim(CsRecordClean.operator), "") == "",
func.trim(func.coalesce(CsRecord.description, "")).op("~")("^[A-Za-z0-9_-]{20,}"), func.trim(func.coalesce(CsRecordClean.description, "")).op("~")("^[A-Za-z0-9_-]{20,}"),
), ),
SourceKind.CUSTOMER_SELF.value, SourceKind.CUSTOMER_SELF.value,
), ),
@@ -155,13 +155,13 @@ def today_kpi(db: Session, date: str | None = None, start: str | None = None, en
select( select(
_cs_kind_case, _cs_kind_case,
func.count().label("sessions"), func.count().label("sessions"),
func.count(func.distinct(func.nullif(func.trim(CsRecord.device_sn), ""))).label("devices"), func.count(func.distinct(func.nullif(func.trim(CsRecordClean.device_sn), ""))).label("devices"),
func.count(func.distinct( func.count(func.distinct(
func.concat(func.coalesce(CsRecord.device_sn, ""), "|", func.concat(func.coalesce(CsRecordClean.device_sn, ""), "|",
func.to_char(CsRecord.operation_time, "YYYY-MM-DD HH24:MI:SS")) func.to_char(CsRecordClean.operation_time, "YYYY-MM-DD HH24:MI:SS"))
)).label("events"), )).label("events"),
) )
.where(_between(CsRecord.operation_time, ts, te)) .where(_between(CsRecordClean.operation_time, ts, te))
.group_by(_cs_kind_case) .group_by(_cs_kind_case)
).all() ).all()
cs_breakdown: dict[str, dict[str, int]] = { cs_breakdown: dict[str, dict[str, int]] = {
@@ -177,26 +177,26 @@ def today_kpi(db: Session, date: str | None = None, start: str | None = None, en
# AI 接管率: 会话口径(全量占比) + 设备口径(AI 独有 SN 占比) # AI 接管率: 会话口径(全量占比) + 设备口径(AI 独有 SN 占比)
_ai_sessions = cs_breakdown[SourceKind.AI_AUTO.value]["sessions"] _ai_sessions = cs_breakdown[SourceKind.AI_AUTO.value]["sessions"]
_human_sn_rows = db.execute( _human_sn_rows = db.execute(
select(func.count(func.distinct(func.nullif(func.trim(CsRecord.device_sn), "")))) select(func.count(func.distinct(func.nullif(func.trim(CsRecordClean.device_sn), ""))))
.where(_between(CsRecord.operation_time, ts, te), .where(_between(CsRecordClean.operation_time, ts, te),
func.lower(func.trim(func.coalesce(CsRecord.operator, ""))) != "wechat", func.lower(func.trim(func.coalesce(CsRecordClean.operator, ""))) != "wechat",
func.coalesce(func.trim(CsRecord.operator), "") != "") func.coalesce(func.trim(CsRecordClean.operator), "") != "")
).scalar() or 0 ).scalar() or 0
_ai_sn_rows = db.execute( _ai_sn_rows = db.execute(
select(func.count(func.distinct(func.nullif(func.trim(CsRecord.device_sn), "")))) select(func.count(func.distinct(func.nullif(func.trim(CsRecordClean.device_sn), ""))))
.where(_between(CsRecord.operation_time, ts, te), .where(_between(CsRecordClean.operation_time, ts, te),
func.lower(func.trim(func.coalesce(CsRecord.operator, ""))) == "wechat") func.lower(func.trim(func.coalesce(CsRecordClean.operator, ""))) == "wechat")
).scalar() or 0 ).scalar() or 0
_overlap = db.execute( _overlap = db.execute(
select(func.count()).select_from( select(func.count()).select_from(
select(func.nullif(func.trim(CsRecord.device_sn), "").label("sn")) select(func.nullif(func.trim(CsRecordClean.device_sn), "").label("sn"))
.where(_between(CsRecord.operation_time, ts, te)) .where(_between(CsRecordClean.operation_time, ts, te))
.group_by(func.nullif(func.trim(CsRecord.device_sn), "")) .group_by(func.nullif(func.trim(CsRecordClean.device_sn), ""))
.having(and_( .having(and_(
func.count().filter(func.lower(func.trim(func.coalesce(CsRecord.operator, ""))) == "wechat") > 0, func.count().filter(func.lower(func.trim(func.coalesce(CsRecordClean.operator, ""))) == "wechat") > 0,
func.count().filter(and_( func.count().filter(and_(
func.lower(func.trim(func.coalesce(CsRecord.operator, ""))) != "wechat", func.lower(func.trim(func.coalesce(CsRecordClean.operator, ""))) != "wechat",
func.coalesce(func.trim(CsRecord.operator), "") != "", func.coalesce(func.trim(CsRecordClean.operator), "") != "",
)) > 0, )) > 0,
)).subquery() )).subquery()
) )
@@ -261,24 +261,24 @@ def today_kpi(db: Session, date: str | None = None, start: str | None = None, en
done_y, real_done_y = _ticket_effective_finish_time_where(db, ys, ye) done_y, real_done_y = _ticket_effective_finish_time_where(db, ys, ye)
# 6. 今日在岗人数 (cs_records distinct operator) # 6. 今日在岗人数 (cs_records distinct operator)
staff_t = db.execute(select(func.count(func.distinct(CsRecord.operator))) staff_t = db.execute(select(func.count(func.distinct(CsRecordClean.operator)))
.where(_between(CsRecord.operation_time, ts, te), .where(_between(CsRecordClean.operation_time, ts, te),
CsRecord.operator.isnot(None))).scalar() or 0 CsRecordClean.operator.isnot(None))).scalar() or 0
staff_y = db.execute(select(func.count(func.distinct(CsRecord.operator))) staff_y = db.execute(select(func.count(func.distinct(CsRecordClean.operator)))
.where(_between(CsRecord.operation_time, ys, ye), .where(_between(CsRecordClean.operation_time, ys, ye),
CsRecord.operator.isnot(None))).scalar() or 0 CsRecordClean.operator.isnot(None))).scalar() or 0
# 6b. ★ #309 窗口充值(真实现金流充入): device_combos 按 created_at_raw 落窗, # 6b. ★ #309 窗口充值(真实现金流充入): device_combos 按 created_at_raw 落窗,
# 剔除工厂质检专用套餐流水(package_name ILIKE '%质检%') —— 与分品类运营看板 # 剔除工厂质检专用套餐流水(package_name ILIKE '%质检%') —— 与分品类运营看板
# #293 同判据(出厂质检单笔 999/9999 元非真实营收, 2026-10 曾占当期流水 57.5%)。 # #293 同判据(出厂质检单笔 999/9999 元非真实营收, 2026-10 曾占当期流水 57.5%)。
# 环比 = 上一等长窗口(ys~ye), 与其他 KPI 卡同一对比语义。 # 环比 = 上一等长窗口(ys~ye), 与其他 KPI 卡同一对比语义。
_not_qc = DeviceCombo.package_name.notilike("%质检%") _not_qc = DeviceComboClean.package_name.notilike("%质检%")
_rc_t = db.execute(select( _rc_t = db.execute(select(
func.count(), func.coalesce(func.sum(DeviceCombo.actual_amount), 0), func.count(), func.coalesce(func.sum(DeviceComboClean.actual_amount), 0),
).where(_between(DeviceCombo.created_at_raw, ts, te), _not_qc)).first() ).where(_between(DeviceComboClean.created_at_raw, ts, te), _not_qc)).first()
_rc_y = db.execute(select( _rc_y = db.execute(select(
func.count(), func.coalesce(func.sum(DeviceCombo.actual_amount), 0), func.count(), func.coalesce(func.sum(DeviceComboClean.actual_amount), 0),
).where(_between(DeviceCombo.created_at_raw, ys, ye), _not_qc)).first() ).where(_between(DeviceComboClean.created_at_raw, ys, ye), _not_qc)).first()
rc_cnt_t, rc_amt_t = int(_rc_t[0] or 0), round(float(_rc_t[1] or 0), 2) rc_cnt_t, rc_amt_t = int(_rc_t[0] or 0), round(float(_rc_t[1] or 0), 2)
rc_cnt_y, rc_amt_y = int(_rc_y[0] or 0), round(float(_rc_y[1] or 0), 2) rc_cnt_y, rc_amt_y = int(_rc_y[0] or 0), round(float(_rc_y[1] or 0), 2)

View File

@@ -15,19 +15,19 @@ from typing import Any
from sqlalchemy import and_, case, func, literal, or_, select, text, union from sqlalchemy import and_, case, func, literal, or_, select, text, union
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from app.models.card_complaint import CardComplaint from app.models.ods_clean import CardComplaintClean
from app.models.cs_record import CsRecord from app.models.ods_clean import CsRecordClean
from app.models.external_complaint import ExternalComplaint from app.models.ods_clean import ExternalComplaintClean
from app.models.group import Group, PersonGroupMembership from app.models.group import Group, PersonGroupMembership
from app.models.incident import Incident from app.models.incident import Incident
from app.models.person import Person, PersonAlias from app.models.person import Person, PersonAlias
from app.models.refund import RefundClean from app.models.ods_clean import RefundClean
from app.models.refund_record_report import RefundRecordReport from app.models.ods_clean import RefundRecordReportClean
from app.models.specialty import PersonSpecialtyAssignment, Specialty from app.models.specialty import PersonSpecialtyAssignment, Specialty
from app.models.task import SpecialTask from app.models.task import SpecialTask
from app.models.ticket import ServiceTicket from app.models.ticket import ServiceTicket
from app.models.todo import TodoItem from app.models.todo import TodoItem
from app.models.wallet import WalletWithdrawal from app.models.ods_clean import WalletWithdrawalClean
from app.services.complaint_scope import ( from app.services.complaint_scope import (
ALL_BUCKETS, ALL_BUCKETS,
BUCKET_CARD, BUCKET_CARD,

View File

@@ -15,19 +15,19 @@ from typing import Any
from sqlalchemy import and_, case, func, literal, or_, select, text, union from sqlalchemy import and_, case, func, literal, or_, select, text, union
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from app.models.card_complaint import CardComplaint from app.models.ods_clean import CardComplaintClean
from app.models.cs_record import CsRecord from app.models.ods_clean import CsRecordClean
from app.models.external_complaint import ExternalComplaint from app.models.ods_clean import ExternalComplaintClean
from app.models.group import Group, PersonGroupMembership from app.models.group import Group, PersonGroupMembership
from app.models.incident import Incident from app.models.incident import Incident
from app.models.person import Person, PersonAlias from app.models.person import Person, PersonAlias
from app.models.refund import RefundClean from app.models.ods_clean import RefundClean
from app.models.refund_record_report import RefundRecordReport from app.models.ods_clean import RefundRecordReportClean
from app.models.specialty import PersonSpecialtyAssignment, Specialty from app.models.specialty import PersonSpecialtyAssignment, Specialty
from app.models.task import SpecialTask from app.models.task import SpecialTask
from app.models.ticket import ServiceTicket from app.models.ticket import ServiceTicket
from app.models.todo import TodoItem from app.models.todo import TodoItem
from app.models.wallet import WalletWithdrawal from app.models.ods_clean import WalletWithdrawalClean
from app.services.complaint_scope import ( from app.services.complaint_scope import (
ALL_BUCKETS, ALL_BUCKETS,
BUCKET_CARD, BUCKET_CARD,
@@ -90,8 +90,8 @@ def today_reception_breakdown(db: Session, date: str | None = None, start: str |
else: else:
d_iso = (_dt.datetime.now(TZ).date() if date is None else _parse_date(date)).isoformat() d_iso = (_dt.datetime.now(TZ).date() if date is None else _parse_date(date)).isoformat()
d_end_iso = None d_end_iso = None
between = and_(CsRecord.operation_time.isnot(None), between = and_(CsRecordClean.operation_time.isnot(None),
CsRecord.operation_time >= ts, CsRecord.operation_time <= te) CsRecordClean.operation_time >= ts, CsRecordClean.operation_time <= te)
tk_between = and_(ServiceTicket.register_time.isnot(None), tk_between = and_(ServiceTicket.register_time.isnot(None),
ServiceTicket.register_time >= ts, ServiceTicket.register_time <= te) ServiceTicket.register_time >= ts, ServiceTicket.register_time <= te)
@@ -104,7 +104,7 @@ def today_reception_breakdown(db: Session, date: str | None = None, start: str |
.group_by("category_bucket") .group_by("category_bucket")
).all() ).all()
cat_device_rows = db.execute( cat_device_rows = db.execute(
select(bucket, func.count(func.distinct(func.nullif(func.trim(CsRecord.device_sn), ""))).label("dcnt")) select(bucket, func.count(func.distinct(func.nullif(func.trim(CsRecordClean.device_sn), ""))).label("dcnt"))
.where(between) .where(between)
.group_by("category_bucket") .group_by("category_bucket")
).all() ).all()
@@ -133,15 +133,15 @@ def today_reception_breakdown(db: Session, date: str | None = None, start: str |
op_person = _operator_to_person_cte() op_person = _operator_to_person_cte()
top_rows = db.execute( top_rows = db.execute(
select( select(
CsRecord.operator.label("raw_op"), CsRecordClean.operator.label("raw_op"),
op_person.c.display_name.label("display_name"), op_person.c.display_name.label("display_name"),
op_person.c.department.label("department"), op_person.c.department.label("department"),
op_person.c.group_name.label("group_name"), op_person.c.group_name.label("group_name"),
func.count().label("cnt"), func.count().label("cnt"),
func.count(func.distinct(func.nullif(func.trim(CsRecord.device_sn), ""))).label("dcnt"), func.count(func.distinct(func.nullif(func.trim(CsRecordClean.device_sn), ""))).label("dcnt"),
) )
.outerjoin(op_person, CsRecord.operator == op_person.c.operator) .outerjoin(op_person, CsRecordClean.operator == op_person.c.operator)
.where(between, CsRecord.operator.isnot(None)) .where(between, CsRecordClean.operator.isnot(None))
.group_by("raw_op", op_person.c.display_name, op_person.c.department, op_person.c.group_name) .group_by("raw_op", op_person.c.display_name, op_person.c.department, op_person.c.group_name)
.order_by(func.count().desc()) .order_by(func.count().desc())
.limit(10) .limit(10)
@@ -179,13 +179,13 @@ def today_reception_breakdown(db: Session, date: str | None = None, start: str |
# 叠加 external_complaints 今日 (按 complaint_category 聚合,未分类则归入 "外部投诉(监督局/315等)") # 叠加 external_complaints 今日 (按 complaint_category 聚合,未分类则归入 "外部投诉(监督局/315等)")
# 2026-08-14: _INCLUDE_COMPLAINT_FILES=False 时跳过, ec_between=literal(False) 使下游查询返回空 # 2026-08-14: _INCLUDE_COMPLAINT_FILES=False 时跳过, ec_between=literal(False) 使下游查询返回空
if _INCLUDE_COMPLAINT_FILES: if _INCLUDE_COMPLAINT_FILES:
ec_between = and_(ExternalComplaint.complaint_date.isnot(None), ec_between = and_(ExternalComplaintClean.complaint_date.isnot(None),
ExternalComplaint.complaint_date >= ts, ExternalComplaint.complaint_date <= te) ExternalComplaintClean.complaint_date >= ts, ExternalComplaintClean.complaint_date <= te)
ec_rows = db.execute( ec_rows = db.execute(
select(func.coalesce(ExternalComplaint.complaint_category, "外部投诉(监督局/315等)"), select(func.coalesce(ExternalComplaintClean.complaint_category, "外部投诉(监督局/315等)"),
func.count().label("cnt")) func.count().label("cnt"))
.where(ec_between) .where(ec_between)
.group_by(func.coalesce(ExternalComplaint.complaint_category, "外部投诉(监督局/315等)")) .group_by(func.coalesce(ExternalComplaintClean.complaint_category, "外部投诉(监督局/315等)"))
.order_by(text("cnt DESC")) .order_by(text("cnt DESC"))
).all() ).all()
for r in ec_rows: for r in ec_rows:
@@ -195,13 +195,13 @@ def today_reception_breakdown(db: Session, date: str | None = None, start: str |
"value": int(r[1] or 0), "value": int(r[1] or 0),
}) })
# 叠加 card_complaints 今日 (按 problem_category 聚合, 未分类归入 "卡商群投诉") # 叠加 card_complaints 今日 (按 problem_category 聚合, 未分类归入 "卡商群投诉")
cc_between = and_(CardComplaint.complaint_date.isnot(None), cc_between = and_(CardComplaintClean.complaint_date.isnot(None),
CardComplaint.complaint_date >= ts, CardComplaint.complaint_date <= te) CardComplaintClean.complaint_date >= ts, CardComplaintClean.complaint_date <= te)
cc_rows = db.execute( cc_rows = db.execute(
select(func.coalesce(CardComplaint.problem_category, BUCKET_CARD), select(func.coalesce(CardComplaintClean.problem_category, BUCKET_CARD),
func.count().label("cnt")) func.count().label("cnt"))
.where(cc_between) .where(cc_between)
.group_by(func.coalesce(CardComplaint.problem_category, BUCKET_CARD)) .group_by(func.coalesce(CardComplaintClean.problem_category, BUCKET_CARD))
.order_by(text("cnt DESC")) .order_by(text("cnt DESC"))
).all() ).all()
for r in cc_rows: for r in cc_rows:
@@ -234,9 +234,9 @@ def today_reception_breakdown(db: Session, date: str | None = None, start: str |
_complaint_source_type_case() == BUCKET_CARD) _complaint_source_type_case() == BUCKET_CARD)
).scalar() or 0 ).scalar() or 0
_ov_card_cc = db.execute( _ov_card_cc = db.execute(
select(func.count()).select_from(CardComplaint) select(func.count()).select_from(CardComplaintClean)
.where(cc_between, CardComplaint.matched_ticket_no.isnot(None), .where(cc_between, CardComplaintClean.matched_ticket_no.isnot(None),
CardComplaint.matched_ticket_no.in_( CardComplaintClean.matched_ticket_no.in_(
select(ServiceTicket.ticket_no).where( select(ServiceTicket.ticket_no).where(
tk_between, _ticket_category_case() == "投诉", tk_between, _ticket_category_case() == "投诉",
_complaint_source_type_case() == BUCKET_CARD, _complaint_source_type_case() == BUCKET_CARD,
@@ -287,10 +287,10 @@ def today_reception_breakdown(db: Session, date: str | None = None, start: str |
card_by_category: list[dict] = [] card_by_category: list[dict] = []
if _cnt_cc > 0: if _cnt_cc > 0:
_cc_rows = db.execute( _cc_rows = db.execute(
select(func.coalesce(CardComplaint.problem_category, CARD_SCOPE), select(func.coalesce(CardComplaintClean.problem_category, CARD_SCOPE),
func.count().label("cnt")) func.count().label("cnt"))
.where(cc_between) .where(cc_between)
.group_by(func.coalesce(CardComplaint.problem_category, CARD_SCOPE)) .group_by(func.coalesce(CardComplaintClean.problem_category, CARD_SCOPE))
.order_by(text("cnt DESC")) .order_by(text("cnt DESC"))
).all() ).all()
card_by_category = [ card_by_category = [
@@ -345,7 +345,7 @@ def today_reception_breakdown(db: Session, date: str | None = None, start: str |
-- 是否匹配到 cs_record: 1=匹配, 0=未匹配 -- 是否匹配到 cs_record: 1=匹配, 0=未匹配
CASE WHEN c.problem_category IS NOT NULL THEN 1 ELSE 0 END AS has_match CASE WHEN c.problem_category IS NOT NULL THEN 1 ELSE 0 END AS has_match
FROM comp_tickets ct FROM comp_tickets ct
LEFT JOIN cs_records c LEFT JOIN cs_record_clean c
ON ct.device_sn = c.device_sn ON ct.device_sn = c.device_sn
AND c.operation_time IS NOT NULL AND c.operation_time IS NOT NULL
AND c.operation_time BETWEEN (ct.register_time - INTERVAL '24 hours') AND c.operation_time BETWEEN (ct.register_time - INTERVAL '24 hours')
@@ -403,7 +403,7 @@ def today_reception_breakdown(db: Session, date: str | None = None, start: str |
WHEN COALESCE(service_item, '') ~* '微信投诉' WHEN COALESCE(service_item, '') ~* '微信投诉'
THEN '微信商户号投诉类' THEN '微信商户号投诉类'
-- A3-4: 外部平台名(12315/工信部/消协/公共服务/市监/政务) → 外部平台投诉受理类 -- A3-4: 外部平台名(12315/工信部/消协/公共服务/市监/政务) → 外部平台投诉受理类
-- 注意: 这里不重复 外部投诉(ExternalComplaint)表, 这些工单是 service_tickets -- 注意: 这里不重复 外部投诉(ExternalComplaintClean)表, 这些工单是 service_tickets
-- 中通过 "投诉子类" 字段直接录入"全国12315平台"等名称的记录 -- 中通过 "投诉子类" 字段直接录入"全国12315平台"等名称的记录
WHEN COALESCE(service_item, '') ~* '(12315|工信部|消协|消费者协会|公共服务|监督局|市场监管|市监|12345|政务平台)' WHEN COALESCE(service_item, '') ~* '(12315|工信部|消协|消费者协会|公共服务|监督局|市场监管|市监|12345|政务平台)'
THEN '外部平台投诉受理类' THEN '外部平台投诉受理类'
@@ -445,9 +445,9 @@ def today_reception_breakdown(db: Session, date: str | None = None, start: str |
return "外部平台投诉受理类" return "外部平台投诉受理类"
ec_list_rows = db.execute( ec_list_rows = db.execute(
select(ExternalComplaint.complaint_category, select(ExternalComplaintClean.complaint_category,
ExternalComplaint.platform, ExternalComplaintClean.platform,
ExternalComplaint.keyword) ExternalComplaintClean.keyword)
.where(ec_between) .where(ec_between)
).all() ).all()
for r in ec_list_rows: for r in ec_list_rows:
@@ -472,8 +472,8 @@ def today_reception_breakdown(db: Session, date: str | None = None, start: str |
return "卡商群待标注类" return "卡商群待标注类"
cc_list_rows = db.execute( cc_list_rows = db.execute(
select(CardComplaint.problem_category, select(CardComplaintClean.problem_category,
CardComplaint.problem) CardComplaintClean.problem)
.where(cc_between) .where(cc_between)
).all() ).all()
for r in cc_list_rows: for r in cc_list_rows:
@@ -505,7 +505,7 @@ def today_reception_breakdown(db: Session, date: str | None = None, start: str |
# ═══════════════════════════════════════════════════════════ # ═══════════════════════════════════════════════════════════
# 2026-08-13 新增: 投诉处理人 × 处理状态 统计 (三张投诉表合并) # 2026-08-13 新增: 投诉处理人 × 处理状态 统计 (三张投诉表合并)
# - 处理人来源: service_tickets.handler / ExternalComplaint.handler / CardComplaint.handler # - 处理人来源: service_tickets.handler / ExternalComplaintClean.handler / CardComplaintClean.handler
# - 统一归一化: "企微-洪佳" → "洪佳",空值 → "未分配" # - 统一归一化: "企微-洪佳" → "洪佳",空值 → "未分配"
# - 状态口径: 3表统一映射为 [待处理/处理中/已完成/已终止/未标注] 5 桶 # - 状态口径: 3表统一映射为 [待处理/处理中/已完成/已终止/未标注] 5 桶
# ═══════════════════════════════════════════════════════════ # ═══════════════════════════════════════════════════════════
@@ -545,7 +545,7 @@ def today_reception_breakdown(db: Session, date: str | None = None, start: str |
END AS raw_status, END AS raw_status,
'外部投诉'::text AS source_type, '外部投诉'::text AS source_type,
COUNT(*)::int AS cnt COUNT(*)::int AS cnt
FROM external_complaints ec FROM external_complaint_clean ec
WHERE ec.complaint_date IS NOT NULL WHERE ec.complaint_date IS NOT NULL
AND ec.complaint_date >= :ts AND ec.complaint_date <= :te AND ec.complaint_date >= :ts AND ec.complaint_date <= :te
GROUP BY 1, 2, 3 GROUP BY 1, 2, 3
@@ -564,7 +564,7 @@ def today_reception_breakdown(db: Session, date: str | None = None, start: str |
END AS raw_status, END AS raw_status,
'卡商群投诉'::text AS source_type, '卡商群投诉'::text AS source_type,
COUNT(*)::int AS cnt COUNT(*)::int AS cnt
FROM card_complaints cc FROM card_complaint_clean cc
WHERE cc.complaint_date IS NOT NULL WHERE cc.complaint_date IS NOT NULL
AND cc.complaint_date >= :ts AND cc.complaint_date <= :te AND cc.complaint_date >= :ts AND cc.complaint_date <= :te
GROUP BY 1, 2, 3 GROUP BY 1, 2, 3
@@ -837,27 +837,27 @@ def today_reception_breakdown(db: Session, date: str | None = None, start: str |
# 注意: complaint_pending 的三表查询直接使用表模型,不受 _INCLUDE_COMPLAINT_FILES 标志影响 # 注意: complaint_pending 的三表查询直接使用表模型,不受 _INCLUDE_COMPLAINT_FILES 标志影响
# 该标志仅控制投诉子类饼图/根因等聚合查询,并不阻止未闭环列表查询 # 该标志仅控制投诉子类饼图/根因等聚合查询,并不阻止未闭环列表查询
ec_where = and_( ec_where = and_(
ExternalComplaint.complaint_date.isnot(None), ExternalComplaintClean.complaint_date.isnot(None),
ExternalComplaint.complaint_date >= ts_pending, ExternalComplaintClean.complaint_date >= ts_pending,
ExternalComplaint.complaint_date <= te_pending, ExternalComplaintClean.complaint_date <= te_pending,
) )
ec_pending_rows = db.execute( ec_pending_rows = db.execute(
select( select(
ExternalComplaint.complaint_no, ExternalComplaintClean.complaint_no,
ExternalComplaint.complaint_date, ExternalComplaintClean.complaint_date,
ExternalComplaint.complainant, ExternalComplaintClean.complainant,
ExternalComplaint.complainant_phone, ExternalComplaintClean.complainant_phone,
ExternalComplaint.device_sn, ExternalComplaintClean.device_sn,
ExternalComplaint.platform, ExternalComplaintClean.platform,
ExternalComplaint.problem, ExternalComplaintClean.problem,
ExternalComplaint.complaint_category, ExternalComplaintClean.complaint_category,
ExternalComplaint.result, ExternalComplaintClean.result,
ExternalComplaint.handle_time, ExternalComplaintClean.handle_time,
ExternalComplaint.handle_detail, ExternalComplaintClean.handle_detail,
ExternalComplaint.follower, ExternalComplaintClean.follower,
) )
.where(ec_where) .where(ec_where)
.order_by(ExternalComplaint.complaint_date.asc()) .order_by(ExternalComplaintClean.complaint_date.asc())
# ⚠️ 移除 LIMIT (和 service_tickets 同理, 统一放到 Python 层 过滤完 bucket 后再 LIMIT) # ⚠️ 移除 LIMIT (和 service_tickets 同理, 统一放到 Python 层 过滤完 bucket 后再 LIMIT)
).mappings().all() ).mappings().all()
for r in ec_pending_rows: for r in ec_pending_rows:
@@ -905,27 +905,27 @@ def today_reception_breakdown(db: Session, date: str | None = None, start: str |
# ── C. card_complaints 卡商群投诉 (未闭环) ── # ── C. card_complaints 卡商群投诉 (未闭环) ──
cc_where = and_( cc_where = and_(
CardComplaint.complaint_date.isnot(None), CardComplaintClean.complaint_date.isnot(None),
CardComplaint.complaint_date >= ts_pending, CardComplaintClean.complaint_date >= ts_pending,
CardComplaint.complaint_date <= te_pending, CardComplaintClean.complaint_date <= te_pending,
) )
cc_pending_rows = db.execute( cc_pending_rows = db.execute(
select( select(
CardComplaint.id, CardComplaintClean.id,
CardComplaint.complaint_date, CardComplaintClean.complaint_date,
CardComplaint.complainant, CardComplaintClean.complainant,
CardComplaint.complainant_phone, CardComplaintClean.complainant_phone,
CardComplaint.device_sn, CardComplaintClean.device_sn,
CardComplaint.complaint_group, CardComplaintClean.complaint_group,
CardComplaint.problem, CardComplaintClean.problem,
CardComplaint.problem_category, CardComplaintClean.problem_category,
CardComplaint.result, CardComplaintClean.result,
CardComplaint.handle_result, CardComplaintClean.handle_result,
CardComplaint.solution, CardComplaintClean.solution,
CardComplaint.handler, CardComplaintClean.handler,
) )
.where(cc_where) .where(cc_where)
.order_by(CardComplaint.complaint_date.asc()) .order_by(CardComplaintClean.complaint_date.asc())
# ⚠️ 移除 LIMIT (和 service_tickets 同理, 统一放到 Python 层 过滤完 bucket 后再 LIMIT) # ⚠️ 移除 LIMIT (和 service_tickets 同理, 统一放到 Python 层 过滤完 bucket 后再 LIMIT)
).mappings().all() ).mappings().all()
for r in cc_pending_rows: for r in cc_pending_rows:

View File

@@ -15,19 +15,19 @@ from typing import Any
from sqlalchemy import and_, case, func, literal, or_, select, text, union from sqlalchemy import and_, case, func, literal, or_, select, text, union
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from app.models.card_complaint import CardComplaint from app.models.ods_clean import CardComplaintClean
from app.models.cs_record import CsRecord from app.models.ods_clean import CsRecordClean
from app.models.external_complaint import ExternalComplaint from app.models.ods_clean import ExternalComplaintClean
from app.models.group import Group, PersonGroupMembership from app.models.group import Group, PersonGroupMembership
from app.models.incident import Incident from app.models.incident import Incident
from app.models.person import Person, PersonAlias from app.models.person import Person, PersonAlias
from app.models.refund import RefundClean from app.models.ods_clean import RefundClean
from app.models.refund_record_report import RefundRecordReport from app.models.ods_clean import RefundRecordReportClean
from app.models.specialty import PersonSpecialtyAssignment, Specialty from app.models.specialty import PersonSpecialtyAssignment, Specialty
from app.models.task import SpecialTask from app.models.task import SpecialTask
from app.models.ticket import ServiceTicket from app.models.ticket import ServiceTicket
from app.models.todo import TodoItem from app.models.todo import TodoItem
from app.models.wallet import WalletWithdrawal from app.models.ods_clean import WalletWithdrawalClean
from app.services.complaint_scope import ( from app.services.complaint_scope import (
ALL_BUCKETS, ALL_BUCKETS,
BUCKET_CARD, BUCKET_CARD,

View File

@@ -15,19 +15,19 @@ from typing import Any
from sqlalchemy import and_, case, func, literal, or_, select, text, union from sqlalchemy import and_, case, func, literal, or_, select, text, union
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from app.models.card_complaint import CardComplaint from app.models.ods_clean import CardComplaintClean
from app.models.cs_record import CsRecord from app.models.ods_clean import CsRecordClean
from app.models.external_complaint import ExternalComplaint from app.models.ods_clean import ExternalComplaintClean
from app.models.group import Group, PersonGroupMembership from app.models.group import Group, PersonGroupMembership
from app.models.incident import Incident from app.models.incident import Incident
from app.models.person import Person, PersonAlias from app.models.person import Person, PersonAlias
from app.models.refund import RefundClean from app.models.ods_clean import RefundClean
from app.models.refund_record_report import RefundRecordReport from app.models.ods_clean import RefundRecordReportClean
from app.models.specialty import PersonSpecialtyAssignment, Specialty from app.models.specialty import PersonSpecialtyAssignment, Specialty
from app.models.task import SpecialTask from app.models.task import SpecialTask
from app.models.ticket import ServiceTicket from app.models.ticket import ServiceTicket
from app.models.todo import TodoItem from app.models.todo import TodoItem
from app.models.wallet import WalletWithdrawal from app.models.ods_clean import WalletWithdrawalClean
from app.services.complaint_scope import ( from app.services.complaint_scope import (
ALL_BUCKETS, ALL_BUCKETS,
BUCKET_CARD, BUCKET_CARD,
@@ -144,36 +144,36 @@ def today_staff_efficiency(
def _sys_auto_filter(): def _sys_auto_filter():
"""返回 SQL 条件 排除 cs_record 中系统自动产生的记录""" """返回 SQL 条件 排除 cs_record 中系统自动产生的记录"""
kw = ("系统自动", "自动发送", "自动回复", "自动推送", "自动补发", "自动触发", "自动生成") kw = ("系统自动", "自动发送", "自动回复", "自动推送", "自动补发", "自动触发", "自动生成")
kw_like = [CsRecord.description.ilike(f"%{k}%") for k in kw] \ kw_like = [CsRecordClean.description.ilike(f"%{k}%") for k in kw] \
+ [CsRecord.execution_result.ilike(f"%{k}%") for k in kw] \ + [CsRecordClean.execution_result.ilike(f"%{k}%") for k in kw] \
+ [CsRecord.solution.ilike(f"%{k}%") for k in kw] + [CsRecordClean.solution.ilike(f"%{k}%") for k in kw]
return and_( return and_(
CsRecord.operator.isnot(None), CsRecordClean.operator.isnot(None),
CsRecord.operator.notin_(SYS_OP_BLACKLIST), CsRecordClean.operator.notin_(SYS_OP_BLACKLIST),
~or_(*kw_like), # 反向: 不包含任何自动关键词 ~or_(*kw_like), # 反向: 不包含任何自动关键词
) )
# 1. 会话接待量 + 接待人数(去重SN) 总量 per operator # 1. 会话接待量 + 接待人数(去重SN) 总量 per operator
# v2: 排除系统自动记录 # v2: 排除系统自动记录
cs_q = (select(CsRecord.operator.label("op"), cs_q = (select(CsRecordClean.operator.label("op"),
func.count().label("c"), func.count().label("c"),
func.count(func.distinct(func.nullif(func.trim(CsRecord.device_sn), ""))).label("dc")) func.count(func.distinct(func.nullif(func.trim(CsRecordClean.device_sn), ""))).label("dc"))
.where(CsRecord.operation_time.isnot(None), .where(CsRecordClean.operation_time.isnot(None),
CsRecord.operation_time >= ts, CsRecord.operation_time <= te, CsRecordClean.operation_time >= ts, CsRecordClean.operation_time <= te,
_sys_auto_filter()) _sys_auto_filter())
.group_by(CsRecord.operator)).cte("cs_q") .group_by(CsRecordClean.operator)).cte("cs_q")
# 2. 会话接待 按 6 个 category_bucket 分桶 per operator: 会话数 + 接待数(SN去重) 两套 # 2. 会话接待 按 6 个 category_bucket 分桶 per operator: 会话数 + 接待数(SN去重) 两套
# v2: 同样排除系统自动记录 # v2: 同样排除系统自动记录
cs_cat_q = (select( cs_cat_q = (select(
CsRecord.operator.label("op"), CsRecordClean.operator.label("op"),
_category_case().label("bucket"), _category_case().label("bucket"),
func.count().label("scnt"), func.count().label("scnt"),
func.count(func.distinct(func.nullif(func.trim(CsRecord.device_sn), ""))).label("pcnt"), func.count(func.distinct(func.nullif(func.trim(CsRecordClean.device_sn), ""))).label("pcnt"),
).where( ).where(
CsRecord.operation_time.isnot(None), CsRecordClean.operation_time.isnot(None),
CsRecord.operation_time >= ts, CsRecord.operation_time <= te, CsRecordClean.operation_time >= ts, CsRecordClean.operation_time <= te,
_sys_auto_filter(), _sys_auto_filter(),
).group_by(CsRecord.operator, "bucket")).cte("cs_cat_q") ).group_by(CsRecordClean.operator, "bucket")).cte("cs_cat_q")
# 3. 退款挽回实际执行: 笔数 + 去重设备个数 # 3. 退款挽回实际执行: 笔数 + 去重设备个数
# 已有 sys/system 排除, 保留 # 已有 sys/system 排除, 保留
# ⚠️ 2026-09-08: 工作量口径(无金额)保持全状态(any_status), 不随经营口径收紧到已退款, # ⚠️ 2026-09-08: 工作量口径(无金额)保持全状态(any_status), 不随经营口径收紧到已退款,
@@ -361,7 +361,7 @@ def today_staff_efficiency(
ELSE '咨询接待' ELSE '咨询接待'
END AS spec, END AS spec,
c.device_sn AS sn c.device_sn AS sn
FROM cs_records c FROM cs_record_clean c
WHERE c.operation_time IS NOT NULL WHERE c.operation_time IS NOT NULL
AND c.operation_time >= :ts AND c.operation_time <= :te AND c.operation_time >= :ts AND c.operation_time <= :te
AND c.operator IS NOT NULL AND c.operator IS NOT NULL
@@ -446,7 +446,7 @@ def today_staff_efficiency(
END AS spec, END AS spec,
st.device_sn AS sn st.device_sn AS sn
FROM service_tickets st FROM service_tickets st
JOIN cs_records c2 JOIN cs_record_clean c2
ON c2.device_sn = st.device_sn ON c2.device_sn = st.device_sn
AND c2.operator IS NOT NULL AND c2.operator IS NOT NULL
AND c2.operator NOT IN ('sys','system','SYS','System','用户自助','客户自助','企微用户自助','WeChat','wechat','WECHAT','AI客服','智能客服','机器人客服','客户','班牛','系统','微信投诉') AND c2.operator NOT IN ('sys','system','SYS','System','用户自助','客户自助','企微用户自助','WeChat','wechat','WECHAT','AI客服','智能客服','机器人客服','客户','班牛','系统','微信投诉')
@@ -537,7 +537,7 @@ def today_staff_efficiency(
st.service_item AS service_item, st.service_item AS service_item,
st.service_category AS service_category st.service_category AS service_category
FROM service_tickets st FROM service_tickets st
JOIN cs_records c2 JOIN cs_record_clean c2
ON c2.device_sn = st.device_sn ON c2.device_sn = st.device_sn
AND c2.operator IS NOT NULL AND c2.operator IS NOT NULL
AND c2.operator NOT IN ('sys','system','SYS','System','用户自助','客户自助','企微用户自助','WeChat','wechat','WECHAT','AI客服','智能客服','机器人客服','客户','班牛','系统','微信投诉') AND c2.operator NOT IN ('sys','system','SYS','System','用户自助','客户自助','企微用户自助','WeChat','wechat','WECHAT','AI客服','智能客服','机器人客服','客户','班牛','系统','微信投诉')
@@ -1080,14 +1080,14 @@ def today_staff_efficiency(
"sys", "system", "SYS", "System", "系统", "班牛", "用户自助", "客户自助"] "sys", "system", "SYS", "System", "系统", "班牛", "用户自助", "客户自助"]
_virtual_agg = db.execute( _virtual_agg = db.execute(
select( select(
CsRecord.operator.label("op"), CsRecordClean.operator.label("op"),
func.count().label("rc"), func.count().label("rc"),
func.count(func.distinct(func.nullif(func.trim(CsRecord.device_sn), ""))).label("rdc"), func.count(func.distinct(func.nullif(func.trim(CsRecordClean.device_sn), ""))).label("rdc"),
) )
.where(CsRecord.operation_time.isnot(None), .where(CsRecordClean.operation_time.isnot(None),
CsRecord.operation_time >= ts, CsRecord.operation_time <= te, CsRecordClean.operation_time >= ts, CsRecordClean.operation_time <= te,
CsRecord.operator.in_(_AI_SYS_OPS)) CsRecordClean.operator.in_(_AI_SYS_OPS))
.group_by(CsRecord.operator) .group_by(CsRecordClean.operator)
).all() ).all()
# ───────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────
# ★ 2026-09-14 用户要求: 「AI 接待也纳入效率值核算,算法与专项人员一致」 # ★ 2026-09-14 用户要求: 「AI 接待也纳入效率值核算,算法与专项人员一致」
@@ -1131,14 +1131,14 @@ def today_staff_efficiency(
_category_case().label("bucket"), _category_case().label("bucket"),
func.count().label("scnt"), func.count().label("scnt"),
func.count( func.count(
func.distinct(func.nullif(func.trim(CsRecord.device_sn), "")) func.distinct(func.nullif(func.trim(CsRecordClean.device_sn), ""))
).label("pcnt"), ).label("pcnt"),
) )
.where( .where(
CsRecord.operation_time.isnot(None), CsRecordClean.operation_time.isnot(None),
CsRecord.operation_time >= ts, CsRecordClean.operation_time >= ts,
CsRecord.operation_time <= te, CsRecordClean.operation_time <= te,
CsRecord.operator == op_name, CsRecordClean.operator == op_name,
) )
.group_by("bucket") .group_by("bucket")
).all() ).all()

View File

@@ -15,19 +15,19 @@ from typing import Any
from sqlalchemy import and_, case, func, literal, or_, select, text, union from sqlalchemy import and_, case, func, literal, or_, select, text, union
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from app.models.card_complaint import CardComplaint from app.models.ods_clean import CardComplaintClean
from app.models.cs_record import CsRecord from app.models.ods_clean import CsRecordClean
from app.models.external_complaint import ExternalComplaint from app.models.ods_clean import ExternalComplaintClean
from app.models.group import Group, PersonGroupMembership from app.models.group import Group, PersonGroupMembership
from app.models.incident import Incident from app.models.incident import Incident
from app.models.person import Person, PersonAlias from app.models.person import Person, PersonAlias
from app.models.refund import RefundClean from app.models.ods_clean import RefundClean
from app.models.refund_record_report import RefundRecordReport from app.models.ods_clean import RefundRecordReportClean
from app.models.specialty import PersonSpecialtyAssignment, Specialty from app.models.specialty import PersonSpecialtyAssignment, Specialty
from app.models.task import SpecialTask from app.models.task import SpecialTask
from app.models.ticket import ServiceTicket from app.models.ticket import ServiceTicket
from app.models.todo import TodoItem from app.models.todo import TodoItem
from app.models.wallet import WalletWithdrawal from app.models.ods_clean import WalletWithdrawalClean
from app.services.complaint_scope import ( from app.services.complaint_scope import (
ALL_BUCKETS, ALL_BUCKETS,
BUCKET_CARD, BUCKET_CARD,
@@ -218,11 +218,11 @@ def today_hourly_trend(db: Session, date: str | None = None, start: str | None =
x_label = "日期 (跨度 {} 天)".format(len(slots_days)) x_label = "日期 (跨度 {} 天)".format(len(slots_days))
n_slots = len(slots_days) n_slots = len(slots_days)
cs_rows = db.execute( cs_rows = db.execute(
select(_time_trunc(CsRecord.operation_time), select(_time_trunc(CsRecordClean.operation_time),
func.count().label("cnt"), func.count().label("cnt"),
func.count(func.distinct(func.nullif(func.trim(CsRecord.device_sn), ""))).label("dev_cnt")) func.count(func.distinct(func.nullif(func.trim(CsRecordClean.device_sn), ""))).label("dev_cnt"))
.where(CsRecord.operation_time.isnot(None), .where(CsRecordClean.operation_time.isnot(None),
CsRecord.operation_time >= ts, CsRecord.operation_time <= te) CsRecordClean.operation_time >= ts, CsRecordClean.operation_time <= te)
.group_by("trunc_t") .group_by("trunc_t")
).all() ).all()
cs_map: dict[int, int] = {} cs_map: dict[int, int] = {}
@@ -261,11 +261,11 @@ def today_hourly_trend(db: Session, date: str | None = None, start: str | None =
# --- 路线B: 钱包提现(按粒度 笔数+金额) --- # --- 路线B: 钱包提现(按粒度 笔数+金额) ---
wb_rows = db.execute( wb_rows = db.execute(
select(_time_trunc(WalletWithdrawal.op_time), select(_time_trunc(WalletWithdrawalClean.op_time),
func.count().label("cnt"), func.count().label("cnt"),
func.coalesce(func.sum(WalletWithdrawal.op_amount), 0).label("amt")) func.coalesce(func.sum(WalletWithdrawalClean.op_amount), 0).label("amt"))
.where(WalletWithdrawal.op_time.isnot(None), .where(WalletWithdrawalClean.op_time.isnot(None),
WalletWithdrawal.op_time >= ts, WalletWithdrawal.op_time <= te, WalletWithdrawalClean.op_time >= ts, WalletWithdrawalClean.op_time <= te,
*_route_b_withdraw_filter()) *_route_b_withdraw_filter())
.group_by("trunc_t") .group_by("trunc_t")
).all() ).all()
@@ -289,11 +289,11 @@ def today_hourly_trend(db: Session, date: str | None = None, start: str | None =
# --- 路线E: 商户号实际现金流(按粒度 笔数+金额, 数据源=refund_record_reports) --- # --- 路线E: 商户号实际现金流(按粒度 笔数+金额, 数据源=refund_record_reports) ---
e_rows = db.execute( e_rows = db.execute(
select(_time_trunc(RefundRecordReport.refund_time), select(_time_trunc(RefundRecordReportClean.refund_time),
func.count().label("cnt"), func.count().label("cnt"),
func.coalesce(func.sum(RefundRecordReport.refund_amount), 0).label("amt")) func.coalesce(func.sum(RefundRecordReportClean.refund_amount), 0).label("amt"))
.where(RefundRecordReport.refund_time.isnot(None), .where(RefundRecordReportClean.refund_time.isnot(None),
RefundRecordReport.refund_time >= ts, RefundRecordReport.refund_time <= te, RefundRecordReportClean.refund_time >= ts, RefundRecordReportClean.refund_time <= te,
*_route_e_cashflow_filter()) *_route_e_cashflow_filter())
.group_by("trunc_t") .group_by("trunc_t")
).all() ).all()
@@ -313,7 +313,7 @@ def today_hourly_trend(db: Session, date: str | None = None, start: str | None =
# **与 #81 核心指标同源**(分桶 + 扣 315 已 matched 重叠 + 剥卡商) # **与 #81 核心指标同源**(分桶 + 扣 315 已 matched 重叠 + 剥卡商)
# #
# 【修什么】原实现(2026-09-14 前): # 【修什么】原实现(2026-09-14 前):
# `工单投诉类 + ExternalComplaint 全表 + CardComplaint 全表` # `工单投诉类 + 外部投诉全表 + 卡商群投诉全表`
# → ① **含卡商**(与 #53「卡商独立、不纳入总量」直接冲突) # → ① **含卡商**(与 #53「卡商独立、不纳入总量」直接冲突)
# ② **不扣重叠** → 315 里已 matched 到工单的记录被重复计入 # ② **不扣重叠** → 315 里已 matched 到工单的记录被重复计入
# ③ 与同页「核心指标·投诉数」**对不上**(实测 1127 vs 1111,差 16) # ③ 与同页「核心指标·投诉数」**对不上**(实测 1127 vs 1111,差 16)
@@ -387,12 +387,12 @@ def today_hourly_trend(db: Session, date: str | None = None, start: str | None =
func.extract("epoch", ServiceTicket.finish_time - ServiceTicket.register_time) >= 180, func.extract("epoch", ServiceTicket.finish_time - ServiceTicket.register_time) >= 180,
ServiceTicket.finish_time >= ts, ServiceTicket.finish_time <= te) ServiceTicket.finish_time >= ts, ServiceTicket.finish_time <= te)
ec_done_rows = db.execute( ec_done_rows = db.execute(
select(_time_trunc(ExternalComplaint.complaint_date), func.count().label("cnt"), select(_time_trunc(ExternalComplaintClean.complaint_date), func.count().label("cnt"),
func.count(ExternalComplaint.matched_ticket_no) func.count(ExternalComplaintClean.matched_ticket_no)
.filter(ExternalComplaint.matched_ticket_no.in_(_done_sub)).label("ov")) .filter(ExternalComplaintClean.matched_ticket_no.in_(_done_sub)).label("ov"))
.where(ExternalComplaint.complaint_date.isnot(None), .where(ExternalComplaintClean.complaint_date.isnot(None),
ExternalComplaint.complaint_date >= ts, ExternalComplaint.complaint_date <= te, ExternalComplaintClean.complaint_date >= ts, ExternalComplaintClean.complaint_date <= te,
ExternalComplaint.result.in_(_EXTERNAL_DONE_RESULTS_LOCAL)) ExternalComplaintClean.result.in_(_EXTERNAL_DONE_RESULTS_LOCAL))
.group_by("trunc_t") .group_by("trunc_t")
).all() ).all()
for r in ec_done_rows: for r in ec_done_rows:
@@ -408,16 +408,16 @@ def today_hourly_trend(db: Session, date: str | None = None, start: str | None =
# 315 补充表**净新增**并入「外部投诉」(与 /summary 合并口径一致) # 315 补充表**净新增**并入「外部投诉」(与 /summary 合并口径一致)
# 净新增 = 当日总数 − 已 matched 到本范围内工单的条数(避免与工单双算) # 净新增 = 当日总数 − 已 matched 到本范围内工单的条数(避免与工单双算)
if _INCLUDE_COMPLAINT_FILES: if _INCLUDE_COMPLAINT_FILES:
_ec_cond = [ExternalComplaint.complaint_date.isnot(None), _ec_cond = [ExternalComplaintClean.complaint_date.isnot(None),
ExternalComplaint.complaint_date >= ts, ExternalComplaint.complaint_date <= te] ExternalComplaintClean.complaint_date >= ts, ExternalComplaintClean.complaint_date <= te]
_sub_tk = select(ServiceTicket.ticket_no).where( _sub_tk = select(ServiceTicket.ticket_no).where(
_ticket_category_case() == "投诉", _ticket_category_case() == "投诉",
ServiceTicket.register_time.isnot(None), ServiceTicket.register_time.isnot(None),
ServiceTicket.register_time >= ts, ServiceTicket.register_time <= te) ServiceTicket.register_time >= ts, ServiceTicket.register_time <= te)
ec_rows = db.execute( ec_rows = db.execute(
select(_time_trunc(ExternalComplaint.complaint_date), func.count().label("cnt"), select(_time_trunc(ExternalComplaintClean.complaint_date), func.count().label("cnt"),
func.count(ExternalComplaint.matched_ticket_no) func.count(ExternalComplaintClean.matched_ticket_no)
.filter(ExternalComplaint.matched_ticket_no.in_(_sub_tk)).label("ov")) .filter(ExternalComplaintClean.matched_ticket_no.in_(_sub_tk)).label("ov"))
.where(*_ec_cond) .where(*_ec_cond)
.group_by("trunc_t") .group_by("trunc_t")
).all() ).all()
@@ -431,7 +431,7 @@ def today_hourly_trend(db: Session, date: str | None = None, start: str | None =
net = int(r.cnt or 0) - int(r.ov or 0) net = int(r.cnt or 0) - int(r.ov or 0)
if net: if net:
cp_map[idx] = cp_map.get(idx, 0) + net cp_map[idx] = cp_map.get(idx, 0) + net
# ⚠️ CardComplaint **不再并入**(#53 卡商独立,见上方说明) # ⚠️ 卡商群投诉 **不再并入**(#53 卡商独立,见上方说明)
# ── 填充到槽位: 单日填当前小时之前, 多日填全部(最后一天可能实时) ── # ── 填充到槽位: 单日填当前小时之前, 多日填全部(最后一天可能实时) ──
slots = list(range(n_slots)) slots = list(range(n_slots))

View File

@@ -0,0 +1,6 @@
"""数据质量审核域服务包(#322 P2 逐域切换)。
原 `app/services/analytics/refund.py` 内的「手动退款登记质量审核」含**写路径**
(直接 UPDATE `ods.manual_refunds` 基表), analytics 层已切到 public 干净视图读路径,
门禁 C2 禁止 analytics 再引用 ods 基表/基类, 故将写路径移出到本包。
"""

View File

@@ -0,0 +1,107 @@
"""手动退款登记 数据质量审核 (L3: 自动纠错记录 + 待人工确认)。
## 为什么单独成模块(#322 P2 方案A)
原实现位于 `app/services/analytics/refund.py`, 但其中 `manual_refund_quality_confirm/fix`
是**写路径** —— 需要直接 UPDATE `ods.manual_refunds` 基表(`ManualRefund` 只能映射基表,
视图不可写)。analytics 层已整体切到 `public` 干净视图读路径, 门禁 C2 禁止 analytics 再引用
ods 基表/基类, 故把写路径**移出 analytics**:
- 读(列表): 走 `ManualRefundClean`(薄视图 `public.manual_refund_clean`)
- 写(确认/修正): 走 ods 基表 `ManualRefund`(可写)
行为与签名逐字不变, 仅数据源归属调整。
"""
import datetime as _dt
from typing import Any
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from app.models.manual_refund import ManualRefund
from app.models.ods_clean import ManualRefundClean
from app.services.analytics._core import TZ, D
def _mr_quality_out(r: ManualRefund) -> dict[str, Any]:
"""手动退款登记行 → 质量审核输出"""
def _sh(dt):
return dt.astimezone(TZ).strftime("%Y-%m-%d %H:%M:%S") if dt else None
return {
"id": r.id,
"register_time": _sh(r.register_time),
"op_time": _sh(r.op_time),
"is_refunded": r.is_refunded,
"refund_amount": float(r.refund_amount or 0),
"order_amount": float(r.order_amount or 0),
"device_sn": r.device_sn,
"order_no": r.order_no,
"merchant_id": r.merchant_id,
"registrant": r.registrant,
"refund_operator": r.refund_operator,
"remark": r.remark,
"quality_status": r.quality_status,
"quality_note": r.quality_note,
}
def manual_refund_quality_list(db: Session, status: str = "all", page: int = 1, page_size: int = 50) -> dict[str, Any]:
"""手动退款登记质量审核列表: status = all/pending/auto_fixed/ok
统计摘要: 各状态计数(供前端徽标)。列表按登记日期倒序。
"""
summary = {}
for st in ("pending", "auto_fixed", "ok"):
summary[st] = db.scalar(
select(func.count()).select_from(ManualRefundClean).where(ManualRefundClean.quality_status == st)
) or 0
q = select(ManualRefundClean)
if status in ("pending", "auto_fixed", "ok"):
q = q.where(ManualRefundClean.quality_status == status)
total = db.scalar(select(func.count()).select_from(q.subquery())) or 0
rows = db.execute(
q.order_by(ManualRefundClean.register_time.desc())
.offset((page - 1) * page_size).limit(page_size)
).scalars().all()
return {"total": total, "summary": summary,
"list": [_mr_quality_out(r) for r in rows]}
def manual_refund_quality_confirm(db: Session, rid: int) -> dict[str, Any]:
"""人工确认通过: quality_status → ok (保留纠错说明, 追加确认标记)"""
r = db.get(ManualRefund, rid)
if not r:
raise ValueError(f"手动退款记录不存在: id={rid}")
r.quality_status = "ok"
r.quality_note = ((r.quality_note or "") + " | 人工确认通过") if r.quality_note else "人工确认通过"
db.commit()
return {"id": rid, "quality_status": "ok"}
def manual_refund_quality_fix(db: Session, rid: int, payload: dict[str, Any]) -> dict[str, Any]:
"""人工修正: 更新指定字段(register_time/op_time/is_refunded/refund_amount/order_no/device_sn等),
修正后置 ok。仅接受白名单字段, 时间字段支持 'YYYY-MM-DD' / 'YYYY-MM-DD HH:MM:SS' / ISO 格式。
"""
r = db.get(ManualRefund, rid)
if not r:
raise ValueError(f"手动退款记录不存在: id={rid}")
allowed = {"register_time", "op_time", "is_refunded", "refund_amount", "order_amount",
"order_no", "device_sn", "merchant_id", "registrant", "refund_operator", "remark"}
changes = []
for k, v in (payload or {}).items():
if k not in allowed or v is None:
continue
if k in ("register_time", "op_time"):
setattr(r, k, _dt.datetime.fromisoformat(str(v)) if isinstance(v, str) and "T" in str(v)
else _dt.datetime.strptime(str(v), "%Y-%m-%d %H:%M:%S")
if isinstance(v, str) and len(str(v)) > 10
else _dt.datetime.strptime(str(v), "%Y-%m-%d"))
changes.append(k)
elif k in ("refund_amount", "order_amount"):
setattr(r, k, D(v))
changes.append(k)
else:
setattr(r, k, str(v))
changes.append(k)
r.quality_status = "ok"
r.quality_note = ((r.quality_note or "") + f" | 人工修正字段: {','.join(changes)}") if changes else (r.quality_note or "")
db.commit()
return {"id": rid, "quality_status": "ok", "changed": changes}

View File

@@ -111,6 +111,10 @@ def _engine():
import app.models # noqa: F401 确保所有模型已注册到 Base.metadata import app.models # noqa: F401 确保所有模型已注册到 Base.metadata
from app.core.database import Base from app.core.database import Base
# #322 方案A: 干净层视图清单(视图名 → (源表全限定名, 列清单))。迁移/conftest
# 共用同一来源, 保证「视图 DDL 与模型/迁移逐字一致」。
from app.models.ods_clean import CLEAN_VIEWS
# 2026-09-23: create_all 不 ALTER 已有表; 旧测试库的 governance_prompts / # 2026-09-23: create_all 不 ALTER 已有表; 旧测试库的 governance_prompts /
# onboarding_prompts 缺新列(change_reason/references/activated_at/by)且 code # onboarding_prompts 缺新列(change_reason/references/activated_at/by)且 code
# 仍带旧 unique 约束(多版本行 INSERT 会冲突)。先 DROP 让 create_all 按新模型重建。 # 仍带旧 unique 约束(多版本行 INSERT 会冲突)。先 DROP 让 create_all 按新模型重建。
@@ -143,47 +147,37 @@ def _engine():
conn.execute(text(f'DROP TABLE IF EXISTS public."{_t}" CASCADE')) conn.execute(text(f'DROP TABLE IF EXISTS public."{_t}" CASCADE'))
conn.commit() conn.commit()
# #322 方案A: create_all 会按 RefundClean 模型把 refund_clean 建成**表**(见下方说明), # #322 方案A: create_all 会按各 *Clean 模型把视图名建成**表**(见下方说明),
# 故先移除同名残留对象(可能是上一轮遗留的视图或表), 让 create_all 能干净地按模型建表。 # 故先移除同名残留对象(可能是上一轮遗留的视图或表), 让 create_all 能干净地按模型建表。
# PG 的 DROP VIEW/TABLE 在对象类型不符时会报 WrongObjectType(即使带 IF EXISTS), # PG 的 DROP VIEW/TABLE 在对象类型不符时会报 WrongObjectType(即使带 IF EXISTS),
# 故用 DO 块按 pg_class.relkind 判定后再删。 # 故先按 pg_class.relkind 判定对象类型, 再删对应类型。
with engine.connect() as conn: with engine.connect() as conn:
conn.execute(text( for _view in CLEAN_VIEWS:
""" _relkind = conn.execute(
DO $$ text(
BEGIN "SELECT c.relkind FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace "
IF EXISTS (SELECT 1 FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace "WHERE n.nspname = 'public' AND c.relname = :rn"
WHERE n.nspname = 'public' AND c.relname = 'refund_clean' ),
AND c.relkind = 'v') THEN {"rn": _view},
DROP VIEW public.refund_clean; ).scalar()
END IF; if _relkind == "v":
IF EXISTS (SELECT 1 FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace conn.execute(text(f'DROP VIEW public."{_view}"'))
WHERE n.nspname = 'public' AND c.relname = 'refund_clean' elif _relkind in ("r", "p"):
AND c.relkind IN ('r', 'p')) THEN conn.execute(text(f'DROP TABLE public."{_view}" CASCADE'))
DROP TABLE public.refund_clean CASCADE;
END IF;
END $$;
"""
))
conn.commit() conn.commit()
Base.metadata.create_all(bind=engine) Base.metadata.create_all(bind=engine)
# #322 方案A: create_all 会按 RefundClean 模型把 refund_clean 建成**表**, # #322 方案A: create_all 会把各 *Clean 模型建成**表**(视图无法由 create_all 建),
# 与线上的**视图**(over ods.refunds)不符; 且 analytics 现已全部改读 # 与线上的**视图**(over ods 源表)不符; 且 analytics 现已全部改读这些视图,
# refund_clean, 若留 create_all 的空表会口径错位/用例全红。 # 若留 create_all 的空表会口径错位/用例全红。
# 测试库不跑 Alembic 迁移, 故在此拆表重建为视图。 # 测试库不跑 Alembic 迁移, 故在此拆表重建为视图。
# CREATE VIEW 的列清单与迁移 a9b0c1d2e3f4 逐字一致(改一处须同步另一处)。 # CREATE VIEW 的列清单与迁移 c3d4e5f6a7b8 共用 CLEAN_VIEWS(改一处即同步)。
# 顺序: create_all 先建出视图依赖的 ods.refunds → 再建视图 → 最后 stamp。 # 顺序: create_all 先建出视图依赖的 ods 源表 → 再建视图 → 最后 stamp。
with engine.connect() as conn: with engine.connect() as conn:
conn.execute(text("DROP TABLE IF EXISTS public.refund_clean CASCADE")) for _view, (_src, _cols) in CLEAN_VIEWS.items():
conn.execute(text( conn.execute(text(f'DROP TABLE IF EXISTS public."{_view}" CASCADE'))
"CREATE VIEW refund_clean AS SELECT " conn.execute(text(f'CREATE VIEW public."{_view}" AS SELECT {_cols} FROM {_src}'))
"refund_no, platform_order_no, third_order_no, device_sn, refund_reason, package_name, "
"merchant_name, refund_amount, actual_amount, remark, status, refund_method, "
"audit_remark, auditor, operator, created_at_raw, updated_at_raw, actual_refund_time, "
"quality_status, created_at, updated_at FROM ods.refunds"
))
conn.commit() conn.commit()
# ⚠️ 2026-09-07 修复: create_all 建表后 stamp alembic 到 head。 # ⚠️ 2026-09-07 修复: create_all 建表后 stamp alembic 到 head。

View File

@@ -1,66 +1,3 @@
# C2 棘轮基线: analytics 读 ods 源表的存量违规 (#319 C2 / P2 逐域收敛) # C2 棘轮基线: analytics 读 ods 源表的存量违规 (#319 C2 / P2 逐域收敛)
# 每行一签: <文件>|<类型>|<token>;P2 每切换一个域, 该域条目应减少。 # 每行一签: <文件>|<类型>|<token>;P2 每切换一个域, 该域条目应减少。
backend/app/services/analytics/_core.py|orm_model|BmsOrder
backend/app/services/analytics/_core.py|orm_model|CardComplaint
backend/app/services/analytics/_core.py|orm_model|DeviceCombo
backend/app/services/analytics/_core.py|orm_model|ExternalComplaint
backend/app/services/analytics/_core.py|orm_model|RefundRecordReport
backend/app/services/analytics/_core.py|orm_model|WalletWithdrawal
backend/app/services/analytics/cash.py|orm_model|ManualRefund
backend/app/services/analytics/cash.py|orm_model|RefundRecordReport
backend/app/services/analytics/cash.py|orm_model|WalletWithdrawal
backend/app/services/analytics/category.py|orm_model|DeviceCombo
backend/app/services/analytics/category.py|orm_model|DeviceFlowUsage
backend/app/services/analytics/category.py|sql_table|device_combos
backend/app/services/analytics/complaint.py|orm_model|BmsOrder
backend/app/services/analytics/complaint.py|orm_model|CsRecord
backend/app/services/analytics/reception.py|orm_model|CsRecord
backend/app/services/analytics/refund.py|orm_model|BmsOrder
backend/app/services/analytics/refund.py|orm_model|ManualRefund
backend/app/services/analytics/refund.py|sql_table|cs_records
backend/app/services/analytics/refund.py|sql_table|device_combos
backend/app/services/analytics/refund.py|sql_table|manual_refunds
backend/app/services/analytics/refund.py|sql_table|refund_record_reports
backend/app/services/analytics/refund.py|sql_table|shanmao_refunds
backend/app/services/analytics/refund.py|sql_table|wallet_withdrawals
backend/app/services/analytics/refund_chain.py|orm_model|ManualRefund
backend/app/services/analytics/refund_chain.py|orm_model|WalletWithdrawal
backend/app/services/analytics/refund_chain.py|sql_table|wallet_withdrawals
backend/app/services/analytics/rr_views.py|orm_model|CardComplaint
backend/app/services/analytics/rr_views.py|orm_model|CsRecord
backend/app/services/analytics/rr_views.py|orm_model|ExternalComplaint
backend/app/services/analytics/today/_kpi.py|orm_model|CardComplaint
backend/app/services/analytics/today/_kpi.py|orm_model|CsRecord
backend/app/services/analytics/today/_kpi.py|orm_model|DeviceCombo
backend/app/services/analytics/today/_kpi.py|orm_model|ExternalComplaint
backend/app/services/analytics/today/_kpi.py|orm_model|RefundRecordReport
backend/app/services/analytics/today/_kpi.py|orm_model|WalletWithdrawal
backend/app/services/analytics/today/_queue.py|orm_model|CardComplaint
backend/app/services/analytics/today/_queue.py|orm_model|CsRecord
backend/app/services/analytics/today/_queue.py|orm_model|ExternalComplaint
backend/app/services/analytics/today/_queue.py|orm_model|RefundRecordReport
backend/app/services/analytics/today/_queue.py|orm_model|WalletWithdrawal
backend/app/services/analytics/today/_reception.py|orm_model|CardComplaint
backend/app/services/analytics/today/_reception.py|orm_model|CsRecord
backend/app/services/analytics/today/_reception.py|orm_model|ExternalComplaint
backend/app/services/analytics/today/_reception.py|orm_model|RefundRecordReport
backend/app/services/analytics/today/_reception.py|orm_model|WalletWithdrawal
backend/app/services/analytics/today/_reception.py|sql_table|card_complaints
backend/app/services/analytics/today/_reception.py|sql_table|cs_records
backend/app/services/analytics/today/_reception.py|sql_table|external_complaints
backend/app/services/analytics/today/_refund.py|orm_model|CardComplaint
backend/app/services/analytics/today/_refund.py|orm_model|CsRecord
backend/app/services/analytics/today/_refund.py|orm_model|ExternalComplaint
backend/app/services/analytics/today/_refund.py|orm_model|RefundRecordReport
backend/app/services/analytics/today/_refund.py|orm_model|WalletWithdrawal
backend/app/services/analytics/today/_staff.py|orm_model|CardComplaint
backend/app/services/analytics/today/_staff.py|orm_model|CsRecord
backend/app/services/analytics/today/_staff.py|orm_model|ExternalComplaint
backend/app/services/analytics/today/_staff.py|orm_model|RefundRecordReport
backend/app/services/analytics/today/_staff.py|orm_model|WalletWithdrawal
backend/app/services/analytics/today/_staff.py|sql_table|cs_records
backend/app/services/analytics/today/_trend.py|orm_model|CardComplaint
backend/app/services/analytics/today/_trend.py|orm_model|CsRecord
backend/app/services/analytics/today/_trend.py|orm_model|ExternalComplaint
backend/app/services/analytics/today/_trend.py|orm_model|RefundRecordReport
backend/app/services/analytics/today/_trend.py|orm_model|WalletWithdrawal