- schema: expected_record unique key (site,waybill_no) -> (site,handover_no,waybill_no),
so the same waybill can belong to multiple handover batches without being overwritten
- store: dedup by (handover_no,waybill_no) instead of waybill_no alone; UPSERT conflict
key matches the new unique key; handover_no no longer force-overwritten on conflict
- store: add purge_expected, called on force re-download; deletes by business_date
(download day, same source as task date) rather than batch_out_date, which is offset
from the download day for sites like yunda where download day != outbound day
- store/runtime: thread force + target_date through ingest_task / _persist_to_db /
_site_undelivered_handler so force re-download purges then re-ingests in one tx
- db_compare: undelivered_pieces now sums per-waybill shortfall (total - arrived)
instead of expected_total - arrived_total, unaffected by unrelated/extra scans
Replace Excel-based undelivered comparison with DB queries for all four
sites. The engine anchors on actual scan_time, reverse-lookups handover
batches, and compares expected vs actual waybill-by-waybill.
Shunxin SF waybills: use COUNT(*) instead of COUNT(DISTINCT piece_no)
since SF piece numbers are random and not derivable from the waybill.
Changes:
- db_compare.py: new module with compare_site_date(), compare_site_batch(),
write_result_excel(), and POST /compare API endpoint
- runtime.py: switch _site_undelivered_handler from compare.write_site_file
(Excel) to db_compare (DB); downloads succeed independently of comparison
- server.py: add POST /compare endpoint with date validation
- docs: implementation plan for Shunxin DB comparison
Co-Authored-By: Claude <noreply@anthropic.com>
