114 Commits

Author SHA1 Message Date
Misaka_Company
05539ff293 1.15.0 2026-08-04 09:44:05 +08:00
Misaka_Company
ab790a6be5 docs: add release notes for version 1.15.0 2026-08-04 09:43:59 +08:00
Misaka_Company
056bf3908c feat(cleaner): add configurable row number protection toggle 2026-08-04 09:43:31 +08:00
Misaka_Company
98567d69f0 docs: clarify project rename is planned only, not yet applied to code
- Update rename notice in all 6 doc files to state that the rename from
  ERPAuto to BIPMaterialManager is currently a plan only, with no actual
  code changes implemented
- Documentation names updated for forward compatibility; all code-level
  configs, paths, and artifact names remain unchanged

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-03 16:38:57 +08:00
Misaka
cf9976f605 fix: add AT and ZONE to PostgreSQL SQL keywords for prepareSql
The prepareSql function quotes any word not in SQL_KEYWORDS as an
identifier. Since AT and ZONE were missing from the set, the expression
(NOW() AT TIME ZONE 'UTC') was mangled into (NOW() "AT" TIME "ZONE"
'UTC'), causing INSERT failures on PostgreSQL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 22:00:06 +08:00
Misaka
a6c2e2ccc2 fix: use explicit UTC timestamp in PostgreSQL dialect
PostgreSQL's CURRENT_TIMESTAMP returns session-local time, unlike
SYSUTCDATETIME() (SQL Server) and UTC_TIMESTAMP() (MySQL) which
explicitly return UTC. Switch to (NOW() AT TIME ZONE 'UTC') to
keep operation history timestamps consistent across all databases.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 21:48:28 +08:00
Misaka_Company
21089e8b40 perf(import): bypass intermediate Excel file in extraction pipeline
Replace the Extract → Write Excel → Read Excel → Import DB flow with
direct record-to-database persistence. The extractor now builds
MaterialPlanRecord[] from parsed orders and imports them without the
round-trip through a merged Excel file.

Key changes:
- Add importFromRecords() to DataImportService for record-based import
- Add SQL Server OPENJSON batch insert and atomic replace operations
  in DiscreteMaterialPlanDAO for efficient bulk writes
- Extract common import logic into private importRecords() method
- Configure explicit request/connection timeouts for SQL Server
- Add unit tests for direct record import path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 17:11:30 +08:00
Misaka_Company
f36c88aa89 fix(extractor): display operation time in local timezone
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 13:53:54 +08:00
Misaka_Company
dbb8e4904e perf: optimize order resolution history writes 2026-04-28 13:39:08 +08:00
Misaka_Company
1ffa0650a6 1.14.1 2026-04-28 12:17:15 +08:00
Misaka_Company
72dba32a52 docs: add release notes for version 1.14.1
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 12:17:11 +08:00
Misaka_Company
98865f5d7e fix(cleaner): preserve production IDs in operation history
The 总排号 field was always empty because getCleanerData() resolved
production IDs to order numbers before passing them to the cleaner,
losing the original inputs. Now originalInputs are carried through
the full chain so the resolver can properly set productionId.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 11:57:47 +08:00
Misaka_Company
5ff99cdd0f refactor: use dot notation for table name config (schema.table instead of schema_table)
Replace underscore-based table name splitting with dot-based splitting
to match the standard schema.tablename format, removing MySQL compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 11:10:03 +08:00
Misaka
664f26d63f fix(cleaner): guard missing session refresh config 2026-04-24 19:11:21 +08:00
Misaka_Company
e39fc87869 1.14.0 2026-04-24 15:50:33 +08:00
Misaka_Company
4b071e4331 docs: add release notes for version 1.14.0 2026-04-24 15:50:29 +08:00
Misaka_Company
28a632d0a7 Merge branch 'dev-log-enhance' into dev 2026-04-24 15:41:23 +08:00
Misaka_Company
ac43790127 Add cleaner session refresh and ERP diagnostics 2026-04-24 15:40:47 +08:00
Misaka_Company
723d6de0ae 1.13.0 2026-04-17 12:43:45 +08:00
Misaka_Company
9d7fe8f4e7 docs: add release notes for version 1.13.0
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 12:43:39 +08:00
Misaka_Company
f49f99fc0c perf(cleaner-history): parallelize batch fetching in searchBatches
Replace sequential for-loop with Promise.all so that matched batches
are fetched concurrently instead of one-by-one, reducing total query
latency from O(n) serial round-trips to a single parallel batch.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 12:42:07 +08:00
Misaka_Company
622543fff4 fix(cleaner-history): highlight username and status in batch summary during search
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 11:30:08 +08:00
Misaka_Company
74d9b4042e feat(cleaner-history): integrate search UI into history modal
Add search bar to CleanerOperationHistoryModal with keyword search
across batch IDs, order numbers, and material codes/names. Search
results auto-expand with preloaded data and highlight matched text.
Also add searchHistoryRecords to the CleanerAPI type definition.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 11:21:02 +08:00
Misaka_Company
ed9058c93d feat(cleaner-history): add renderer search types and highlight utility
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 11:11:26 +08:00
Misaka_Company
9167359c6e feat(cleaner-history): expose search API in preload
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 11:09:31 +08:00
Misaka_Company
5faf26df3f feat(cleaner-history): add search IPC handler
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 11:07:20 +08:00
Misaka_Company
3a30694684 feat(cleaner-history): add searchBatches DAO method
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 11:05:17 +08:00
Misaka_Company
c61d62fd98 feat(cleaner-history): add search types and IPC channel
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 10:59:46 +08:00
Misaka_Company
aeb3595b36 docs: add implementation plan for cleaner history search
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 10:52:05 +08:00
Misaka_Company
b8925926cb docs: add design for cleaner history full-level search
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 10:46:34 +08:00
Misaka_Company
2936f1fca3 perf: optimize MaterialTypeManagementDialog with memo, parallel fetch, and stable callbacks
- Use Promise.all for parallel managers + records loading (async-parallel)
- Wrap KeywordCard in memo to skip unnecessary list item re-renders
- Stabilize handlers with useCallback + functional setState pattern
- Hoist generateId to module scope to avoid per-render recreation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-16 10:25:00 +08:00
Misaka_Company
f57fdf69f7 refactor: modernize MaterialTypeManagementDialog UI and fix admin hover overlap
Restructure the dialog with a card-grid layout, KeywordCard subcomponent,
and smooth hover animations. Fix admin view where manager badge and delete
button overlapped by using flex layout with translate and max-width transitions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-16 08:55:08 +08:00
Misaka_Company
bb495c7a93 1.12.4 2026-04-15 15:44:08 +08:00
Misaka_Company
33ffc0406d docs: add release notes for version 1.12.4
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 15:42:53 +08:00
Misaka
8fa4d6c16d fix: support postgres material upserts without unique constraints 2026-04-14 21:55:05 +08:00
Misaka
cb59dda727 refactor: unify operation history delete dialogs 2026-04-14 21:23:54 +08:00
Misaka
fbcaa11b1c refactor: unify cleaner history status display 2026-04-14 21:16:05 +08:00
Misaka
b5b8af078d feat: improve cleaner history pagination and report states 2026-04-14 21:10:05 +08:00
Misaka
5b43d5a60c fix: restore cleaner history in postgresql 2026-04-14 20:49:29 +08:00
Misaka_Company
936c98a023 1.12.3 2026-04-14 15:37:09 +08:00
Misaka_Company
c661a12287 docs: add release notes for version 1.12.3
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 15:36:53 +08:00
Misaka_Company
1cd6660774 chore: remove unused playwright config and debug scripts
Remove playwright.config.ts (no longer using Playwright for E2E),
and delete test-s3-playwright.js / test-s3-playwright-simple.js
(one-off S3 debug scripts).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 15:33:25 +08:00
Misaka_Company
1f06fd275e 1.12.2 2026-04-14 15:20:34 +08:00
Misaka_Company
c86508989b docs: add release notes for version 1.12.2
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 15:20:27 +08:00
Misaka_Company
838783e384 fix(auth): clear cached silentLoginPromise on logout to allow re-authentication
After logout, the cached silentLoginPromise caused silentLogin() to return
a stale result instead of re-executing loginByComputerName(), leaving
sessionManager.currentUser as null and making subsequent switchUser() calls fail.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 15:17:34 +08:00
Misaka_Company
d5028bfcf4 1.12.1 2026-04-14 14:54:29 +08:00
Misaka_Company
b2b29e9754 docs: add release notes for version 1.12.1
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 14:53:23 +08:00
Misaka_Company
cba3c8c4f0 feat(ui): use icons with tooltips for material results
- Replace text badges with icons for cleaner UI
- Map database values: 'success'→deleted, 'skipped', 'uncertain', 'failed_*'→failed
- Add hover tooltip showing status name (Deleted, Skipped, Uncertain, Failed)
- Icons: CheckCircle (green), CircleMinus (gray), AlertTriangle (amber), XCircle (red)
- Add cursor-help to indicate hoverable elements
2026-04-14 14:34:32 +08:00
Misaka_Company
343cb24234 chore: run pretier format across project
- Format TypeScript source files
- Format documentation files
- Update eslint config formatting
2026-04-14 14:03:58 +08:00
Misaka_Company
4ce5b91340 feat(ui): add serial number columns to cleaner operation history
- Add order-level serial number column in order table
- Add material-level serial number column in material details table
- Update colSpan from 10 to 11 to accommodate new column
2026-04-14 14:01:21 +08:00
Misaka_Company
1f033eb315 docs: add plans/ directory naming conventions
- Add date-prefixed naming format: YYYY-MM-DD-description-type.md
- Document -plan.md and -design.md type suffixes
- Add examples from existing plan files
- Update classification examples to include plans/ naming
2026-04-14 12:25:46 +08:00
Misaka_Company
681f3ba517 refactor(docs): reorganize documentation directory structure
- Create user/ - User guides and configuration documentation
- Create features/ - Feature specifications and business flows
- Create debugging/ - Debug guides and quick references
- Create testing/ - Test infrastructure, reports, and plans
- Create internal/ - Internal plans, analyses, and templates
- Move cleaner/*.md to cleaner/ directory
- Move LOGGING_*.md to developer/guides/

Add docs/README.md as documentation index with category navigation
and quick lookup guide.

The reorganized structure makes it easier for users and developers
to quickly locate relevant documentation.
2026-04-14 12:15:09 +08:00
Misaka_Company
0c6bb85e67 1.12.0 2026-04-14 10:51:21 +08:00
Misaka_Company
1a48dca57c docs: add release notes for version 1.12.0
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 10:50:05 +08:00
Misaka_Company
e91b7308a7 fix(tests): sync test expectations with current implementation
Update dialect tests for UTC timestamp functions and cleaner-handler
test for runCleaner's extended parameter signature.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 10:47:04 +08:00
Misaka_Company
35cad8baa9 style(cleaner-history): widen operation history modal to 140%
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 10:29:54 +08:00
Misaka_Company
6f49596467 feat(cleaner-history): record missing orders with production ID tracking
Record ALL input orders in history, including resolution failures (not_found)
and ERP query misses (erp_not_found). Add ProductionId column to track original
总排号 input. Add 总排号 column and new status styles to the history UI. Fix
empty result caching that prevented retry on transient query failures.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 10:16:49 +08:00
Misaka
95eb44979a refactor(cleaner-history): extract BatchItem with React.memo and fix colSpan bug
- Fix colSpan mismatch: material detail row now correctly spans 9 columns
- Use lazy state initialization for Set/Map useState to avoid re-creation
- Remove data-duplicating refs (batchExecutionsRef, batchOrdersRef, orderMaterialsRef)
  and replace with lightweight tracking refs (detailsLoadedRef, loadedMaterialsRef)
- Extract per-batch rendering into BatchItem with React.memo to prevent
  sibling re-renders when expanding/collapsing one batch
- Reduce parent component state from 13 to 5 variables

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 23:09:18 +08:00
Misaka_Company
420f811488 fix(timezone): use UTC for database storage and local time for UI display
Backend changes:
- SQL Server dialect: GETDATE() → SYSUTCDATETIME()
- MySQL dialect: NOW() → UTC_TIMESTAMP()
- Ensures OperationTime and EndTime use consistent UTC timezone

Frontend changes:
- formatDateTime: display UTC timestamps in user's local timezone
- Uses getFullYear/getMonth/getDate/getHours (local) instead of UTC methods

Data migration:
- Executed migration script to fix historical OperationTime records
- All existing records now have correct UTC timestamps
- Execution duration now accurate (minutes, not hours)

Impact:
- New executions store UTC timestamps correctly
- UI displays times in user's local timezone (UTC+8 for CN users)
- Historical data corrected via migration
- Time difference between OperationTime and EndTime now accurate
2026-04-13 17:52:50 +08:00
Misaka_Company
6aa1fc29e5 feat(cleaner-history): display retry information in order history UI
- Add 'Retry' column to order history table
- Show retry count badge with refresh icon
- Display retry success/failure status with visual indicators
- Purple badge for retry count, green/red for success/failure
2026-04-13 16:09:39 +08:00
Misaka_Company
151485caed feat(cleaner): track skipped materials and skip DB writes on dry run
Record materials not in the deletion list as "skipped" with reason
instead of just logging them. Skip inserting material details to
database during dry runs to avoid phantom records.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 15:58:00 +08:00
Misaka_Company
116539ff42 feat(cleaner): record all material operations in database, including successful deletions
Previously only skipped and failed materials were persisted. Now every
material (deleted, uncertain, skipped, failed) is recorded in
CleanerMaterialDetail for full audit traceability.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 14:24:18 +08:00
Misaka_Company
0286df94dd fix(cleaner): cast BIT to INT for MAX() in getBatches query
SQL Server does not support MAX() on BIT columns, causing the
getBatches query to fail silently and return empty results.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 14:17:26 +08:00
Misaka_Company
32931cecad style: format changed files with prettier
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 12:35:31 +08:00
Misaka_Company
b95f12fca1 chore(cleaner): clean up legacy report viewer references
Remove ReportViewerDialog and ReportAnalysisDialog lazy imports, state
variables, Suspense wrappers, and the "查看报告" toolbar button. These
components were for the old Markdown file-based report viewer which has
been replaced by database-backed operation history.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 12:30:10 +08:00
Misaka_Company
4c40457c71 feat(cleaner): add operation history modal with database-backed records
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 12:23:57 +08:00
Misaka_Company
bd68444a74 feat(cleaner): add renderer types for cleaner operation history
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 12:09:47 +08:00
Misaka_Company
7dfa88c2a3 refactor(cleaner): remove Markdown report generator
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 12:06:19 +08:00
Misaka_Company
998edb4b84 refactor(cleaner): replace report generation with database persistence
Remove generateExecutionId(), generateAndUploadReport(), and all
executionId references from CleanerApplicationService. The service
now accepts batchId, historyDao, and appVersion from the IPC handler
and writes execution/order/material records to the database via
CleanerOperationHistoryDAO instead of generating Markdown reports.

All execution paths (success, failure, outer retry, retry-login-failure)
persist their results to the database with appropriate status tracking.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 12:02:43 +08:00
Misaka_Company
74096fbfa0 feat(cleaner): add preload API for cleaner operation history
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 11:52:14 +08:00
Misaka_Company
73656dada8 feat(cleaner): add IPC handlers for cleaner operation history
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 11:38:54 +08:00
Misaka_Company
9a91658121 feat(cleaner): add IPC channels for cleaner operation history
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 11:33:16 +08:00
Misaka_Company
a924e8a4e8 feat(cleaner): add CleanerOperationHistoryDAO for three-table persistence
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 11:29:54 +08:00
Misaka_Company
b363a53d8a feat(cleaner): add type definitions for cleaner operation history
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 11:18:56 +08:00
Misaka_Company
e1d55b8b39 feat(cleaner): add outer-level retry on fatal crash with execution ID
When CleanerService hits a fatal error (browser crash, timeout), the
outer catch now sets result.crashed=true. CleanerApplicationService
detects this, closes the dead browser session, re-logs into ERP, and
re-runs all orders once. An execution ID (CLN-yyyyMMddHHmmss-XXXX)
generated at startup ensures report files are deduplicated across
retries. Reports now display execution ID and app version.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 10:07:29 +08:00
Misaka
8b173890fa feat(cleaner): add multi-signal deletion verification with material-level retry
Replace fragile single-signal (row change only) deletion verification
with a robust multi-signal approach using row change + material count +
ERP message detection. Add material-level retry (up to 3 attempts) for
transient failures, with detailed tracking of failed/uncertain deletions.

- Add DeletionOutcome/DeletionErrorCategory enums and FailedMaterial type
- Add deleteWithVerification() core method with retry logic
- Add evaluateDeletionSignals() pure logic (unit tested, 9 cases)
- Add helper methods: readMaterialCount, checkErpMessages, handleConfirmDialog
- Extend CleanerResult/OrderCleanDetail with failed/uncertain tracking
- Update report generator with failed materials detail section
- Update ExecutionReportDialog to display failed/uncertain stats

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-07 21:55:47 +08:00
Misaka_Company
b065e23306 1.11.1 2026-04-07 08:53:22 +08:00
Misaka_Company
1c0a000a67 fix(preload): add missing selectedManagers param to getCleanerData type declaration
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-07 08:52:58 +08:00
Misaka
6b3c62268a 1.11.0 2026-04-06 19:22:32 +08:00
Misaka
bb86208d32 docs: add release notes for version 1.11.0
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-06 19:22:20 +08:00
Misaka
3be7959067 feat(cleaner): support selectedManagers filtering for admin cleaner execution
Admin can now pass selectedManagers to getCleanerData so material codes
are queried from MaterialsToBeDeleted by ManagerName IN (selectedManagers).
When no managers are selected, fallback to DiscreteMaterialPlanData by
orderNumbers. User behavior is unchanged. Includes updated tests and
role-based flow documentation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-06 19:20:22 +08:00
Misaka
91f29a1167 refactor(audit): unify computerName source to cached os.hostname()
Export cachedHostname from audit-logger and use it in process-guards,
replacing process.env.COMPUTERNAME so all audit entries use the same value.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-06 17:41:36 +08:00
Misaka
abad61758c refactor(audit): type-safe enums, expanded coverage, and crash-safe logging
Replace magic strings with AuditAction/AuditStatus enums across all consumers,
add logAuditWithCurrentUser() convenience wrapper, extend audit coverage to
data import, result export, app update, and ERP credentials operations, and
harden crash handlers with try/catch to prevent audit failures from cascading.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-06 17:40:30 +08:00
Misaka
d0c745e243 refactor(test): migrate e2e to Playwright and remove duplicate unit tests
Switch extractor-workflow e2e test from vitest to Playwright test runner
for consistency with playwright.config.ts. Remove redundant unit tests
(cleaner, erp-auth, extractor) that have been superseded by more thorough
replacements under tests/unit/services/erp/. Enable test isolation
unconditionally to prevent cross-file state pollution.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-06 11:55:58 +08:00
Misaka
fe6cdbf076 fix(test): add mock-driver tests for database services
- Add connected-path tests for mysql, sql-server, and postgresql using
  mocked drivers (mysql2/promise, mssql, pg) covering
  connect, query, transaction, and disconnect scenarios
- Fix tautological assertion in auth-flow.test.ts (hasError >= 0 was always true)
- Add tests/integration to vitest exclude list to prevent
  module cache pollution under isolate:false
- Set isolate to true for CI, false for local dev (was: isolate false)
2026-04-06 11:11:04 +08:00
Misaka
188117e5ce refactor(test): replace module-level mutable state with vi.fn() mocks
Replace fragile module-level let variables and SQL string parsing
with vi.hoisted() mock functions that are reset and configured
per-test in beforeEach via mockResolvedValue/mockResolvedValueOnce.

- Remove 8 module-level mutable state variables
- Remove matchQuery() SQL parser
- Use vi.hoisted() for shared mock functions across vi.mock() factories
- Each test explicitly controls mock return values with mockResolvedValueOnce
- Fix getCleanerData error test to use direct mock instead of dynamic import

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-06 09:21:04 +08:00
Misaka
0560b3c84a style: apply prettier formatting to test files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-06 09:03:16 +08:00
Misaka
fb3bdbc493 test: fix mock configuration and lint errors in unit tests
- Fix logger mock missing default export in extractor.test.ts
- Fix performance-monitor mock configuration
- Fix prefer-const in validation-database.test.ts
- Fix no-unsafe-function-type in cleaner-handler.test.ts
- Fix no-empty-function in cleaner-application-service.test.ts
- Run prettier format on test files

All 622 tests now passing (59 files, 3 skipped)
2026-04-05 22:04:32 +08:00
Misaka
c6f67e49a4 1.10.0 2026-04-05 21:41:08 +08:00
Misaka
d16f2d1af0 docs: add release notes for version 1.10.0
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 21:40:55 +08:00
Misaka
4150a13175 fix(test): improve test isolation and reduce noise in unit tests
- Add logger/error-utils mocks to cleaner-handler test to suppress IPC error log noise
- Move setupServiceMocks into beforeEach for consistent default mocking in cleaner tests
- Replace vi.waitFor (2s timeout) with setImmediate microtask flush in extractor test
- Remove dead activeType assignments in validation-database test
- Align TestUser.id type with UserInfo.id (string → number) and use deterministic counter

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 21:38:20 +08:00
Misaka
d0f8ad0fef test(erp): add unit tests for ERP services and test coverage docs
Add unit tests for core ERP service modules including ErpBrowserManager,
cleaner, erp-auth, extractor-core, extractor, and order-resolver. Also
includes test coverage improvement plan and quality review report.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 17:23:00 +08:00
Misaka
4a7c220baa fix(extractor): resolve SQL syntax error from double-quoted table names 2026-04-05 13:51:18 +08:00
Misaka
f51cae0f6f fix(db): complete PostgreSQL integration in validation and cleaner services
OrderNumberResolver, validation, and cleaner services had incomplete
PostgreSQL support - they only handled SQL Server and MySQL, causing
PostgreSQL to fall through to MySQL code paths with invalid syntax
(backticks, ? placeholders) and missing schema.table name splitting.

Changes:
- Add PostgreSQL SQL generation ($N params, double-quoted identifiers)
  in OrderNumberResolver, validation-application-service,
  production-input-service, and validation-database
- Add PostgreSQL to database factory functions in validation-database
  and cleaner-application-service
- Add UPPER, LOWER, and 40+ common SQL functions to SQL_KEYWORDS to
  prevent prepareSql() from quoting them as identifiers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 13:46:37 +08:00
Misaka
7601b5f176 fix(db): PostgreSQL P0 fixes - SQL_KEYWORDS expansion, timeout config, and tests
- Expand SQL_KEYWORDS from ~120 to 226+ words covering:
  - Window functions (ROW_NUMBER, RANK, LAG, LEAD, etc.)
  - CTEs (WITH, RECURSIVE, MATERIALIZED, etc.)
  - Advanced grouping (ROLLUP, CUBE, GROUPING SETS)
  - JSON operations, types, table sampling
  - Transaction control and other PostgreSQL-specific keywords
- Add connection pool timeout configuration:
  - connectionTimeoutMillis: 10s
  - statement_timeout: 30s (PostgreSQL level)
  - idleTimeoutMillis: 30s (connection cleanup)
  - query_timeout: 60s (driver-level fallback)
- Add 12 comprehensive edge case tests covering:
  - Window functions, CTEs, advanced grouping
  - CASE expressions, set operations, JSON operators
- All 38 tests pass

Production-ready: prevents hung queries and supports complex SQL.
2026-04-05 13:09:01 +08:00
Misaka
e2669af870 fix: remove unused imports and fix logger test isolation
- Remove unused imports (run, trackDuration, PerformanceTracker,
  ConfigManager, disconnectDb) flagged by ESLint
- Remove unused isSlow variable in performance-monitor catch block
- Add eslint-disable for require() in Playwright JS script
- Fix logger-performance test flakiness by using vi.resetModules()
  with dynamic imports to prevent cached logger references across
  test files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 12:15:18 +08:00
Misaka
e54d94fce2 style: apply formatter to docs, types, and test files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 11:56:22 +08:00
Misaka
9791a84047 fix(db): auto-quote SQL identifiers for PostgreSQL case-sensitivity
Add prepareSql() to PostgreSqlService that quotes unquoted column names
before execution. PostgreSQL lowercases unquoted identifiers, but
SSMA-migrated tables have uppercase column names requiring double-quoting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 11:54:30 +08:00
Misaka
b5ba18b595 refactor(db): migrate BIPUsersDAO to use DatabaseFactory and SqlDialect
Replace hardcoded MySqlService/SqlServerService with DatabaseFactory,
enabling PostgreSQL support for user authentication and management.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 11:20:17 +08:00
Misaka
13fb7bcf46 style: fix lint errors in dialect files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:37:32 +08:00
Misaka
0ca17a1807 fix(db): correct dialect import paths and extend bip-users-dao type
- Fix dialect files to use relative paths instead of @types alias
- Add 'postgresql' to BIPUsersDAO dbType union

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:34:03 +08:00
Misaka
54a3ac680a feat(db): integrate PostgreSQL into factory, config, and TypeORM data source
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:29:57 +08:00
Misaka
16b2882729 style: fix extra blank line after formatting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:26:29 +08:00
Misaka
e97ec63433 refactor(db): use SqlDialect in ExtractorOperationHistoryDAO
Replace all isSqlServer checks, buildPlaceholders, and hardcoded table names
with the SqlDialect abstraction. The dialect now handles parameter placeholders,
table name quoting, current timestamp functions, and pagination across all
supported database types.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:25:38 +08:00
Misaka
9556891dea refactor(db): use SqlDialect in MaterialsTypeToBeDeletedDAO
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:20:04 +08:00
Misaka
fa57f9e564 refactor(db): use SqlDialect in MaterialsToBeDeletedDAO
Replace all isSqlServer/if-else branches with SqlDialect calls:
- Table name via dialect.quoteTableName()
- Placeholders via dialect.param() and dialect.params()
- UPSERT via dialect.upsert() in upsertMaterial(), upsertBatch(), updateManager()
- Remove buildPlaceholders(), TABLE_NAME_SQLSERVER, TABLE_NAME_MYSQL
- Re-export SqlDialect type from dialects barrel

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:17:21 +08:00
Misaka
9300f3455f refactor(db): use SqlDialect in DiscreteMaterialPlanDAO
Replace all manual isSqlServer checks and inline SQL dialect logic with the
SqlDialect abstraction. Removes buildPlaceholders(), TABLE_NAME_SQLSERVER,
and TABLE_NAME_MYSQL in favor of dialect.params(), dialect.param(), and
dialect.quoteTableName(). Batch size logic now uses dialect.maxBatchRows().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:10:54 +08:00
Misaka
7e521da3f1 feat(db): add PostgreSqlService with pg driver
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:04:48 +08:00
Misaka
130e0602d1 feat(db): implement SqlDialect with MySQL, SQL Server, PostgreSQL dialects
Add three SqlDialect implementations with a factory function:
- MySqlDialect: positional ?, ON DUPLICATE KEY UPDATE, LIMIT/OFFSET
- SqlServerDialect: @pN params, MERGE USING, OFFSET/FETCH
- PostgreSqlDialect: $N (1-based), ON CONFLICT DO UPDATE, LIMIT/OFFSET

TDD approach: 43 tests written first, all passing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:01:24 +08:00
Misaka
0956bf907f feat(db): add SqlDialect interface and PostgreSQL type definitions
- Add 'postgresql' to DatabaseType union in database.types.ts
- Add PostgreSqlConfig interface extending DatabaseConfig
- Add postgresqlConfigSchema Zod schema with host, port, database,
  username, password, and maxPoolSize fields
- Add 'postgresql' to databaseConfigSchema and type exports
- Create SqlDialect interface with methods for quoteTableName,
  param, params, currentTimestamp, upsert, paginate, maxBatchRows

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 09:19:49 +08:00
Misaka
6c730616b8 docs: add PostgreSQL integration implementation plan
6-task TDD plan covering SqlDialect abstraction, PostgreSqlService,
DAO refactoring, and config/factory integration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 09:14:33 +08:00
Misaka
4f4e5fd91a docs: add PostgreSQL integration design document
Design for integrating PostgreSQL as a third database option using
a SqlDialect abstraction layer to unify SQL dialect differences
across MySQL, SQL Server, and PostgreSQL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 09:09:39 +08:00
Misaka
ae29f38d24 fix(test): improve logger mock path and add behavior-based repository tests
- Fix logger-performance test mock path to use bare module specifier
- Replace meaningless "should be defined" assertions in repositories test
  with behavior-based tests covering upsert, batch operations, queries,
  deletes, and error handling for both MaterialsToBeDeletedRepository
  and DiscreteMaterialPlanRepository

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 08:16:40 +08:00
Misaka
406a8dfd2f fix(test): replace duplicated business logic in cleaner test with real CleanerService
The shouldDeleteMaterial tests had a mockCleaner that reimplemented the
production logic inline, meaning bugs in the real code would never be caught.
Now uses an actual CleanerService instance instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 08:16:02 +08:00
178 changed files with 21477 additions and 2695 deletions

4
.gitignore vendored
View File

@@ -45,3 +45,7 @@ nul
# TypeScript incremental compilation cache
*.tsbuildinfo
# temporary files
tmp/
temp/

View File

@@ -5,7 +5,9 @@
## 项目概览
ERPAuto 是一个基于 Electron 的桌面应用,用于自动化处理 ERP 系统中的数据提取、清理、校验和配置管理。
BIPMaterialManager 是一个基于 Electron 的桌面应用,用于自动化处理 BIP 系统中的物料数据提取、清理、校验和配置管理。
> **项目重命名说明**:本项目计划从 `ERPAuto` 重命名为 `BIPMaterialManager`。本次重命名仅停留在计划阶段,尚未落实到任何具体代码。目前仅对文档中的项目名称进行了更新,代码层面的配置、路径、产物名称等均保持原样,将在后续阶段统一处理。
技术栈:

View File

@@ -1,6 +1,8 @@
# ERPAuto - ERP 数据自动化处理工具
# BIPMaterialManager - BIP 物料管理工具
一个基于 Electron 的桌面应用程序,用于自动化处理 ERP 系统中的数据提取和清理任务
> **项目重命名说明**:本项目计划从 `ERPAuto` 重命名为 `BIPMaterialManager`。本次重命名仅停留在计划阶段,尚未落实到任何具体代码。目前仅对文档中的项目名称进行了更新,代码层面的配置、路径、产物名称等均保持原样,将在后续阶段统一处理
一个基于 Electron 的桌面应用程序,用于自动化处理 BIP 系统中的物料数据提取和清理任务。
## 功能特性
@@ -22,7 +24,7 @@
```bash
# 克隆项目
git clone <repository-url>
cd ERPAuto
cd BIPMaterialManager
# 安装依赖
npm install
@@ -143,7 +145,7 @@ const config = createMockConfigManager({ logging: { level: 'debug' } })
## 项目结构
```
ERPAuto/
BIPMaterialManager/
├── src/
│ ├── main/ # 主进程代码
│ │ ├── services/ # 业务服务
@@ -158,6 +160,8 @@ ERPAuto/
└── docs/ # 文档
```
> **注意**:由于项目正在进行重命名,代码层面的配置文件、路径和产物名称暂时仍使用 `erpauto`,将在后续阶段统一更新。
## 技术栈
- **框架**Electron 39

View File

@@ -4,7 +4,7 @@
# 部署说明:
# 1. 复制此文件为 config.yaml
# 2. 根据实际环境修改配置值
# 3. 设置 database.activeType 为 mysqlsqlserver
# 3. 设置 database.activeType 为 mysqlsqlserver 或 postgresql
# ================================
# 注意ERP 认证信息存储在数据库 (dbo_BIPUsers) 中,按用户管理
# ================================
@@ -29,6 +29,14 @@ database:
driver: 'ODBC Driver 18 for SQL Server'
trustServerCertificate: true
postgresql:
host: <PG_HOST>
port: 5432
database: <DATABASE_NAME>
username: <USERNAME>
password: <PASSWORD>
maxPoolSize: 10
paths:
dataDir: './data/'
defaultOutput: 'output.xlsx'
@@ -57,6 +65,8 @@ orderResolution:
cleaner:
queryBatchSize: 100
processConcurrency: 1
sessionRefreshOrderThreshold: 160 # 会在 batch 边界检查;达到或超过阈值后,在当前 batch 完成后重建浏览器会话
enableRowProtection: true # 行号保护:禁止删除行号 2000-7999 范围内的物料,关闭后不再检查行号范围
logging:
level: info

290
docs/README.md Normal file
View File

@@ -0,0 +1,290 @@
# BIPMaterialManager 文档指南
> **项目重命名说明**:本项目计划从 `ERPAuto` 重命名为 `BIPMaterialManager`。本次重命名仅停留在计划阶段,尚未落实到任何具体代码。目前仅对文档中的项目名称进行了更新,代码层面的配置、路径、产物名称等均保持原样,将在后续阶段统一处理。
本文档是 BIPMaterialManager 项目文档的**分类指南和编写规范**,用于:
- 指导文档的分类和归档
- 规范新文档的命名和格式
- 帮助开发者快速定位应创建的文档类型
---
## 📚 文档分类体系
### 一、按受众分类
| 分类 | 目录 | 受众 | 内容示例 |
| -------------- | ------------ | ---------- | ---------------------------- |
| **用户文档** | `user/` | 最终用户 | 使用指南、配置说明、迁移指南 |
| **开发者文档** | `developer/` | 开发人员 | 架构设计、开发指南、模块说明 |
| **内部文档** | `internal/` | 项目维护者 | 分析报告、优化计划、模板 |
### 二、按内容类型分类
| 分类 | 目录 | 内容特点 |
| ------------ | ----------------------------------- | -------------------------------- |
| **功能特性** | `features/` | 功能说明、业务流程、重构概览 |
| **调试指南** | `debugging/` | 调试指南、快速参考、故障排查 |
| **测试文档** | `testing/` | 测试计划、测试报告、测试基础设施 |
| **模块文档** | `cleaner/`, `browser/`, `database/` | 特定模块的详细文档 |
| **计划文档** | `plans/` | 设计方案、实施计划 |
| **发布说明** | `releases/` | 版本发布记录 |
---
## 📝 文档命名规范
### 文件名格式
```
<主题>-<子主题>-<类型>.md
```
**规则:**
- 使用**小写字母**和**连字符** (`-`)
- 不使用空格、下划线或大写字母
- 保持简短但有描述性
**示例:**
```
✅ user-override-match-feature.md
✅ settings-partial-save.md
✅ cleaner-validation-flow.md
✅ test-improvement-plan.md
❌ UserOverrideMatchFeature.md # 驼峰命名
❌ user_override_match.md # 下划线
❌ user override match.md # 空格
```
### 类型后缀约定
| 后缀 | 用途 | 示例 |
| -------------- | ---------- | ----------------------------------------- |
| `-guide.md` | 指南类文档 | `erp-login-debug-guide.md` |
| `-quickref.md` | 快速参考 | `erp-login-debug-quickref.md` |
| `-flow.md` | 流程说明 | `settings-save-button-flow.md` |
| `-feature.md` | 功能特性 | `user-override-match-feature.md` |
| `-plan.md` | 计划方案 | `test-improvement-plan.md` |
| `-report.md` | 报告总结 | `TEST_REVIEW_REPORT.md` |
| `-template.md` | 模板文件 | `cleaner-execution-report-template.md` |
| `-overview.md` | 概览说明 | `validation-handler-refactor-overview.md` |
### Plans 路径专用命名规范
`plans/` 目录使用**日期前缀**命名法,便于按时间排序和管理:
```
<YYYY-MM-DD>-<描述>-<类型>.md
```
**类型标识:**
| 类型后缀 | 用途 | 内容重点 |
| ------------ | -------- | -------------------------------------- |
| `-plan.md` | 实施计划 | 任务分解、时间线、资源分配、风险评估 |
| `-design.md` | 设计方案 | 技术架构、接口设计、数据模型、决策理由 |
**示例:**
```
✅ 2026-04-13-cleaner-db-persistence-plan.md
✅ 2026-04-13-cleaner-db-persistence-design.md
✅ 2026-04-05-postgresql-integration-plan.md
✅ 2026-04-05-postgresql-integration-design.md
❌ cleaner-db-plan.md # 缺少日期
❌ 2026-4-13-cleaner-db-plan.md # 日期格式不正确(应为 2026-04-13
❌ 2026-04-13-plan-cleaner-db.md # 类型应在最后
```
**相关文件对:**
同一个项目通常会有配对的计划和设计文档:
- `2026-04-13-cleaner-db-persistence-plan.md` - 实施计划
- `2026-04-13-cleaner-db-persistence-design.md` - 设计方案
使用相同的日期和描述,便于关联查找。
---
## 🗂️ 分类决策流程
创建新文档时,按以下流程确定分类:
```
1. 文档的读者是谁?
├─ 最终用户 → user/
├─ 开发者 → developer/
└─ 项目维护者 → internal/ 或其他专业目录
2. 文档的内容类型是什么?
├─ 功能说明 → features/
├─ 调试帮助 → debugging/
├─ 测试相关 → testing/
├─ 模块特定 → cleaner/, browser/, database/
├─ 设计计划 → plans/
└─ 发布记录 → releases/
3. 是否需要快速参考?
└─ 是 → 使用 -quickref.md 后缀,放入 debugging/
```
### 分类示例
| 文档主题 | 正确分类 | 理由 |
| ----------------- | ----------------------------------------------------- | ------------ |
| 如何配置 ERP 连接 | `user/config-erp-guide.md` | 用户操作指南 |
| 日志系统设计 | `developer/architecture/logging-design.md` | 架构设计 |
| 登录失败排查 | `debugging/erp-login-quickref.md` | 调试快速参考 |
| 测试覆盖率分析 | `testing/coverage-analysis-report.md` | 测试报告 |
| 物料清理模块说明 | `cleaner/module-overview.md` | 模块文档 |
| 新功能实施计划 | `plans/2026-04-14-new-feature-implementation-plan.md` | 实施计划 |
| 数据库设计文档 | `plans/2026-04-14-database-schema-design.md` | 设计方案 |
---
## 📋 文档模板
### 指南类文档模板
```markdown
# <功能> 指南
## 概述
简要说明文档目的和适用范围。
## 前置条件
列出使用该功能的前提条件。
## 操作步骤
1. 步骤一
2. 步骤二
3. 步骤三
## 常见问题
- Q: 问题描述
- A: 解决方案
## 相关文档
- [相关文档 1](link)
- [相关文档 2](link)
```
### 功能特性文档模板
```markdown
# <功能名称> 特性说明
## 背景
为什么需要这个功能。
## 功能描述
功能的具体行为和预期结果。
## 用户流程
用户使用该功能的完整流程。
## 技术实现
关键实现细节(可选)。
## 影响范围
对其他模块的影响。
```
### 计划文档模板
```markdown
# <项目名称> 实施计划
## 目标
项目要达成的目标。
## 范围
包含和不包含的内容。
## 任务分解
- [ ] 任务 1
- [ ] 任务 2
- [ ] 任务 3
## 时间线
预计开始和结束时间。
## 风险
可能的风险和应对措施。
```
---
## 🔧 文档维护
### 文档更新
- **功能变更时**:同步更新相关文档
- **发现错误时**:立即修正并提交
- **版本发布时**:更新 `releases/` 中的发布说明
### 文档审查
新文档创建后,应检查:
- [ ] 分类是否正确
- [ ] 命名是否符合规范
- [ ] 是否使用了模板
- [ ] 链接是否有效
- [ ] 是否添加到相关索引
### 废弃文档
过时的文档应:
1. 在文件顶部添加 `> ⚠️ 已废弃` 标记
2. 说明废弃原因和替代文档
3. 在下一个版本发布时移至 `archive/` 目录
---
## 📖 根目录文档
`docs/` 根目录仅保留**跨category的项目级文档**
| 文档 | 用途 |
| -------------------------------------- | ----------------- |
| `README.md` | 本文档 - 分类指南 |
| `build-and-release-guide.md` | 构建和发布流程 |
| `portable-auto-update-architecture.md` | 便携版更新架构 |
**原则**:如果文档不属于特定分类,且对项目整体重要,可放在根目录。
---
## 🔍 找不到合适的分类?
如果现有分类无法容纳你的文档:
1. 检查是否可以归入 `internal/`(内部文档)
2. 考虑是否应该创建新的子目录
3. 在提交 PR 时说明分类理由
---
_最后更新2026-04-14_

View File

@@ -1,6 +1,8 @@
# Playwright 部署说明
本文档聚焦“如何让 ERPAuto 在目标机器上拥有可用的 Playwright Chromium 浏览器”,适合作为实际部署操作说明
> **项目重命名说明**:本项目计划从 `ERPAuto` 重命名为 `BIPMaterialManager`。本次重命名仅停留在计划阶段,尚未落实到任何具体代码。目前仅对文档中的项目名称进行了更新,代码层面的配置、路径、产物名称等均保持原样,将在后续阶段统一处理
本文档聚焦”如何让 BIPMaterialManager 在目标机器上拥有可用的 Playwright Chromium 浏览器”,适合作为实际部署操作说明。
如果你想看版本信息,请同时参考:

View File

@@ -1,6 +1,8 @@
# 构建与发布流程
本文档说明 ERPAuto Windows 便携版的当前构建与发布方式,包括推荐的一键发布命令、分步命令,以及发布产物在对象存储中的结构
> **项目重命名说明**:本项目计划从 `ERPAuto` 重命名为 `BIPMaterialManager`。本次重命名仅停留在计划阶段,尚未落实到任何具体代码。目前仅对文档中的项目名称进行了更新,代码层面的配置、路径、产物名称等均保持原样,将在后续阶段统一处理
本文档说明 BIPMaterialManager Windows 便携版的当前构建与发布方式,包括推荐的一键发布命令、分步命令,以及发布产物在对象存储中的结构。
## 概览

View File

@@ -0,0 +1,309 @@
# 清理器角色差异流程 — Admin vs User
**文档版本**: 1.1
**创建日期**: 2026-04-06
**面向对象**: 开发人员
## 概述
清理器Cleaner在决定"哪些物料需要被清除"时Admin 和 User 两个角色存在系统性的差异。这些差异贯穿三个阶段:**初始化 → 校验确认 → 执行清理**。
本文档使用 Mermaid 图表说明每个阶段的角色分支逻辑。
---
## 全局流程概览
```mermaid
flowchart TB
subgraph init["阶段一:页面初始化"]
I1([页面加载]) --> I2{角色判断}
I2 -->|Admin| I3["管理员列表 ← 全部负责人<br/>默认选中全部"]
I2 -->|User| I4["管理员列表 ← 空<br/>默认选中仅自己"]
end
subgraph validate["阶段二:校验 → 勾选 → 同步数据库"]
V1([点击校验]) --> V2["后端查询物料<br/>(不区分角色)"]
V2 --> V3["物料匹配算法<br/>User 有覆盖匹配)"]
V3 --> V4{角色判断}
V4 -->|Admin| V5["显示全部物料<br/>侧边栏可按负责人筛选"]
V4 -->|User| V6["仅显示自己的物料<br/>+ 无负责人的物料"]
V5 --> V7["用户勾选/取消勾选"]
V6 --> V7
V7 --> V8{点击同步数据库}
V8 --> V9{角色判断}
V9 -->|Admin| V10["处理范围:全部校验结果"]
V9 -->|User| V11["处理范围:仅筛选后结果"]
end
subgraph execute["阶段三执行清理ERP 删除)"]
E1([点击执行清理]) --> E2["getCleanerData(selectedManagers)<br/>获取物料代码"]
E2 --> E3{角色判断}
E3 -->|Admin| E3a{selectedManagers<br/>非空?}
E3a -->|"是"| E4["SQL WHERE ManagerName IN (选中)<br/>从 MaterialsToBeDeleted 获取"]
E3a -->|"否"| E4b["从 DiscreteMaterialPlanData<br/>按 orderNumbers 获取"]
E3 -->|User| E5["SQL WHERE ManagerName = 用户<br/>仅获取自己的物料代码"]
E4 --> E6["传递给 runCleaner 执行"]
E4b --> E6
E5 --> E6
E6 --> E7([在 ERP 中删除物料])
end
init --> validate --> execute
```
---
## 阶段一:页面初始化
**源码位置**: `src/renderer/src/hooks/cleaner/api.ts:25-52``src/renderer/src/hooks/useCleaner.ts:98-112`
```mermaid
flowchart TB
Start([页面加载]) --> GetAdmin["调用 auth:isAdmin<br/>判断是否管理员"]
GetAdmin --> GetUser["调用 auth:getCurrentUser<br/>获取当前用户名"]
GetUser --> RoleCheck{isAdmin?}
RoleCheck -->|Admin| GetManagers["调用 materials:getManagers<br/>获取全部负责人列表"]
GetManagers --> SelectAll["selectedManagers ← 全部负责人<br/>(默认全选)"]
SelectAll --> RenderSidebar["渲染 CleanerSidebar<br/>显示负责人复选框"]
RoleCheck -->|User| SetSelf["selectedManagers ← {currentUsername}<br/>(仅选中自己)"]
SetSelf --> NoSidebar["不渲染 CleanerSidebar<br/>无侧边栏"]
RenderSidebar --> Ready([就绪])
NoSidebar --> Ready
```
**差异总结**:
| 维度 | Admin | User |
| ---------- | ----------------- | ------ |
| 侧边栏 | 有 CleanerSidebar | 无 |
| 管理员列表 | 查询全部负责人 | 不查询 |
| 默认选中 | 所有负责人 | 仅自己 |
---
## 阶段二:校验 → 勾选 → 同步数据库
### 2.1 物料校验(后端,不区分角色)
**源码位置**: `src/main/services/validation/validation-application-service.ts`
校验阶段后端查询不区分角色Admin 和 User 拿到相同的物料数据。区别在于**匹配算法**
```mermaid
flowchart TB
Start([遍历每条物料记录]) --> P1{"优先级1<br/>MaterialsToBeDeleted<br/>精确匹配 MaterialCode?"}
P1 -->|"匹配"| SetManager["managerName ← 表中记录<br/>isMarkedForDeletion = true"]
P1 -->|"未匹配"| P2{"优先级2<br/>MaterialsTypeToBeDeleted<br/>MaterialName 包含匹配?"}
P2 -->|"匹配"| SetType["managerName ← 类型关键词负责人<br/>matchedTypeKeyword ← 匹配项"]
P2 -->|"未匹配"| SetNull["managerName = null"]
SetManager --> RoleCheck{角色?}
SetType --> RoleCheck
SetNull --> RoleCheck
RoleCheck -->|"Admin"| Skip["跳过覆盖<br/>使用当前结果"]
RoleCheck -->|"User"| P3{"优先级3User 覆盖)<br/>自己的类型关键词匹配?"}
P3 -->|"匹配"| Override["强制覆盖<br/>managerName ← 当前用户"]
P3 -->|"未匹配"| Keep["保持当前结果"]
Skip --> Next(["下一条物料"])
Override --> Next
Keep --> Next
```
**匹配优先级说明**:
| 优先级 | 数据源 | 匹配方式 | 适用角色 |
| -------------- | -------------------------- | --------------------- | -------- |
| 1最高 | `MaterialsToBeDeleted` | MaterialCode 精确匹配 | 全部 |
| 2 | `MaterialsTypeToBeDeleted` | MaterialName 包含匹配 | 全部 |
| 3User 覆盖) | 当前用户的类型关键词 | MaterialName 包含匹配 | 仅 User |
> **优先级 3 的作用**:当某个物料按优先级 2 被分配给其他负责人,但当前 User 有匹配的类型关键词时,会强制覆盖为自己的。这确保 User 不会为他人操作物料。
### 2.2 前端显示过滤
**源码位置**: `src/renderer/src/hooks/cleaner/helpers.ts:34-57`
校验结果返回前端后,会根据角色进行显示过滤:
```mermaid
flowchart TB
Input([校验结果 validationResults]) --> RoleCheck{角色判断}
RoleCheck -->|Admin| FilterManagers["按侧边栏选中的负责人过滤<br/>selectedManagers.has(managerName)<br/>|| !managerName"]
RoleCheck -->|User| FilterSelf["仅显示自己的 + 无负责人的<br/>managerName === currentUsername<br/>|| !managerName"]
FilterManagers --> FilterHidden["排除已隐藏的物料<br/>!hiddenItems.has(materialCode)"]
FilterSelf --> FilterHidden
FilterHidden --> Output([filteredResults<br/>用于表格显示])
```
### 2.3 确认删除(同步数据库)
**源码位置**: `src/renderer/src/hooks/useCleaner.ts:289-344`
```mermaid
flowchart TB
Start([点击确认删除]) --> RoleScope{角色判断}
RoleScope -->|Admin| UseAll["resultsToProcess = validationResults<br/>处理全部校验结果"]
RoleScope -->|User| UseFiltered["resultsToProcess = filteredResults<br/>仅处理筛选后结果"]
UseAll --> BuildPlan["buildDeletionPlan(resultsToProcess, selectedItems)"]
UseFiltered --> BuildPlan
BuildPlan --> Loop["遍历 resultsToProcess"]
Loop --> Check{物料是否勾选?}
Check -->|"已勾选"| HasManager{有负责人?}
Check -->|"未勾选"| ToDelete["加入 materialsToDelete<br/>从数据库移除标记"]
HasManager -->|"有"| ToUpsert["加入 materialsToUpsert<br/>写入/更新到数据库"]
HasManager -->|"无"| Missing["加入 missingManager<br/>阻止操作"]
ToUpsert --> Save["调用 materials:upsertBatch"]
ToDelete --> Del["调用 materials:delete"]
Missing --> Warn(["弹窗警告:缺少负责人"])
Save --> Done([完成])
Del --> Done
```
**关键代码**:
```typescript
// Admin 处理全部结果User 只处理筛选后的结果
const resultsToProcess = isAdmin ? validationResults : filteredResults
```
**差异总结**:
| 维度 | Admin | User |
| ---------------- | --------------------------- | -------------------------------------- |
| 处理范围 | `validationResults`(全部) | `filteredResults`(自己的+无负责人的) |
| 可操作物料 | 所有负责人的物料 | 仅自己的 + 无负责人的 |
| 能否修改他人数据 | 是 | 否 |
---
## 阶段三执行清理ERP 删除)
**源码位置**:
- 前端调用: `src/renderer/src/hooks/cleaner/api.ts:116-166`
- 获取数据: `src/main/services/validation/validation-application-service.ts:497-655`
- 执行删除: `src/main/services/cleaner/cleaner-application-service.ts`
```mermaid
sequenceDiagram
participant UI as 前端 useCleaner
participant API as api.ts
participant Main as 主进程
participant DB as 数据库
participant ERP as ERP 系统
UI->>API: runCleanerExecution({ dryRun, selectedManagers, ... })
API->>Main: getCleanerData({ selectedManagers })
alt Admin + selectedManagers 非空
Main->>DB: SELECT MaterialCode FROM MaterialsToBeDeleted<br/>WHERE ManagerName IN (@manager0, @manager1, ...)
Note over Main,DB: 按选中的负责人过滤<br/>从 MaterialsToBeDeleted 获取
else Admin + selectedManagers 为空
Main->>DB: SELECT DISTINCT MaterialCode FROM DiscreteMaterialPlanData<br/>WHERE SourceNumber IN (orderNumbers)
Note over Main,DB: 按订单号查询<br/>从 DiscreteMaterialPlanData 获取
else User
Main->>DB: SELECT MaterialCode FROM MaterialsToBeDeleted<br/>WHERE ManagerName = @username
Note over Main,DB: 按 ManagerName 过滤<br/>仅获取自己的物料代码
end
DB-->>Main: materialCodes[]
Main-->>API: { orderNumbers, materialCodes }
Note over API: 传入角色过滤后的 materialCodes
API->>Main: cleaner.runCleaner({ orderNumbers, materialCodes, ... })
Main->>ERP: 按订单遍历,删除指定物料
ERP-->>Main: 删除结果
Main-->>API: CleanerResult
API-->>UI: 显示执行报告
```
**SQL 差异**:
```mermaid
flowchart TB
subgraph AdminWithMgr["Admin + selectedManagers 非空"]
A1["SELECT MaterialCode<br/>FROM MaterialsToBeDeleted<br/>WHERE ManagerName IN (@manager0, ...)<br/>AND MaterialCode IS NOT NULL"]
end
subgraph AdminNoMgr["Admin + selectedManagers 为空"]
A2["SELECT DISTINCT MaterialCode<br/>FROM DiscreteMaterialPlanData<br/>WHERE SourceNumber IN (orderNumbers)"]
end
subgraph User["User 查询"]
U1["SELECT MaterialCode<br/>FROM MaterialsToBeDeleted<br/>WHERE ManagerName = @username<br/>AND MaterialCode IS NOT NULL"]
end
AdminWithMgr --> |"按选中负责人过滤"| Result([传入 runCleaner])
AdminNoMgr --> |"按订单号查 DiscreteMaterialPlanData"| Result
User --> |"仅返回自己的物料代码"| Result
```
**差异总结**:
| 维度 | Admin有 selectedManagers | Admin无 selectedManagers | User |
| ---------- | ---------------------------- | -------------------------------------- | ------------------------------- |
| 数据源 | `MaterialsToBeDeleted` | `DiscreteMaterialPlanData` | `MaterialsToBeDeleted` |
| 查询条件 | `WHERE ManagerName IN (...)` | `WHERE SourceNumber IN (orderNumbers)` | `WHERE ManagerName = @username` |
| 可删除物料 | 选中负责人的物料 | 订单关联的全部物料 | 仅自己标记的物料 |
| 无订单号时 | — | 返回空数组 | — |
---
## 数据安全边界
角色隔离在**三个层面**同时生效,形成纵深防御:
```mermaid
flowchart TB
subgraph layer1["第一层:前端过滤"]
L1["filterValidationResults()<br/>User 仅看到自己的物料"]
end
subgraph layer2["第二层:同步范围"]
L2["handleConfirmDeletion()<br/>User 仅同步 filteredResults"]
end
subgraph layer3["第三层:后端查询"]
L3["loadMaterialCodesForCleaner()<br/>Admin: WHERE ManagerName IN (selectedManagers)<br/>User: SQL WHERE ManagerName = user"]
end
L1 -->|"防止误操作"| L2
L2 -->|"缩小同步范围"| L3
L3 -->|"最终保证"| Safe([User 无法删除他人物料])
```
> **注意**`runCleaner()` 本身不做角色过滤,它信任上游传入的 `materialCodes` 已经过角色过滤。安全性由 `getCleanerData()` 的 SQL 查询保证。
---
## 涉及文件索引
| 文件 | 关键函数/逻辑 | 行号 |
| ---------------------------------------------------------------- | ------------------------------------------------------------------------------------- | ------------------------- |
| `src/renderer/src/hooks/cleaner/api.ts` | `initializeCleanerPage()`, `runCleanerExecution()` | 25-52, 116-166 |
| `src/renderer/src/hooks/useCleaner.ts` | `handleConfirmDeletion()`, 初始化逻辑 | 98-120, 289-345 |
| `src/renderer/src/hooks/cleaner/helpers.ts` | `filterValidationResults()`, `buildDeletionPlan()` | 34-57, 59-92 |
| `src/main/services/validation/validation-application-service.ts` | `getCleanerData()`, `loadMaterialCodesForCleaner()`, `queryMaterialCodesByManagers()` | 232-305, 497-604, 606-655 |
| `src/main/services/cleaner/cleaner-application-service.ts` | `runCleaner()` | 31-168 |
| `src/main/ipc/cleaner-handler.ts` | `CLEANER_RUN` handler | 16-22 |
| `src/main/ipc/validation-handler.ts` | `getCleanerData` handler | 194-223 |
| `src/preload/api/validation.ts` | `getCleanerData()` IPC 桥接 | 11-12 |
| `src/renderer/src/pages/CleanerPage.tsx` | 页面组件,条件渲染侧边栏 | 74-82 |

View File

@@ -188,21 +188,26 @@ flowchart LR
**表名转换逻辑**:
```typescript
// MySQL: dbo_MaterialsToBeDeleted
// 输入格式: dbo.MaterialsToBeDeleted
// SQL Server: [dbo].[MaterialsToBeDeleted]
function getTableName(mysqlTableName: string): string {
const dbType = process.env.DB_TYPE?.toLowerCase()
if (dbType === 'sqlserver' || dbType === 'mssql') {
// 找到第一个下划线分割schema和表名
const firstUnderscoreIndex = mysqlTableName.indexOf('_')
if (firstUnderscoreIndex > 0) {
const schema = mysqlTableName.substring(0, firstUnderscoreIndex)
const tableName = mysqlTableName.substring(firstUnderscoreIndex + 1)
// PostgreSQL: "dbo"."MaterialsToBeDeleted"
function getValidationTableName(dottedTableName: string): string {
const configManager = ConfigManager.getInstance()
const dbType = configManager.getDatabaseType()
const dotIndex = dottedTableName.indexOf('.')
if (dotIndex > 0) {
const schema = dottedTableName.substring(0, dotIndex)
const tableName = dottedTableName.substring(dotIndex + 1)
if (dbType === 'sqlserver') {
return `[${schema}].[${tableName}]`
}
return `[dbo].[${mysqlTableName}]`
return `"${schema}"."${tableName}"`
}
return mysqlTableName
if (dbType === 'sqlserver') {
return `[dbo].[${dottedTableName}]`
}
return `"public"."${dottedTableName}"`
}
```

View File

@@ -541,7 +541,7 @@ graph LR
### 10.2 默认配置示例
```env
DB_TABLE_NAME=productionContractData_26 年压力表合同数据
DB_TABLE_NAME=ERPAuto.vw_productionContractData
DB_FIELD_PRODUCTION_ID=总排号
DB_FIELD_ORDER_NUMBER=生产订单号
```

View File

@@ -0,0 +1,143 @@
# PostgreSQL 集成设计文档
**日期:** 2026-04-05
**状态:** 已批准
**分支:** dev-logging
## 目标
将 PostgreSQL 作为第三种可选数据库类型集成到 ERPAuto 中,与现有 MySQL、SQL Server 并列。通过引入 SqlDialect 抽象层,统一管理三种数据库的 SQL 方言差异,同时重构现有 DAO 层消除散落的 `isSqlServer` 判断。
## 背景
- PostgreSQL 数据库已通过 SSMA 从 SQL Server 迁移完成表结构、schema 组织、列名完全一致
- 连接信息:`postgresql://admin:***@192.168.31.83:5432/postgres`,数据库 `CompanyDB`
- 共 15 个 schema、151 张表,`dbo` schema 包含 ERPAuto 直接使用的表
## 方案:抽象数据库方言层
### 1. SqlDialect 接口
新建 `src/main/types/sql-dialect.types.ts`
```typescript
export interface SqlDialect {
readonly dbType: DatabaseType
// 表名引用
quoteTableName(schema: string, table: string): string
// 参数占位符
param(index: number): string
params(count: number): string
// SQL 函数
currentTimestamp(): string
// UPSERT
upsert(p: {
table: string
keyColumns: string[]
valueColumns: string[]
placeholderCount: number
startParamIndex: number
}): string
// 分页
paginate(p: { sql: string; limit: number; offset?: number; paramIndex: number }): {
sql: string
paramIndex: number
}
// 批量限制
maxBatchRows(columnsPerRow: number): number
}
```
### 2. 三种方言实现
新建 `src/main/services/database/dialects/` 目录:
| 文件 | 数据库 | param(n) | quoteTableName | currentTimestamp | upsert | paginate |
| ----------------------- | ---------- | -------- | --------------- | ------------------- | ------------------ | ------------------ |
| `mysql-dialect.ts` | MySQL | `?` | `dbo_Table` | `NOW()` | `ON DUPLICATE KEY` | `LIMIT x OFFSET y` |
| `sqlserver-dialect.ts` | SQL Server | `@p{n}` | `[dbo].[Table]` | `GETDATE()` | `MERGE` | `OFFSET/FETCH` |
| `postgresql-dialect.ts` | PostgreSQL | `${n+1}` | `"dbo"."Table"` | `CURRENT_TIMESTAMP` | `ON CONFLICT` | `LIMIT x OFFSET y` |
方言工厂 `dialects/index.ts`
```typescript
export function createDialect(type: DatabaseType): SqlDialect
```
### 3. DAO 层重构
每个 DAO 新增 `dialect` 成员,替代原有的 `getTableName()``buildPlaceholders()` 和所有 `isSqlServer` 分支:
**删除:**
- `getTableName()` 私有方法
- `buildPlaceholders()` 私有方法
- 所有 `isSqlServer` 局部变量和条件分支
- `*_CONFIG` 中的 `TABLE_NAME_SQLSERVER` / `TABLE_NAME_MYSQL` → 合并为 `TABLE_SCHEMA` + `TABLE_NAME`
**新增:**
- `private dialect: SqlDialect | null = null`
- `private getDialect(): SqlDialect`
**涉及 DAO**
- `DiscreteMaterialPlanDAO` — 占位符、表名、批量大小
- `MaterialsToBeDeletedDAO` — 占位符、表名、MERGE/ON DUPLICATE KEY → `upsert()`
- `MaterialsTypeToBeDeletedDAO` — 同上
- `ExtractorOperationHistoryDAO` — 占位符、表名、GETDATE()/NOW() → `currentTimestamp()`、分页 → `paginate()`
### 4. PostgreSQL 服务层
新建 `src/main/services/database/postgresql.ts`
- 使用 `pg` 驱动,`Pool` 连接池
- 实现 `IDatabaseService` 接口
- `query()` 直接传递参数数组给 `pg`
- `transaction()` 使用 `client.query('BEGIN/COMMIT/ROLLBACK')`
### 5. 工厂、配置、TypeORM
**database/index.ts** `create()` 新增 `'postgresql'` 分支,新增 `createPostgreSqlConfig()`
**database.types.ts** `DatabaseType` 扩展为 `'mysql' | 'sqlserver' | 'postgresql'`,新增 `PostgreSqlConfig`
**data-source.ts** TypeORM `type` 映射新增 `'postgres'`
**config.template.yaml** 新增 `postgresql` 配置段
**package.json** 新增 `pg` 依赖
## 改动范围
| 层 | 文件 | 动作 |
| ------- | ---------------------------------------------- | ---- |
| 类型 | `types/database.types.ts` | 修改 |
| 方言 | `database/dialects/index.ts` | 新建 |
| 方言 | `database/dialects/mysql-dialect.ts` | 新建 |
| 方言 | `database/dialects/sqlserver-dialect.ts` | 新建 |
| 方言 | `database/dialects/postgresql-dialect.ts` | 新建 |
| 服务 | `database/postgresql.ts` | 新建 |
| 工厂 | `database/index.ts` | 修改 |
| TypeORM | `database/data-source.ts` | 修改 |
| DAO | `database/discrete-material-plan-dao.ts` | 重构 |
| DAO | `database/materials-to-be-deleted-dao.ts` | 重构 |
| DAO | `database/materials-type-to-be-deleted-dao.ts` | 重构 |
| DAO | `database/extractor-operation-history-dao.ts` | 重构 |
| 配置 | `config.template.yaml` | 修改 |
| 依赖 | `package.json` | 修改 |
**4 个新文件 + 10 个修改文件**
## 不在范围内
- IPC 处理器新增(前端暂不需要直接切换 PostgreSQL
- Entity/Repository 的 TypeScript 类型适配TypeORM 内部处理方言差异)
- 数据迁移工具
- 前端 UI 变更

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,239 @@
# Cleaner 数据库持久化设计
## 背景
Cleaner 当前使用 Markdown 文件做执行记录持久化,通过 RustFS 上传存储。存在以下问题:
- 报告是非结构化文本,无法程序化查询和统计
- 历史记录无法按用户、时间、状态筛选
- 重试时依赖文件名去重,覆盖了首次执行的崩溃信息
- 前端需要通过 RustFS 下载报告再解析展示,链路长且脆弱
Extractor 已有成熟的数据库持久化模式(`ExtractorOperationHistory` 表 + DAO + 前端弹窗Cleaner 应复用相同模式。
## 设计决策
| 决策项 | 选择 | 理由 |
| -------- | ------------------------------------------------------------ | ----------------------------------------------------------------------------------- |
| 表结构 | 独立建表,不与 Extractor 共用 | Cleaner 数据结构差异大(双层、物料级详情),独立更清晰 |
| 记录粒度 | 执行 + 订单 + 物料三层 | 执行表存全局信息,订单表存订单汇总,物料表存操作明细 |
| 批次标识 | `BatchId`UUID与 Extractor 一致 | 标准、简洁,不需要嵌入时间戳 |
| 重试记录 | 不覆盖,每次尝试独立写入,用 `AttemptNumber` 区分 | 保留完整审计链,为后续智能跳过提供数据基础 |
| 报告文件 | 移除 Markdown 报告和 RustFS 上传 | 数据库完全替代报告相关代码CleanerReportGenerator、generateAndUploadReport删除 |
| 前端历史 | 独立 CleanerOperationHistoryModal复用 Extractor 的 UI 模式 | 放在 CleanerPage 上,与 Extractor 的"操作历史"按钮对齐 |
## 数据库表结构
所有表的 schema 为 `ERPAuto`
### 1. `CleanerExecution`(执行级)
全限定名:`ERPAuto.CleanerExecution`
一次清理操作(含重试)的全局信息。每次尝试一行记录。
| 列名 | 类型 | 说明 |
| ----------------------- | ---------------- | ---------------------------------------------- |
| ID | INT IDENTITY | 自增主键 |
| BatchId | UNIQUEIDENTIFIER | 批次 ID一次清理操作含重试共享 |
| AttemptNumber | INT | 第几次尝试1=首次2=外层重试) |
| UserId | INT | 操作用户 ID |
| Username | NVARCHAR(255) | 操作用户名 |
| OperationTime | DATETIME | 操作时间 |
| EndTime | DATETIME | 结束时间 |
| Status | NVARCHAR(50) | pending / success / failed / partial / crashed |
| IsDryRun | BIT | 是否模拟运行 |
| TotalOrders | INT | 订单总数 |
| OrdersProcessed | INT | 已处理订单数 |
| TotalMaterialsDeleted | INT | 总删除物料数 |
| TotalMaterialsSkipped | INT | 总跳过物料数 |
| TotalMaterialsFailed | INT | 总失败物料数 |
| TotalUncertainDeletions | INT | 总不确定删除数 |
| ErrorMessage | NVARCHAR(MAX) | 全局错误信息(如外层崩溃原因) |
| AppVersion | NVARCHAR(20) | 应用版本号 |
### 2. `CleanerOrderHistory`(订单级)
全限定名:`ERPAuto.CleanerOrderHistory`
每个订单在每次尝试中的执行结果。每个订单每次尝试一行记录。
| 列名 | 类型 | 说明 |
| ------------------ | ---------------- | -------------------------- |
| ID | INT IDENTITY | 自增主键 |
| BatchId | UNIQUEIDENTIFIER | 关联执行表 BatchId |
| AttemptNumber | INT | 关联执行表 AttemptNumber |
| OrderNumber | NVARCHAR(255) | 订单号 |
| Status | NVARCHAR(50) | pending / success / failed |
| MaterialsDeleted | INT | 删除物料数 |
| MaterialsSkipped | INT | 跳过物料数 |
| MaterialsFailed | INT | 删除失败物料数 |
| UncertainDeletions | INT | 不确定删除数 |
| RetryCount | INT | 内层重试次数 |
| RetrySuccess | BIT | 内层重试是否成功 |
| ErrorMessage | NVARCHAR(MAX) | 错误信息 |
关联方式:`BatchId + AttemptNumber` 关联执行表。
### 3. `CleanerMaterialDetail`(物料级)
全限定名:`ERPAuto.CleanerMaterialDetail`
每个物料在每次尝试中的操作明细。
| 列名 | 类型 | 说明 |
| ------------------ | ---------------- | -------------------------------------- |
| ID | INT IDENTITY | 自增主键 |
| BatchId | UNIQUEIDENTIFIER | 关联执行表 BatchId |
| AttemptNumber | INT | 关联执行表 AttemptNumber |
| OrderNumber | NVARCHAR(255) | 所属订单号 |
| MaterialCode | NVARCHAR(255) | 物料代码 |
| MaterialName | NVARCHAR(255) | 物料名称 |
| RowNumber | INT | 行号 |
| Result | NVARCHAR(50) | deleted / skipped / failed / uncertain |
| Reason | NVARCHAR(MAX) | 跳过/失败原因 |
| AttemptCount | INT | 删除尝试次数 |
| FinalErrorCategory | NVARCHAR(50) | 最终错误分类 |
关联方式:`BatchId + AttemptNumber + OrderNumber` 关联订单表。
### 数据示例
首次执行到第 80 个订单时崩溃,外层重试成功完成全部 211 个订单:
**CleanerExecution**
```
BatchId=uuid-1, Attempt=1, Status=crashed, TotalOrders=211, Processed=80, ...
BatchId=uuid-1, Attempt=2, Status=success, TotalOrders=211, Processed=211, ...
```
**CleanerOrderHistory**Attempt=1 中部分记录)
```
BatchId=uuid-1, Attempt=1, Order=SC001, Status=success, Deleted=5, Skipped=1
BatchId=uuid-1, Attempt=1, Order=SC080, Status=crashed, Error=查询超时
```
**CleanerOrderHistory**Attempt=2 中部分记录)
```
BatchId=uuid-1, Attempt=2, Order=SC001, Status=success, Deleted=5, Skipped=1
BatchId=uuid-1, Attempt=2, Order=SC080, Status=success, Deleted=3, Skipped=0
BatchId=uuid-1, Attempt=2, Order=SC211, Status=success, Deleted=2, Skipped=0
```
**CleanerMaterialDetail**SC080 在 Attempt=2 中的物料)
```
BatchId=uuid-1, Attempt=2, Order=SC080, Material=MAT-001, Result=deleted
BatchId=uuid-1, Attempt=2, Order=SC080, Material=MAT-002, Result=skipped, Reason=不可删除
```
## 写入时机
```
用户点击"执行清理"
→ IPC: cleaner:run
→ cleaner-handler.ts
→ ① BatchId = randomUUID()
→ ② 插入 CleanerExecutionStatus=pending
→ ③ 插入 CleanerOrderHistory所有订单Status=pending
→ ④ 执行清理CleanerApplicationService.runCleaner
→ ⑤ 更新 CleanerExecutionStatus=success/failed/partial/crashed
→ ⑥ 更新 CleanerOrderHistory每个订单的结果
→ ⑦ 插入 CleanerMaterialDetail每个物料的操作明细
→ ⑧ 如果 crashed → 外层重试
→ 插入新的 CleanerExecutionAttemptNumber=2, Status=pending
→ 插入新的 CleanerOrderHistoryAttemptNumber=2, Status=pending
→ 重新执行
→ 更新执行表和订单表状态
→ 插入物料明细
```
- 步骤 ②③:在 `cleaner-handler.ts` 中,执行前写入,记录操作人、全局配置、待处理订单
- 步骤 ⑤⑥⑦:在 `CleanerApplicationService` 中,执行完成后回调 DAO 写入结果
- 步骤 ⑧:外层重试时,三张表都新增 AttemptNumber=2 的记录,首次尝试的数据完整保留
## 变更清单
### 新增文件
1. **`src/main/services/database/cleaner-operation-history-dao.ts`**
- `CleanerOperationHistoryDAO`
- 执行表操作insertExecution、updateExecutionStatus
- 订单表操作insertOrderRecords、updateOrderStatus
- 物料表操作insertMaterialDetails
- 查询操作getBatches、getBatchDetails含订单+物料、deleteBatch
- 参考 `ExtractorOperationHistoryDAO` 的模式,表名使用 `ERPAuto.CleanerExecution``ERPAuto.CleanerOrderHistory``ERPAuto.CleanerMaterialDetail`
2. **`src/main/types/cleaner-history.types.ts`**
- `CleanerExecutionRecord``CleanerOrderRecord``CleanerMaterialRecord`
- `CleanerBatchStats``InsertCleanerExecutionInput``InsertOrderInput``InsertMaterialDetailInput`
3. **`src/renderer/src/components/CleanerOperationHistoryModal.tsx`**
- 操作历史弹窗,复用 ExtractorOperationHistoryModal 的 UI 模式
- 批次列表(按 BatchId 聚合,显示操作时间、用户、状态、成功/失败数,区分多次尝试)
- 展开明细(订单列表,每订单的删除/跳过/失败数)
- 物料级详情(第二层展开,显示每个物料的操作结果)
- 管理员可按用户筛选、可删除批次
### 修改文件
4. **`src/main/ipc/cleaner-handler.ts`**
- `CLEANER_RUN` handler 中:执行前插入 execution + order 的 pending 记录,执行后更新结果
- 新增 IPC handlers`CLEANER_HISTORY_BATCHES``CLEANER_HISTORY_DETAILS``CLEANER_HISTORY_DELETE`
5. **`src/main/services/cleaner/cleaner-application-service.ts`**
- `runCleaner` 接收 `batchId` 参数
- 移除 `generateExecutionId()` 函数
- 移除 `generateAndUploadReport()` 方法
- 移除 `executionId` 相关逻辑
- 外层重试时,通过 DAO 写入 AttemptNumber=2 的执行记录和订单记录,不覆盖首次尝试
- 执行完成后回调 DAO 写入订单结果和物料明细
6. **`src/main/ipc/index.ts`**
- 注册新的 cleaner history IPC handlers
7. **`src/preload/api/cleaner.ts`**
- 新增 IPC 调用方法getBatches、getBatchDetails、deleteBatch
8. **`src/preload/index.d.ts`**
- `CleanerAPI` 接口新增 getBatches、getBatchDetails、deleteBatch 类型声明
9. **`src/renderer/src/pages/CleanerPage.tsx`**
- 新增"操作历史"按钮
- 引入 CleanerOperationHistoryModal
### 删除文件
10. **`src/main/services/report/cleaner-report-generator.ts`**
- 整个文件删除,报告生成逻辑不再需要
### 可选清理
11. **`src/renderer/src/components/ReportViewerDialog.tsx`**
- 基于 RustFS 文件的报告查看器Cleaner 不再使用
- 如果 Extractor 不共用此组件,可删除
12. **`src/renderer/src/components/ReportAnalysisDialog.tsx`**
- 基于报告文件的分析Cleaner 不再使用
- 后续可基于数据库重新实现统计分析
## 移除的概念
| 概念 | 原因 |
| ------------------------------ | --------------------------------- |
| ExecutionIdCLN-时间戳-随机) | 为文件名设计,数据库用 UUID |
| generateExecutionId() | 随 ExecutionId 一起移除 |
| CleanerReportGenerator | Markdown 报告生成器,被数据库替代 |
| generateAndUploadReport() | RustFS 上传链路,被数据库写入替代 |
| 报告文件名去重 | 数据库 UUID 天然唯一 |
| 重试覆盖旧报告 | 数据库保留所有尝试记录 |
## 不涉及的部分
- Extractor 的持久化逻辑不变
- 数据库 schema 迁移(需 DBA 创建表,应用层只做 CRUD
- 后续智能跳过功能(基于已有 success 记录跳过已成功的订单)
- 内层重试逻辑(订单级/物料级)不变

View File

@@ -0,0 +1,699 @@
# Cleaner 数据库持久化实施计划
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** 将 Cleaner 的执行记录从 Markdown 文件持久化迁移到数据库(三张表:执行级、订单级、物料级),并在前端新增操作历史弹窗。
**Architecture:** 新建 `CleanerOperationHistoryDAO` 操作三张表(`ERPAuto.CleanerExecution``ERPAuto.CleanerOrderHistory``ERPAuto.CleanerMaterialDetail`),通过新增 IPC handlers 暴露给前端。执行前写入 pending 记录,执行后更新结果和物料明细。外层重试时新增 AttemptNumber=2 的记录,不覆盖首次尝试。移除 Markdown 报告生成和 RustFS 上传链路。
**Tech Stack:** TypeScript, Electron IPC, SQL (MySQL/SQL Server/PostgreSQL via existing DAO+dialect pattern), React
---
## Task 1: 新增类型定义
**Files:**
- Create: `src/main/types/cleaner-history.types.ts`
**Step 1: 创建类型文件**
```typescript
// src/main/types/cleaner-history.types.ts
/**
* Cleaner 操作历史类型定义
*/
/** 执行级记录 */
export interface CleanerExecutionRecord {
id?: number
batchId: string
attemptNumber: number
userId: number
username: string
operationTime: Date
endTime: Date | null
status: string
isDryRun: boolean
totalOrders: number
ordersProcessed: number
totalMaterialsDeleted: number
totalMaterialsSkipped: number
totalMaterialsFailed: number
totalUncertainDeletions: number
errorMessage: string | null
appVersion: string | null
}
/** 订单级记录 */
export interface CleanerOrderRecord {
id?: number
batchId: string
attemptNumber: number
orderNumber: string
status: string
materialsDeleted: number
materialsSkipped: number
materialsFailed: number
uncertainDeletions: number
retryCount: number
retrySuccess: boolean
errorMessage: string | null
}
/** 物料级记录 */
export interface CleanerMaterialRecord {
id?: number
batchId: string
attemptNumber: number
orderNumber: string
materialCode: string
materialName: string
rowNumber: number
result: string
reason: string | null
attemptCount: number
finalErrorCategory: string | null
}
/** 批次统计(前端列表展示用) */
export interface CleanerBatchStats {
batchId: string
userId: number
username: string
operationTime: string
/** 最终一次尝试的状态 */
status: string
totalAttempts: number
totalOrders: number
ordersProcessed: number
totalMaterialsDeleted: number
totalMaterialsFailed: number
successCount: number
failedCount: number
isDryRun: boolean
}
/** 插入执行记录的输入 */
export interface InsertCleanerExecutionInput {
batchId: string
attemptNumber: number
userId: number
username: string
isDryRun: boolean
totalOrders: number
appVersion: string
}
/** 插入订单记录的输入 */
export interface InsertOrderInput {
orderNumber: string
}
/** 插入物料明细的输入 */
export interface InsertMaterialDetailInput {
orderNumber: string
materialCode: string
materialName: string
rowNumber: number
result: string
reason: string | null
attemptCount: number
finalErrorCategory: string | null
}
/** 查询批次的选项 */
export interface GetCleanerBatchesOptions {
limit?: number
offset?: number
usernames?: string[]
}
```
**Step 2: 验证类型检查通过**
Run: `npm run typecheck`
Expected: PASS新文件不影响现有代码
**Step 3: Commit**
```
feat(cleaner): add type definitions for cleaner operation history
```
---
## Task 2: 新增 DAO 层
**Files:**
- Create: `src/main/services/database/cleaner-operation-history-dao.ts`
**Step 1: 创建 DAO 文件**
参考 `extractor-operation-history-dao.ts` 的模式(`create()` 获取数据库连接、`createDialect()` 处理 SQL 方言、`trackDuration()` 记录耗时)。表名使用 `ERPAuto` schema。
关键方法:
```typescript
export class CleanerOperationHistoryDAO {
private dbService: IDatabaseService | null = null
private dialect: SqlDialect | null = null
// ===== 执行表 =====
private getExecutionTableName(): string {
return this.getDialect().quoteTableName('ERPAuto', 'CleanerExecution')
}
async insertExecution(input: InsertCleanerExecutionInput): Promise<boolean>
async updateExecutionStatus(
batchId: string,
attemptNumber: number,
status: string,
ordersProcessed: number,
materialsDeleted: number,
materialsSkipped: number,
materialsFailed: number,
uncertainDeletions: number,
endTime: Date,
errorMessage?: string
): Promise<boolean>
// ===== 订单表 =====
private getOrderTableName(): string {
return this.getDialect().quoteTableName('ERPAuto', 'CleanerOrderHistory')
}
async insertOrderRecords(
batchId: string,
attemptNumber: number,
orders: InsertOrderInput[]
): Promise<boolean>
async updateOrderStatus(
batchId: string,
attemptNumber: number,
orderNumber: string,
status: string,
materialsDeleted: number,
materialsSkipped: number,
materialsFailed: number,
uncertainDeletions: number,
retryCount: number,
retrySuccess: boolean,
errorMessage?: string
): Promise<boolean>
// ===== 物料表 =====
private getMaterialTableName(): string {
return this.getDialect().quoteTableName('ERPAuto', 'CleanerMaterialDetail')
}
async insertMaterialDetails(
batchId: string,
attemptNumber: number,
details: InsertMaterialDetailInput[]
): Promise<boolean>
// ===== 查询 =====
async getBatches(
userId?: number,
options?: GetCleanerBatchesOptions
): Promise<CleanerBatchStats[]>
async getBatchDetails(
batchId: string
): Promise<{ executions: CleanerExecutionRecord[]; orders: CleanerOrderRecord[] }>
async getMaterialDetails(
batchId: string,
attemptNumber: number,
orderNumber: string
): Promise<CleanerMaterialRecord[]>
// ===== 删除 =====
async deleteBatch(
batchId: string,
requestingUserId: number,
isAdmin: boolean
): Promise<{ success: boolean; error?: string }>
// ===== 列询执行级记录 =====
async getMaterialDetails(
batchId: string,
attemptNumber: number,
orderNumber: string
): Promise<CleanerMaterialRecord[]>
// ===== 删除 =====
async deleteBatch(
batchId: string,
requestingUserId: number,
isAdmin: boolean
): Promise<{ success: boolean; error?: string }>
async disconnect(): Promise<void>
}
```
`getBatches` 查询逻辑:
- `GROUP BY BatchId`,取 `MAX(AttemptNumber)` 对应的执行记录状态作为最终状态
- 汇总订单级的 success/failed 计数
- 支持 userId 过滤(普通用户)和 usernames 过滤(管理员)
- 支持分页
`getBatchDetails` 查询逻辑:
- 返回某 BatchId 下所有 execution 记录 + order 记录
- 前端用 attemptNumber 区分不同尝试
每个 INSERT/UPDATE 使用 `trackDuration()` 包裹error handling 与 Extractor DAO 一致。
**Step 2: 验证类型检查通过**
Run: `npm run typecheck`
Expected: PASS
**Step 3: Commit**
```
feat(cleaner): add CleanerOperationHistoryDAO for three-table persistence
```
---
## Task 3: 新增 IPC channels
**Files:**
- Modify: `src/shared/ipc-channels.ts`
**Step 1: 添加 cleaner history channels**
在现有的 `CLEANER_PROGRESS` 之后添加:
```typescript
// Cleaner history
CLEANER_HISTORY_GET_BATCHES: 'cleanerHistory:getBatches',
CLEANER_HISTORY_GET_BATCH_DETAILS: 'cleanerHistory:getBatchDetails',
CLEANER_HISTORY_GET_MATERIAL_DETAILS: 'cleanerHistory:getMaterialDetails',
CLEANER_HISTORY_DELETE_BATCH: 'cleanerHistory:deleteBatch',
```
**Step 2: Commit**
```
feat(cleaner): add IPC channels for cleaner operation history
```
---
## Task 4: 新增 IPC handler
**Files:**
- Create: `src/main/ipc/cleaner-history-handler.ts`
- Modify: `src/main/ipc/index.ts` — 注册新 handler
**Step 1: 创建 cleaner-history-handler.ts**
参考 `operation-history-handler.ts` 的模式。四个 handler
- `CLEANER_HISTORY_GET_BATCHES`获取批次列表Admin 看全部User 看自己的
- `CLEANER_HISTORY_GET_BATCH_DETAILS`:获取某个批次的执行记录和订单记录
- `CLEANER_HISTORY_GET_MATERIAL_DETAILS`:获取某个订单的物料明细
- `CLEANER_HISTORY_DELETE_BATCH`:删除批次,权限校验与 Extractor 一致
```typescript
export function registerCleanerHistoryHandlers(): void {
const dao = new CleanerOperationHistoryDAO()
ipcMain.handle(
IPC_CHANNELS.CLEANER_HISTORY_GET_BATCHES,
async (event, options?: GetCleanerBatchesOptions): Promise<IpcResult<CleanerBatchStats[]>> => {
return withErrorHandling(async () => {
const currentUser = SessionManager.getInstance().getUserInfo()
if (!currentUser) throw new Error('用户未登录')
const userId = currentUser.userType === 'Admin' ? undefined : currentUser.id
return dao.getBatches(userId, options)
}, 'cleanerHistory:getBatches')
}
)
ipcMain.handle(
IPC_CHANNELS.CLEANER_HISTORY_GET_BATCH_DETAILS,
async (
event,
batchId: string
): Promise<
IpcResult<{ executions: CleanerExecutionRecord[]; orders: CleanerOrderRecord[] }>
> => {
// ... 与 operation-history-handler 的 getBatchDetails 模式一致
}
)
ipcMain.handle(
IPC_CHANNELS.CLEANER_HISTORY_GET_MATERIAL_DETAILS,
async (
event,
batchId: string,
attemptNumber: number,
orderNumber: string
): Promise<IpcResult<CleanerMaterialRecord[]>> => {
// ...
}
)
ipcMain.handle(
IPC_CHANNELS.CLEANER_HISTORY_DELETE_BATCH,
async (event, batchId: string): Promise<IpcResult<{ deleted: boolean }>> => {
// ... 权限校验后删除三张表的记录
}
)
}
```
**Step 2: 在 index.ts 中注册**
`registerIpcHandlers()` 中添加 `registerCleanerHistoryHandlers()` 调用,并在顶部添加 import。
**Step 3: 验证类型检查通过**
Run: `npm run typecheck`
Expected: PASS
**Step 4: Commit**
```
feat(cleaner): add IPC handlers for cleaner operation history
```
---
## Task 5: 新增 Preload API
**Files:**
- Modify: `src/preload/api/cleaner.ts` — 新增 history 方法
- Modify: `src/preload/index.d.ts` — 新增类型声明
**Step 1: 在 cleaner.ts 中新增 history 方法**
```typescript
import type {
CleanerBatchStats,
CleanerExecutionRecord,
CleanerOrderRecord,
CleanerMaterialRecord,
GetCleanerBatchesOptions
} from '../../main/types/cleaner-history.types'
// 在 cleanerApi 对象中追加:
getHistoryBatches: (options?: GetCleanerBatchesOptions): Promise<IpcResult<CleanerBatchStats[]>> =>
invokeIpc(IPC_CHANNELS.CLEANER_HISTORY_GET_BATCHES, options),
getHistoryBatchDetails: (batchId: string): Promise<IpcResult<{
executions: CleanerExecutionRecord[]
orders: CleanerOrderRecord[]
}>> =>
invokeIpc(IPC_CHANNELS.CLEANER_HISTORY_GET_BATCH_DETAILS, batchId),
getHistoryMaterialDetails: (batchId: string, attemptNumber: number, orderNumber: string): Promise<IpcResult<CleanerMaterialRecord[]>> =>
invokeIpc(IPC_CHANNELS.CLEANER_HISTORY_GET_MATERIAL_DETAILS, batchId, attemptNumber, orderNumber),
deleteHistoryBatch: (batchId: string): Promise<IpcResult<{ deleted: boolean }>> =>
invokeIpc(IPC_CHANNELS.CLEANER_HISTORY_DELETE_BATCH, batchId),
```
**Step 2: 在 index.d.ts 中更新 CleanerAPI 接口**
`CleanerAPI` 接口中添加对应的类型声明,与实际 API 对齐。
**Step 3: 验证类型检查通过**
Run: `npm run typecheck`
Expected: PASS
**Step 4: Commit**
```
feat(cleaner): add preload API for cleaner operation history
```
---
## Task 6: 改造 CleanerApplicationService — 写入数据库记录
**Files:**
- Modify: `src/main/services/cleaner/cleaner-application-service.ts`
这是核心变更。`runCleaner` 方法需要:
**Step 1: 修改 runCleaner 签名,接收 batchId 和 DAO**
```typescript
async runCleaner(
eventSender: WebContents,
input: CleanerInput,
batchId: string,
historyDao: CleanerOperationHistoryDAO
): Promise<CleanerResult>
```
**Step 2: 移除报告相关代码**
- 删除 `import { app } from 'electron'`(仅用于 `app.getVersion()`
- 删除 `generateExecutionId()` 函数
- 删除 `generateAndUploadReport()` 方法
- 删除所有 `executionId` 相关变量和日志
**Step 3: 插入 pending 订单记录**
在登录成功后、执行清理前,调用 `historyDao.insertOrderRecords(batchId, 1, orders)` 写入 pending 状态的订单记录。
**Step 4: 执行后更新订单记录和写入物料明细**
清理完成后遍历 `result.details``OrderCleanDetail[]`),对每个订单:
- 调用 `historyDao.updateOrderStatus(...)` 更新订单结果
- 调用 `historyDao.insertMaterialDetails(...)` 写入物料明细skipped + failed 材料全部写入)
**Step 5: 更新执行记录状态**
调用 `historyDao.updateExecutionStatus(batchId, 1, ...)` 更新为最终状态。
**Step 6: 外层重试改造**
`result.crashed` 时:
1. 调用 `historyDao.updateExecutionStatus(batchId, 1, 'crashed', ...)` 标记首次尝试为 crashed
2. 调用 `historyDao.insertExecution({ batchId, attemptNumber: 2, ... })` 创建第二次尝试
3. 调用 `historyDao.insertOrderRecords(batchId, 2, orders)` 写入第二次尝试的 pending 订单
4. 重新登录并执行
5. 执行后更新 AttemptNumber=2 的订单和物料记录
**Step 7: 验证类型检查通过**
Run: `npm run typecheck`
Expected: PASS
**Step 8: Commit**
```
refactor(cleaner): replace report generation with database persistence
```
---
## Task 7: 改造 cleaner-handler.ts — 执行前后写入
**Files:**
- Modify: `src/main/ipc/cleaner-handler.ts`
**Step 1: 修改 CLEANER_RUN handler**
在调用 `cleanerService.runCleaner()` 之前:
1. 获取当前用户信息
2. `batchId = randomUUID()`
3. 创建 `CleanerOperationHistoryDAO` 实例
4. 调用 `dao.insertExecution({ batchId, attemptNumber: 1, userId, username, isDryRun, totalOrders, appVersion })`
`batchId``dao` 传入 `runCleaner()`
执行完成后(无论成功失败),更新执行记录的最终状态。
**Step 2: 移除 app.getVersion() 调用**
`appVersion` 改为在 handler 层获取(因为 handler 已有 electron 访问权限),传给 DAO。
**Step 3: 验证类型检查通过**
Run: `npm run typecheck`
Expected: PASS
**Step 4: Commit**
```
refactor(cleaner): write execution records to database in IPC handler
```
---
## Task 8: 删除 Markdown 报告生成器
**Files:**
- Delete: `src/main/services/report/cleaner-report-generator.ts`
**Step 1: 删除文件**
删除 `cleaner-report-generator.ts`
**Step 2: 检查是否有其他文件引用它**
搜索 `cleaner-report-generator``CleanerReportGenerator`,如有引用则一并移除(主要是 `cleaner-application-service.ts` 中已删除的 import
**Step 3: 验证编译通过**
Run: `npm run typecheck`
Expected: PASS
**Step 4: Commit**
```
refactor(cleaner): remove Markdown report generator
```
---
## Task 9: 前端 — 新增操作历史弹窗
**Files:**
- Create: `src/renderer/src/components/CleanerOperationHistoryModal.tsx`
- Modify: `src/renderer/src/pages/CleanerPage.tsx`
**Step 1: 创建 CleanerOperationHistoryModal**
参考 `ExtractorOperationHistoryModal.tsx` 的 UI 模式和代码结构。关键差异:
- 数据源使用 `window.electron.cleaner.getHistoryBatches()` 等新 API
- 批次列表增加"尝试次数"列和"模拟运行"标识
- 展开明细时顶部显示执行级信息尝试次数、crashed 状态等)
- 订单表格增加 deleted/skipped/failed/uncertain 列
- 订单行可再次展开查看物料明细(调用 `getHistoryMaterialDetails`
- 管理员按用户筛选、删除功能与 Extractor 一致
**Step 2: 在 CleanerPage 中添加"操作历史"按钮和弹窗**
-`CleanerToolbar` 中添加"操作历史"按钮(或直接在 CleanerPage 添加)
- 引入 `CleanerOperationHistoryModal` 组件
- 传入 `user``isOpen/onClose` 控制
**Step 3: 验证类型检查通过**
Run: `npm run typecheck`
Expected: PASS
**Step 4: Commit**
```
feat(cleaner): add operation history modal with database-backed records
```
---
## Task 10: 更新 renderer 类型定义
**Files:**
- Modify: `src/renderer/src/hooks/cleaner/types.ts`
**Step 1: 添加 history 相关类型**
在 types.ts 中添加前端需要的类型(或直接从 `cleaner-history.types.ts` import根据项目的前端类型引用模式决定
**Step 2: 验证类型检查通过**
Run: `npm run typecheck`
Expected: PASS
**Step 3: Commit**
```
feat(cleaner): add renderer types for cleaner operation history
```
---
## Task 11: 清理旧代码
**Files:**
- Modify: `src/renderer/src/hooks/cleaner/types.ts` — 移除 `CleanerReportData.crashed`(如果不再需要)
- 检查 `ReportViewerDialog.tsx``ReportAnalysisDialog.tsx` 是否仍被 Cleaner 使用
**Step 1: 清理 renderer 中不再需要的类型**
- `CleanerReportData` 中如果 `crashed` 字段已无用,移除
- 确认 `CleanerPhase``'retry'` 值是否仍需要(前端进度通知仍在使用,保留)
**Step 2: 评估 ReportViewerDialog 和 ReportAnalysisDialog**
这两个组件目前用于查看 Markdown 报告文件。如果 Cleaner 不再使用它们:
- 在 CleanerPage 中移除相关按钮和引用
- 不删除组件本身Extractor 可能仍在使用,后续统一清理)
**Step 3: 验证编译和类型检查通过**
Run: `npm run typecheck && npm run lint`
Expected: PASS
**Step 4: Commit**
```
chore(cleaner): clean up legacy report-related code
```
---
## Task 12: 集成测试
**Step 1: 运行完整类型检查**
Run: `npm run typecheck`
Expected: PASS
**Step 2: 运行 lint**
Run: `npm run lint`
Expected: PASS
**Step 3: 运行单元测试**
Run: `npm run test`
Expected: PASS
**Step 4: 手动验证**
1. 启动 `npm run dev`
2. 在 Cleaner 页面执行一次清理(模拟运行)
3. 检查数据库三张表是否正确写入
4. 点击"操作历史"按钮,验证批次列表和详情展示
5. 模拟崩溃场景(如果可以),验证外层重试写入 AttemptNumber=2 的记录
6. 用管理员账号验证用户筛选和删除功能
---
## 执行顺序
```
Task 1 (types) → Task 2 (DAO) → Task 3 (IPC channels) → Task 4 (IPC handler)
→ Task 5 (preload) → Task 6 (CleanerApplicationService) → Task 7 (cleaner-handler)
→ Task 8 (删除报告生成器) → Task 10 (renderer types) → Task 9 (前端弹窗)
→ Task 11 (清理) → Task 12 (集成测试)
```
Task 9 和 Task 10 可以并行。Task 8 必须在 Task 6、7 之后。

View File

@@ -0,0 +1,152 @@
# Cleaner 外层重试机制设计
## 背景
当 CleanerService.performCleanup 的主循环抛出未捕获异常时(如查询超时、浏览器崩溃),代码进入 outer catch 块,直接返回 partial result。位于 try 块后半段的订单级重试逻辑retryFailedOrders永远没有机会执行。
典型场景211 个订单中处理到第 80 个时,查询列表页等待表格行超时 → Cleaner failed → 浏览器被关闭 → 剩余 131 个订单未处理 → 无重试。
## 设计决策
| 决策项 | 选择 | 理由 |
| ------------ | ------------------------- | -------------------------------- |
| 重试层级 | CleanerApplicationService | 崩溃后浏览器不可用,必须重新登录 |
| 重试范围 | 全部订单重新跑 | 简单可靠,物料删除是幂等操作 |
| 最大重试次数 | 1 次 | 覆盖瞬态故障,不过度消耗时间 |
| 触发条件 | result.crashed === true | 仅 outer catch 触发时才重试 |
| 报告去重 | 执行 ID | 用户点击执行时生成,重试不变 |
## 变更清单
### 1. CleanerResult 新增字段
**文件**: `src/main/types/cleaner.types.ts`
```typescript
export interface CleanerResult {
// ... 现有字段
crashed?: boolean // true = outer catch triggered, 流程级崩溃
}
```
同步更新 `src/shared/types/cleaner.types.ts`(如有独立定义)和 preload 暴露的类型声明。
### 2. CleanerService 标记崩溃
**文件**: `src/main/services/erp/cleaner.ts`line 375 的 catch 块
```typescript
} catch (error) {
const message = error instanceof Error ? error.message : 'Unknown error'
log.error('Cleaner failed', { ... })
result.errors.push(`Clean failed: ${message}`)
result.crashed = true // ← 新增
}
```
### 3. CleanerApplicationService 重试逻辑
**文件**: `src/main/services/cleaner/cleaner-application-service.ts`
`runCleaner()` 中,`cleaner.clean()` 返回后增加重试判断:
```
runCleaner(eventSender, input) {
const executionId = generateExecutionId() // 用户点击时生成
const startTime = Date.now()
// 1. 获取 ERP 配置、数据库连接、订单解析(不变)
// 2. 登录 ERP不变
let result = await cleaner.clean(modifiedInput)
// === 外层重试 ===
if (result.crashed) {
log.warn('检测到流程级崩溃,准备外层重试', { executionId })
await authService.close() // 关闭不可用的浏览器
authService = new ErpAuthService({...})
await authService.login() // 重新登录
cleaner = new CleanerService(authService)
result = await cleaner.clean(modifiedInput) // 全部订单重新跑
}
// 3. 生成报告(使用 executionId 作为文件名一部分,避免重复)
await this.generateAndUploadReport(input, result, startTime, executionId)
return result
}
```
### 4. 执行 ID 生成规则
格式: `CLN-{yyyyMMddHHmmss}-{4位随机字母}`
示例: `CLN-20260410112930-A7FK`
生成时机: `runCleaner()` 入口处,在 ERP 登录之前。重试时同一个 executionId 不变。
用途:
- 报告文件名: `cleaner-report-CLN-20260410112930-A7FK.md`
- RustFS 存储路径中包含该 ID重试时覆盖同一文件
- 报告内容中显示该 ID
### 5. 报告增强
**文件**: `src/main/services/report/cleaner-report-generator.ts`
在执行摘要表格中新增字段:
```markdown
| 项目 | 值 |
| ------------ | ------------------------- | ------ |
| **执行 ID** | `CLN-20260410112930-A7FK` | ← 新增 |
| **应用版本** | `1.11.1` | ← 新增 |
| **执行时间** | `2026-04-10 11:29:30` |
| **执行模式** | `正式执行` |
| ... | ... |
```
- **执行 ID**: 从 ReportOptions 传入
- **应用版本**: `app.getVersion()`,沿用 logger 中已有的获取方式
**ReportOptions 变更**:
```typescript
export interface ReportOptions {
dryRun: boolean
username: string
startTime: number
endTime: number
executionId: string // ← 新增
appVersion: string // ← 新增
}
```
**报告文件名变更**:
```
旧: cleaner-report-2026-04-10-03-30-12.md
新: cleaner-report-CLN-20260410112930-A7FK.md
```
重试时同一个 executionId 生成相同的文件名,本地文件和 RustFS 上传都会覆盖旧报告,无需额外去重逻辑。
### 6. 进度通知增强
重试时向前端发送进度通知,让用户知道正在重试:
```typescript
this.sendProgress(eventSender, '流程崩溃,正在重新登录并重试...', 0, {
phase: 'retry',
...
})
```
## 不涉及的部分
- 前端 UI 变更(后续可单独做,展示重试状态)
- IPC channel 变更
- 内层重试逻辑(订单级/物料级)不变
- 数据库 schema 变更

View File

@@ -0,0 +1,292 @@
# Cleaner v1.11.1 之后更新内容改进计划
本文档基于 `v1.11.1..v1.12.3` 区间内已完成的前端审查结果整理而成,目标不是重复提交记录,而是为后续实现人员提供一份可以直接排期和落地的改进路线图。计划范围仅覆盖 Cleaner 相关前端改进,不扩展到主进程 DAO、IPC 或数据库结构重构。
## 1. 背景与范围
本计划覆盖 `v1.11.1` 之后到当前最新版本 `v1.12.3` 的 Cleaner 前端相关更新,重点关注以下变化:
- 新增 Cleaner 操作历史弹窗
- 用数据库持久化替代原有 Markdown 报告查看路径
- 为执行结果补充失败与不确定删除统计
- 引入 `React.lazy``BatchItem` 拆分来降低页面负担
本次计划的核心目标是:
- 先修复当前历史弹窗与执行结果展示中的稳定性问题
- 再优化首屏加载和复杂列表交互性能
- 最后补齐长期可维护性和可扩展性基础
默认审查区间固定为 `v1.11.1..v1.12.3`,默认文档语言为中文,默认落点为 `docs/plans/`
## 2. 当前状态总结
这轮更新已经做对了几件重要的事情:
- Cleaner 历史记录已经完成数据库化,前端不再依赖旧的 Markdown 报告浏览流
- `CleanerOperationHistoryModal` 被独立成单独组件,并通过 `BatchItem` 局部拆分降低兄弟节点联动重渲染
- `CleanerPage` 已经开始使用 `React.lazy` 引入历史弹窗与执行报告相关组件
- `ExecutionReportDialog` 已经补充 `materialsFailed``uncertainDeletions` 的展示能力
这些改动说明整体方向是正确的,但从 React 最佳实践和后续维护成本看,当前实现仍然存在几个明确的改进空间:异步缓存策略不够稳、按需加载没有完全生效、复杂列表的扩展能力有限、前端回归保护不足。
## 3. 主要改进项
### P0 立刻修
#### 3.1 修正历史详情与物料详情的缓存时机
问题:
- 当前历史批次详情和物料详情会在请求发起前就标记为“已加载”
- 如果首次请求失败,后续再次展开不会重试,用户会长期看到空详情或误导性空状态
目标:
- 只在请求成功后写入缓存
- 失败后允许再次展开重新请求
- 在 UI 上保留现有交互风格,不做视觉重设计
建议方向:
- 将详情加载状态拆成 `idle / loading / success / error`
- `detailsLoadedRef``loadedMaterialsRef` 只在成功后更新
- 对失败场景提供自然重试路径,优先采用“再次展开即重试”的方式
预期收益:
- 避免瞬时请求失败被错误地永久缓存
- 提高历史查看功能的稳定性和用户信任感
#### 3.2 将 Cleaner 历史弹窗改成真正条件挂载
问题:
- 当前 `CleanerPage` 虽然使用了 `React.lazy`,但历史弹窗组件仍然会在页面渲染时被挂入树中
- 这会导致对应 chunk 仍在首屏阶段就被加载,未达到真正按需加载的效果
目标:
- 历史弹窗只在用户打开时才参与渲染和加载
- 避免进入 Cleaner 页面就提前下载历史功能代码
建议方向:
- 采用条件渲染而不是仅保留 `isOpen` 控制
- 延续当前交互样式和打开方式,不调整页面布局
预期收益:
- 降低 Cleaner 页面的首屏负担
- 更符合 `bundle-conditional` 类最佳实践
#### 3.3 补最小前端回归测试
问题:
- 本轮新增了历史弹窗、异步详情展开和执行结果增强,但前端侧缺少对应测试保护
目标:
- 为关键行为建立最小可行回归测试
- 优先补组件/行为测试,不新增端到端测试要求
建议方向:
- 覆盖历史弹窗未打开时不触发懒加载模块请求
- 覆盖批次详情和物料详情首次失败后再次展开可重试
- 覆盖管理员筛选切换后请求参数与结果一致
预期收益:
- 降低后续修复和优化时的回归风险
- 为后续分页、交互优化提供安全网
### P1 本周优化
#### 3.4 为历史列表增加分页能力
问题:
- 当前历史列表和明细表格按全量数据渲染,随着批次数量、订单数量和物料数量增加,性能风险会上升
目标:
- 让历史列表在数据增长后仍保持可接受的打开和滚动体验
建议方向:
- 默认优先采用分页,不先引入虚拟列表库
- 先做批次列表分页,再评估是否需要对订单或物料明细做进一步优化
预期收益:
- 控制渲染体量
- 降低复杂列表在中等数据规模下的卡顿风险
#### 3.5 管理员筛选切换使用 `startTransition`
问题:
- 管理员切换用户筛选时会立即触发批次列表刷新,后续数据量增长后可能影响点击反馈
目标:
- 保持筛选按钮点击响应流畅
- 将非紧急更新降级处理
建议方向:
- 将筛选触发的列表刷新包装到 `startTransition`
- 保持现有筛选交互模型不变
预期收益:
- 降低筛选切换时的阻塞感
- 更符合 React 对非紧急更新的建议用法
#### 3.6 收敛重复派生计算
问题:
- 当前实现中存在多处基于 `orders``currentAttempt` 的重复 `filter/map`
- 数据规模扩大后,这些重复遍历会逐步放大渲染成本
目标:
- 让渲染中的数据派生更集中、更可读
建议方向:
- 将当前 attempt 对应订单集合收敛成单一派生结果
- 复制列内容等行为复用同一份派生数据
预期收益:
- 降低不必要的重复计算
-`BatchItem` 的渲染路径更容易维护
#### 3.7 优化执行报告的结果语义
问题:
- 当前执行报告的标题和成功态仍主要依赖 `errors`
- 当存在 `materialsFailed``uncertainDeletions` 时,结果表达仍可能显得过于乐观
目标:
- 让执行结果清楚区分成功、部分成功、失败、需人工确认
建议方向:
- 重新定义结果态判定优先级
- 在不重做 UI 视觉设计的前提下,优化标题、说明文案和结果提示条
预期收益:
- 降低误判执行结果的风险
- 让失败和不确定删除场景更容易被用户注意到
### P2 后续演进
#### 3.8 统一状态映射定义
问题:
- 当前状态的 label、icon、style 已有集中趋势,但仍是组件内局部定义
- 后续新增状态时容易出现展示不一致
目标:
- 用统一的受类型约束的映射管理状态展示
建议方向:
- 抽离共享状态映射
- 覆盖 batch、execution、order、material 这几类状态展示
预期收益:
- 降低重复定义
- 提高新增状态时的一致性和可维护性
#### 3.9 补无障碍语义
问题:
- 当前批次展开和订单展开更多依赖点击容器,语义和键盘可达性还有提升空间
目标:
- 让复杂历史弹窗具备更清晰的交互语义
建议方向:
- 使用真实按钮作为展开触发器
- 增加 `aria-expanded``aria-controls` 等属性
预期收益:
- 提升键盘交互和屏幕阅读器兼容性
- 为后续复杂交互维护提供更稳定语义基础
#### 3.10 规划历史查询的扩展能力
问题:
- 当前查询能力主要围绕固定数量批次列表和基础筛选
- 如果历史功能继续增强,前端会越来越依赖更丰富的查询条件
目标:
- 为后续历史功能演进预留明确方向
建议方向:
- 预留时间范围筛选
- 预留状态筛选
- 延续服务端分页方向,而不是继续扩大前端一次性加载量
预期收益:
- 让后续功能迭代有稳定扩展路径
- 避免复杂度持续堆积在当前单一弹窗实现中
## 4. 推荐执行顺序
建议按以下顺序推进:
1. 先修 `P0`,优先处理缓存时机错误和按需加载未完全生效的问题
2.`P0` 修复完成后补最小前端回归测试,锁住关键行为
3. 再做 `P1`,先分页,再处理 `startTransition` 和重复派生计算
4. 最后进入 `P2`,统一状态映射、补无障碍语义,并规划历史查询扩展能力
这个顺序的原则是:先修稳定性,再做性能,再做长期演进。
## 5. 完成标准
本计划相关改进完成后,至少应满足以下验收标准:
- 历史弹窗未打开时,不触发对应懒加载模块请求
- 批次详情或物料详情首次请求失败后,用户再次展开可重新请求
- 用户筛选切换后,列表数据与筛选条件一致
- 执行报告在存在 `materialsFailed``uncertainDeletions` 时,不再展示为完全成功
- `npm run typecheck` 通过
- 相关前端测试通过
- Cleaner 页面关键路径手工验证通过,包括:
- 打开历史弹窗
- 展开批次详情
- 展开订单物料详情
- 切换管理员筛选
- 查看执行结果提示
## 6. 默认方案与实施约束
为避免后续实现阶段再次做不必要决策,本计划固定以下默认方案:
- 历史列表优先采用分页,不先引入虚拟列表库
- 历史弹窗继续保留现有交互样式,不做视觉重设计
- 测试优先补组件/行为测试,不新增端到端测试要求
- 本计划只覆盖 Cleaner 相关前端改进,不扩展到主进程 DAO、IPC、数据库结构重构
如果后续版本继续围绕 Cleaner 历史功能扩展,可以在本计划基础上继续追加更细的实施文档,但不应改变本计划中 `P0 / P1 / P2` 的优先级顺序。

View File

@@ -0,0 +1,94 @@
# Cleaner Operation History - Full-Level Search Design
Date: 2026-04-17
## Summary
Add a full-level search feature to `CleanerOperationHistoryModal` that allows users to search across batches, orders, and materials by entering a single keyword. A new backend search API returns pre-joined three-level nested data, and the frontend renders it with keyword highlighting.
## Interaction Design
- **Search bar**: placed in the toolbar area, above the user filter chips, with a search icon and clear button.
- **Trigger**: press Enter or click the search button (no per-keystroke requests).
- **Search mode behavior**:
- Hides pagination controls (results are cross-page).
- Matching batches auto-expand with orders and materials displayed directly.
- Non-matching levels are hidden.
- Clearing the search box returns to normal browse mode.
- **Highlighting**: matched text wrapped in `<mark>` with yellow background.
- **Empty result**: shows "未找到匹配的记录" message.
- **Result cap**: backend limits to 20 batches; if truncated, shows a hint.
## Search Fields
| Level | Searchable fields |
|-------|-------------------|
| Batch | `batchId`, `username`, `status` |
| Order | `orderNumber`, `productionId` |
| Material | `materialCode`, `materialName` |
## Data Flow
```
renderer: window.electron.cleaner.searchHistoryRecords(query, options)
→ preload: expose searchHistoryRecords
→ main IPC handler: cleaner:searchHistoryRecords
→ service/DAO: searchCleanerHistory(searchQuery, options)
→ DB query (JOIN batches + orders + materials, LIKE filter)
```
### Input Types
```typescript
interface SearchCleanerHistoryOptions {
query: string
usernames?: string[] // admin-only user scope
limit?: number // default 20
}
```
### Response Type
```typescript
interface CleanerHistorySearchResult {
batches: Array<{
batch: CleanerHistoryBatchStats
executions: ExecutionRecord[]
orders: Array<{
order: CleanerHistoryOrderRecord
materials: CleanerHistoryMaterialRecord[]
}>
}>
totalMatches: number
}
```
## Frontend Changes
1. **Modal top-level**: add `searchMode` / `searchQuery` state; switch data source between search API and paginated API.
2. **Toolbar**: add search input with Search icon and clear button.
3. **BatchItem**: accept optional pre-loaded `orders` + `materials` props; skip lazy-loading in search mode.
4. **Highlight utility**: `highlightText(text: string, query: string)` wraps matches in `<mark>` tags.
5. **Footer**: hide pagination in search mode; show "找到 X 个批次" + "清除搜索" button.
## Backend Changes
| File | Change |
|------|--------|
| `src/main/types/cleaner-history.types.ts` | Add `SearchCleanerHistoryOptions`, `CleanerHistorySearchResult` types |
| DAO (cleaner history) | Add `searchCleanerHistory` method with SQL LIKE across joined tables |
| Service (cleaner) | Add `searchHistoryRecords` method |
| IPC handler | Register `cleaner:searchHistoryRecords` channel |
| Preload | Expose `searchHistoryRecords` method |
| `src/renderer/src/hooks/cleaner/types.ts` | Sync search result types |
## Affected Files
- `src/main/types/cleaner-history.types.ts` — new types
- `src/main/services/database/cleaner-history-dao.ts` (or similar) — new search method
- `src/main/services/cleaner-service.ts` (or similar) — new search method
- `src/main/ipc/cleaner-handler.ts` (or similar) — new IPC channel
- `src/preload/index.ts` (or cleaner-specific) — expose search API
- `src/renderer/src/hooks/cleaner/types.ts` — sync types
- `src/renderer/src/components/CleanerOperationHistoryModal.tsx` — search UI + state
- `src/renderer/src/components/cleaner-history-highlight.ts` — highlight utility (new file)

View File

@@ -0,0 +1,675 @@
# Cleaner History Full-Level Search Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Add full-level search (batch + order + material) to the CleanerOperationHistoryModal, using a new backend search API that returns pre-joined three-level nested data, with keyword highlighting in the frontend.
**Architecture:** New `searchHistoryRecords` IPC channel goes through the existing DAO pattern. The DAO method performs a UNION-based SQL query across all three tables to find matching BatchIds, then fetches the full nested data for those batches. The frontend switches between browse mode (paginated) and search mode (full results) based on whether a search query is active.
**Tech Stack:** TypeScript, React, SQL (MySQL/PostgreSQL/SQL Server via dialect abstraction), Electron IPC
---
### Task 1: Add search types to main process
**Files:**
- Modify: `src/main/types/cleaner-history.types.ts` (append at end)
- Modify: `src/shared/ipc-channels.ts` (add new channel)
**Step 1: Add search types to cleaner-history.types.ts**
Append after the existing `GetCleanerBatchesOptions` interface:
```typescript
/** Search options for full-level history search */
export interface SearchCleanerHistoryOptions {
query: string
usernames?: string[]
limit?: number
}
/** A single batch's full nested data for search results */
export interface CleanerSearchBatchResult {
batch: CleanerBatchStats
executions: CleanerExecutionRecord[]
orders: Array<{
order: CleanerOrderRecord
materials: CleanerMaterialRecord[]
}>
}
/** Search response */
export interface CleanerHistorySearchResult {
batches: CleanerSearchBatchResult[]
totalMatches: number
}
```
**Step 2: Add IPC channel to ipc-channels.ts**
In the `// Cleaner operation history` section, after `CLEANER_HISTORY_DELETE_BATCH`, add:
```typescript
CLEANER_HISTORY_SEARCH: 'cleanerHistory:search',
```
**Step 3: Verify TypeScript compiles**
Run: `npx tsc --noEmit --project src/main/tsconfig.json 2>&1 | head -20`
Expected: No new errors related to these types
**Step 4: Commit**
```bash
git add src/main/types/cleaner-history.types.ts src/shared/ipc-channels.ts
git commit -m "feat(cleaner-history): add search types and IPC channel"
```
---
### Task 2: Add search DAO method
**Files:**
- Modify: `src/main/services/database/cleaner-operation-history-dao.ts`
**Step 1: Add imports for new types**
At the top of the file, add to the existing import from `../../types/cleaner-history.types`:
```typescript
import type {
// ... existing imports ...
SearchCleanerHistoryOptions,
CleanerSearchBatchResult,
CleanerHistorySearchResult
} from '../../types/cleaner-history.types'
```
**Step 2: Add the searchBatches method to the DAO class**
Add this method after the `getBatches` method (around line 707). The strategy:
1. First, find matching BatchIds via a UNION query across all three tables using LIKE.
2. Then fetch full nested data (batch stats, executions, orders, materials) for those batch IDs.
```typescript
// ==================== QUERY: SEARCH ====================
/**
* Full-level search across batches, orders, and materials
* Uses UNION to find matching BatchIds, then fetches full nested data
*/
async searchBatches(
userId: number | undefined,
options: SearchCleanerHistoryOptions
): Promise<CleanerHistorySearchResult> {
try {
const dbService = await this.getDatabaseService()
const execTable = this.getExecutionTableName()
const orderTable = this.getOrderTableName()
const materialTable = this.getMaterialTableName()
const dialect = this.getDialect()
const likeValue = `%${options.query}%`
const limit = options.limit ?? 20
// Step 1: Find distinct BatchIds matching the query across all tables
const batchIdSql = `
SELECT DISTINCT BatchId FROM (
SELECT e.BatchId FROM ${execTable} e
WHERE e.BatchId LIKE ${dialect.param(0)}
OR e.Username LIKE ${dialect.param(0)}
OR e.Status LIKE ${dialect.param(0)}
UNION ALL
SELECT o.BatchId FROM ${orderTable} o
WHERE o.OrderNumber LIKE ${dialect.param(0)}
OR o.ProductionId LIKE ${dialect.param(0)}
UNION ALL
SELECT m.BatchId FROM ${materialTable} m
WHERE m.MaterialCode LIKE ${dialect.param(0)}
OR m.MaterialName LIKE ${dialect.param(0)}
) AS matched
${userId !== undefined ? `WHERE BatchId IN (SELECT BatchId FROM ${execTable} WHERE UserId = ${dialect.param(1)})` : ''}
${options.usernames && options.usernames.length > 0 ? `WHERE BatchId IN (SELECT BatchId FROM ${execTable} WHERE Username IN (${dialect.params(options.usernames.length)}))` : ''}
`
const batchIdParams: (string | number)[] = [likeValue]
if (userId !== undefined) {
batchIdParams.push(userId)
}
if (options.usernames && options.usernames.length > 0) {
batchIdParams.push(...options.usernames)
}
const batchIdResult = await trackDuration(
async () => await dbService.query(batchIdSql, batchIdParams),
{
operationName: 'CleanerOperationHistoryDAO.searchBatches.batchIds',
context: { operationType: 'SELECT', query: options.query }
}
)
const matchedBatchIds = batchIdResult.result.rows.map((r) => r.BatchId as string)
if (matchedBatchIds.length === 0) {
return { batches: [], totalMatches: 0 }
}
// Apply limit
const limitedBatchIds = matchedBatchIds.slice(0, limit)
// Step 2: For each batch, fetch full nested data in parallel
const batches: CleanerSearchBatchResult[] = []
for (const batchId of limitedBatchIds) {
// Fetch batch stats
const batchStatsArr = await this.getBatches(userId, { limit: 1 })
const batchStats = batchStatsArr.find((b) => b.batchId === batchId)
if (!batchStats) continue
// Fetch executions + orders
const details = await this.getBatchDetails(batchId)
// Fetch materials for all orders
const ordersWithMaterials = await Promise.all(
details.orders.map(async (order) => {
const materials = await this.getMaterialDetails(
batchId,
order.attemptNumber,
order.orderNumber
)
return { order, materials }
})
)
batches.push({
batch: batchStats,
executions: details.executions,
orders: ordersWithMaterials
})
}
return {
batches,
totalMatches: matchedBatchIds.length
}
} catch (error) {
log.error('Search batches error', {
operationType: 'SELECT',
requestId: getRequestId(),
query: options.query,
error: error instanceof Error ? error.message : String(error)
})
return { batches: [], totalMatches: 0 }
}
}
```
**Step 3: Verify TypeScript compiles**
Run: `npx tsc --noEmit --project src/main/tsconfig.json 2>&1 | head -20`
Expected: No errors related to the DAO
**Step 4: Commit**
```bash
git add src/main/services/database/cleaner-operation-history-dao.ts
git commit -m "feat(cleaner-history): add searchBatches DAO method"
```
---
### Task 3: Add IPC handler for search
**Files:**
- Modify: `src/main/ipc/cleaner-history-handler.ts`
**Step 1: Add new IPC handler**
In `registerCleanerHistoryHandlers()`, after the `CLEANER_HISTORY_DELETE_BATCH` handler (before the closing log statement at the end), add:
```typescript
/**
* Search across all history levels (batches, orders, materials)
* Admin users search all records, regular users search only their own
*/
ipcMain.handle(
IPC_CHANNELS.CLEANER_HISTORY_SEARCH,
async (
_event,
options: SearchCleanerHistoryOptions
): Promise<IpcResult<CleanerHistorySearchResult>> => {
return withErrorHandling(async () => {
const currentUser = SessionManager.getInstance().getUserInfo()
if (!currentUser) {
throw new Error('用户未登录')
}
if (!options.query || options.query.trim().length === 0) {
return { batches: [], totalMatches: 0 }
}
const userId = currentUser.userType === 'Admin' ? undefined : currentUser.id
log.info('Searching cleaner history', {
userId: currentUser.id,
userType: currentUser.userType,
query: options.query
})
return await dao.searchBatches(userId, {
...options,
query: options.query.trim()
})
}, 'cleanerHistory:search')
}
)
```
**Step 2: Update imports in cleaner-history-handler.ts**
Add to the existing import from `../../types/cleaner-history.types`:
```typescript
import type {
// ... existing imports ...
SearchCleanerHistoryOptions,
CleanerHistorySearchResult
} from '../types/cleaner-history.types'
```
**Step 3: Verify TypeScript compiles**
Run: `npx tsc --noEmit --project src/main/tsconfig.json 2>&1 | head -20`
**Step 4: Commit**
```bash
git add src/main/ipc/cleaner-history-handler.ts
git commit -m "feat(cleaner-history): add search IPC handler"
```
---
### Task 4: Expose search API in preload
**Files:**
- Modify: `src/preload/api/cleaner.ts`
**Step 1: Add search method to cleanerApi**
After the `deleteHistoryBatch` method, add:
```typescript
searchHistoryRecords: (
options: SearchCleanerHistoryOptions
): Promise<IpcResult<CleanerHistorySearchResult>> =>
invokeIpc(IPC_CHANNELS.CLEANER_HISTORY_SEARCH, options),
```
**Step 2: Add imports**
Add to the existing import from `../../main/types/cleaner-history.types`:
```typescript
import type {
// ... existing imports ...
SearchCleanerHistoryOptions,
CleanerHistorySearchResult
} from '../../main/types/cleaner-history.types'
```
**Step 3: Verify TypeScript compiles**
Run: `npx tsc --noEmit --project src/preload/tsconfig.json 2>&1 | head -20`
**Step 4: Commit**
```bash
git add src/preload/api/cleaner.ts
git commit -m "feat(cleaner-history): expose search API in preload"
```
---
### Task 5: Add renderer-side types and highlight utility
**Files:**
- Modify: `src/renderer/src/hooks/cleaner/types.ts` (add search result types)
- Create: `src/renderer/src/components/cleaner-history-highlight.tsx`
**Step 1: Add search result types to renderer types**
Append to `src/renderer/src/hooks/cleaner/types.ts`:
```typescript
// Search result types (mirrors main process types)
export interface CleanerHistorySearchOrderResult {
order: CleanerHistoryOrderRecord
materials: CleanerHistoryMaterialRecord[]
}
export interface CleanerHistorySearchBatchResult {
batch: CleanerHistoryBatchStats
executions: CleanerHistoryExecutionRecord[]
orders: CleanerHistorySearchOrderResult[]
}
export interface CleanerHistorySearchResult {
batches: CleanerHistorySearchBatchResult[]
totalMatches: number
}
```
**Step 2: Create highlight utility**
Create `src/renderer/src/components/cleaner-history-highlight.tsx`:
```tsx
import React from 'react'
/**
* Highlight matching text with a <mark> tag
* Case-insensitive matching of the query within text
*/
export function highlightText(text: string, query: string): React.ReactNode {
if (!query || !text) return text
const lowerText = text.toLowerCase()
const lowerQuery = query.toLowerCase()
const index = lowerText.indexOf(lowerQuery)
if (index === -1) return text
const before = text.substring(0, index)
const match = text.substring(index, index + query.length)
const after = text.substring(index + query.length)
return (
<>
{before}
<mark className="bg-yellow-200 text-inherit rounded px-0.5">{match}</mark>
{highlightText(after, query)}
</>
)
}
```
**Step 3: Commit**
```bash
git add src/renderer/src/hooks/cleaner/types.ts src/renderer/src/components/cleaner-history-highlight.tsx
git commit -m "feat(cleaner-history): add renderer search types and highlight utility"
```
---
### Task 6: Integrate search into CleanerOperationHistoryModal
**Files:**
- Modify: `src/renderer/src/components/CleanerOperationHistoryModal.tsx`
This is the largest task. The changes are:
1. Add search state variables
2. Add search bar UI to the toolbar
3. Modify BatchItem to accept pre-loaded data in search mode
4. Switch footer between pagination and search result summary
**Step 1: Add new imports**
Add to the lucide-react import:
```typescript
import { Search, X } from 'lucide-react'
```
Add the highlight utility and new types:
```typescript
import { highlightText } from './cleaner-history-highlight'
import type { CleanerHistorySearchResult } from '../hooks/cleaner/types'
```
**Step 2: Add search state in the main modal component**
After the existing state declarations (around line 651), add:
```typescript
const [searchQuery, setSearchQuery] = useState('')
const [searchInput, setSearchInput] = useState('')
const [searchResult, setSearchResult] = useState<CleanerHistorySearchResult | null>(null)
const [isSearching, setIsSearching] = useState(false)
```
**Step 3: Add the search execution function**
Add after `clearUserFilters`:
```typescript
const executeSearch = useCallback(async () => {
const trimmed = searchInput.trim()
if (!trimmed) {
setSearchQuery('')
setSearchResult(null)
return
}
setIsSearching(true)
setSearchQuery(trimmed)
try {
const options =
isAdmin && selectedUsers.length > 0
? { query: trimmed, usernames: selectedUsers }
: { query: trimmed }
const result = await window.electron.cleaner.searchHistoryRecords(options)
if (result.success && result.data) {
setSearchResult(result.data)
} else {
setSearchResult({ batches: [], totalMatches: 0 })
}
} catch {
setSearchResult({ batches: [], totalMatches: 0 })
} finally {
setIsSearching(false)
}
}, [searchInput, isAdmin, selectedUsers])
const clearSearch = () => {
setSearchInput('')
setSearchQuery('')
setSearchResult(null)
}
```
**Step 4: Add search bar UI**
In the toolbar section, before the user filter `<div className="mb-3">` block, add the search input:
```tsx
{/* Search bar */}
<div className="flex items-center gap-2 mb-3">
<div className="relative flex-1">
<Search size={16} className="absolute left-3 top-1/2 -translate-y-1/2 text-gray-400" />
<input
type="text"
value={searchInput}
onChange={(e) => setSearchInput(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter') void executeSearch()
}}
placeholder="搜索批次ID、订单号、物料编码/名称..."
className="w-full pl-9 pr-8 py-2 border border-gray-300 rounded-lg text-sm focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-transparent"
disabled={loading}
/>
{searchInput && (
<button
className="absolute right-2 top-1/2 -translate-y-1/2 text-gray-400 hover:text-gray-600"
onClick={clearSearch}
>
<X size={16} />
</button>
)}
</div>
<button
className="px-4 py-2 bg-blue-600 text-white rounded-lg text-sm font-medium hover:bg-blue-700 disabled:opacity-50 transition-colors"
onClick={() => void executeSearch()}
disabled={isSearching || !searchInput.trim()}
>
{isSearching ? '搜索中...' : '搜索'}
</button>
</div>
```
**Step 5: Modify the batch list section to support search mode**
Replace the batch list section (the `<div className="flex-1 overflow-y-auto">` block) with conditional rendering:
```tsx
{/* Batch list */}
<div className="flex-1 overflow-y-auto">
{searchQuery ? (
// Search mode
isSearching ? (
<div className="flex items-center justify-center h-32 text-gray-500">...</div>
) : searchResult && searchResult.batches.length > 0 ? (
<div className="flex flex-col gap-3">
{searchResult.batches.map((result) => (
<BatchItem
key={result.batch.batchId}
batch={result.batch}
isAdmin={isAdmin}
onDelete={handleDeleteBatch}
onRequestDelete={requestDeleteConfirmation}
searchQuery={searchQuery}
preloadedExecutions={result.executions}
preloadedOrders={result.orders}
/>
))}
</div>
) : (
<div className="flex items-center justify-center h-32 text-gray-500">
{searchQuery}
</div>
)
) : (
// Browse mode (existing logic)
<>
{loading && batches.length === 0 ? (
<div className="flex items-center justify-center h-32 text-gray-500">...</div>
) : batches.length === 0 ? (
<div className="flex items-center justify-center h-32 text-gray-500"></div>
) : (
<div className="flex flex-col gap-3">
{batches.map((batch) => (
<BatchItem
key={batch.batchId}
batch={batch}
isAdmin={isAdmin}
onDelete={handleDeleteBatch}
onRequestDelete={requestDeleteConfirmation}
/>
))}
</div>
)}
</>
)}
</div>
```
**Step 6: Modify footer for search mode**
Replace the footer section to conditionally show pagination or search summary:
```tsx
{/* Footer */}
<div className="pt-4 border-t border-gray-200 flex justify-center">
{searchQuery && searchResult ? (
<div className="flex items-center gap-4">
<span className="text-sm text-gray-600">
{searchResult.totalMatches}
{searchResult.totalMatches > (searchResult.batches.length) &&
`(显示前 ${searchResult.batches.length} 个)`}
</span>
<button
className="px-4 py-2 text-sm text-gray-600 hover:text-gray-800 underline"
onClick={clearSearch}
>
</button>
</div>
) : (
<div className="inline-flex items-center rounded-full border border-slate-200 bg-white p-1 shadow-sm">
{/* ... existing pagination buttons ... */}
</div>
)}
</div>
```
**Step 7: Update BatchItem props and search mode rendering**
Update `BatchItemProps` interface to support optional preloaded data:
```typescript
interface BatchItemProps {
batch: CleanerHistoryBatchStats
isAdmin: boolean
onDelete: (batchId: string) => void
onRequestDelete: (batchId: string) => Promise<boolean>
searchQuery?: string
preloadedExecutions?: ExecutionRecord[]
preloadedOrders?: Array<{
order: CleanerHistoryOrderRecord
materials: CleanerHistoryMaterialRecord[]
}>
}
```
In the BatchItem component, when `searchQuery` is set and preloaded data is available:
- Start expanded (`isExpanded` initial state: `!!searchQuery`)
- Use preloaded executions/orders directly instead of fetching
- Pass `searchQuery` to text rendering for highlighting
**Step 8: Verify typecheck**
Run: `npm run typecheck`
**Step 9: Commit**
```bash
git add src/renderer/src/components/CleanerOperationHistoryModal.tsx
git commit -m "feat(cleaner-history): integrate search UI into history modal"
```
---
### Task 7: Final verification
**Step 1: Run full typecheck**
Run: `npm run typecheck`
**Step 2: Run linter**
Run: `npm run lint`
**Step 3: Build the project**
Run: `npm run build`
**Step 4: Manual test checklist**
- [ ] Open the Cleaner Operation History Modal
- [ ] Verify search bar appears at top of toolbar
- [ ] Type a keyword and press Enter — results should load
- [ ] Matching batches auto-expand with orders and materials
- [ ] Highlighted text appears with yellow background
- [ ] Clear button (X) resets to browse mode
- [ ] Pagination hidden during search, shown after clearing
- [ ] Admin: user filter combined with search works
- [ ] Regular user: only their own records searched
- [ ] Empty search query does nothing
- [ ] Non-matching query shows "未找到匹配" message

View File

@@ -1,4 +1,6 @@
# ERPAuto 便携版自动更新说明
# BIPMaterialManager 便携版自动更新说明
> **项目重命名说明**:本项目计划从 `ERPAuto` 重命名为 `BIPMaterialManager`。本次重命名仅停留在计划阶段,尚未落实到任何具体代码。目前仅对文档中的项目名称进行了更新,代码层面的配置、路径、产物名称等均保持原样,将在后续阶段统一处理。
## 概览

16
docs/releases/1.10.0.md Normal file
View File

@@ -0,0 +1,16 @@
# 1.10.0
## 数据库
- **新增 PostgreSQL 支持**:应用现可连接 PostgreSQL 数据库,与 MySQL、SQL Server 并列可选。
- 数据库方言自动适配SQL 语句根据数据库类型生成正确的标识符引用格式。
## 稳定性
- 修复 PostgreSQL 环境下表名双引号导致的 SQL 语法错误。
- 修复 PostgreSQL 关键字冲突和大小写敏感问题,自动处理标识符转义。
## 质量改进
- 扩展核心业务模块(认证、清理、校验)的单元测试覆盖,提升回归检测能力。
- 改进测试隔离性,减少跨用例状态泄漏和测试日志噪音。

16
docs/releases/1.11.0.md Normal file
View File

@@ -0,0 +1,16 @@
# 1.11.0
## 物料清理
- 管理员执行清理时可按负责人筛选物料,仅处理指定负责人的数据,避免误删其他人的标记。
- 未选择负责人时自动按订单号关联查询物料,保证清理范围准确。
## 审计日志
- 统一审计记录中的计算机名称来源,消除多来源不一致的情况。
- 增强审计日志的类型安全性和覆盖范围,异常情况下不再丢失日志。
## 质量改进
- 端到端测试迁移至 Playwright 框架,提升测试稳定性和执行效率。
- 改进单元测试的隔离性和模拟驱动覆盖,减少跨用例状态干扰。

5
docs/releases/1.11.1.md Normal file
View File

@@ -0,0 +1,5 @@
# 1.11.1
## 问题修复
- 修复管理员按负责人筛选清理时,因类型声明缺失导致构建失败的问题。

19
docs/releases/1.12.0.md Normal file
View File

@@ -0,0 +1,19 @@
# 1.12.0
## 清理操作历史
- 新增操作历史面板,每次清理的执行记录、订单结果、物料明细均可回溯查看。
- 历史记录按批次归档,支持管理员查看所有用户记录、普通用户查看自己的记录。
- 批次支持展开查看多层详情:执行概况、订单状态、物料操作明细。
## 订单追踪
- 所有输入的订单(含总排号)均会记录在历史中,不再遗漏未找到或未匹配的订单。
- 总排号与订单号并列显示,未匹配的总排号标注为"未找到"ERP 中不存在的订单标注为"ERP 不存在"。
- 内层重试和外层崩溃重试信息在订单详情中完整展示。
## 改进
- 数据库时间统一使用 UTC 存储,界面显示本地时间。
- 试运行模式下跳过物料级别的数据库写入,避免产生无效记录。
- 操作历史面板加宽至 140%,改善订单表格的阅读体验。

6
docs/releases/1.12.1.md Normal file
View File

@@ -0,0 +1,6 @@
# 1.12.1
## 界面与交互
- 操作历史面板新增序号列,订单和物料明细表均可直观查看行号。
- 物料操作结果改用图标显示(已删除 / 已跳过 / 不确定 / 失败),悬停可查看状态名称。

5
docs/releases/1.12.2.md Normal file
View File

@@ -0,0 +1,5 @@
# 1.12.2
## 问题修复
- 修复管理员切换用户后登出,再次选择用户无法进入应用的问题。

5
docs/releases/1.12.3.md Normal file
View File

@@ -0,0 +1,5 @@
# 1.12.3
## 内部优化
- 清理项目根目录无用文件,移除已弃用的 Playwright 配置和调试脚本。

11
docs/releases/1.12.4.md Normal file
View File

@@ -0,0 +1,11 @@
# 1.12.4
## 问题修复
- 修复清理器操作历史在 PostgreSQL 数据库下无法正常加载的问题。
- 修复 PostgreSQL 环境下物料数据写入失败的问题,支持无唯一约束的表。
## 改进
- 优化清理器操作历史的分页加载和执行报告展示,提升大数据量下的响应速度。
- 统一清理器和提取器的操作历史删除确认交互,保持一致的体验。

18
docs/releases/1.13.0.md Normal file
View File

@@ -0,0 +1,18 @@
# 1.13.0
## 清理操作历史
- 新增历史记录全局搜索,支持按批次 ID、订单号、物料编码/名称、用户名、状态等关键字检索。
- 搜索结果高亮显示匹配关键字,快速定位目标记录。
- 搜索结果自动展开批次详情、订单和物料明细,无需逐层手动点击。
- 搜索支持与用户筛选联动,管理员可限定搜索范围到指定用户。
## 界面与交互
- 物料类型管理面板样式更新,改善整体视觉一致性。
- 修复管理员模式下物料类型管理面板的悬浮重叠问题。
## 改进
- 优化历史搜索查询性能,多个批次数据并行获取,减少等待时间。
- 物料类型管理面板加载速度优化,减少不必要的重渲染。

16
docs/releases/1.14.0.md Normal file
View File

@@ -0,0 +1,16 @@
# 1.14.0
## 核心功能
- 清理任务新增长批次自动重建会话机制,在大批量订单处理中可按批次边界自动重开浏览器并继续执行,降低长时间运行中断的概率。
- 会话重建阈值支持配置,可结合批次大小灵活调整重建时机,避免在批次处理中途打断流程。
## 问题修复
- 修复无头模式下大批量清理任务更容易在后半程停滞的问题,提升真实数据处理场景下的完成率。
- 优化清理流程在登录态异常场景下的识别能力,减少详情页长时间等待后才暴露问题的情况。
## 改进
- 新增 ERP 页面状态诊断日志,可区分登录页、首页、查询页、详情页等关键页面状态,方便快速定位异常发生位置。
- 补充详情页打开、页面跳转、弹窗创建和并发等待等关键日志,排查间歇性问题时可以更快还原现场。

10
docs/releases/1.14.1.md Normal file
View File

@@ -0,0 +1,10 @@
# 1.14.1
## 问题修复
- 修复清理任务操作历史中「总排号」始终为空的问题,原始生产编号现在能正确保留并显示在历史记录中。
- 修复会话重建配置缺失时清理任务可能异常中断的问题,提升配置不完整场景下的运行稳定性。
## 改进
- 表名配置统一使用 `schema.tablename` 标准点分写法,与数据库标准格式保持一致。

16
docs/releases/1.15.0.md Normal file
View File

@@ -0,0 +1,16 @@
# 1.15.0
## 核心功能
- 物料清理新增「行号保护」开关,可在执行设置中随时开启或关闭 2000-7999 行号范围的保护,关闭后不再限制该范围内物料的删除,灵活适配不同业务场景。
- 离散物料计划导入链路重构,提取结果直接写入数据库,不再经过中间 Excel 文件,大批量导入整体耗时显著下降。
## 改进
- 订单解析历史记录写入流程优化,批量场景下写入更高效,任务整体耗时更短。
## 问题修复
- 修复 ERP 操作历史中操作时间显示与本地时区不一致的问题,时间展示更直观。
- 修复 PostgreSQL 数据源中时间戳未显式按 UTC 写入的问题,避免跨时区数据偏差。
- 修复 PostgreSQL 预编译 SQL 对 AT、ZONE 等关键字处理错误的问题,相关查询可正常执行。

View File

@@ -11,20 +11,20 @@
### 失败测试分布
| 测试文件 | 失败数量 | 根因分类 | 预计工时 |
|---------|---------|---------|---------|
| `logger.test.ts` | 11 failures | Winston format mock + 循环依赖 | 2-3h |
| `update-service.test.ts` | 1 failure | Mock 参数不匹配 | 30min |
| `update-installer.test.ts` | 1 failure | 路径断言错误 | 15min |
| **总计** | **13 failures** | - | **~3-4h** |
| 测试文件 | 失败数量 | 根因分类 | 预计工时 |
| -------------------------- | --------------- | ------------------------------ | --------- |
| `logger.test.ts` | 11 failures | Winston format mock + 循环依赖 | 2-3h |
| `update-service.test.ts` | 1 failure | Mock 参数不匹配 | 30min |
| `update-installer.test.ts` | 1 failure | 路径断言错误 | 15min |
| **总计** | **13 failures** | - | **~3-4h** |
### 测试通过率
| 指标 | 当前 | 修复后 |
|------|------|--------|
| 失败套件 | 3 suites | 0 suites |
| 失败测试 | 13 tests | 0 tests |
| 通过率 | 95% (311/327) | 100% (327/327) |
| 指标 | 当前 | 修复后 |
| -------- | ------------- | -------------- |
| 失败套件 | 3 suites | 0 suites |
| 失败测试 | 13 tests | 0 tests |
| 通过率 | 95% (311/327) | 100% (327/327) |
---
@@ -42,6 +42,7 @@
#### 问题诊断
**失败模式**:
```
TypeError: __vite_ssr_import_0__.default.format(...) is not a function
at src/main/services/logger/index.ts:114:4
@@ -54,6 +55,7 @@ TypeError: __vite_ssr_import_0__.default.format(...) is not a function
3. **具体表现**: 第 114 行的 `winston.format()` 链式调用在 mock 环境中返回 undefined
**调用栈**:
```
logger.test.ts
→ imports logger.ts
@@ -63,6 +65,7 @@ logger.test.ts
```
**文件位置**:
- 测试文件:`tests/unit/logger.test.ts`
- 被 mock 文件:`src/main/services/logger/index.ts:100-116`
- Setup mock: `tests/setup.ts` (无 winston mock 冲突)
@@ -135,11 +138,13 @@ vi.mock('winston', () => ({
**方案 B: 将 logger.test.ts 转为集成测试 (2 小时)**
如果 mock 过于复杂,可以考虑:
- 使用 vi.resetModules() 确保每次测试都重新加载
- 使用 vi.mock(importOriginal) 混合真实模块
- 或完全重写测试,只测试 logger 的公共 API
**预期结果**:
- ✅ 18/18 tests passing
- ✅ format().combine().timestamp().printf() 链式调用正常工作
- ✅ logger 创建、子 logger、日志输出测试全部通过
@@ -165,8 +170,9 @@ vi.mock('winston', () => ({
**失败测试**: `checks updates for user and auto-downloads available recommendation`
**错误信息**:
```
AssertionError: expected "vi.fn()" to be called with arguments:
AssertionError: expected "vi.fn()" to be called with arguments:
['stable/1.1.0.exe', 'preview/1.1.0.exe']
Number of calls: 0
@@ -175,6 +181,7 @@ Number of calls: 0
**根因**: Mock 调用参数与实际调用不匹配
**代码位置**:
- 测试文件:`tests/unit/update-service.test.ts:165-175`
- 被测文件:`src/main/services/update/update-service.ts`
@@ -200,12 +207,14 @@ it('checks updates for user and auto-downloads available recommendation', async
**步骤 2.2.3**: 更新测试断言
**选项 A: 匹配实际调用**
```typescript
// 如果实际只调用了一个参数
expect(mockDownload).toHaveBeenCalledWith('stable/1.1.0.exe')
```
**选项 B: 使用更松散的断言**
```typescript
// 如果参数顺序或数量有变化
expect(mockDownload).toHaveBeenCalled()
@@ -213,14 +222,12 @@ expect(mockDownload.mock.calls[0]).toContain('stable/1.1.0.exe')
```
**选项 C: 调整 mock 设置**
```typescript
// 确保 mock 正确设置
mockDownload.mockClear()
// ... 触发动作 ...
expect(mockDownload).toHaveBeenCalledWith(
expect.stringContaining('stable'),
expect.any(String)
)
expect(mockDownload).toHaveBeenCalledWith(expect.stringContaining('stable'), expect.any(String))
```
#### 成功标准
@@ -243,8 +250,9 @@ expect(mockDownload).toHaveBeenCalledWith(
**失败测试**: `builds downloaded package path under userData pending-update`
**错误信息**:
```
AssertionError: expected 'D:\...\test-user-data\pending-update\stable-1.2.3.exe'
AssertionError: expected 'D:\...\test-user-data\pending-update\stable-1.2.3.exe'
to contain 'logs\pending-update'
Expected: "logs\pending-update"
@@ -254,6 +262,7 @@ Received: "D:\...\test-user-data\pending-update\stable-1.2.3.exe"
**根因**: Electron mock 的 `app.getPath('userData')` 返回 `test-user-data`,但测试期望路径包含 `logs`
**代码位置**:
- 测试文件:`tests/unit/update-installer.test.ts:13-16`
- Setup mock: `tests/setup.ts:17-24`
@@ -286,6 +295,7 @@ userData: path.join(process.cwd(), 'logs')
```
**推荐**: 方案 2.3.1 (测试适应 mock)
- 理由mock 是为了测试隔离,测试应该适应 mock 环境
#### 成功标准
@@ -310,6 +320,7 @@ npm run test:run tests/unit/logger.test.ts
```
**失败时排查**:
1. 检查 vi.mock 是否在文件顶部 (hoisted)
2. 清除 vitest 缓存:`npx vitest --clearCache`
3. 检查是否有多个 winston mock 冲突
@@ -354,11 +365,11 @@ npm run test:run 2>&1 | Select-String "Test Files.*failed"
### 技术指标
| 指标 | 修复前 | 修复后 | 验证命令 |
|------|-------|-------|---------|
| 失败套件 | 3 suites | 0 suites | `npm run test:run` |
| 失败测试 | 13 tests | 0 tests | `npm run test:run` |
| 通过率 | 95% (311/327) | **100%** (327/327) | 测试报告 |
| 指标 | 修复前 | 修复后 | 验证命令 |
| -------- | ------------- | ------------------ | ------------------ |
| 失败套件 | 3 suites | 0 suites | `npm run test:run` |
| 失败测试 | 13 tests | 0 tests | `npm run test:run` |
| 通过率 | 95% (311/327) | **100%** (327/327) | 测试报告 |
### 验收条件
@@ -373,11 +384,11 @@ npm run test:run 2>&1 | Select-String "Test Files.*failed"
### 技术风险
| 风险 | 可能性 | 影响 | 缓解措施 |
|------|--------|------|---------|
| logger mock 实现复杂 | 中 | 高 | 采用延迟 mock先跑通一部分测试 |
| 链式调用 mock 不完整 | 高 | 中 | 使用 createFormatFn 工厂函数 |
| 循环依赖难解耦 | 低 | 高 | 只修复 mock不重构依赖关系 |
| 风险 | 可能性 | 影响 | 缓解措施 |
| -------------------- | ------ | ---- | ------------------------------- |
| logger mock 实现复杂 | 中 | 高 | 采用延迟 mock先跑通一部分测试 |
| 链式调用 mock 不完整 | 高 | 中 | 使用 createFormatFn 工厂函数 |
| 循环依赖难解耦 | 低 | 高 | 只修复 mock不重构依赖关系 |
### 时间风险
@@ -386,6 +397,7 @@ npm run test:run 2>&1 | Select-String "Test Files.*failed"
- **保守估计**: 6 小时 (遇到意外问题)
**风险缓解**: 如果 logger mock 问题超过 3 小时无法解决,考虑:
1. 暂时跳过 logger.test.ts (保持 95% 通过率)
2. 先修复简单的 update 测试 (13 failures → 2 failures)
3. 记录问题,后续专门花精力解决
@@ -401,6 +413,7 @@ npm run test:run 2>&1 | Select-String "Test Files.*failed"
**实际工时**: X 小时
**修复步骤**:
1. [ ] 诊断 mock 问题
2. [ ] 实现 formatFn 工厂
3. [ ] 添加所有链式方法
@@ -408,10 +421,12 @@ npm run test:run 2>&1 | Select-String "Test Files.*failed"
5. [ ] 验证测试通过
**遇到的问题**:
- 问题 1: [描述] → 解决方案: [方案]
- 问题 2: [描述] → 解决方案: [方案]
**关键代码**:
```typescript
// 最终有效的 mock 实现
```
@@ -424,7 +439,8 @@ npm run test:run 2>&1 | Select-String "Test Files.*failed"
**结束时间**: HH:MM
**实际工时**: X 分钟
**修复方式**:
**修复方式**:
- [ ] 修改断言
- [ ] 修改 mock 参数
- [ ] 其他: [描述]
@@ -439,7 +455,8 @@ npm run test:run 2>&1 | Select-String "Test Files.*failed"
**结束时间**: HH:MM
**实际工时**: X 分钟
**修复方式**:
**修复方式**:
- [ ] 修改断言
- [ ] 修改 mock
- [ ] 其他: [描述]

View File

@@ -0,0 +1,781 @@
# ERPAuto 测试覆盖率提升计划
## 1. 执行摘要
### 1.1 当前状态评估
| 指标 | 当前值 | 目标值 | 差距 |
| ------------------ | ------ | ------ | ------- |
| **总体行覆盖率** | 11.36% | 70% | -58.64% |
| **总体函数覆盖率** | 21.29% | 70% | -48.71% |
| **总体分支覆盖率** | 10.08% | 60% | -49.92% |
| **测试文件总数** | 54 | 100+ | -46+ |
**关键模块覆盖率差距:**
| 模块 | 当前覆盖率 | 要求阈值 | 优先级 |
| ---------------------------------------- | ---------- | -------- | ------------- |
| ERP 服务 (`src/main/services/erp/**`) | 11.68% | 80% | P0 |
| 更新服务 (`src/main/services/update/**`) | 42.45% | 80% | P0 |
| 数据库服务 | 17.24% | 70% | P1 |
| 配置管理 | 20.56% | 70% | P1 |
| 日志服务 | 70.67% | 70% | P2 (已达标的) |
### 1.2 提升目标
**阶段性目标:**
- **Phase 1 (4 周)**ERP 服务达到 60%,更新服务达到 70%
- **Phase 2 (4 周)**:数据库服务达到 60%,配置管理达到 60%
- **Phase 3 (4 周)**:所有关键模块达到目标阈值,总体覆盖率达到 70%
**最终目标:**
- 全局覆盖率70% 行 / 70% 函数 / 60% 分支
- ERP 服务80% 行 / 80% 函数 / 70% 分支
- 更新服务80% 行 / 80% 函数 / 70% 分支
### 1.3 时间线估算
| 阶段 | 持续时间 | 里程碑 |
| -------- | --------- | ---------------------- |
| Phase 1 | 4 周 | ERP 核心服务测试完成 |
| Phase 2 | 4 周 | 数据层与配置层测试完成 |
| Phase 3 | 4 周 | 集成测试与 E2E 补全 |
| 缓冲期 | 2 周 | 修复与优化 |
| **总计** | **14 周** | **达到目标覆盖率** |
---
## 2. 分阶段提升计划
### Phase 1: ERP 核心服务测试攻坚(第 1-4 周)
**目标:** ERP 服务覆盖率从 11.68% 提升至 60%
**工作内容:**
| 模块 | 文件数 | 新增测试数 | 优先级 |
| ---------------------- | ------ | ---------- | ------ |
| `erp-auth.ts` | 1 | 15 | P0 |
| `extractor.ts` | 1 | 20 | P0 |
| `extractor-core.ts` | 1 | 15 | P0 |
| `cleaner.ts` | 1 | 12 | P0 |
| `ErpBrowserManager.ts` | 1 | 10 | P1 |
| `order-resolver.ts` | 1 | 8 | P1 |
| `page-diagnostics.ts` | 1 | 6 | P2 |
| `erp-error-context.ts` | 1 | 5 | P2 |
| `locators.ts` | 1 | 8 | P1 |
**预计投入:** 80-100 小时
**成功标准:**
- [ ] ERP 服务行覆盖率 ≥ 60%
- [ ] ERP 服务函数覆盖率 ≥ 70%
- [ ] 新增测试文件9 个
- [ ] 所有 P0 模块有完整测试覆盖
---
### Phase 2: 数据层与配置层测试(第 5-8 周)
**目标:** 数据库服务与配置管理覆盖率达标
**工作内容:**
#### 2.1 数据库服务17.24% → 60%
| 模块 | 文件数 | 新增测试数 | 优先级 |
| ---------------------------------------------- | ------ | ---------- | ------ |
| `mysql.ts` / `sql-server.ts` / `postgresql.ts` | 3 | 18 | P0 |
| `data-source.ts` | 1 | 8 | P0 |
| `data-importer.ts` | 1 | 10 | P0 |
| DAO 层文件 | 4 | 16 | P1 |
| Repository 层 | 2 | 8 | P1 |
| 数据库实体 | 2 | 6 | P2 |
#### 2.2 配置管理20.56% → 60%
| 模块 | 文件数 | 新增测试数 | 优先级 |
| ------------------- | ------ | ---------- | ------ |
| `config-manager.ts` | 1 | 20 | P0 |
| 配置 Schema 验证 | 1 | 10 | P1 |
#### 2.3 用户服务(新增)
| 模块 | 文件数 | 新增测试数 | 优先级 |
| ---------------------------- | ------ | ---------- | ------ |
| `session-manager.ts` | 1 | 8 | P1 |
| `user-erp-config-service.ts` | 1 | 10 | P1 |
| `bip-users-dao.ts` | 1 | 6 | P2 |
**预计投入:** 100-120 小时
**成功标准:**
- [ ] 数据库服务行覆盖率 ≥ 60%
- [ ] 配置管理行覆盖率 ≥ 60%
- [ ] 新增测试文件15 个
- [ ] 所有数据库方言有完整测试
---
### Phase 3: 更新服务与其他模块补全(第 9-12 周)
**目标:** 更新服务达到 80%,其他服务达到 70%
**工作内容:**
#### 3.1 更新服务42.45% → 80%
| 模块 | 文件数 | 新增测试数 | 优先级 |
| ---------------------------- | ------ | ---------- | ------ |
| `update-service.ts` | 1 | 15 | P0 |
| `update-catalog-service.ts` | 1 | 12 | P0 |
| `update-installer.ts` | 1 | 10 | P0 |
| `update-storage-client.ts` | 1 | 10 | P0 |
| `update-status-publisher.ts` | 1 | 6 | P1 |
| `update-support.ts` | 1 | 5 | P1 |
| `update-utils.ts` | 1 | 5 | P2 |
#### 3.2 其他关键服务
| 模块 | 文件数 | 新增测试数 | 优先级 |
| -------------------------- | ------ | ---------- | ------ |
| 验证服务 (`validation/**`) | 3 | 15 | P1 |
| 清理服务 (`cleaner/**`) | 2 | 10 | P1 |
| Excel 服务 | 2 | 8 | P2 |
| 报告生成 | 1 | 6 | P2 |
| Playwright 浏览器服务 | 2 | 10 | P1 |
| RustFS 服务 | 2 | 8 | P2 |
**预计投入:** 100-120 小时
**成功标准:**
- [ ] 更新服务行覆盖率 ≥ 80%
- [ ] 更新服务函数覆盖率 ≥ 80%
- [ ] 新增测试文件17 个
- [ ] 所有 P0/P1 模块覆盖率达标
---
### Phase 4: 集成测试与 E2E 强化(第 13-14 周)
**目标:** 强化集成测试与端到端测试
**工作内容:**
#### 4.1 集成测试扩展7 → 20 个)
| 测试场景 | 优先级 | 描述 |
| -------------------------- | ------ | ---------------------- |
| ERP 登录 + 提取完整流程 | P0 | 验证认证与数据提取集成 |
| 数据库事务完整流程 | P0 | 验证 TypeORM 事务边界 |
| 配置热加载与验证 | P1 | 验证配置更新传播 |
| 更新检查 + 下载 + 安装流程 | P0 | 验证更新完整链路 |
| 日志异步写入与轮转 | P1 | 验证日志系统 |
| 用户会话切换流程 | P1 | 验证多用户场景 |
| Excel 导入导出完整流程 | P2 | 验证文件处理链 |
#### 4.2 E2E 测试扩展3 → 15 个)
| 用户旅程 | 优先级 | 描述 |
| -------------------- | ------ | -------------------------------- |
| 管理员完整工作流程 | P0 | 登录 → 提取 → 清理 → 验证 → 登出 |
| 普通用户数据提取流程 | P0 | 登录 → 提取 → 查看结果 |
| Guest 只读访问流程 | P1 | 登录 → 查看历史记录 |
| 配置管理流程 | P1 | 修改配置 → 保存 → 验证生效 |
| 自动更新流程 | P0 | 检查更新 → 下载 → 安装 → 重启 |
| 错误恢复流程 | P1 | 断网重连、会话过期恢复 |
| 批量处理流程 | P1 | 大批量订单处理性能验证 |
**预计投入:** 60-80 小时
**成功标准:**
- [ ] 集成测试文件20 个
- [ ] E2E 测试文件15 个
- [ ] 关键用户旅程 100% 覆盖
- [ ] 整体覆盖率达到 70%
---
## 3. 逐模块测试计划
### 3.1 ERP 服务模块
#### 3.1.1 `erp-auth.ts` (P0)
**当前覆盖率:** < 20%
**目标覆盖率:** 80%
| 测试场景 | 测试类型 | Mock 对象 | 预期结果 |
| ------------------- | -------- | ------------------------------- | ------------------- |
| 成功登录流程 | 单元 | Playwright Browser/Context/Page | 返回有效 ErpSession |
| 登录失败 - 网络错误 | 单元 | Playwright + 模拟网络错误 | 抛出连接错误 |
| 登录失败 - 凭证错误 | 单元 | Page + 模拟错误消息 | 抛出认证错误 |
| 会话复用 - 已登录 | 单元 | Session Mock | 直接返回现有会话 |
| 登出流程 | 单元 | Browser/Context Mock | 资源正确释放 |
| 会话超时检测 | 单元 | Page + 超时 Mock | 返回未登录状态 |
| 页面元素定位失败 | 单元 | Page + Selector 失败 | 抛出元素未找到错误 |
| SSL 证书错误处理 | 集成 | 真实 Browser + 自签名证书 | 成功建立连接 |
**预计测试数:** 15
---
#### 3.1.2 `extractor.ts` (P0)
**当前覆盖率:** ~30%
**目标覆盖率:** 80%
| 测试场景 | 测试类型 | Mock 对象 | 预期结果 |
| -------------- | -------- | -------------------------- | -------------------- |
| 单订单提取成功 | 单元 | ErpAuthService + Page | 返回 ExtractorResult |
| 批量订单提取 | 单元 | ErpAuthService + 循环 Mock | 正确分批处理 |
| 订单号无效处理 | 单元 | Page + 错误响应 | 记录错误,继续处理 |
| 下载文件合并 | 单元 | ExcelJS + fs Mock | 生成合并文件 |
| 数据库持久化 | 集成 | DatabaseService Mock | 记录成功导入 |
| 并发限制控制 | 单元 | 信号量 Mock | 不超过并发上限 |
| 提取中断恢复 | 集成 | 模拟中断 + 恢复 | 从断点继续 |
| 结果统计准确性 | 单元 | 完整 Mock 链 | 统计数字准确 |
**预计测试数:** 20
---
#### 3.1.3 `extractor-core.ts` (P0)
**当前覆盖率:** < 10%
**目标覆盖率:** 80%
| 测试场景 | 测试类型 | Mock 对象 | 预期结果 |
| ---------------- | -------- | ---------------- | ------------ |
| 页面导航到列表页 | 单元 | Page + Frame | 成功导航 |
| 订单号输入 | 单元 | Locator Mock | 正确填充 |
| 查询按钮点击 | 单元 | Locator Mock | 触发查询 |
| 表格数据解析 | 单元 | Table Locator | 返回物料列表 |
| 分页处理 | 单元 | Page + 多页 Mock | 遍历所有页 |
| 下载按钮点击 | 单元 | Locator + Dialog | 触发下载 |
| 下载完成等待 | 单元 | fs + 文件事件 | 文件落地 |
| 错误弹窗检测 | 单元 | Page + 错误元素 | 捕获错误消息 |
**预计测试数:** 15
---
#### 3.1.4 `cleaner.ts` (P0)
**当前覆盖率:** ~25%
**目标覆盖率:** 80%
| 测试场景 | 测试类型 | Mock 对象 | 预期结果 |
| ---------------- | -------- | ------------------ | ------------ |
| 单物料删除成功 | 单元 | Page + Locator | 删除成功 |
| 批量物料删除 | 单元 | 循环删除 Mock | 全部删除 |
| 物料不存在处理 | 单元 | Page + 空结果 | 跳过并记录 |
| 删除按钮失效处理 | 单元 | Locator + disabled | 跳过该物料 |
| 干运行模式 | 单元 | 不执行实际删除 | 返回预览结果 |
| 并发控制 | 单元 | 信号量 Mock | 限制并发数 |
| 错误重试机制 | 集成 | 失败→成功 Mock | 重试成功 |
| 删除结果统计 | 单元 | 完整 Mock 链 | 统计准确 |
**预计测试数:** 12
---
### 3.2 数据库服务模块
#### 3.2.1 数据库连接服务 (P0)
**文件:** `mysql.ts`, `sql-server.ts`, `postgresql.ts`
**当前覆盖率:** ~20%
**目标覆盖率:** 70%
| 测试场景 | 测试类型 | Mock 对象 | 预期结果 |
| ------------------- | -------- | ----------------------- | ------------ |
| MySQL 连接成功 | 单元 | mysql2 Pool Mock | 返回连接实例 |
| SQL Server 连接成功 | 单元 | mssql Connection Mock | 返回连接实例 |
| PostgreSQL 连接成功 | 单元 | pg Pool Mock | 返回连接实例 |
| 连接失败处理 | 单元 | 模拟连接拒绝 | 抛出错误 |
| 查询执行成功 | 集成 | 数据库 Mock + 返回结果 | 正确返回数据 |
| 事务提交 | 集成 | Transaction Mock | 成功提交 |
| 事务回滚 | 集成 | Transaction Mock + 错误 | 正确回滚 |
| 连接池释放 | 单元 | Pool Mock | 正确关闭 |
**预计测试数:** 18 (3 个数据库 × 6 场景)
---
#### 3.2.2 数据源管理 (P0)
**文件:** `data-source.ts`
**当前覆盖率:** < 10%
**目标覆盖率:** 70%
| 测试场景 | 测试类型 | Mock 对象 | 预期结果 |
| --------------- | -------- | --------------- | -------------- |
| TypeORM 初始化 | 单元 | DataSource Mock | 成功初始化 |
| 数据源销毁 | 单元 | DataSource Mock | 正确释放 |
| Repository 获取 | 单元 | Repository Mock | 返回对应仓库 |
| 实体注册验证 | 单元 | Entity Mock | 所有实体已注册 |
| 多次初始化防护 | 单元 | 状态检查 Mock | 不重复初始化 |
**预计测试数:** 8
---
#### 3.2.3 数据导入器 (P0)
**文件:** `data-importer.ts`
**当前覆盖率:** < 15%
**目标覆盖率:** 70%
| 测试场景 | 测试类型 | Mock 对象 | 预期结果 |
| -------------- | -------- | ------------------------ | ------------ |
| Excel 读取成功 | 集成 | ExcelJS + 测试文件 | 解析数据结构 |
| 数据验证通过 | 单元 | Schema 验证 Mock | 数据合法 |
| 数据验证失败 | 单元 | Schema 验证 Mock | 抛出验证错误 |
| 批量插入 | 集成 | Repository Mock | 正确分批插入 |
| 重复数据处理 | 单元 | Repository + exists 检查 | 跳过或更新 |
| 插入失败回滚 | 集成 | Transaction Mock + 错误 | 全部回滚 |
| 导入进度追踪 | 单元 | EventEmitter Mock | 发送进度事件 |
| 导入结果统计 | 单元 | 完整 Mock 链 | 统计准确 |
**预计测试数:** 10
---
### 3.3 配置管理模块
#### 3.3.1 `config-manager.ts` (P0)
**当前覆盖率:** ~25%
**目标覆盖率:** 70%
| 测试场景 | 测试类型 | Mock 对象 | 预期结果 |
| ---------------- | -------- | -------------------- | ------------ |
| 配置文件加载成功 | 单元 | fs + yaml Mock | 返回有效配置 |
| 配置文件不存在 | 单元 | fs Mock + 不存在 | 使用默认配置 |
| 配置文件格式错误 | 单元 | yaml Mock + 解析失败 | 抛出解析错误 |
| Zod 验证失败 | 单元 | 无效配置数据 | 抛出验证错误 |
| 配置更新 | 单元 | fs + yaml Mock | 文件正确写入 |
| 重置为默认值 | 单元 | 完整 Mock 链 | 恢复默认 |
| 导出为 YAML | 单元 | yaml.stringify Mock | 格式正确 |
| 数据库类型切换 | 单元 | 状态 Mock | 返回正确配置 |
| 日志配置应用 | 集成 | Winston Mock | 日志级别生效 |
| 审计配置应用 | 集成 | AuditLogger Mock | 审计配置生效 |
| 单例模式验证 | 单元 | 多次 getInstance | 返回同一实例 |
| 并发读取安全 | 集成 | 并发 Mock + 竞争 | 数据一致 |
**预计测试数:** 20
---
### 3.4 更新服务模块
#### 3.4.1 `update-service.ts` (P0)
**当前覆盖率:** ~50%
**目标覆盖率:** 80%
| 测试场景 | 测试类型 | Mock 对象 | 预期结果 |
| ------------------- | -------- | ------------------------- | ------------ |
| 服务初始化 | 单元 | ConfigManager + 依赖 Mock | 服务就绪 |
| 获取更新状态 | 单元 | 状态 Mock | 返回当前状态 |
| 获取更新目录 | 单元 | CatalogService Mock | 返回目录结构 |
| 检查更新 - 有新版本 | 集成 | S3Client Mock + 新版本 | 返回更新列表 |
| 检查更新 - 无新版本 | 集成 | S3Client Mock + 最新版 | 返回空列表 |
| 下载更新 - 成功 | 集成 | S3Client + fs Mock | 文件下载成功 |
| 下载更新 - 失败 | 集成 | S3Client + 网络错误 | 抛出错误 |
| 校验 SHA256 - 通过 | 单元 | crypto Mock | 校验通过 |
| 校验 SHA256 - 失败 | 单元 | crypto Mock + 不匹配 | 抛出校验错误 |
| 安装更新 | 集成 | child_process Mock | 启动安装器 |
| 用户权限检查 | 单元 | UserType Mock | 正确过滤 |
| 定期自动检查 | 集成 | setInterval Mock | 按时检查 |
**预计测试数:** 15
---
#### 3.4.2 `update-catalog-service.ts` (P0)
**当前覆盖率:** ~40%
**目标覆盖率:** 80%
| 测试场景 | 测试类型 | Mock 对象 | 预期结果 |
| ------------ | -------- | ------------------ | ------------ |
| 构建更新目录 | 单元 | StorageClient Mock | 返回分类目录 |
| 稳定版过滤 | 单元 | UserType + 目录 | 只看 stable |
| 管理员全访问 | 单元 | AdminType + 目录 | 看全部通道 |
| 更新历史记录 | 单元 | Repository Mock | 返回历史记录 |
| 限制记录数量 | 单元 | 数据截断 | 不超过上限 |
**预计测试数:** 12
---
#### 3.4.3 `update-storage-client.ts` (P0)
**当前覆盖率:** ~35%
**目标覆盖率:** 80%
| 测试场景 | 测试类型 | Mock 对象 | 预期结果 |
| --------------- | -------- | ------------------- | -------------- |
| S3 客户端初始化 | 单元 | AWS SDK Mock | 客户端创建成功 |
| 列出更新包 | 单元 | S3 listObjects Mock | 返回对象列表 |
| 下载文件 | 单元 | S3 getObject Mock | 返回文件流 |
| 下载失败处理 | 单元 | S3 + 网络错误 | 抛出错误 |
| 计算 SHA256 | 单元 | crypto Mock | 哈希值正确 |
| 重试机制 | 集成 | 失败→成功 Mock | 重试成功 |
**预计测试数:** 10
---
## 4. 测试类别实施指南
### 4.1 单元测试
**适用范围:**
- 服务类Service的业务逻辑
- 工具函数Utility Functions
- 数据处理函数
- 类型转换函数
**Mock 策略:**
```typescript
// 使用现有 Mock 库
import {
createMockLogger,
createMockConfigManager,
createMockErpAuthService,
createMockDatabaseService,
createMockDataSource,
createMockRepository
} from '@/tests/mocks'
// 示例ERP Auth 测试
describe('ErpAuthService', () => {
const mockConfig = { url: 'https://test.com', username: 'test', password: 'test' }
const mockPage = createMockPage() // 来自 mocks/index.ts
it('should login successfully', async () => {
mockPage.goto.mockResolvedValue(undefined)
mockPage.waitForSelector.mockResolvedValue(undefined)
const authService = new ErpAuthService(mockConfig)
// 注入 mock (需要构造函数支持或使用 vi.mock)
const session = await authService.login()
expect(session.isLoggedIn).toBe(true)
})
})
```
**测试覆盖重点:**
1. **正常路径:** 主要业务流程成功执行
2. **异常路径:** 错误处理、回滚、重试
3. **边界条件:** 空输入、极大值、极小值
4. **分支覆盖:** if/else、switch/case 所有分支
---
### 4.2 集成测试
**适用范围:**
- 多服务协作场景
- 数据库事务边界
- 文件系统交互
- 外部服务调用(需 Stub
**测试模式:**
```typescript
import { describe, it, expect, beforeEach, afterEach } from 'vitest'
import { DatabaseService } from '@/main/services/database'
import { ConfigManager } from '@/main/services/config'
describe('Database + Config Integration', () => {
let db: DatabaseService
let configManager: ConfigManager
beforeEach(async () => {
// 使用内存数据库或测试配置
configManager = ConfigManager.getInstance()
db = new DatabaseService(configManager)
await db.connect()
})
afterEach(async () => {
await db.disconnect()
})
it('should persist and retrieve data', async () => {
// 实际数据库操作
await db.query('INSERT INTO ...')
const result = await db.query('SELECT ...')
expect(result.rows).toHaveLength(1)
})
})
```
**集成测试清单:**
| 集成场景 | 涉及模块 | 预期时间 |
| --------------- | --------------------------- | -------- |
| ERP 登录 + 提取 | ErpAuth + Extractor | < 5s |
| 数据库事务 | DataSource + Repository | < 2s |
| 配置更新传播 | ConfigManager + Logger | < 1s |
| 文件导入导出 | ExcelParser + fs | < 3s |
| 更新下载校验 | UpdateService + S3 + crypto | < 10s |
---
### 4.3 E2E 测试
**适用范围:**
- 完整用户旅程
- UI 交互验证
- 真实浏览器行为
- 跨进程通信
**Playwright 测试模式:**
```typescript
import { test, expect } from '@playwright/test'
test('complete extraction workflow', async ({ page }) => {
// 1. 导航到登录页
await page.goto('http://localhost:5173/login')
// 2. 登录
await page.getByPlaceholder('用户名').fill('admin')
await page.getByPlaceholder('密码').fill('admin123')
await page.getByRole('button', { name: '登录' }).click()
// 3. 等待跳转
await expect(page).toHaveURL(/dashboard/)
// 4. 进入提取页面
await page.getByText('数据提取').click()
// 5. 输入订单号
await page.getByPlaceholder('请输入订单号').fill('SC202601001')
// 6. 开始提取
await page.getByRole('button', { name: '开始提取' }).click()
// 7. 等待完成
await expect(page.getByText('提取完成')).toBeVisible({ timeout: 30000 })
// 8. 验证结果
await expect(page.getByText('记录数:')).toBeVisible()
})
```
**E2E 测试关键场景:**
| 用户旅程 | 步骤数 | 预期时间 | 优先级 |
| ---------------- | ------ | -------- | ------ |
| 管理员完整工作流 | 15 | < 60s | P0 |
| 普通用户提取 | 8 | < 45s | P0 |
| 配置管理 | 10 | < 30s | P1 |
| 自动更新 | 8 | < 90s | P0 |
| 错误恢复 | 6 | < 40s | P1 |
---
## 5. 资源与工作量估算
### 5.1 人员配置建议
| 角色 | 人数 | 职责 |
| -------------- | -------- | ----------------------- |
| 测试开发工程师 | 2 人 | 单元测试、集成测试编写 |
| 全栈工程师 | 1 人 | E2E 测试、Mock 基础设施 |
| 代码审查员 | 1 人 | 测试代码质量审查 |
| **总计** | **4 人** | **14 周完成** |
**单人模式调整:**
若只有 1 人负责,时间调整为:
- 周投入20-25 小时
- 总周期20-24 周
- 优先级P0 → P1 → P2
---
### 5.2 工作量分解
| 阶段 | 任务 | 估算小时 |
| -------- | ----------------- | ---------------- |
| Phase 1 | ERP 服务单元测试 | 80-100 |
| | Mock 基础设施优化 | 10-15 |
| Phase 2 | 数据库单元测试 | 60-80 |
| | 配置单元测试 | 20-30 |
| | 集成测试 | 20-30 |
| Phase 3 | 更新服务测试 | 60-80 |
| | 其他服务测试 | 40-50 |
| Phase 4 | E2E 测试 | 40-60 |
| | 覆盖率优化 | 20-30 |
| **总计** | | **350-475 小时** |
---
### 5.3 风险因素
| 风险 | 可能性 | 影响 | 缓解措施 |
| --------------------------- | ------ | ---- | ------------------------ |
| Playwright 浏览器兼容性问题 | 中 | 高 | 提前验证浏览器版本 |
| 数据库连接不稳定 | 低 | 中 | 使用内存数据库或容器 |
| Mock 与实现不同步 | 高 | 中 | 定期同步,添加类型检查 |
| 测试维护成本过高 | 中 | 中 | 使用工厂模式,避免硬编码 |
| 覆盖率工具性能影响 | 低 | 低 | CI 中仅对变更文件检查 |
---
## 6. 成功度量标准
### 6.1 覆盖率指标
| 里程碑 | 总体行覆盖率 | ERP 服务 | 更新服务 | 数据库 |
| ------------ | ------------ | -------- | -------- | ------- |
| Phase 1 完成 | 25% | 60% | 50% | 25% |
| Phase 2 完成 | 45% | 65% | 60% | 60% |
| Phase 3 完成 | 65% | 75% | 80% | 65% |
| Phase 4 完成 | **70%** | **80%** | **80%** | **70%** |
---
### 6.2 测试数量目标
| 类型 | 当前 | Phase 1 | Phase 2 | Phase 3 | Phase 4 |
| -------------- | ------ | ------- | ------- | ------- | ------- |
| 单元测试文件 | 40 | 50 | 60 | 75 | 85 |
| 集成测试文件 | 7 | 8 | 12 | 15 | 20 |
| E2E 测试文件 | 3 | 3 | 3 | 5 | 15 |
| **总测试文件** | **50** | **61** | **75** | **95** | **120** |
---
### 6.3 质量门禁
**每个 PR 必须满足:**
1. **新增代码覆盖率 ≥ 80%** (使用 `vitest --coverage --changed`)
2. **无测试失败**
3. **测试执行时间 < 30s** (单元测试) / < 120s (集成) / < 5min (E2E)
4. **无 Mock 滥用** (真实逻辑必须有真实测试)
**CI/CD 检查:**
```yaml
# GitHub Actions 示例
- name: Test & Coverage
run: |
npm run test:coverage
# 检查覆盖率阈值
npx vitest --coverage --thresholds
# 生成报告
npx vitest --coverage --reporter=html
# 上传覆盖率
uses: codecov/codecov-action@v4
```
---
## 7. 立即行动项(本周)
### 7.1 优先级 P0 - 必须完成
| 任务 | 负责人 | 截止日期 | 状态 |
| -------------------------------- | ------ | -------- | ---- |
| 创建 ERP Auth 测试文件框架 | - | Day 2 | ☐ |
| 创建 Extractor Core 测试文件框架 | - | Day 3 | ☐ |
| 扩展现有 Mock 库支持新增场景 | - | Day 4 | ☐ |
| 运行首次覆盖率基准测试 | - | Day 1 | ☐ |
### 7.2 优先级 P1 - 建议完成
| 任务 | 负责人 | 截止日期 | 状态 |
| -------------------------- | ------ | -------- | ---- |
| 整理现有测试文件结构 | - | Day 3 | ☐ |
| 创建测试模板和最佳实践文档 | - | Day 5 | ☐ |
| 设置覆盖率 CI 报告 | - | Day 5 | ☐ |
### 7.3 技术准备清单
```bash
# 1. 安装覆盖率报告工具
npm install --save-dev @vitest/coverage-v8
# 2. 运行基准测试
npm run test:coverage
# 3. 查看 HTML 报告
npm run test:coverage
# 打开 coverage/index.html
# 4. 按文件查看详细覆盖率
npx vitest --coverage --reporter=verbose
```
### 7.4 第一个 Sprint 目标Week 1-2
**目标ERP Auth 测试完成 50%**
- [ ] `tests/unit/services/erp/erp-auth.test.ts` 创建
- [ ] 成功登录场景测试3 个)
- [ ] 失败场景测试5 个)
- [ ] 会话管理测试3 个)
- [ ] Mock 优化支持 Page 生命周期事件
- [ ] 运行测试,覆盖率 ≥ 40%
---
## 附录
### A. 现有测试资源
| 资源 | 路径 | 状态 |
| --------- | ---------------------------- | ------------------ |
| 测试设置 | `tests/setup.ts` | 完整 Electron Mock |
| 测试工厂 | `tests/fixtures/factory.ts` | 8 个工厂类 |
| Mock 库 | `tests/mocks/index.ts` | 15+ Mock 函数 |
| 测试文档 | `docs/TEST_FACTORY_USAGE.md` | 工厂使用指南 |
| Mock 文档 | `docs/MOCK_LIBRARY_USAGE.md` | Mock 使用指南 |
### B. 推荐测试工具
| 工具 | 用途 |
| ---------------------- | ------------- |
| `vitest` | 单元测试框架 |
| `@playwright/test` | E2E 测试框架 |
| `@vitest/coverage-v8` | V8 覆盖率引擎 |
| `vitest-html-reporter` | HTML 报告生成 |
### C. 相关文件
- `vitest.config.ts` - Vitest 配置与覆盖率阈值
- `package.json` - 测试脚本定义
- `.github/workflows/test.yml` - CI 测试工作流
---
**文档版本:** 1.0
**创建日期:** 2026-04-05
**最后更新:** 2026-04-05
**维护者:** ERPAuto 开发团队

View File

@@ -0,0 +1,931 @@
# ERPAuto 测试质量审查报告
**审查日期**: 2026-04-05
**审查范围**: 新增的 ERP 服务单元测试文件
**审查者**: AI Code Review Agent
---
## 执行摘要
本次审查覆盖了 6 个新增的 ERP 服务单元测试文件,共计 **117 个测试用例**114 个通过3 个待实现)。测试整体质量**优秀**,符合企业级测试标准。
### 总体评分:**A (90/100)**
| 评估维度 | 得分 | 权重 | 加权分 |
| ------------ | ------ | -------- | -------- |
| 测试覆盖率 | 85/100 | 30% | 25.5 |
| 测试设计质量 | 92/100 | 25% | 23.0 |
| Mock 策略 | 90/100 | 20% | 18.0 |
| 可维护性 | 88/100 | 15% | 13.2 |
| 错误处理测试 | 95/100 | 10% | 9.5 |
| **总计** | | **100%** | **89.2** |
---
## 1. 测试文件概览
### 1.1 文件统计
| 测试文件 | 测试用例数 | 通过 | 失败 | 跳过/Todo | 行数 |
| --------------------------- | ---------- | ------- | ----- | --------- | -------- |
| `erp-auth.test.ts` | 11 | 11 | 0 | 0 | 216 |
| `cleaner.test.ts` | 20 | 20 | 0 | 0 | 272 |
| `ErpBrowserManager.test.ts` | 20 | 20 | 0 | 0 | 252 |
| `extractor-core.test.ts` | 11 | 8 | 0 | 3 | 265 |
| `extractor.test.ts` | 17 | 17 | 0 | 0 | 350 |
| `order-resolver.test.ts` | 26 | 26 | 0 | 0 | 363 |
| `page-diagnostics.test.ts` | 6 | 6 | 0 | 0 | - |
| `erp-error-context.test.ts` | 7 | 7 | 0 | 0 | - |
| **总计** | **118** | **115** | **0** | **3** | **1718** |
### 1.2 测试执行结果
```
✓ 8 个测试文件全部通过
✓ 114 个测试用例通过
✓ 0 个测试失败
⚠ 3 个测试标记为 todo需要集成测试环境
✓ 执行时间:< 1.5 秒(优秀)
```
---
## 2. 详细质量评估
### 2.1 `erp-auth.test.ts` - **A+ (95/100)**
**测试对象**: `ErpAuthService` - ERP 认证服务
#### 优点 ✅
1. **完整的生命周期测试**
- 构造函数初始化验证
- 登录流程(成功/失败)
- 会话复用机制
- 登出/关闭处理
2. **优秀的 Mock 策略**
```typescript
vi.mock('playwright', () => ({
chromium: { launch: vi.fn() }
}))
```
- 外部依赖完全隔离
- 模拟对象结构清晰
3. **边界条件覆盖**
- `contentFrame` 返回 `null` 的异常处理
- 重复登录的会话复用
- 未登录时调用 `getSession()` 的错误处理
4. **测试命名规范**
- 使用 `should/could` 语义
- 清晰表达测试意图
#### 改进建议 🔧
1. **缺少真实场景集成测试**
```typescript
// TODO: 添加集成测试
it('should login with real browser (integration)', async () => {
// 使用真实 Playwright 浏览器测试
})
```
2. **错误消息验证不够精确**
```typescript
// 当前
expect(() => service.getSession()).toThrow('Not logged in')
// 建议
expect(() => service.getSession()).toThrow('Not logged in. Call login() first.')
```
3. **缺少性能测试**
```typescript
it('should complete login within 5 seconds', async () => {
const start = Date.now()
await service.login()
expect(Date.now() - start).toBeLessThan(5000)
})
```
#### 覆盖率评估
| 方法 | 测试覆盖 | 评价 |
| --------------- | ----------------- | ---- |
| `constructor()` | ✓ 完全覆盖 | 优秀 |
| `login()` | ✓ 主要路径 + 异常 | 优秀 |
| `getSession()` | ✓ 覆盖 | 良好 |
| `isActive()` | ✓ 覆盖 | 良好 |
| `close()` | ✓ 覆盖 | 良好 |
---
### 2.2 `cleaner.test.ts` - **A (90/100)**
**测试对象**: `CleanerService` - 物料清理服务
#### 优点 ✅
1. **纯函数测试设计优秀**
```typescript
describe('shouldDeleteMaterial()', () => {
it('should return true when material matches all deletion criteria', () => {
const result = cleaner.shouldDeleteMaterial({...})
expect(result).toBe(true)
})
})
```
- 无副作用,易于测试
- 输入输出明确
2. **边界值测试完备**
```typescript
it('should respect boundary row numbers', () => {
// Row 1999: can delete
expect(...).toBe(true)
// Row 2000: protected
expect(...).toBe(false)
// Row 7999: protected
expect(...).toBe(false)
// Row 8000: can delete
expect(...).toBe(true)
})
```
3. **辅助函数测试充分**
- `createBatches()`: 数组分批逻辑
- `runWithConcurrency()`: 并发控制验证
- `getMissingOrders()`: 集合差集计算
4. **并发测试验证**
```typescript
it('should limit parallelism to specified concurrency', async () => {
let running = 0
let peak = 0
await runWithConcurrency(items, 2, async () => {
running += 1
peak = Math.max(peak, running)
await new Promise((resolve) => setTimeout(resolve, 10))
running -= 1
})
expect(peak).toBeLessThanOrEqual(2)
expect(peak).toBe(2)
})
```
#### 改进建议 🔧
1. **缺少 `clean()` 主方法测试**
- 文件顶部有 TODO 注释说明需要集成测试
- 建议补充:
```typescript
describe('clean() - Integration', () => {
it('should complete full cleanup workflow', async () => {
// 完整流程集成测试
})
})
```
2. **错误场景测试不足**
```typescript
// 建议添加
it('should handle page navigation failure', async () => {
// Mock 导航失败场景
})
```
3. **干运行模式测试可以更详细**
```typescript
it('should not delete materials in dry-run mode', async () => {
// 验证 dryRun=true 时不执行实际删除
})
```
---
### 2.3 `ErpBrowserManager.test.ts` - **A+ (95/100)**
**测试对象**: `ErpBrowserManager` - 浏览器管理器
#### 优点 ✅
1. **状态管理测试完备**
```typescript
it('should return existing browser if running', async () => {
const firstBrowser = await manager.launch()
const secondBrowser = await manager.launch()
expect(firstBrowser).toBe(secondBrowser)
expect(chromium.launch).toHaveBeenCalledTimes(1)
})
```
2. **参数化测试**
```typescript
it.each([true, false])('should launch with headless=%s', async (headless) => {
const manager = new ErpBrowserManager({ headless })
await manager.launch()
expect(chromium.launch).toHaveBeenCalledWith(expect.objectContaining({ headless }))
})
```
3. **错误恢复测试**
```typescript
it('should close browser even if context.close fails', async () => {
mockContext.close.mockRejectedValue(new Error('Context close error'))
await manager.close()
expect(mockBrowser.close).toHaveBeenCalled()
})
```
4. **生命周期覆盖全面**
- 启动 → 初始化 → 导航 → 创建上下文 → 关闭
- 所有公开方法都有测试
#### 改进建议 🔧
1. **缺少超时测试**
```typescript
it('should timeout on slow page navigation', async () => {
mockPage.goto.mockImplementation(() => new Promise((resolve) => setTimeout(resolve, 60000)))
await expect(manager.navigate('http://slow.com')).rejects.toThrow('timeout')
})
```
2. **可以添加内存泄漏检测**
```typescript
it('should release all resources after close', async () => {
await manager.launch()
await manager.close()
// 验证没有悬空引用
})
```
---
### 2.4 `extractor-core.test.ts` - **B+ (85/100)**
**测试对象**: `ExtractorCore` - 提取核心逻辑
#### 优点 ✅
1. **私有方法测试策略合理**
```typescript
// @ts-ignore - accessing private method for testing
await extractorCore.waitForLoading(mockWorkFrame)
```
- 使用 `@ts-ignore` 测试私有方法是可接受的
- 避免了为了测试而暴露内部实现
2. **进度回调测试精确**
```typescript
it('should calculate progress correctly', async () => {
await extractorCore.downloadAllBatches(input)
expect(progressCallback).toHaveBeenNthCalledWith(1, '处理批次 1/2', 40, {...})
expect(progressCallback).toHaveBeenNthCalledWith(2, '处理批次 2/2', 60, {...})
})
```
3. **错误处理验证**
```typescript
it('should handle errors in batch download gracefully', async () => {
vi.spyOn(extractorCore as any, 'downloadBatch')
.mockResolvedValueOnce('/path/file1.xlsx')
.mockRejectedValueOnce(new Error('Network error'))
const result = await extractorCore.downloadAllBatches(input)
expect(result.errors).toHaveLength(1)
})
```
#### 不足 ⚠️
1. **3 个测试标记为 TODO**
```typescript
it.todo('TODO: needs integration test setup - should handle complete navigation flow')
it.todo('TODO: needs integration test setup - should handle download events correctly')
it.todo('TODO: needs integration test setup - should verify locator interactions')
```
- **影响**: 核心功能缺少完整流程测试
- **建议**: 优先级 P0尽快补充集成测试
2. **Mock 过于复杂**
- `navigateToExtractorPage` 和 `downloadBatch` 都被 Mock
- 实际只测试了流程编排,未测试真实逻辑
#### 改进建议 🔧
**高优先级**:
```typescript
// 集成测试示例
describe('ExtractorCore - Integration', () => {
it('should handle real iframe navigation', async () => {
// 使用真实 Playwright 浏览器
// 测试完整的 iframe 查找和内容帧获取
})
})
```
---
### 2.5 `extractor.test.ts` - **A (90/100)**
**测试对象**: `ExtractorService` - 提取服务
#### 优点 ✅
1. **依赖注入测试**
```typescript
beforeEach(() => {
mockExcelParserInstance = { parse: vi.fn().mockResolvedValue(undefined) }
mockDataImportInstance = { importFromExcel: vi.fn().mockResolvedValue({...}) }
mockExtractorCoreInstance = { downloadAllBatches: vi.fn().mockResolvedValue({...}) }
})
```
2. **私有方法测试合理**
```typescript
// @ts-ignore - accessing private method for testing
const result = await service.mergeFiles(['./file1.xlsx'], ['ORD001'])
```
3. **错误传播测试**
```typescript
it('should handle extraction errors gracefully', async () => {
mockExtractorCoreInstance.downloadAllBatches.mockRejectedValue(new Error('Network error'))
const result = await service.extract({ orderNumbers: ['ORD001'] })
expect(Array.isArray(result.errors)).toBe(true)
})
```
4. **性能监控集成测试**
```typescript
it('should wrap import in trackDuration', async () => {
await service.importToDatabaseWithLogging('./merged.xlsx', onLog)
expect(trackDuration).toHaveBeenCalledWith(
expect.any(Function),
expect.objectContaining({ operationName: 'Database Import' })
)
})
```
#### 改进建议 🔧
1. **缺少 `extract()` 主方法完整流程测试**
- 只有基础行为测试
- 建议添加完整 E2E 流程
2. **Mock 重置策略可以更清晰**
```typescript
// 建议在每个测试前明确重置所有 Mock
beforeEach(() => {
vi.clearAllMocks()
mockExcelParserInstance.lastOrders = [] // 显式清空
})
```
---
### 2.6 `order-resolver.test.ts` - **A+ (95/100)**
**测试对象**: `OrderNumberResolver` - 订单号解析器
#### 优点 ✅
1. **测试覆盖率最高**
- 26 个测试用例,覆盖所有公开方法
- 包含性能测试
2. **类型识别测试完备**
```typescript
describe('isProductionId()', () => {
it('should recognize valid production IDs', () => {
expect(resolver.isProductionId('22A1')).toBe(true)
expect(resolver.isProductionId('26B10617')).toBe(true)
})
it('should reject invalid formats', () => {
expect(resolver.isProductionId('SC70202602120085')).toBe(false)
expect(resolver.isProductionId('abc')).toBe(false)
})
})
```
3. **去重逻辑测试**
```typescript
it('deduplicates identical inputs', async () => {
const results = await resolver.resolve(['22A1', '22A1', '22A1'])
expect(results).toHaveLength(1) // deduplicated
})
```
4. **性能测试**
```typescript
it('performance with large order sets', async () => {
const largeInput = Array.from({ length: 100 }, (_, i) => `22A${i}`)
const startTime = Date.now()
const results = await resolver.resolve(largeInput)
const elapsed = Date.now() - startTime
expect(elapsed).toBeLessThan(5000)
})
```
5. **统计和报告测试**
- `getStats()`: 统计数据准确性
- `getWarnings()`: 警告消息格式化
- `getDeduplicationReport()`: 去重报告生成
#### 改进建议 🔧
1. **可以添加数据库连接失败的重试测试**
```typescript
it('should retry on transient database errors', async () => {
// Mock 第一次失败,第二次成功
// 验证重试逻辑
})
```
2. **缓存策略测试可以更详细**
```typescript
it('should cache resolved mappings', async () => {
// 验证相同输入不会重复查询数据库
})
```
---
## 3. 共性问题与建议
### 3.1 Mock 策略优化
**当前做法**:
```typescript
vi.mock('playwright', () => ({
chromium: { launch: vi.fn() }
}))
```
**建议改进**:
```typescript
// 使用工厂函数创建可重置的 Mock
const createMockPlaywright = () => ({
chromium: {
launch: vi.fn().mockResolvedValue(createMockBrowser()),
connect: vi.fn()
}
})
beforeEach(() => {
vi.mocked(chromium.launch).mockResolvedValue(createMockBrowser())
})
```
**好处**:
- 每个测试独立的 Mock 状态
- 避免测试间的相互影响
- 更易维护
### 3.2 测试数据工厂
**当前**: 手动创建测试数据
```typescript
const config = {
url: 'https://test-erp.com',
username: 'testuser',
password: 'testpass',
headless: true
}
```
**建议**: 使用工厂函数
```typescript
// tests/fixtures/factory.ts
const ErpConfigFactory = {
create: (overrides?: Partial<ErpConfig>) => ({
url: 'https://test-erp.com',
username: 'testuser',
password: 'testpass',
headless: true,
...overrides
})
}
// 测试中
const config = ErpConfigFactory.create({ headless: false })
```
### 3.3 错误消息断言
**当前**:
```typescript
await expect(service.login()).rejects.toThrow('Failed to access')
```
**建议**: 使用更精确的匹配
```typescript
await expect(service.login()).rejects.toThrow(
expect.objectContaining({
message: expect.stringContaining('Failed to access forwardFrame')
})
)
```
### 3.4 集成测试缺失
**问题**: 多个文件有 TODO 注释说明需要集成测试
**建议优先级**:
1. **P0**: `extractor-core.test.ts` - 3 个 TODO
2. **P1**: `extractor.test.ts` - `extract()` 完整流程
3. **P1**: `cleaner.test.ts` - `clean()` 完整流程
**集成测试框架建议**:
```typescript
// tests/integration/erp/extractor.integration.test.ts
import { test, expect } from '@playwright/test'
test('complete extraction workflow', async () => {
// 使用真实浏览器
// 测试完整提取流程
})
```
---
## 4. 测试设计模式评估
### 4.1 AAA 模式 (Arrange-Act-Assert)
**评分**: **优秀** ✅
所有测试都遵循 AAA 模式:
```typescript
it('should create session on successful login', async () => {
// Arrange
service = new ErpAuthService(config)
// Act
const session = await service.login()
// Assert
expect(chromium.launch).toHaveBeenCalledWith(...)
expect(session.isLoggedIn).toBe(true)
})
```
### 4.2 测试独立性
**评分**: **良好** ⚠️
**优点**:
- 每个测试使用 `beforeEach` 重置状态
- `vi.clearAllMocks()` 调用普遍
**改进点**:
- 部分测试依赖前一个测试的 Mock 状态
- 建议在每个测试中完全独立设置 Mock
### 4.3 测试可读性
**评分**: **优秀** ✅
- 测试命名清晰:`should/could` 语义
- 分组合理:`describe` 层次分明
- 注释充分:关键步骤有说明
### 4.4 测试可维护性
**评分**: **良好** ⚠️
**优点**:
- 代码结构清晰
- 重复代码较少
**改进点**:
- 缺少测试数据工厂
- Mock 设置代码重复
- 魔法数字(如 `40`, `60` 进度值)缺少常量定义
---
## 5. 覆盖率分析
### 5.1 方法覆盖率
| 服务 | 公开方法 | 已测试 | 覆盖率 |
| --------------------- | -------- | ------ | ------ |
| `ErpAuthService` | 5 | 5 | 100% |
| `CleanerService` | 7 | 4 | 57% ⚠️ |
| `ErpBrowserManager` | 9 | 9 | 100% |
| `ExtractorCore` | 3 | 2 | 67% ⚠️ |
| `ExtractorService` | 5 | 4 | 80% |
| `OrderNumberResolver` | 10 | 10 | 100% |
### 5.2 分支覆盖率估算
| 服务 | 条件分支 | 已覆盖 | 估算覆盖率 |
| --------------------- | -------- | ------ | ---------- |
| `ErpAuthService` | 8 | 7 | 87% |
| `CleanerService` | 15 | 12 | 80% |
| `ErpBrowserManager` | 10 | 9 | 90% |
| `ExtractorCore` | 12 | 8 | 67% |
| `ExtractorService` | 14 | 11 | 78% |
| `OrderNumberResolver` | 20 | 18 | 90% |
### 5.3 未覆盖的关键路径
1. **CleanerService**
- `clean()` 主方法的完整流程
- 重试机制 (`retryFailedOrders`)
- 进度发布 (`publishProgress`)
2. **ExtractorCore**
- `navigateToExtractorPage()` 完整导航逻辑
- `downloadBatch()` 实际下载流程
- iframe 交互的真实场景
3. **ExtractorService**
- `extract()` 方法的完整编排流程
- 并发控制在实际场景中的表现
---
## 6. 性能测试评估
### 6.1 现有性能测试
**优秀示例**:
```typescript
it('performance with large order sets', async () => {
const largeInput = Array.from({ length: 100 }, (_, i) => `22A${i}`)
const startTime = Date.now()
const results = await resolver.resolve(largeInput)
const elapsed = Date.now() - startTime
expect(elapsed).toBeLessThan(5000)
})
```
### 6.2 缺失的性能测试
1. **并发性能**
```typescript
it('should handle 1000 concurrent orders', async () => {
const orders = Array.from({ length: 1000 }, (_, i) => `ORD${i}`)
const start = Date.now()
await resolver.resolve(orders)
expect(Date.now() - start).toBeLessThan(10000)
})
```
2. **内存使用**
```typescript
it('should not leak memory on repeated calls', async () => {
const initialMemory = process.memoryUsage().heapUsed
for (let i = 0; i < 100; i++) {
await service.extract({ orderNumbers: ['ORD001'] })
}
const finalMemory = process.memoryUsage().heapUsed
expect(finalMemory - initialMemory).toBeLessThan(10 * 1024 * 1024) // < 10MB
})
```
---
## 7. 错误处理测试评估
### 7.1 优秀实践 ✅
1. **网络错误处理**
```typescript
mockExtractorCoreInstance.downloadAllBatches.mockRejectedValue(new Error('Network error'))
```
2. **数据库连接失败**
```typescript
vi.mocked(mockDbService.query).mockRejectedValue(new Error('Database connection failed'))
```
3. **元素未找到**
```typescript
mockPage.locator = vi.fn().mockReturnValue({
contentFrame: vi.fn().mockResolvedValue(null)
})
await expect(service.login()).rejects.toThrow('Failed to access')
```
### 7.2 改进建议 🔧
1. **添加错误类型验证**
```typescript
it('should throw specific error types', async () => {
await expect(service.login()).rejects.toThrow(ErpAuthenticationError)
})
```
2. **错误上下文验证**
```typescript
it('should include context in error messages', async () => {
try {
await service.login()
} catch (error) {
expect(error.context).toEqual({
url: 'https://test-erp.com',
step: 'login'
})
}
})
```
---
## 8. 与测试覆盖率提升计划对标
### 8.1 计划目标回顾
根据 `TEST_COVERAGE_IMPROVEMENT_PLAN.md`:
| 模块 | 当前覆盖率 | 目标覆盖率 | 优先级 |
| ---------------------- | ---------- | ---------- | ------ |
| `erp-auth.ts` | < 20% | 80% | P0 |
| `extractor.ts` | ~30% | 80% | P0 |
| `extractor-core.ts` | < 10% | 80% | P0 |
| `cleaner.ts` | ~25% | 80% | P0 |
| `ErpBrowserManager.ts` | N/A | 80% | P1 |
| `order-resolver.ts` | N/A | 80% | P1 |
### 8.2 当前进展
**估算覆盖率提升**:
| 模块 | 测试前 | 测试后(估算) | 提升 | 达标状态 |
| ---------------------- | ------ | -------------- | ---- | ------------------- |
| `erp-auth.ts` | < 20% | ~75% | +55% | ⚠️ 接近达标 |
| `extractor.ts` | ~30% | ~70% | +40% | ⚠️ 接近达标 |
| `extractor-core.ts` | < 10% | ~55% | +45% | ❌ 需补充集成测试 |
| `cleaner.ts` | ~25% | ~65% | +40% | ⚠️ 需补充主方法测试 |
| `ErpBrowserManager.ts` | N/A | ~85% | N/A | ✅ 已达标 |
| `order-resolver.ts` | N/A | ~90% | N/A | ✅ 已达标 |
### 8.3 下一步行动
**P0 - 立即执行**:
1. 补充 `extractor-core.test.ts` 的 3 个 TODO 测试
2. 添加 `cleaner.ts` 的 `clean()` 方法集成测试
3. 补充 `extractor.ts` 的 `extract()` 完整流程测试
**P1 - 本周执行**:
1. 为所有错误路径添加断言
2. 添加性能测试覆盖关键路径
3. 创建测试数据工厂减少重复代码
---
## 9. 总体评价与建议
### 9.1 优点总结
1. **测试设计优秀**
- AAA 模式遵循良好
- 测试命名清晰
- 分组合理
2. **Mock 策略成熟**
- 外部依赖完全隔离
- Mock 对象结构清晰
- 参数化测试使用得当
3. **错误处理充分**
- 主要错误场景都有覆盖
- 异常传播验证到位
4. **边界条件重视**
- 边界值测试普遍
- 特殊情况考虑周全
### 9.2 改进优先级
**P0 - 必须完成(本周)**:
1. ✅ 补充 `extractor-core.test.ts` 的集成测试
2. ✅ 添加 `cleaner()` 主方法测试
3. ✅ 完成 `extractor.extract()` 完整流程测试
**P1 - 强烈建议(下周)**:
1. 创建测试数据工厂
2. 统一 Mock 设置模式
3. 添加性能基准测试
**P2 - 建议(本月)**:
1. 添加内存泄漏检测测试
2. 补充错误类型验证
3. 完善并发场景测试
### 9.3 测试文化建议
1. **测试审查流程**
- 将测试审查纳入 PR 必选项
- 使用本报告的评分标准
2. **测试文档**
- 编写《测试最佳实践》文档
- 建立测试模式库
3. **覆盖率门禁**
- CI/CD 中设置覆盖率阈值
- 新增代码覆盖率要求 ≥ 80%
---
## 10. 结论
本次审查的测试文件整体质量**优秀**,展现了团队对测试工作的重视和高超的测试设计能力。主要优势在于:
- ✅ 测试设计模式成熟AAA 模式)
- ✅ Mock 策略合理,依赖隔离充分
- ✅ 错误处理和边界条件覆盖全面
- ✅ 测试可读性和可维护性良好
需要改进的方面:
- ⚠️ 集成测试缺失3 个 TODO 待实现)
- ⚠️ 部分主方法测试不完整
- ⚠️ 缺少性能基准测试
- ⚠️ 测试数据工厂可进一步优化
**总体评分A (90/100)**
按照本报告的改进建议执行后,预计可将 ERP 服务模块的测试覆盖率提升至 **75-85%**,达到项目设定的阶段性目标。
---
**附录 A: 测试运行统计**
```
Test Files: 8 passed (8)
Tests: 114 passed | 3 todo (117)
Duration: ~1.0s
Setup: ~259ms
Transform: ~708ms
```
**附录 B: 审查工具**
- Vitest 测试运行器
- Playwright Mock 库
- TypeScript 类型检查
- ESLint 代码规范检查
---
**报告结束**

View File

@@ -180,7 +180,7 @@ validation:
# 订单号解析配置
orderResolution:
tableName: 'productionContractData_26 年压力表合同数据'
tableName: 'ERPAuto.vw_productionContractData'
productionIdField: '总排号'
orderNumberField: '生产订单号'
```

165
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{
"name": "erpauto",
"version": "1.9.0",
"version": "1.15.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "erpauto",
"version": "1.9.0",
"version": "1.15.0",
"hasInstallScript": true,
"dependencies": {
"@aws-sdk/client-s3": "^3.929.0",
@@ -24,6 +24,7 @@
"lucide-react": "^0.575.0",
"mssql": "^12.2.0",
"mysql2": "^3.18.2",
"pg": "^8.20.0",
"playwright": "^1.58.2",
"playwright-core": "^1.58.2",
"react-focus-lock": "^2.13.7",
@@ -48,6 +49,7 @@
"@playwright/test": "^1.58.2",
"@types/mssql": "^9.1.9",
"@types/node": "^22.19.13",
"@types/pg": "^8.20.0",
"@types/react": "^19.2.7",
"@types/react-dom": "^19.2.3",
"@types/uuid": "^10.0.0",
@@ -5020,6 +5022,18 @@
"undici-types": "~6.21.0"
}
},
"node_modules/@types/pg": {
"version": "8.20.0",
"resolved": "https://registry.npmjs.org/@types/pg/-/pg-8.20.0.tgz",
"integrity": "sha512-bEPFOaMAHTEP1EzpvHTbmwR8UsFyHSKsRisLIHVMXnpNefSbGA1bD6CVy+qKjGSqmZqNqBDV2azOBo8TgkcVow==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/node": "*",
"pg-protocol": "*",
"pg-types": "^2.2.0"
}
},
"node_modules/@types/plist": {
"version": "3.0.5",
"resolved": "https://registry.npmjs.org/@types/plist/-/plist-3.0.5.tgz",
@@ -13532,6 +13546,96 @@
"integrity": "sha512-F3asv42UuXchdzt+xXqfW1OGlVBe+mxa2mqI0pg5yAHZPvFmY3Y6drSf/GQ1A86WgWEN9Kzh/WrgKa6iGcHXLg==",
"license": "MIT"
},
"node_modules/pg": {
"version": "8.20.0",
"resolved": "https://registry.npmjs.org/pg/-/pg-8.20.0.tgz",
"integrity": "sha512-ldhMxz2r8fl/6QkXnBD3CR9/xg694oT6DZQ2s6c/RI28OjtSOpxnPrUCGOBJ46RCUxcWdx3p6kw/xnDHjKvaRA==",
"license": "MIT",
"peer": true,
"dependencies": {
"pg-connection-string": "^2.12.0",
"pg-pool": "^3.13.0",
"pg-protocol": "^1.13.0",
"pg-types": "2.2.0",
"pgpass": "1.0.5"
},
"engines": {
"node": ">= 16.0.0"
},
"optionalDependencies": {
"pg-cloudflare": "^1.3.0"
},
"peerDependencies": {
"pg-native": ">=3.0.1"
},
"peerDependenciesMeta": {
"pg-native": {
"optional": true
}
}
},
"node_modules/pg-cloudflare": {
"version": "1.3.0",
"resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.3.0.tgz",
"integrity": "sha512-6lswVVSztmHiRtD6I8hw4qP/nDm1EJbKMRhf3HCYaqud7frGysPv7FYJ5noZQdhQtN2xJnimfMtvQq21pdbzyQ==",
"license": "MIT",
"optional": true
},
"node_modules/pg-connection-string": {
"version": "2.12.0",
"resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.12.0.tgz",
"integrity": "sha512-U7qg+bpswf3Cs5xLzRqbXbQl85ng0mfSV/J0nnA31MCLgvEaAo7CIhmeyrmJpOr7o+zm0rXK+hNnT5l9RHkCkQ==",
"license": "MIT"
},
"node_modules/pg-int8": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz",
"integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==",
"license": "ISC",
"engines": {
"node": ">=4.0.0"
}
},
"node_modules/pg-pool": {
"version": "3.13.0",
"resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.13.0.tgz",
"integrity": "sha512-gB+R+Xud1gLFuRD/QgOIgGOBE2KCQPaPwkzBBGC9oG69pHTkhQeIuejVIk3/cnDyX39av2AxomQiyPT13WKHQA==",
"license": "MIT",
"peerDependencies": {
"pg": ">=8.0"
}
},
"node_modules/pg-protocol": {
"version": "1.13.0",
"resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.13.0.tgz",
"integrity": "sha512-zzdvXfS6v89r6v7OcFCHfHlyG/wvry1ALxZo4LqgUoy7W9xhBDMaqOuMiF3qEV45VqsN6rdlcehHrfDtlCPc8w==",
"license": "MIT"
},
"node_modules/pg-types": {
"version": "2.2.0",
"resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz",
"integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==",
"license": "MIT",
"dependencies": {
"pg-int8": "1.0.1",
"postgres-array": "~2.0.0",
"postgres-bytea": "~1.0.0",
"postgres-date": "~1.0.4",
"postgres-interval": "^1.1.0"
},
"engines": {
"node": ">=4"
}
},
"node_modules/pgpass": {
"version": "1.0.5",
"resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz",
"integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==",
"license": "MIT",
"dependencies": {
"split2": "^4.1.0"
}
},
"node_modules/picocolors": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz",
@@ -13641,6 +13745,45 @@
"dev": true,
"license": "MIT"
},
"node_modules/postgres-array": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz",
"integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==",
"license": "MIT",
"engines": {
"node": ">=4"
}
},
"node_modules/postgres-bytea": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz",
"integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==",
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/postgres-date": {
"version": "1.0.7",
"resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz",
"integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==",
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/postgres-interval": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz",
"integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==",
"license": "MIT",
"dependencies": {
"xtend": "^4.0.0"
},
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/postject": {
"version": "1.0.0-alpha.6",
"resolved": "https://registry.npmjs.org/postject/-/postject-1.0.0-alpha.6.tgz",
@@ -14912,6 +15055,15 @@
"url": "https://github.com/sponsors/wooorm"
}
},
"node_modules/split2": {
"version": "4.2.0",
"resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",
"integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",
"license": "ISC",
"engines": {
"node": ">= 10.x"
}
},
"node_modules/sprintf-js": {
"version": "1.1.3",
"resolved": "https://registry.npmjs.org/sprintf-js/-/sprintf-js-1.1.3.tgz",
@@ -17836,6 +17988,15 @@
"integrity": "sha512-JZnDKK8B0RCDw84FNdDAIpZK+JuJw+s7Lz8nksI7SIuU3UXJJslUthsi+uWBUYOwPFwW7W7PRLRfUKpxjtjFCw==",
"license": "MIT"
},
"node_modules/xtend": {
"version": "4.0.2",
"resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
"integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==",
"license": "MIT",
"engines": {
"node": ">=0.4"
}
},
"node_modules/y18n": {
"version": "5.0.8",
"resolved": "https://registry.npmjs.org/y18n/-/y18n-5.0.8.tgz",

View File

@@ -1,6 +1,6 @@
{
"name": "erpauto",
"version": "1.9.0",
"version": "1.15.0",
"description": "An Electron application with React and TypeScript",
"main": "./out/main/index.js",
"author": "example.com",
@@ -48,6 +48,7 @@
"lucide-react": "^0.575.0",
"mssql": "^12.2.0",
"mysql2": "^3.18.2",
"pg": "^8.20.0",
"playwright": "^1.58.2",
"playwright-core": "^1.58.2",
"react-focus-lock": "^2.13.7",
@@ -72,6 +73,7 @@
"@playwright/test": "^1.58.2",
"@types/mssql": "^9.1.9",
"@types/node": "^22.19.13",
"@types/pg": "^8.20.0",
"@types/react": "^19.2.7",
"@types/react-dom": "^19.2.3",
"@types/uuid": "^10.0.0",

View File

@@ -1,23 +0,0 @@
import { defineConfig } from '@playwright/test'
export default defineConfig({
testDir: './tests/e2e',
timeout: 120000,
fullyParallel: false,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
workers: 1,
reporter: 'html',
use: {
trace: 'on-first-retry',
screenshot: 'only-on-failure'
},
// Test configuration for Electron
projects: [
{
name: 'electron',
testMatch: '**/*.test.ts'
}
]
})

View File

@@ -1,34 +1,42 @@
import { app } from 'electron'
import logger from '../services/logger/index'
import { logAudit, closeAuditLogger } from '../services/logger/audit-logger'
import { logAudit, closeAuditLogger, cachedHostname } from '../services/logger/audit-logger'
import { AuditAction, AuditStatus } from '../types/audit.types'
import { serializeError } from '../services/logger/error-utils'
export function setupProcessGuards(): void {
process.on('uncaughtException', (err) => {
logger.error('Uncaught exception', { error: err })
logAudit('SYSTEM_CRASH', 'system', {
username: 'system',
computerName: process.env.COMPUTERNAME || 'unknown',
resource: 'main-process',
status: 'failure',
metadata: { error: err.message, stack: err.stack }
})
setTimeout(() => process.exit(1), 1000)
try {
logAudit(AuditAction.SYSTEM_CRASH, 'system', {
username: 'system',
computerName: cachedHostname,
resource: 'main-process',
status: AuditStatus.FAILURE,
metadata: { error: err.message, stack: err.stack }
})
} catch (auditError) {
logger.error('Failed to write crash audit log', { error: auditError })
} finally {
setTimeout(() => process.exit(1), 1000)
}
})
process.on('unhandledRejection', (reason) => {
const errorMeta =
reason instanceof Error
? { error: serializeError(reason) }
: { reason: String(reason) }
reason instanceof Error ? { error: serializeError(reason) } : { reason: String(reason) }
logger.error('Unhandled Rejection', errorMeta)
logAudit('SYSTEM_ERROR', 'system', {
username: 'system',
computerName: process.env.COMPUTERNAME || 'unknown',
resource: 'main-process',
status: 'failure',
metadata: errorMeta
})
try {
logAudit(AuditAction.SYSTEM_ERROR, 'system', {
username: 'system',
computerName: cachedHostname,
resource: 'main-process',
status: AuditStatus.FAILURE,
metadata: errorMeta
})
} catch (auditError) {
logger.error('Failed to write unhandled rejection audit log', { error: auditError })
}
})
app.on('render-process-gone', (_, webContents, details) => {

View File

@@ -1,3 +1,5 @@
import { randomUUID } from 'crypto'
import { app } from 'electron'
import { ipcMain } from 'electron'
import { withErrorHandling, type IpcResult } from './index'
import type {
@@ -8,6 +10,7 @@ import type {
} from '../types/cleaner.types'
import { IPC_CHANNELS } from '../../shared/ipc-channels'
import { CleanerApplicationService } from '../services/cleaner/cleaner-application-service'
import { CleanerOperationHistoryDAO } from '../services/database/cleaner-operation-history-dao'
export function registerCleanerHandlers(): void {
const cleanerService = new CleanerApplicationService()
@@ -15,10 +18,21 @@ export function registerCleanerHandlers(): void {
ipcMain.handle(
IPC_CHANNELS.CLEANER_RUN,
async (event, input: CleanerInput): Promise<IpcResult<CleanerResult>> => {
return withErrorHandling(
async () => cleanerService.runCleaner(event.sender, input),
'cleaner:run'
)
return withErrorHandling(async () => {
const batchId = randomUUID()
const historyDao = new CleanerOperationHistoryDAO()
const appVersion = app.getVersion()
const result = await cleanerService.runCleaner(
event.sender,
input,
batchId,
historyDao,
appVersion
)
return result
}, 'cleaner:run')
}
)

View File

@@ -0,0 +1,195 @@
/**
* IPC Handler for Cleaner Operation History
*
* Handles IPC requests for cleaner operation history management:
* - Get batch list (filtered by user for non-admin users)
* - Get batch details (executions + orders)
* - Get material details for a specific order
* - Delete batches
*/
import { ipcMain } from 'electron'
import { CleanerOperationHistoryDAO } from '../services/database/cleaner-operation-history-dao'
import { SessionManager } from '../services/user/session-manager'
import { withErrorHandling, type IpcResult } from './index'
import { IPC_CHANNELS } from '../../shared/ipc-channels'
import { createLogger } from '../services/logger'
import type {
CleanerBatchStats,
CleanerExecutionRecord,
CleanerOrderRecord,
CleanerMaterialRecord,
GetCleanerBatchesOptions,
SearchCleanerHistoryOptions,
CleanerHistorySearchResult
} from '../types/cleaner-history.types'
const log = createLogger('CleanerHistoryHandler')
/**
* Register IPC handlers for cleaner operation history
*/
export function registerCleanerHistoryHandlers(): void {
const dao = new CleanerOperationHistoryDAO()
/**
* Get batches list
* Admin users get all batches, regular users get only their own
*/
ipcMain.handle(
IPC_CHANNELS.CLEANER_HISTORY_GET_BATCHES,
async (_event, options?: GetCleanerBatchesOptions): Promise<IpcResult<CleanerBatchStats[]>> => {
return withErrorHandling(async () => {
const currentUser = SessionManager.getInstance().getUserInfo()
if (!currentUser) {
throw new Error('用户未登录')
}
// Admin gets all batches, User gets only their own
const userId = currentUser.userType === 'Admin' ? undefined : currentUser.id
log.info('Getting cleaner history batches', {
userId: currentUser.id,
userType: currentUser.userType,
filtered: userId !== undefined
})
return await dao.getBatches(userId, options)
}, 'cleanerHistory:getBatches')
}
)
/**
* Get batch details (executions + orders)
* Users can only view their own batch details, admins can view all
*/
ipcMain.handle(
IPC_CHANNELS.CLEANER_HISTORY_GET_BATCH_DETAILS,
async (
_event,
batchId: string
): Promise<
IpcResult<{ executions: CleanerExecutionRecord[]; orders: CleanerOrderRecord[] }>
> => {
return withErrorHandling(async () => {
const currentUser = SessionManager.getInstance().getUserInfo()
if (!currentUser) {
throw new Error('用户未登录')
}
log.info('Getting cleaner batch details', { batchId, userId: currentUser.id })
const details = await dao.getBatchDetails(batchId)
// For non-admin users, verify they own this batch
if (currentUser.userType !== 'Admin' && details.executions.length > 0) {
const batchOwnerId = details.executions[0].userId
if (batchOwnerId !== currentUser.id) {
throw new Error('没有权限查看此批次详情')
}
}
return details
}, 'cleanerHistory:getBatchDetails')
}
)
/**
* Get material details for a specific order
*/
ipcMain.handle(
IPC_CHANNELS.CLEANER_HISTORY_GET_MATERIAL_DETAILS,
async (
_event,
batchId: string,
attemptNumber: number,
orderNumber: string
): Promise<IpcResult<CleanerMaterialRecord[]>> => {
return withErrorHandling(async () => {
const currentUser = SessionManager.getInstance().getUserInfo()
if (!currentUser) {
throw new Error('用户未登录')
}
log.info('Getting cleaner material details', { batchId, attemptNumber, orderNumber })
return await dao.getMaterialDetails(batchId, attemptNumber, orderNumber)
}, 'cleanerHistory:getMaterialDetails')
}
)
/**
* Delete a batch
* Users can only delete their own batches, admins can delete any
*/
ipcMain.handle(
IPC_CHANNELS.CLEANER_HISTORY_DELETE_BATCH,
async (_event, batchId: string): Promise<IpcResult<{ deleted: boolean }>> => {
return withErrorHandling(async () => {
const currentUser = SessionManager.getInstance().getUserInfo()
if (!currentUser) {
throw new Error('用户未登录')
}
const isAdmin = currentUser.userType === 'Admin'
log.info('Deleting cleaner batch', {
batchId,
userId: currentUser.id,
isAdmin
})
const result = await dao.deleteBatch(batchId, currentUser.id, isAdmin)
if (!result.success) {
throw new Error(result.error || '删除批次失败')
}
return { deleted: true }
}, 'cleanerHistory:deleteBatch')
}
)
/**
* Search across all history levels (batches, orders, materials)
* Admin users search all records, regular users search only their own
*/
ipcMain.handle(
IPC_CHANNELS.CLEANER_HISTORY_SEARCH,
async (
_event,
options: SearchCleanerHistoryOptions
): Promise<IpcResult<CleanerHistorySearchResult>> => {
return withErrorHandling(async () => {
const currentUser = SessionManager.getInstance().getUserInfo()
if (!currentUser) {
throw new Error('用户未登录')
}
if (!options.query || options.query.trim().length === 0) {
return { batches: [], totalMatches: 0 }
}
const userId = currentUser.userType === 'Admin' ? undefined : currentUser.id
log.info('Searching cleaner history', {
userId: currentUser.id,
userType: currentUser.userType,
query: options.query
})
return await dao.searchBatches(userId, {
...options,
query: options.query.trim()
})
}, 'cleanerHistory:search')
}
)
log.info('Cleaner history IPC handlers registered')
}

View File

@@ -5,7 +5,8 @@ import { OrderNumberResolver } from '../services/erp/order-resolver'
import { create, type IDatabaseService } from '../services/database'
import { ExtractorOperationHistoryDAO } from '../services/database/extractor-operation-history-dao'
import { createLogger } from '../services/logger'
import { logAudit } from '../services/logger/audit-logger'
import { logAuditWithCurrentUser } from '../services/logger/audit-logger'
import { AuditAction, AuditStatus } from '../types/audit.types'
import { SessionManager } from '../services/user/session-manager'
import { withErrorHandling, type IpcResult } from './index'
import { ErpConnectionError, ValidationError, DatabaseQueryError } from '../types/errors'
@@ -125,7 +126,9 @@ export function registerExtractorHandlers(): void {
sendLog(sender, 'info', '正在解析订单号...')
const resolver = new OrderNumberResolver(dbService)
const resolutionStart = Date.now()
const mappings = await resolver.resolve(input.orderNumbers)
const resolutionDurationMs = Date.now() - resolutionStart
// Get valid order numbers and warnings
const validOrderNumbers = resolver.getValidOrderNumbers(mappings)
@@ -145,7 +148,16 @@ export function registerExtractorHandlers(): void {
)
}
log.info('Resolved order numbers', { count: validOrderNumbers.length })
log.info('Resolved order numbers', {
inputCount: input.orderNumbers.length,
count: validOrderNumbers.length,
durationMs: resolutionDurationMs
})
sendLog(
sender,
'info',
`订单号解析完成:${validOrderNumbers.length}/${input.orderNumbers.length} 个有效,耗时 ${(resolutionDurationMs / 1000).toFixed(2)}`
)
// Initialize operation history recording
const currentUser = SessionManager.getInstance().getUserInfo()
@@ -154,6 +166,7 @@ export function registerExtractorHandlers(): void {
// Save order records to history (preserve productionId -> orderNumber mapping)
if (currentUser) {
const historyInsertStart = Date.now()
const orderRecords = mappings.map((m) => ({
productionId: m.productionId || null,
orderNumber: m.orderNumber || m.input
@@ -164,9 +177,11 @@ export function registerExtractorHandlers(): void {
currentUser.username,
orderRecords
)
const historyInsertDurationMs = Date.now() - historyInsertStart
log.info('Operation history batch created', {
batchId,
recordCount: orderRecords.length
recordCount: orderRecords.length,
durationMs: historyInsertDurationMs
})
}
@@ -278,24 +293,17 @@ export function registerExtractorHandlers(): void {
}
// Audit log: EXTRACT (non-blocking)
const os = await import('os')
if (currentUser) {
const auditStatus: 'success' | 'failure' | 'partial' =
const auditStatus: AuditStatus =
result.errors.length > 0 && result.recordCount > 0
? 'partial'
? AuditStatus.PARTIAL
: result.errors.length > 0
? 'failure'
: 'success'
logAudit('EXTRACT', String(currentUser.id), {
username: currentUser.username,
computerName: os.hostname(),
resource: 'MATERIAL_PLAN',
status: auditStatus,
metadata: {
orderCount: validOrderNumbers.length,
recordCount: result.recordCount,
errorCount: result.errors.length
}
? AuditStatus.FAILURE
: AuditStatus.SUCCESS
logAuditWithCurrentUser(AuditAction.EXTRACT, 'MATERIAL_PLAN', auditStatus, {
orderCount: validOrderNumbers.length,
recordCount: result.recordCount,
errorCount: result.errors.length
})
}

View File

@@ -18,6 +18,7 @@ import { registerReportHandlers } from './report-handler'
import { registerUpdateHandlers } from './update-handler'
import { registerPlaywrightBrowserHandlers } from './playwright-browser'
import { registerOperationHistoryHandlers } from './operation-history-handler'
import { registerCleanerHistoryHandlers } from './cleaner-history-handler'
import { createLogger, logError } from '../services/logger'
import { serializeError, sanitizeError } from '../services/logger/error-utils'
import { getErrorMessage, getErrorCode, isBaseError } from '../types/errors'
@@ -115,5 +116,6 @@ export function registerIpcHandlers(): void {
registerUpdateHandlers()
registerPlaywrightBrowserHandlers()
registerOperationHistoryHandlers()
registerCleanerHistoryHandlers()
log.info('All IPC handlers registered')
}

View File

@@ -5,7 +5,8 @@ import { UserErpConfigService } from '../services/user/user-erp-config-service'
import { MySqlService } from '../services/database/mysql'
import { SqlServerService } from '../services/database/sql-server'
import { createLogger } from '../services/logger'
import { logAudit } from '../services/logger/audit-logger'
import { logAuditWithCurrentUser } from '../services/logger/audit-logger'
import { AuditAction, AuditStatus } from '../types/audit.types'
import type { UserType, ConnectionTestResult, SaveSettingsResult } from '../types/settings.types'
import { IPC_CHANNELS } from '../../shared/ipc-channels'
import { ValidationError } from '../types/errors'
@@ -67,13 +68,9 @@ export function registerSettingsHandlers(): void {
})
// Audit log: SETTINGS_CHANGE (non-blocking)
const os = await import('os')
logAudit('SETTINGS_CHANGE', String(currentUser.id), {
username: currentUser.username,
computerName: os.hostname(),
resource: 'ERP_CONFIG',
status: 'success',
metadata: { changeType: 'erp_credentials', usernameChanged: !!settings.erp.username }
logAuditWithCurrentUser(AuditAction.SETTINGS_CHANGE, 'ERP_CONFIG', AuditStatus.SUCCESS, {
changeType: 'erp_credentials',
usernameChanged: !!settings.erp.username
})
}

View File

@@ -194,7 +194,8 @@ export function registerValidationHandlers(): void {
ipcMain.handle(
IPC_CHANNELS.VALIDATION_GET_CLEANER_DATA,
async (
event
event,
params?: { selectedManagers?: string[] }
): Promise<{
success: boolean
orderNumbers?: string[]
@@ -213,7 +214,11 @@ export function registerValidationHandlers(): void {
}
}
return validationApplicationService.getCleanerData(userInfo, event.sender.id)
return validationApplicationService.getCleanerData(
userInfo,
event.sender.id,
params?.selectedManagers ?? []
)
}
)
}

View File

@@ -14,7 +14,8 @@ export const CleanerInputSchema = z.object({
materialCodes: z.array(z.string().min(1, 'Material code cannot be empty')),
dryRun: z.boolean(),
queryBatchSize: z.number().int().min(1).max(100).optional().default(100),
processConcurrency: z.number().int().min(1).max(20).optional().default(1)
processConcurrency: z.number().int().min(1).max(20).optional().default(1),
sessionRefreshOrderThreshold: z.number().int().positive().optional().default(160)
// Note: onProgress is a function, not validated via Zod
})

View File

@@ -3,6 +3,7 @@ import { SessionManager } from '../user/session-manager'
import { UpdateService } from '../update/update-service'
import { createLogger, run, getRequestId, getContext } from '../logger'
import { logAudit } from '../logger/audit-logger'
import { AuditAction, AuditStatus } from '../../types/audit.types'
import { ValidationError } from '../../types/errors'
import type { UserInfo } from '../../types/user.types'
import type {
@@ -73,11 +74,11 @@ export class AuthApplicationService {
userId: userInfo.id
})
this.writeAuditLog('LOGIN', String(userInfo.id), {
this.writeAuditLog(AuditAction.LOGIN, String(userInfo.id), {
username: userInfo.username,
computerName: hostname(),
resource: 'ERP_SYSTEM',
status: 'success',
status: AuditStatus.SUCCESS,
metadata: { loginType: 'silent', userType: userInfo.userType }
})
@@ -127,11 +128,11 @@ export class AuthApplicationService {
const userInfo = this.sessionManager.getUserInfo()
if (!success || !userInfo) {
this.writeAuditLog('LOGIN', '0', {
this.writeAuditLog(AuditAction.LOGIN, '0', {
username,
computerName: hostname(),
resource: 'ERP_SYSTEM',
status: 'failure',
status: AuditStatus.FAILURE,
metadata: { loginType: 'credentials', reason: 'invalid_credentials' }
})
@@ -155,11 +156,11 @@ export class AuthApplicationService {
})
await this.updateService.setUserContext(userInfo.userType)
this.writeAuditLog('LOGIN', String(userInfo.id), {
this.writeAuditLog(AuditAction.LOGIN, String(userInfo.id), {
username: userInfo.username,
computerName: hostname(),
resource: 'ERP_SYSTEM',
status: 'success',
status: AuditStatus.SUCCESS,
metadata: { loginType: 'credentials', userType: userInfo.userType }
})
@@ -204,16 +205,17 @@ export class AuthApplicationService {
})
if (userInfo) {
this.writeAuditLog('LOGOUT', String(userInfo.id), {
this.writeAuditLog(AuditAction.LOGOUT, String(userInfo.id), {
username: userInfo.username,
computerName: hostname(),
resource: 'ERP_SYSTEM',
status: 'success',
status: AuditStatus.SUCCESS,
metadata: { userType: userInfo.userType }
})
}
this.sessionManager.logout()
this.silentLoginPromise = null
await this.updateService.setUserContext(null)
},
{ operation: 'logout' }
@@ -310,7 +312,7 @@ export class AuthApplicationService {
}
private writeAuditLog(
action: 'LOGIN' | 'LOGOUT',
action: AuditAction.LOGIN | AuditAction.LOGOUT,
actorId: string,
payload: Parameters<typeof logAudit>[2]
): void {

View File

@@ -1,21 +1,21 @@
import type { WebContents } from 'electron'
import type { MySqlService } from '../database/mysql'
import type { SqlServerService } from '../database/sql-server'
import type { IDatabaseService } from '../../types/database.types'
import { ErpAuthService } from '../erp/erp-auth'
import { CleanerService } from '../erp/cleaner'
import { OrderNumberResolver } from '../erp/order-resolver'
import { MySqlService as MySqlServiceImpl } from '../database/mysql'
import { SqlServerService as SqlServerServiceImpl } from '../database/sql-server'
import { PostgreSqlService as PostgreSqlServiceImpl } from '../database/postgresql'
import { ConfigManager } from '../config/config-manager'
import { ResultExporter } from '../excel/result-exporter'
import { CleanerReportGenerator } from '../report/cleaner-report-generator'
import { RustfsService } from '../rustfs'
import { SessionManager } from '../user/session-manager'
import { UserErpConfigService } from '../user/user-erp-config-service'
import { createLogger } from '../logger'
import { logAudit } from '../logger/audit-logger'
import { logAuditWithCurrentUser } from '../logger/audit-logger'
import { AuditAction, AuditStatus } from '../../types/audit.types'
import { IPC_CHANNELS } from '../../../shared/ipc-channels'
import { DatabaseQueryError, ErpConnectionError, ValidationError } from '../../types/errors'
import { CleanerOperationHistoryDAO } from '../database/cleaner-operation-history-dao'
import type {
CleanerInput,
CleanerProgress,
@@ -23,16 +23,22 @@ import type {
ExportResultItem,
ExportResultResponse
} from '../../types/cleaner.types'
import type { InsertMaterialDetailInput, InsertOrderInput } from '../../types/cleaner-history.types'
import type { OrderMapping } from '../../types/order-resolver.types'
const log = createLogger('CleanerApplicationService')
type DatabaseService = MySqlService | SqlServerService
const DEFAULT_SESSION_REFRESH_ORDER_THRESHOLD = 160
export class CleanerApplicationService {
async runCleaner(eventSender: WebContents, input: CleanerInput): Promise<CleanerResult> {
const startTime = Date.now()
async runCleaner(
eventSender: WebContents,
input: CleanerInput,
batchId: string,
historyDao: CleanerOperationHistoryDAO,
appVersion: string
): Promise<CleanerResult> {
let authService: ErpAuthService | null = null
let dbService: DatabaseService | null = null
let dbService: IDatabaseService | null = null
try {
log.info('Fetching ERP configuration from database...')
@@ -46,7 +52,7 @@ export class CleanerApplicationService {
const configManager = ConfigManager.getInstance()
const dbType = configManager.getDatabaseType()
log.info(
`Connecting to ${dbType === 'sqlserver' ? 'SQL Server' : 'MySQL'} for order resolution...`
`Connecting to ${dbType === 'sqlserver' ? 'SQL Server' : dbType === 'postgresql' ? 'PostgreSQL' : 'MySQL'} for order resolution...`
)
try {
@@ -60,7 +66,8 @@ export class CleanerApplicationService {
}
const resolver = new OrderNumberResolver(dbService)
const mappings = await resolver.resolve(input.orderNumbers)
const inputsToResolve = input.originalInputs?.length ? input.originalInputs : input.orderNumbers
const mappings = await resolver.resolve(inputsToResolve)
const validOrderNumbers = resolver.getValidOrderNumbers(mappings)
const warnings = resolver.getWarnings(mappings)
@@ -68,14 +75,60 @@ export class CleanerApplicationService {
log.warn('Resolution warnings', { warnings })
}
if (validOrderNumbers.length === 0) {
throw new ValidationError(
'没有有效的生产订单号可处理。请检查输入的格式或数据库连接。',
'VAL_INVALID_INPUT'
)
// Build order inputs from ALL mappings (including resolution failures)
const orderInputs = this.buildOrderInputs(mappings)
log.info('Resolved order numbers', {
total: mappings.length,
resolved: validOrderNumbers.length,
failed: mappings.length - validOrderNumbers.length
})
// Insert execution record and ALL order records into database
const currentUser = SessionManager.getInstance().getUserInfo()
if (currentUser) {
await historyDao.insertExecution({
batchId,
attemptNumber: 1,
userId: currentUser.id,
username: currentUser.username,
isDryRun: input.dryRun ?? false,
totalOrders: orderInputs.length,
appVersion
})
await historyDao.insertOrderRecords(batchId, 1, orderInputs)
}
log.info('Resolved order numbers', { count: validOrderNumbers.length })
// If no valid order numbers, update execution to failed and return early
if (validOrderNumbers.length === 0) {
if (currentUser) {
await historyDao.updateExecutionStatus(
batchId,
1,
'failed',
0,
0,
0,
0,
0,
new Date(),
warnings.join('\n') || '没有有效的生产订单号可处理'
)
}
const emptyResult: CleanerResult = {
ordersProcessed: 0,
materialsDeleted: 0,
materialsSkipped: 0,
errors: [...warnings],
details: [],
retriedOrders: 0,
successfulRetries: 0,
materialsFailed: 0,
uncertainDeletions: 0
}
await this.recordCleanupAudit(0, input, emptyResult)
return emptyResult
}
authService = new ErpAuthService({
url: erpConfig.url,
@@ -97,6 +150,8 @@ export class CleanerApplicationService {
log.info('Login successful')
const totalOrders = validOrderNumbers.length
const effectiveSessionRefreshOrderThreshold =
this.resolveSessionRefreshOrderThreshold(input, configManager)
this.sendProgress(eventSender, 'ERP 登录成功', (1 / (1 + totalOrders)) * 100, {
phase: 'login',
currentOrderIndex: 0,
@@ -105,22 +160,105 @@ export class CleanerApplicationService {
totalMaterialsInOrder: 0
})
const cleaner = new CleanerService(authService)
const modifiedInput: CleanerInput = {
...input,
orderNumbers: validOrderNumbers,
sessionRefreshOrderThreshold: effectiveSessionRefreshOrderThreshold,
onProgress: (message, progress, extra) => {
this.sendProgress(eventSender, message, progress ?? 0, extra)
}
}
log.info('Starting cleaning', {
batchId,
orderCount: validOrderNumbers.length,
queryBatchSize: input.queryBatchSize ?? 100,
processConcurrency: input.processConcurrency ?? 1
processConcurrency: input.processConcurrency ?? 1,
sessionRefreshOrderThreshold: effectiveSessionRefreshOrderThreshold
})
const result = await cleaner.clean(modifiedInput)
let cleaner = new CleanerService(authService)
let result = await cleaner.clean(modifiedInput)
// Outer retry: re-login and re-run all orders on fatal crash
if (result.crashed) {
log.warn('检测到流程级崩溃,准备外层重试', { batchId })
// Save attempt 1 result as crashed
await this.saveAttemptToDatabase(historyDao, batchId, 1, result)
this.sendProgress(eventSender, '流程崩溃,正在重新登录并重试...', 0, {
phase: 'retry',
currentOrderIndex: 0,
totalOrders,
currentMaterialIndex: 0,
totalMaterialsInOrder: 0
})
try {
await authService.close()
} catch {
// Browser may already be dead, ignore close errors
}
authService = null
authService = new ErpAuthService({
url: erpConfig.url,
username: erpConfig.username,
password: erpConfig.password,
headless: input.headless ?? true
})
try {
await authService.login()
log.info('Outer retry: re-login successful', { batchId })
} catch (loginError) {
log.error('Outer retry: re-login failed', {
batchId,
error: loginError instanceof Error ? loginError.message : String(loginError)
})
// Return the original crash result if re-login fails
result.errors.push(
`外层重试登录失败: ${loginError instanceof Error ? loginError.message : String(loginError)}`
)
if (warnings.length > 0) {
result.errors = [...warnings, ...result.errors]
}
await this.recordCleanupAudit(validOrderNumbers.length, input, result)
// Attempt 1 already saved as crashed above
return result
}
// Insert execution and order records for attempt 2
if (currentUser) {
await historyDao.insertExecution({
batchId,
attemptNumber: 2,
userId: currentUser.id,
username: currentUser.username,
isDryRun: input.dryRun ?? false,
totalOrders: orderInputs.length,
appVersion
})
await historyDao.insertOrderRecords(batchId, 2, orderInputs)
}
cleaner = new CleanerService(authService)
result = await cleaner.clean(modifiedInput)
// Save attempt 2 result
await this.saveAttemptToDatabase(historyDao, batchId, 2, result)
log.info('Outer retry completed', {
batchId,
processedCount: result.ordersProcessed,
errorCount: result.errors.length,
crashed: result.crashed
})
} else {
// No crash — save attempt 1 result
await this.saveAttemptToDatabase(historyDao, batchId, 1, result)
}
if (warnings.length > 0) {
result.errors = [...warnings, ...result.errors]
@@ -135,12 +273,12 @@ export class CleanerApplicationService {
})
log.info('Cleaning completed', {
batchId,
processedCount: result.ordersProcessed,
errorCount: result.errors.length
})
await this.recordCleanupAudit(validOrderNumbers.length, input, result)
await this.generateAndUploadReport(input, result, startTime)
return result
} finally {
@@ -201,7 +339,7 @@ export class CleanerApplicationService {
}
}
private async getDatabaseService(): Promise<DatabaseService> {
private async getDatabaseService(): Promise<IDatabaseService> {
const configManager = ConfigManager.getInstance()
const config = configManager.getConfig()
const dbType = configManager.getDatabaseType()
@@ -223,6 +361,19 @@ export class CleanerApplicationService {
return sqlServerService
}
if (dbType === 'postgresql') {
const dbConfig = config.database.postgresql
const pgService = new PostgreSqlServiceImpl({
host: dbConfig.host,
port: dbConfig.port,
user: dbConfig.username,
password: dbConfig.password,
database: dbConfig.database
})
await pgService.connect()
return pgService
}
const dbConfig = config.database.mysql
const mysqlService = new MySqlServiceImpl({
host: dbConfig.host,
@@ -257,6 +408,38 @@ export class CleanerApplicationService {
}
}
/**
* Build order inputs from ALL mappings, including resolution failures.
* Deduplicates by orderNumber for resolved mappings, includes all failed mappings.
*/
private buildOrderInputs(mappings: OrderMapping[]): InsertOrderInput[] {
const inputs: InsertOrderInput[] = []
const seenOrderNumbers = new Set<string>()
for (const mapping of mappings) {
if (mapping.resolved && mapping.orderNumber) {
// Deduplicate resolved mappings by order number
if (!seenOrderNumbers.has(mapping.orderNumber)) {
seenOrderNumbers.add(mapping.orderNumber)
inputs.push({
orderNumber: mapping.orderNumber,
productionId: mapping.productionId
})
}
} else {
// Resolution failure: use original input as orderNumber identifier
inputs.push({
orderNumber: mapping.input,
productionId: mapping.productionId,
initialStatus: 'not_found',
errorMessage: mapping.error || '未在数据库中找到对应的订单号'
})
}
}
return inputs
}
private async recordCleanupAudit(
orderCount: number,
input: CleanerInput,
@@ -267,93 +450,162 @@ export class CleanerApplicationService {
return
}
const status: 'success' | 'failure' | 'partial' =
const status: AuditStatus =
result.errors.length > 0 && result.materialsDeleted > 0
? 'partial'
? AuditStatus.PARTIAL
: result.errors.length > 0
? 'failure'
: 'success'
? AuditStatus.FAILURE
: AuditStatus.SUCCESS
const effectiveSessionRefreshOrderThreshold = this.resolveSessionRefreshOrderThreshold(
input,
ConfigManager.getInstance()
)
logAudit('CLEAN', String(currentUser.id), {
username: currentUser.username,
computerName: (await import('os')).hostname(),
resource: 'MATERIAL_PLAN',
status,
metadata: {
orderCount,
dryRun: input.dryRun ?? false,
queryBatchSize: input.queryBatchSize ?? 100,
processConcurrency: input.processConcurrency ?? 1,
materialsDeleted: result.materialsDeleted,
materialsSkipped: result.materialsSkipped,
errorCount: result.errors.length
}
logAuditWithCurrentUser(AuditAction.CLEAN, 'MATERIAL_PLAN', status, {
orderCount,
dryRun: input.dryRun ?? false,
queryBatchSize: input.queryBatchSize ?? 100,
processConcurrency: input.processConcurrency ?? 1,
sessionRefreshOrderThreshold: effectiveSessionRefreshOrderThreshold,
materialsDeleted: result.materialsDeleted,
materialsSkipped: result.materialsSkipped,
errorCount: result.errors.length
})
}
private async generateAndUploadReport(
private resolveSessionRefreshOrderThreshold(
input: CleanerInput,
result: CleanerResult,
startTime: number
configManager: Pick<ConfigManager, 'getConfig'>
): number {
return (
input.sessionRefreshOrderThreshold ??
configManager.getConfig().cleaner?.sessionRefreshOrderThreshold ??
DEFAULT_SESSION_REFRESH_ORDER_THRESHOLD
)
}
/**
* Save attempt results to database: update order statuses, insert material details,
* and update execution status.
*/
private async saveAttemptToDatabase(
historyDao: CleanerOperationHistoryDAO,
batchId: string,
attemptNumber: number,
result: CleanerResult
): Promise<void> {
try {
const endTime = Date.now()
const currentUser = SessionManager.getInstance().getUserInfo()
const username = currentUser?.username ?? 'unknown'
// Query execution record to determine if this is a dry run
const batchDetails = await historyDao.getBatchDetails(batchId)
const execution = batchDetails.executions.find((e) => e.attemptNumber === attemptNumber)
const isDryRun = execution?.isDryRun ?? false
const reportGenerator = new CleanerReportGenerator()
const reportPath = await reportGenerator.generateReport(result, {
dryRun: input.dryRun ?? false,
username,
startTime,
endTime
})
log.info('Report generated', { path: reportPath })
const configManager = ConfigManager.getInstance()
const config = configManager.getConfig()
if (!config.rustfs?.enabled || !config.rustfs.endpoint) {
log.debug('RustFS is not enabled, skipping upload')
return
}
try {
const rustfs = new RustfsService({ config: config.rustfs })
const reportFileName = reportPath.split(/[\\/]/).pop() || 'report.md'
const storageKey = rustfs.generateReportKey(reportFileName, username)
log.info('Uploading report to RustFS', {
localPath: reportPath,
storageKey
})
const uploadResult = await rustfs.uploadFile(
reportPath,
storageKey,
'text/markdown; charset=utf-8'
// Update order statuses and insert material details
for (const detail of result.details) {
await historyDao.updateOrderStatus(
batchId,
attemptNumber,
detail.orderNumber,
detail.notFound ? 'erp_not_found' : detail.errors.length > 0 ? 'failed' : 'success',
detail.materialsDeleted,
detail.materialsSkipped,
detail.materialsFailed,
detail.uncertainDeletions,
detail.retryCount,
detail.retrySuccess ?? false,
detail.errors.length > 0 ? detail.errors.join('\n') : undefined
)
if (uploadResult.success) {
log.info('Report uploaded to RustFS successfully', {
key: storageKey,
etag: uploadResult.etag
})
} else {
log.warn('Failed to upload report to RustFS', {
error: uploadResult.error,
key: storageKey
// Insert material details for all materials
const materialDetails: InsertMaterialDetailInput[] = []
for (const deleted of detail.deletedMaterials) {
materialDetails.push({
orderNumber: detail.orderNumber,
materialCode: deleted.materialCode,
materialName: deleted.materialName,
rowNumber: deleted.rowNumber,
result: deleted.outcome,
reason: null,
attemptCount: 1,
finalErrorCategory: null
})
}
} catch (rustfsError) {
log.error('RustFS upload failed', {
error: rustfsError instanceof Error ? rustfsError.message : String(rustfsError)
})
for (const skipped of detail.skippedMaterials) {
materialDetails.push({
orderNumber: detail.orderNumber,
materialCode: skipped.materialCode,
materialName: skipped.materialName,
rowNumber: skipped.rowNumber,
result: 'skipped',
reason: skipped.reason,
attemptCount: 0,
finalErrorCategory: null
})
}
for (const failed of detail.failedMaterials) {
materialDetails.push({
orderNumber: detail.orderNumber,
materialCode: failed.materialCode,
materialName: failed.materialName,
rowNumber: failed.rowNumber,
result: failed.finalOutcome,
reason:
failed.attempts
.map((a) => a.errorMessage)
.filter(Boolean)
.join('; ') || null,
attemptCount: failed.attempts.length,
finalErrorCategory: failed.finalErrorCategory ?? null
})
}
if (materialDetails.length > 0 && !isDryRun) {
await historyDao.insertMaterialDetails(batchId, attemptNumber, materialDetails)
}
}
} catch (reportError) {
log.warn('Failed to generate report', {
error: reportError instanceof Error ? reportError.message : String(reportError)
// Determine execution status
const execStatus = result.crashed
? 'crashed'
: result.errors.length > 0 && result.materialsDeleted > 0
? 'partial'
: result.errors.length > 0
? 'failed'
: 'success'
// Build error message from execution-level errors (excluding per-order errors)
const execErrorMessage = result.errors.length > 0 ? result.errors.join('\n') : undefined
// Update execution status
await historyDao.updateExecutionStatus(
batchId,
attemptNumber,
execStatus,
result.ordersProcessed,
result.materialsDeleted,
result.materialsSkipped,
result.materialsFailed,
result.uncertainDeletions,
new Date(),
execErrorMessage
)
log.info('Attempt results saved to database', {
batchId,
attemptNumber,
execStatus,
ordersProcessed: result.ordersProcessed
})
} catch (dbError) {
log.error('Failed to save attempt results to database', {
batchId,
attemptNumber,
error: dbError instanceof Error ? dbError.message : String(dbError)
})
// Don't throw — database save failure should not affect the main result
}
}
}

View File

@@ -20,7 +20,7 @@ import { dirname } from 'path'
import { app } from 'electron'
import yaml from 'js-yaml'
import { z } from 'zod'
import { createLogger, applyLoggingConfig, trackDuration } from '../logger'
import { createLogger, applyLoggingConfig } from '../logger'
import { applyAuditConfig } from '../logger/audit-logger'
import {
fullConfigSchema,
@@ -28,6 +28,7 @@ import {
type DatabaseType,
type MySqlConfig,
type SqlServerConfig,
type PostgreSqlConfig,
type LoggingConfig
} from '../../types/config.schema'
@@ -66,6 +67,14 @@ const DEFAULT_CONFIG: FullConfig = {
password: '',
driver: 'ODBC Driver 18 for SQL Server',
trustServerCertificate: true
},
postgresql: {
host: 'localhost',
port: 5432,
database: 'erp_db',
username: 'postgres',
password: '',
maxPoolSize: 10
}
},
paths: {
@@ -90,7 +99,9 @@ const DEFAULT_CONFIG: FullConfig = {
},
cleaner: {
queryBatchSize: 100,
processConcurrency: 1
processConcurrency: 1,
sessionRefreshOrderThreshold: 160,
enableRowProtection: true
},
orderResolution: {
tableName: '',
@@ -299,13 +310,20 @@ export class ConfigManager {
/**
* 获取当前激活的数据库配置
*/
public getActiveDatabaseConfig(): MySqlConfig | SqlServerConfig {
public getActiveDatabaseConfig(): MySqlConfig | SqlServerConfig | PostgreSqlConfig {
if (!this.config) {
throw new Error('Configuration not initialized')
}
const { activeType, mysql, sqlserver } = this.config.database
return activeType === 'mysql' ? mysql : sqlserver
const { activeType, mysql, sqlserver, postgresql } = this.config.database
switch (activeType) {
case 'postgresql':
return postgresql
case 'sqlserver':
return sqlserver
default:
return mysql
}
}
/**

File diff suppressed because it is too large Load Diff

View File

@@ -10,6 +10,8 @@
*/
import { createLogger } from '../logger'
import { logAuditWithCurrentUser } from '../logger/audit-logger'
import { AuditAction, AuditStatus } from '../../types/audit.types'
import { DiscreteMaterialPlanDAO, type MaterialPlanRecord } from './discrete-material-plan-dao'
const log = createLogger('DataImportService')
@@ -94,45 +96,12 @@ export class DataImportService {
// Step 1: Read Excel file
log.info('Reading Excel file...')
const { records, sourceNumbers } = await this.readExcelFile(filePath)
result.recordsRead = records.length
result.uniqueSourceNumbers = sourceNumbers.size
log.info('Excel read completed', {
recordsRead: result.recordsRead,
uniqueSourceNumbers: result.uniqueSourceNumbers
recordsRead: records.length,
uniqueSourceNumbers: sourceNumbers.size
})
if (records.length === 0) {
result.success = true
result.errors.push('Excel file contains no data records')
return result
}
// Step 2: Delete existing records by SourceNumber
log.info('Deleting existing records...', {
sourceNumberCount: sourceNumbers.size
})
const sourceNumberArray = Array.from(sourceNumbers)
result.recordsDeleted = await this.dao.deleteBySourceNumbers(sourceNumberArray)
log.info('Existing records deleted', {
recordsDeleted: result.recordsDeleted
})
// Step 3: Batch insert new records
log.info('Inserting new records...', {
recordCount: records.length,
batchSize
})
result.recordsImported = await this.dao.batchInsert(records, batchSize)
log.info('Records imported successfully', {
recordsImported: result.recordsImported
})
result.success = true
await this.importRecords(records, batchSize, result)
} catch (error) {
const errorMsg = error instanceof Error ? error.message : String(error)
result.errors.push(`Import failed: ${errorMsg}`)
@@ -148,6 +117,106 @@ export class DataImportService {
}
}
// Audit log: DATA_IMPORT
logAuditWithCurrentUser(
AuditAction.DATA_IMPORT,
'MATERIAL_PLAN',
result.success ? AuditStatus.SUCCESS : AuditStatus.FAILURE,
{
recordsRead: result.recordsRead,
recordsDeleted: result.recordsDeleted,
recordsImported: result.recordsImported,
uniqueSourceNumbers: result.uniqueSourceNumbers,
errorCount: result.errors.length
}
)
return result
}
/**
* Import already parsed records to database.
*
* This is the preferred path for extraction: the downloader/parser already has
* structured rows, so database persistence should not require writing and
* reading an intermediate Excel file.
*/
async importFromRecords(records: MaterialPlanRecord[], batchSize = 1000): Promise<ImportResult> {
const result: ImportResult = {
success: false,
recordsRead: 0,
recordsDeleted: 0,
recordsImported: 0,
uniqueSourceNumbers: 0,
errors: []
}
try {
log.info('Starting import from parsed records', {
recordCount: records.length,
batchSize
})
return await this.importRecords(records, batchSize, result)
} catch (error) {
const errorMsg = error instanceof Error ? error.message : String(error)
result.errors.push(`Import failed: ${errorMsg}`)
log.error('Import from records failed', { error: errorMsg })
return result
} finally {
try {
await this.dao.disconnect()
} catch (e) {
log.warn('Error disconnecting DAO', {
error: e instanceof Error ? e.message : String(e)
})
}
logAuditWithCurrentUser(
AuditAction.DATA_IMPORT,
'MATERIAL_PLAN',
result.success ? AuditStatus.SUCCESS : AuditStatus.FAILURE,
{
recordsRead: result.recordsRead,
recordsDeleted: result.recordsDeleted,
recordsImported: result.recordsImported,
uniqueSourceNumbers: result.uniqueSourceNumbers,
errorCount: result.errors.length
}
)
}
}
private async importRecords(
records: MaterialPlanRecord[],
batchSize: number,
result: ImportResult
): Promise<ImportResult> {
const sourceNumbers = new Set(records.map((record) => record.sourceNumber).filter(Boolean))
result.recordsRead = records.length
result.uniqueSourceNumbers = sourceNumbers.size
if (records.length === 0) {
result.success = true
result.errors.push('No data records to import')
return result
}
// Step 1: Replace existing records by SourceNumber
log.info('Replacing existing records...', {
sourceNumberCount: sourceNumbers.size
})
const replaceResult = await this.dao.replaceBySourceNumbers(records, batchSize)
result.recordsDeleted = replaceResult.deleted
result.recordsImported = replaceResult.inserted
log.info('Records replaced successfully', {
recordsDeleted: result.recordsDeleted,
recordsImported: result.recordsImported
})
result.success = true
return result
}

View File

@@ -2,7 +2,7 @@
* TypeORM Data Source Configuration
*
* Provides a centralized database connection for TypeORM entities.
* Supports both MySQL and SQL Server based on configuration.
* Supports MySQL, SQL Server, and PostgreSQL based on configuration.
*
* Note: Configuration is now loaded from config.yaml via ConfigManager,
* not from environment variables.
@@ -18,10 +18,17 @@ const log = createLogger('DataSource')
/**
* Get database type from config manager
*/
function getDatabaseType(): 'mysql' | 'mssql' {
function getDatabaseType(): 'mysql' | 'mssql' | 'postgres' {
const configManager = ConfigManager.getInstance()
const dbType = configManager.getDatabaseType()
return dbType === 'sqlserver' ? 'mssql' : 'mysql'
switch (dbType) {
case 'sqlserver':
return 'mssql'
case 'postgresql':
return 'postgres'
default:
return 'mysql'
}
}
/**
@@ -54,6 +61,17 @@ function buildDataSourceOptions(): DataSourceOptions {
},
...commonOptions
} as DataSourceOptions
} else if (type === 'postgres') {
const dbConfig = config.database.postgresql
return {
type: 'postgres',
host: dbConfig.host,
port: dbConfig.port,
username: dbConfig.username,
password: dbConfig.password,
database: dbConfig.database,
...commonOptions
} as DataSourceOptions
}
const dbConfig = config.database.mysql

View File

@@ -0,0 +1,28 @@
/**
* SQL Dialect Factory
*
* Creates the appropriate SqlDialect implementation based on database type.
*/
import type { DatabaseType } from '../../../types/database.types'
import type { SqlDialect } from '../../../types/sql-dialect.types'
import { MySqlDialect } from './mysql-dialect'
import { PostgreSqlDialect } from './postgresql-dialect'
import { SqlServerDialect } from './sqlserver-dialect'
export { MySqlDialect } from './mysql-dialect'
export { PostgreSqlDialect } from './postgresql-dialect'
export { SqlServerDialect } from './sqlserver-dialect'
export type { SqlDialect } from '../../../types/sql-dialect.types'
export function createDialect(type: DatabaseType): SqlDialect {
switch (type) {
case 'sqlserver':
return new SqlServerDialect()
case 'postgresql':
return new PostgreSqlDialect()
default:
return new MySqlDialect()
}
}

View File

@@ -0,0 +1,71 @@
/**
* MySQL SQL Dialect Implementation
*
* Encapsulates MySQL-specific SQL syntax for:
* - Table name quoting (underscore-separated)
* - Positional parameter placeholders (?)
* - INSERT ... ON DUPLICATE KEY UPDATE upsert
* - LIMIT/OFFSET pagination
*/
import type { SqlDialect } from '../../../types/sql-dialect.types'
export class MySqlDialect implements SqlDialect {
readonly dbType = 'mysql' as const
quoteTableName(schema: string, table: string): string {
return `${schema}_${table}`
}
// eslint-disable-next-line @typescript-eslint/no-unused-vars
param(_index: number): string {
return '?'
}
params(count: number): string {
return Array.from({ length: count }, () => '?').join(',')
}
currentTimestamp(): string {
return 'UTC_TIMESTAMP()'
}
upsert(params: {
table: string
keyColumns: string[]
allColumns: string[]
startParamIndex: number
}): { sql: string; nextParamIndex: number } {
const { table, keyColumns, allColumns, startParamIndex } = params
const columns = allColumns.join(', ')
const placeholders = allColumns.map(() => '?').join(', ')
const nonKeyColumns = allColumns.filter((col) => !keyColumns.includes(col))
const updateClause = nonKeyColumns.map((col) => `${col} = VALUES(${col})`).join(', ')
const sql = `INSERT INTO ${table} (${columns}) VALUES (${placeholders}) ON DUPLICATE KEY UPDATE ${updateClause}`
return {
sql,
nextParamIndex: startParamIndex + allColumns.length
}
}
paginate(params: { sql: string; limit: number; offset?: number; paramIndex: number }): {
sql: string
nextParamIndex: number
} {
const { sql, limit, offset, paramIndex } = params
return {
sql: `${sql} LIMIT ${limit} OFFSET ${offset ?? 0}`,
nextParamIndex: paramIndex
}
}
// eslint-disable-next-line @typescript-eslint/no-unused-vars
maxBatchRows(_columnsPerRow: number): number {
return 1000
}
}

View File

@@ -0,0 +1,76 @@
/**
* PostgreSQL Dialect Implementation
*
* Encapsulates PostgreSQL-specific SQL syntax for:
* - Table name quoting (double-quoted "schema"."table")
* - Positional parameter placeholders ($1, $2, ...) — 1-based
* - INSERT ... ON CONFLICT ... DO UPDATE SET upsert
* - LIMIT/OFFSET pagination
*/
import type { SqlDialect } from '../../../types/sql-dialect.types'
export class PostgreSqlDialect implements SqlDialect {
readonly dbType = 'postgresql' as const
quoteTableName(schema: string, table: string): string {
return `"${schema}"."${table}"`
}
param(index: number): string {
return `$${index + 1}`
}
params(count: number): string {
return Array.from({ length: count }, (_, i) => `$${i + 1}`).join(',')
}
currentTimestamp(): string {
return "(NOW() AT TIME ZONE 'UTC')"
}
upsert(params: {
table: string
keyColumns: string[]
allColumns: string[]
startParamIndex: number
}): { sql: string; nextParamIndex: number } {
const { table, keyColumns, allColumns, startParamIndex } = params
const columns = allColumns.join(', ')
const placeholders = allColumns.map((_, i) => `$${startParamIndex + i + 1}`).join(', ')
const conflictKeys = keyColumns.map((col) => `"${col}"`).join(', ')
const nonKeyColumns = allColumns.filter((col) => !keyColumns.includes(col))
const updateSet = nonKeyColumns.map((col) => `"${col}" = EXCLUDED."${col}"`).join(', ')
const sql = [
`INSERT INTO ${table} (${columns}) VALUES (${placeholders})`,
`ON CONFLICT (${conflictKeys})`,
`DO UPDATE SET ${updateSet}`
].join(' ')
return {
sql,
nextParamIndex: startParamIndex + allColumns.length
}
}
paginate(params: { sql: string; limit: number; offset?: number; paramIndex: number }): {
sql: string
nextParamIndex: number
} {
const { sql, limit, offset, paramIndex } = params
return {
sql: `${sql} LIMIT ${limit} OFFSET ${offset ?? 0}`,
nextParamIndex: paramIndex
}
}
// eslint-disable-next-line @typescript-eslint/no-unused-vars
maxBatchRows(_columnsPerRow: number): number {
return 1000
}
}

View File

@@ -0,0 +1,88 @@
/**
* SQL Server Dialect Implementation
*
* Encapsulates SQL Server-specific SQL syntax for:
* - Table name quoting (bracket notation [schema].[table])
* - Named parameter placeholders (@p0, @p1, ...)
* - MERGE ... USING upsert
* - OFFSET/FETCH pagination
*/
import type { SqlDialect } from '../../../types/sql-dialect.types'
export class SqlServerDialect implements SqlDialect {
readonly dbType = 'sqlserver' as const
quoteTableName(schema: string, table: string): string {
return `[${schema}].[${table}]`
}
param(index: number): string {
return `@p${index}`
}
params(count: number): string {
return Array.from({ length: count }, (_, i) => `@p${i}`).join(',')
}
currentTimestamp(): string {
return 'SYSUTCDATETIME()'
}
upsert(params: {
table: string
keyColumns: string[]
allColumns: string[]
startParamIndex: number
}): { sql: string; nextParamIndex: number } {
const { table, keyColumns, allColumns, startParamIndex } = params
const valueParams = allColumns.map((_, i) => `@p${startParamIndex + i}`).join(', ')
const sourceColumns = allColumns.join(', ')
const joinCondition = keyColumns.map((col) => `target.${col} = source.${col}`).join(' AND ')
const nonKeyColumns = allColumns.filter((col) => !keyColumns.includes(col))
const updateSet = nonKeyColumns.map((col) => `target.${col} = source.${col}`).join(', ')
const insertColumns = allColumns.join(', ')
const insertValues = allColumns.map((col) => `source.${col}`).join(', ')
const sql = [
`MERGE ${table} AS target`,
`USING (VALUES (${valueParams})) AS source (${sourceColumns})`,
`ON ${joinCondition}`,
`WHEN MATCHED THEN UPDATE SET ${updateSet}`,
`WHEN NOT MATCHED THEN INSERT (${insertColumns}) VALUES (${insertValues});`
].join(' ')
return {
sql,
nextParamIndex: startParamIndex + allColumns.length
}
}
paginate(params: { sql: string; limit: number; offset?: number; paramIndex: number }): {
sql: string
nextParamIndex: number
} {
const { sql, offset, paramIndex } = params
void params.limit // used by caller to push param values
if (offset !== undefined) {
return {
sql: `${sql} OFFSET @p${paramIndex} ROWS FETCH NEXT @p${paramIndex + 1} ROWS ONLY`,
nextParamIndex: paramIndex + 2
}
}
return {
sql: `${sql} OFFSET 0 ROWS FETCH NEXT @p${paramIndex} ROWS ONLY`,
nextParamIndex: paramIndex + 1
}
}
maxBatchRows(columnsPerRow: number): number {
return Math.floor(2000 / columnsPerRow)
}
}

View File

@@ -9,9 +9,11 @@
*/
import { create, type IDatabaseService } from './index'
import { createLogger, run, getRequestId, trackDuration } from '../logger'
import { createDialect, type SqlDialect } from './dialects'
import { createLogger, getRequestId, trackDuration } from '../logger'
const log = createLogger('DiscreteMaterialPlanDAO')
const SQLSERVER_REPLACE_SOURCE_NUMBER_BATCH_SIZE = 25
/**
* Material plan record interface
@@ -53,8 +55,6 @@ export interface MaterialPlanRecord {
* Configuration for DiscreteMaterialPlanData table
*/
export const DISCRETE_MATERIAL_PLAN_CONFIG = {
TABLE_NAME_SQLSERVER: '[dbo].[DiscreteMaterialPlanData]',
TABLE_NAME_MYSQL: 'dbo_DiscreteMaterialPlanData',
COLUMNS: {
ID: 'ID',
FACTORY: 'Factory',
@@ -94,15 +94,20 @@ export const DISCRETE_MATERIAL_PLAN_CONFIG = {
*/
export class DiscreteMaterialPlanDAO {
private dbService: IDatabaseService | null = null
private dialect: SqlDialect | null = null
private getDialect(): SqlDialect {
if (!this.dialect) {
this.dialect = createDialect(this.dbService!.type)
}
return this.dialect
}
/**
* Get the appropriate table name based on database type
*/
private getTableName(): string {
const isSqlServer = this.dbService?.type === 'sqlserver'
return isSqlServer
? DISCRETE_MATERIAL_PLAN_CONFIG.TABLE_NAME_SQLSERVER
: DISCRETE_MATERIAL_PLAN_CONFIG.TABLE_NAME_MYSQL
return this.getDialect().quoteTableName('dbo', 'DiscreteMaterialPlanData')
}
/**
@@ -117,15 +122,6 @@ export class DiscreteMaterialPlanDAO {
return this.dbService
}
/**
* Build placeholders for IN clause based on database type
*/
private buildPlaceholders(count: number, isSqlServer: boolean): string {
return isSqlServer
? Array.from({ length: count }, (_, idx) => `@p${idx}`).join(',')
: Array.from({ length: count }, () => '?').join(',')
}
// ==================== QUERY ALL ====================
/**
@@ -219,13 +215,13 @@ export class DiscreteMaterialPlanDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const batchSize = 1500
const allResults: any[] = []
for (let i = 0; i < sourceNumbers.length; i += batchSize) {
const batch = sourceNumbers.slice(i, i + batchSize)
const placeholders = this.buildPlaceholders(batch.length, isSqlServer)
const placeholders = dialect.params(batch.length)
const sqlString = `
SELECT *
@@ -273,13 +269,13 @@ export class DiscreteMaterialPlanDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const batchSize = 1500
const allResults: any[] = []
for (let i = 0; i < sourceNumbers.length; i += batchSize) {
const batch = sourceNumbers.slice(i, i + batchSize)
const placeholders = this.buildPlaceholders(batch.length, isSqlServer)
const placeholders = dialect.params(batch.length)
const sqlString = `
WITH RankedRecords AS (
@@ -338,9 +334,9 @@ export class DiscreteMaterialPlanDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const placeholder = isSqlServer ? '@p0' : '?'
const placeholder = dialect.param(0)
const sqlString = `
SELECT *
FROM ${tableName}
@@ -377,9 +373,9 @@ export class DiscreteMaterialPlanDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const placeholder = isSqlServer ? '@p0' : '?'
const placeholder = dialect.param(0)
const sqlString = `
SELECT *
FROM ${tableName}
@@ -418,13 +414,13 @@ export class DiscreteMaterialPlanDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const batchSize = 1500
const allResults: any[] = []
for (let i = 0; i < planNumbers.length; i += batchSize) {
const batch = planNumbers.slice(i, i + batchSize)
const placeholders = this.buildPlaceholders(batch.length, isSqlServer)
const placeholders = dialect.params(batch.length)
const sqlString = `
SELECT *
@@ -476,7 +472,7 @@ export class DiscreteMaterialPlanDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const batchSize = 2000
// Get unique source numbers
@@ -495,7 +491,7 @@ export class DiscreteMaterialPlanDAO {
for (let i = 0; i < uniqueSourceNumbers.length; i += batchSize) {
const batch = uniqueSourceNumbers.slice(i, i + batchSize)
const batchNumber = Math.floor(i / batchSize) + 1
const placeholders = this.buildPlaceholders(batch.length, isSqlServer)
const placeholders = dialect.params(batch.length)
const sqlString = `
DELETE FROM ${tableName}
@@ -565,23 +561,26 @@ export class DiscreteMaterialPlanDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
if (dbService.type === 'sqlserver') {
return await this.batchInsertSqlServerJson(
dbService,
tableName,
records,
batchSize,
batchId
)
}
// SQL Server has a limit of 2100 parameters per query
// Each record has 28 columns, so max rows per batch = 2100 / 28 = 75
// Leave some margin for query overhead
const columnsPerRow = 28
const sqlServerMaxParams = 2000
const effectiveBatchSize = isSqlServer
? Math.min(batchSize, Math.floor(sqlServerMaxParams / columnsPerRow))
: batchSize
const effectiveBatchSize = Math.min(batchSize, dialect.maxBatchRows(columnsPerRow))
const totalBatches = Math.ceil(records.length / effectiveBatchSize)
log.info('Batch insert started', {
tableName,
operationType: 'INSERT',
requestId: batchId,
isSqlServer,
dbType: dbService.type,
columnsPerRow,
effectiveBatchSize,
@@ -598,7 +597,6 @@ export class DiscreteMaterialPlanDAO {
dbService,
tableName,
batch,
isSqlServer,
batchId,
batchNumber,
totalBatches
@@ -636,6 +634,237 @@ export class DiscreteMaterialPlanDAO {
}
}
async replaceBySourceNumbers(
records: MaterialPlanRecord[],
batchSize = 1000
): Promise<{ deleted: number; inserted: number }> {
if (!records || records.length === 0) {
return { deleted: 0, inserted: 0 }
}
const dbService = await this.getDatabaseService()
const sourceNumbers = [...new Set(records.map((record) => record.sourceNumber).filter(Boolean))]
if (dbService.type === 'sqlserver') {
return await this.replaceSqlServerJson(dbService, records, sourceNumbers)
}
const deleted = await this.deleteBySourceNumbers(sourceNumbers)
const inserted = await this.batchInsert(records, batchSize)
return { deleted, inserted }
}
private async replaceSqlServerJson(
dbService: IDatabaseService,
records: MaterialPlanRecord[],
sourceNumbers: string[]
): Promise<{ deleted: number; inserted: number }> {
const tableName = this.getTableName()
const columns = this.getInsertColumns()
const withColumns = this.getSqlServerJsonWithColumns(columns)
const quotedColumns = columns.map((column) => `[${column}]`).join(', ')
const recordsBySourceNumber = this.groupRecordsBySourceNumber(records)
const totalBatches = Math.ceil(
sourceNumbers.length / SQLSERVER_REPLACE_SOURCE_NUMBER_BATCH_SIZE
)
let totalDeleted = 0
let totalInserted = 0
log.info('SQL Server JSON replace started', {
tableName,
operationType: 'REPLACE',
totalSourceNumbers: sourceNumbers.length,
totalRecords: records.length,
sourceNumberBatchSize: SQLSERVER_REPLACE_SOURCE_NUMBER_BATCH_SIZE,
totalBatches
})
for (
let offset = 0;
offset < sourceNumbers.length;
offset += SQLSERVER_REPLACE_SOURCE_NUMBER_BATCH_SIZE
) {
const sourceNumberBatch = sourceNumbers.slice(
offset,
offset + SQLSERVER_REPLACE_SOURCE_NUMBER_BATCH_SIZE
)
const batchNumber = Math.floor(offset / SQLSERVER_REPLACE_SOURCE_NUMBER_BATCH_SIZE) + 1
const recordBatch = sourceNumberBatch.flatMap(
(sourceNumber) => recordsBySourceNumber.get(sourceNumber) || []
)
const jsonRows = recordBatch.map((record) => this.buildJsonRow(record, columns))
const sqlString = `
DECLARE @deleted int = 0;
DECLARE @inserted int = 0;
BEGIN TRY
BEGIN TRANSACTION;
DELETE target
FROM ${tableName} AS target
INNER JOIN OPENJSON(@p0)
WITH (SourceNumber nvarchar(100) '$') AS source
ON target.SourceNumber = source.SourceNumber;
SET @deleted = @@ROWCOUNT;
INSERT INTO ${tableName} (${quotedColumns})
SELECT ${quotedColumns}
FROM OPENJSON(@p1)
WITH (
${withColumns}
);
SET @inserted = @@ROWCOUNT;
COMMIT TRANSACTION;
END TRY
BEGIN CATCH
IF @@TRANCOUNT > 0
ROLLBACK TRANSACTION;
THROW;
END CATCH;
SELECT @deleted AS deletedCount, @inserted AS insertedCount;
`
const result = await trackDuration(
async () =>
await dbService.query(sqlString, [
JSON.stringify(sourceNumberBatch),
JSON.stringify(jsonRows)
]),
{
operationName: 'DiscreteMaterialPlanDAO.replaceSqlServerJsonBatch',
context: {
tableName,
operationType: 'REPLACE',
batchNumber,
totalBatches,
sourceNumberCount: sourceNumberBatch.length,
recordCount: recordBatch.length
}
}
)
const stats = result.result.rows[0] || {}
totalDeleted += Number(stats.deletedCount || 0)
totalInserted += Number(stats.insertedCount || recordBatch.length)
log.debug('SQL Server JSON replace batch completed', {
tableName,
batchNumber,
totalBatches,
sourceNumberCount: sourceNumberBatch.length,
recordCount: recordBatch.length
})
}
log.info('SQL Server JSON replace completed', {
tableName,
operationType: 'REPLACE',
totalDeleted,
totalInserted,
totalBatches
})
return {
deleted: totalDeleted,
inserted: totalInserted
}
}
private groupRecordsBySourceNumber(
records: MaterialPlanRecord[]
): Map<string, MaterialPlanRecord[]> {
const groups = new Map<string, MaterialPlanRecord[]>()
for (const record of records) {
if (!record.sourceNumber) {
continue
}
const existing = groups.get(record.sourceNumber) || []
existing.push(record)
groups.set(record.sourceNumber, existing)
}
return groups
}
private async batchInsertSqlServerJson(
dbService: IDatabaseService,
tableName: string,
records: MaterialPlanRecord[],
batchSize: number,
batchId: string
): Promise<number> {
const columns = this.getInsertColumns()
const effectiveBatchSize = Math.max(1, batchSize)
const totalBatches = Math.ceil(records.length / effectiveBatchSize)
let totalInserted = 0
log.info('SQL Server JSON batch insert started', {
tableName,
operationType: 'INSERT',
requestId: batchId,
totalRecords: records.length,
effectiveBatchSize,
totalBatches
})
for (let i = 0; i < records.length; i += effectiveBatchSize) {
const batch = records.slice(i, i + effectiveBatchSize)
const batchNumber = Math.floor(i / effectiveBatchSize) + 1
const jsonRows = batch.map((record) => this.buildJsonRow(record, columns))
const withColumns = this.getSqlServerJsonWithColumns(columns)
const quotedColumns = columns.map((column) => `[${column}]`).join(', ')
const sqlString = `
INSERT INTO ${tableName} (${quotedColumns})
SELECT ${quotedColumns}
FROM OPENJSON(@p0)
WITH (
${withColumns}
)
`
const result = await trackDuration(
async () => await dbService.query(sqlString, [JSON.stringify(jsonRows)]),
{
operationName: 'DiscreteMaterialPlanDAO.insertBatchSqlServerJson',
context: {
tableName,
operationType: 'INSERT',
batchId,
batchNumber,
totalBatches,
recordCount: batch.length
}
}
)
totalInserted += result.result.rowCount || batch.length
log.debug('Inserted SQL Server JSON batch', {
batch: batchNumber,
totalBatches,
count: batch.length,
batchId
})
}
log.info('SQL Server JSON batch insert completed', {
tableName,
operationType: 'INSERT',
requestId: batchId,
totalInserted,
batchSize: effectiveBatchSize,
totalBatches
})
return totalInserted
}
/**
* Insert a single batch of records with tracking
*/
@@ -643,7 +872,6 @@ export class DiscreteMaterialPlanDAO {
dbService: IDatabaseService,
tableName: string,
records: MaterialPlanRecord[],
isSqlServer: boolean,
batchId: string,
batchNumber: number,
totalBatches: number
@@ -652,8 +880,67 @@ export class DiscreteMaterialPlanDAO {
return 0
}
// Build column list (excluding id)
const columns = [
const columns = this.getInsertColumns()
// Build parameterized insert
const values: any[] = []
const rowPlaceholders: string[] = []
records.forEach((record, rowIndex) => {
const rowValues = this.buildRowValues(record, columns, rowIndex, values)
rowPlaceholders.push(`(${rowValues.join(',')})`)
})
const sqlString = `
INSERT INTO ${tableName} (${columns.join(', ')})
VALUES ${rowPlaceholders.join(', ')}
`
const result = await trackDuration(async () => await dbService.query(sqlString, values), {
operationName: 'DiscreteMaterialPlanDAO.insertBatch',
context: {
tableName,
operationType: 'INSERT',
batchId,
batchNumber,
totalBatches,
recordCount: records.length
}
})
return result.result.rowCount || records.length
}
/**
* Insert a single batch of records (legacy method - kept for compatibility)
*/
private async insertBatch(
dbService: IDatabaseService,
tableName: string,
records: MaterialPlanRecord[]
): Promise<number> {
return this.insertBatchWithTracking(dbService, tableName, records, 'unknown', 1, 1)
}
/**
* Build parameter values for a single row
*/
private buildRowValues(
record: MaterialPlanRecord,
columns: string[],
_rowIndex: number,
values: any[]
): string[] {
const dialect = this.getDialect()
return columns.map((col) => {
const value = this.getColumnValue(record, col)
values.push(value)
return dialect.param(values.length - 1)
})
}
private getInsertColumns(): string[] {
return [
'Factory',
'MaterialStatus',
'PlanNumber',
@@ -683,67 +970,58 @@ export class DiscreteMaterialPlanDAO {
'UnitUsage',
'CumulativeOutputQuantity'
]
// Build parameterized insert
const values: any[] = []
const rowPlaceholders: string[] = []
records.forEach((record, rowIndex) => {
const rowValues = this.buildRowValues(record, columns, rowIndex, isSqlServer, values)
rowPlaceholders.push(`(${rowValues.join(',')})`)
})
const sqlString = `
INSERT INTO ${tableName} (${columns.join(', ')})
VALUES ${rowPlaceholders.join(', ')}
`
const result = await trackDuration(async () => await dbService.query(sqlString, values), {
operationName: 'DiscreteMaterialPlanDAO.insertBatch',
context: {
tableName,
operationType: 'INSERT',
batchId,
batchNumber,
totalBatches,
recordCount: records.length
}
})
return result.result.rowCount || records.length
}
/**
* Insert a single batch of records (legacy method - kept for compatibility)
*/
private async insertBatch(
dbService: IDatabaseService,
tableName: string,
records: MaterialPlanRecord[],
isSqlServer: boolean
): Promise<number> {
return this.insertBatchWithTracking(dbService, tableName, records, isSqlServer, 'unknown', 1, 1)
private buildJsonRow(record: MaterialPlanRecord, columns: string[]): Record<string, unknown> {
const row: Record<string, unknown> = {}
for (const column of columns) {
const value = this.getColumnValue(record, column)
row[column] = value instanceof Date ? value.toISOString() : value
}
return row
}
/**
* Build parameter values for a single row
*/
private buildRowValues(
record: MaterialPlanRecord,
columns: string[],
_rowIndex: number,
isSqlServer: boolean,
values: any[]
): string[] {
return columns.map((col) => {
const value = this.getColumnValue(record, col)
values.push(value)
private getSqlServerJsonWithColumns(columns: string[]): string {
return columns
.map((column) => `[${column}] ${this.getSqlServerJsonColumnType(column)} '$.${column}'`)
.join(',\n ')
}
if (isSqlServer) {
return `@p${values.length - 1}`
} else {
return '?'
}
})
private getSqlServerJsonColumnType(column: string): string {
const columnTypes: Record<string, string> = {
Factory: 'nvarchar(100)',
MaterialStatus: 'nvarchar(50)',
PlanNumber: 'nvarchar(100)',
SourceNumber: 'nvarchar(100)',
MaterialType: 'nvarchar(100)',
ProductCode: 'nvarchar(100)',
ProductName: 'nvarchar(255)',
ProductUnit: 'nvarchar(50)',
ProductPlanQuantity: 'decimal(18,4)',
UseDepartment: 'nvarchar(100)',
Remark: 'nvarchar(500)',
Creator: 'nvarchar(100)',
CreateDate: 'datetime2',
Approver: 'nvarchar(100)',
ApproveDate: 'datetime2',
SequenceNumber: 'int',
MaterialCode: 'nvarchar(100)',
MaterialName: 'nvarchar(255)',
Specification: 'nvarchar(255)',
Model: 'nvarchar(255)',
DrawingNumber: 'nvarchar(100)',
MaterialQuality: 'nvarchar(100)',
PlanQuantity: 'decimal(18,4)',
Unit: 'nvarchar(50)',
RequiredDate: 'datetime2',
Warehouse: 'nvarchar(100)',
UnitUsage: 'decimal(18,6)',
CumulativeOutputQuantity: 'decimal(18,4)'
}
return columnTypes[column] || 'nvarchar(max)'
}
/**
@@ -793,6 +1071,10 @@ export class DiscreteMaterialPlanDAO {
return null
}
if (value instanceof Date && Number.isNaN(value.getTime())) {
return null
}
// Handle empty strings for string fields
if (typeof value === 'string' && value.trim() === '') {
return null
@@ -839,9 +1121,9 @@ export class DiscreteMaterialPlanDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const placeholder = isSqlServer ? '@p0' : '?'
const placeholder = dialect.param(0)
const sqlString = `
SELECT COUNT(*) as count
FROM ${tableName}
@@ -876,7 +1158,7 @@ export class DiscreteMaterialPlanDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
if (sourceNumbers && sourceNumbers.length > 0) {
const batchSize = 1500
@@ -884,7 +1166,7 @@ export class DiscreteMaterialPlanDAO {
for (let i = 0; i < sourceNumbers.length; i += batchSize) {
const batch = sourceNumbers.slice(i, i + batchSize)
const placeholders = this.buildPlaceholders(batch.length, isSqlServer)
const placeholders = dialect.params(batch.length)
const sqlString = `
SELECT DISTINCT MaterialName
@@ -975,6 +1257,7 @@ export class DiscreteMaterialPlanDAO {
if (this.dbService) {
await this.dbService.disconnect()
this.dbService = null
this.dialect = null
}
}
}

View File

@@ -10,7 +10,8 @@
*/
import { create, type IDatabaseService } from './index'
import { createLogger, run, getRequestId, trackDuration } from '../logger'
import { createDialect, type SqlDialect } from './dialects'
import { createLogger, getRequestId, trackDuration } from '../logger'
import type {
OperationHistoryRecord,
BatchStats,
@@ -36,8 +37,6 @@ function formatDateTime(value: unknown): string {
* Configuration for ExtractorOperationHistory table
*/
export const EXTRACTOR_OPERATION_HISTORY_CONFIG = {
TABLE_NAME_SQLSERVER: '[dbo].[ExtractorOperationHistory]',
TABLE_NAME_MYSQL: 'dbo_ExtractorOperationHistory',
COLUMNS: {
ID: 'ID',
BATCH_ID: 'BatchId',
@@ -57,15 +56,20 @@ export const EXTRACTOR_OPERATION_HISTORY_CONFIG = {
*/
export class ExtractorOperationHistoryDAO {
private dbService: IDatabaseService | null = null
private dialect: SqlDialect | null = null
private getDialect(): SqlDialect {
if (!this.dialect) {
this.dialect = createDialect(this.dbService!.type)
}
return this.dialect
}
/**
* Get the appropriate table name based on database type
*/
private getTableName(): string {
const isSqlServer = this.dbService?.type === 'sqlserver'
return isSqlServer
? EXTRACTOR_OPERATION_HISTORY_CONFIG.TABLE_NAME_SQLSERVER
: EXTRACTOR_OPERATION_HISTORY_CONFIG.TABLE_NAME_MYSQL
return this.getDialect().quoteTableName('dbo', 'ExtractorOperationHistory')
}
/**
@@ -80,15 +84,6 @@ export class ExtractorOperationHistoryDAO {
return this.dbService
}
/**
* Build placeholders for IN clause based on database type
*/
private buildPlaceholders(count: number, isSqlServer: boolean): string {
return isSqlServer
? Array.from({ length: count }, (_, idx) => `@p${idx}`).join(',')
: Array.from({ length: count }, () => '?').join(',')
}
// ==================== INSERT ====================
/**
@@ -119,7 +114,7 @@ export class ExtractorOperationHistoryDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
log.info('Batch records insertion started', {
tableName,
@@ -131,58 +126,47 @@ export class ExtractorOperationHistoryDAO {
recordCount: records.length
})
for (const record of records) {
const columnsPerRecord = 5
const batchSize = Math.max(1, dialect.maxBatchRows(columnsPerRecord))
for (let offset = 0; offset < records.length; offset += batchSize) {
const batch = records.slice(offset, offset + batchSize)
try {
if (isSqlServer) {
const sqlString = `
INSERT INTO ${tableName}
(BatchId, UserId, Username, ProductionId, OrderNumber, OperationTime, Status)
VALUES
(@p0, @p1, @p2, @p3, @p4, GETDATE(), 'pending')
`
await trackDuration(
async () =>
await dbService.query(sqlString, [
batchId,
userId,
username,
record.productionId || null,
record.orderNumber
]),
{
operationName: 'ExtractorOperationHistoryDAO.insertBatchRecords',
context: { tableName, operationType: 'INSERT', batchId }
}
const valuesSql: string[] = []
const params: (string | number | null)[] = []
batch.forEach((record, index) => {
const paramOffset = index * columnsPerRecord
valuesSql.push(
`(${dialect.param(paramOffset)}, ${dialect.param(paramOffset + 1)}, ${dialect.param(paramOffset + 2)}, ${dialect.param(paramOffset + 3)}, ${dialect.param(paramOffset + 4)}, ${dialect.currentTimestamp()}, 'pending')`
)
} else {
const sqlString = `
INSERT INTO ${tableName}
(BatchId, UserId, Username, ProductionId, OrderNumber, OperationTime, Status)
VALUES
(?, ?, ?, ?, ?, NOW(), 'pending')
`
await trackDuration(
async () =>
await dbService.query(sqlString, [
batchId,
userId,
username,
record.productionId || null,
record.orderNumber
]),
{
operationName: 'ExtractorOperationHistoryDAO.insertBatchRecords',
context: { tableName, operationType: 'INSERT', batchId }
}
)
}
params.push(batchId, userId, username, record.productionId || null, record.orderNumber)
})
const sqlString = `
INSERT INTO ${tableName}
(BatchId, UserId, Username, ProductionId, OrderNumber, OperationTime, Status)
VALUES
${valuesSql.join(',\n ')}
`
await trackDuration(async () => await dbService.query(sqlString, params), {
operationName: 'ExtractorOperationHistoryDAO.insertBatchRecords',
context: {
tableName,
operationType: 'INSERT',
batchId,
batchOffset: offset,
batchCount: batch.length
}
})
} catch (error) {
log.error('Error inserting individual record', {
log.error('Error inserting record batch', {
tableName,
operationType: 'INSERT',
requestId,
batchId,
orderNumber: record.orderNumber,
batchOffset: offset,
batchCount: batch.length,
error: error instanceof Error ? error.message : String(error)
})
}
@@ -221,16 +205,16 @@ export class ExtractorOperationHistoryDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const sqlString = `
UPDATE ${tableName}
SET Status = ${isSqlServer ? '@p0' : '?'}
WHERE BatchId = ${isSqlServer ? '@p1' : '?'}
SET Status = ${dialect.param(0)}
WHERE BatchId = ${dialect.param(1)}
`
const params = [status, batchId]
const result = await trackDuration(async () => await dbService.query(sqlString, params), {
await trackDuration(async () => await dbService.query(sqlString, params), {
operationName: 'ExtractorOperationHistoryDAO.updateBatchStatus',
context: { tableName, operationType: 'UPDATE', batchId }
})
@@ -274,7 +258,7 @@ export class ExtractorOperationHistoryDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
let sqlString: string
let params: (string | number | null)[]
@@ -282,20 +266,20 @@ export class ExtractorOperationHistoryDAO {
if (recordCount !== undefined) {
sqlString = `
UPDATE ${tableName}
SET Status = ${isSqlServer ? '@p0' : '?'},
ErrorMessage = ${isSqlServer ? '@p1' : '?'},
RecordCount = ${isSqlServer ? '@p2' : '?'}
WHERE BatchId = ${isSqlServer ? '@p3' : '?'}
AND OrderNumber = ${isSqlServer ? '@p4' : '?'}
SET Status = ${dialect.param(0)},
ErrorMessage = ${dialect.param(1)},
RecordCount = ${dialect.param(2)}
WHERE BatchId = ${dialect.param(3)}
AND OrderNumber = ${dialect.param(4)}
`
params = [status, errorMessage || null, recordCount, batchId, orderNumber]
} else {
sqlString = `
UPDATE ${tableName}
SET Status = ${isSqlServer ? '@p0' : '?'},
ErrorMessage = ${isSqlServer ? '@p1' : '?'}
WHERE BatchId = ${isSqlServer ? '@p2' : '?'}
AND OrderNumber = ${isSqlServer ? '@p3' : '?'}
SET Status = ${dialect.param(0)},
ErrorMessage = ${dialect.param(1)}
WHERE BatchId = ${dialect.param(2)}
AND OrderNumber = ${dialect.param(3)}
`
params = [status, errorMessage || null, batchId, orderNumber]
}
@@ -331,7 +315,7 @@ export class ExtractorOperationHistoryDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
let sqlString = `
SELECT
@@ -350,11 +334,10 @@ export class ExtractorOperationHistoryDAO {
const params: (number | string)[] = []
if (userId !== undefined) {
sqlString += isSqlServer ? ` WHERE UserId = @p0 ` : ` WHERE UserId = ? `
sqlString += ` WHERE UserId = ${dialect.param(params.length)} `
params.push(userId)
} else if (options?.usernames && options.usernames.length > 0) {
const placeholders = this.buildPlaceholders(options.usernames.length, isSqlServer)
sqlString += ` WHERE Username IN (${placeholders}) `
sqlString += ` WHERE Username IN (${dialect.params(options.usernames.length)}) `
params.push(...options.usernames)
}
@@ -367,24 +350,19 @@ export class ExtractorOperationHistoryDAO {
const safeLimit = Math.floor(options.limit)
const safeOffset = options.offset !== undefined ? Math.floor(options.offset) : undefined
if (isSqlServer) {
const offsetIndex = params.length
const result = dialect.paginate({
sql: sqlString,
limit: safeLimit,
offset: safeOffset,
paramIndex: params.length
})
sqlString = result.sql
if (dialect.dbType === 'sqlserver') {
if (safeOffset !== undefined) {
params.push(safeOffset)
}
params.push(safeLimit)
if (safeOffset !== undefined) {
sqlString += ` OFFSET @p${offsetIndex} ROWS FETCH NEXT @p${offsetIndex + 1} ROWS ONLY`
} else {
sqlString += ` OFFSET 0 ROWS FETCH NEXT @p${offsetIndex} ROWS ONLY`
}
} else {
if (safeOffset !== undefined) {
sqlString += ` LIMIT ${safeLimit} OFFSET ${safeOffset}`
} else {
sqlString += ` LIMIT ${safeLimit}`
}
}
}
@@ -425,9 +403,9 @@ export class ExtractorOperationHistoryDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const placeholder = isSqlServer ? '@p0' : '?'
const placeholder = dialect.param(0)
const sqlString = `
SELECT
ID,
@@ -483,9 +461,9 @@ export class ExtractorOperationHistoryDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const placeholder = isSqlServer ? '@p0' : '?'
const placeholder = dialect.param(0)
const sqlString = `
SELECT
BatchId,
@@ -554,7 +532,7 @@ export class ExtractorOperationHistoryDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
// First check if the batch exists and if the user has permission
const batchStats = await this.getBatchStats(batchId)
@@ -569,7 +547,7 @@ export class ExtractorOperationHistoryDAO {
}
// Delete the batch
const placeholder = isSqlServer ? '@p0' : '?'
const placeholder = dialect.param(0)
const sqlString = `
DELETE FROM ${tableName}
WHERE BatchId = ${placeholder}
@@ -612,9 +590,9 @@ export class ExtractorOperationHistoryDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const placeholder = isSqlServer ? '@p0' : '?'
const placeholder = dialect.param(0)
const sqlString = `
DELETE FROM ${tableName}
WHERE UserId = ${placeholder}
@@ -648,9 +626,9 @@ export class ExtractorOperationHistoryDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const placeholder = isSqlServer ? '@p0' : '?'
const placeholder = dialect.param(0)
const sqlString = `
SELECT COUNT(*) as count
FROM ${tableName}
@@ -684,7 +662,7 @@ export class ExtractorOperationHistoryDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
let sqlString = `
SELECT COUNT(DISTINCT BatchId) as count
@@ -694,11 +672,10 @@ export class ExtractorOperationHistoryDAO {
const params: (number | string)[] = []
if (userId !== undefined) {
sqlString += isSqlServer ? ` WHERE UserId = @p0 ` : ` WHERE UserId = ? `
sqlString += ` WHERE UserId = ${dialect.param(params.length)} `
params.push(userId)
} else if (usernames && usernames.length > 0) {
const placeholders = this.buildPlaceholders(usernames.length, isSqlServer)
sqlString += ` WHERE Username IN (${placeholders}) `
sqlString += ` WHERE Username IN (${dialect.params(usernames.length)}) `
params.push(...usernames)
}
@@ -725,6 +702,7 @@ export class ExtractorOperationHistoryDAO {
if (this.dbService) {
await this.dbService.disconnect()
this.dbService = null
this.dialect = null
}
}
}

View File

@@ -2,17 +2,19 @@
* Database Factory
*
* Creates and manages database service instances based on configuration.
* Supports both MySQL and SQL Server databases.
* Supports MySQL, SQL Server, and PostgreSQL databases.
*/
import { ConfigManager } from '../config/config-manager'
import { MySqlService } from './mysql'
import { SqlServerService } from './sql-server'
import { PostgreSqlService } from './postgresql'
import type {
IDatabaseService,
DatabaseType,
MySqlConfig,
SqlServerConfig
SqlServerConfig,
PostgreSqlConfig
} from '../../types/database.types'
import { createLogger } from '../logger'
@@ -65,6 +67,22 @@ export function createSqlServerConfig(): SqlServerConfig {
}
}
/**
* Create PostgreSQL configuration from config manager
*/
export function createPostgreSqlConfig(): PostgreSqlConfig {
const configManager = ConfigManager.getInstance()
const dbConfig = configManager.getConfig().database.postgresql
return {
host: dbConfig.host,
port: dbConfig.port,
user: dbConfig.username,
password: dbConfig.password,
database: dbConfig.database,
maxPoolSize: dbConfig.maxPoolSize
}
}
/**
* Create a database service instance
*
@@ -86,7 +104,10 @@ export async function create(type?: DatabaseType): Promise<IDatabaseService> {
// Create new instance
let service: IDatabaseService
if (dbType === 'sqlserver') {
if (dbType === 'postgresql') {
log.info('Creating PostgreSQL database service')
service = new PostgreSqlService(createPostgreSqlConfig())
} else if (dbType === 'sqlserver') {
log.info('Creating SQL Server database service')
service = new SqlServerService(createSqlServerConfig())
} else {
@@ -175,10 +196,12 @@ export function isConnected(type?: DatabaseType): boolean {
// Re-export types and services
export { MySqlService } from './mysql'
export { SqlServerService } from './sql-server'
export { PostgreSqlService } from './postgresql'
export type {
IDatabaseService,
DatabaseType,
QueryResult,
MySqlConfig,
SqlServerConfig
SqlServerConfig,
PostgreSqlConfig
} from '../../types/database.types'

View File

@@ -9,7 +9,8 @@
*/
import { create, type IDatabaseService } from './index'
import { createLogger, run, getRequestId, trackDuration } from '../logger'
import { createDialect, type SqlDialect } from './dialects'
import { createLogger, getRequestId, trackDuration } from '../logger'
const log = createLogger('MaterialsToBeDeletedDAO')
@@ -44,8 +45,6 @@ export interface MaterialStats {
* Configuration for MaterialsToBeDeleted table
*/
export const MATERIALS_TO_BE_DELETED_CONFIG = {
TABLE_NAME_SQLSERVER: '[dbo].[MaterialsToBeDeleted]',
TABLE_NAME_MYSQL: 'dbo_MaterialsToBeDeleted',
COLUMNS: {
ID: 'ID',
MATERIAL_CODE: 'MaterialCode',
@@ -58,15 +57,20 @@ export const MATERIALS_TO_BE_DELETED_CONFIG = {
*/
export class MaterialsToBeDeletedDAO {
private dbService: IDatabaseService | null = null
private dialect: SqlDialect | null = null
private getDialect(): SqlDialect {
if (!this.dialect) {
this.dialect = createDialect(this.dbService!.type)
}
return this.dialect
}
/**
* Get the appropriate table name based on database type
*/
private getTableName(): string {
const isSqlServer = this.dbService?.type === 'sqlserver'
return isSqlServer
? MATERIALS_TO_BE_DELETED_CONFIG.TABLE_NAME_SQLSERVER
: MATERIALS_TO_BE_DELETED_CONFIG.TABLE_NAME_MYSQL
return this.getDialect().quoteTableName('dbo', 'MaterialsToBeDeleted')
}
/**
@@ -81,15 +85,6 @@ export class MaterialsToBeDeletedDAO {
return this.dbService
}
/**
* Build placeholders for IN clause based on database type
*/
private buildPlaceholders(count: number, isSqlServer: boolean): string {
return isSqlServer
? Array.from({ length: count }, (_, idx) => `@p${idx}`).join(',')
: Array.from({ length: count }, () => '?').join(',')
}
// ==================== UPSERT (MERGE) ====================
/**
@@ -113,34 +108,72 @@ export class MaterialsToBeDeletedDAO {
const tableName = this.getTableName()
const code = materialCode.trim()
const manager = managerName?.trim() || null
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
if (isSqlServer) {
const sqlString = `
MERGE ${tableName} AS target
USING (VALUES (@p0, @p1)) AS source (MaterialCode, ManagerName)
ON target.MaterialCode = source.MaterialCode
WHEN MATCHED THEN UPDATE SET ManagerName = source.ManagerName
WHEN NOT MATCHED THEN INSERT (MaterialCode, ManagerName) VALUES (source.MaterialCode, source.ManagerName);
if (dbService.type === 'postgresql') {
const updateSql = `
UPDATE ${tableName}
SET ManagerName = ${dialect.param(0)}
WHERE MaterialCode = ${dialect.param(1)}
`
await trackDuration(async () => await dbService.query(sqlString, [code, manager]), {
operationName: 'MaterialsToBeDeletedDAO.upsertMaterial',
context: { tableName, operationType: 'MERGE' }
})
} else {
const sqlString = `
const updateResult = await trackDuration(
async () => await dbService.query(updateSql, [manager, code]),
{
operationName: 'MaterialsToBeDeletedDAO.upsertMaterial',
context: { tableName, operationType: 'UPSERT_UPDATE_FIRST' }
}
)
if (updateResult.result.rowCount > 0) {
return true
}
const insertSql = `
INSERT INTO ${tableName} (MaterialCode, ManagerName)
VALUES (?, ?)
ON DUPLICATE KEY UPDATE ManagerName = VALUES(ManagerName)
SELECT ${dialect.param(0)}, ${dialect.param(1)}
WHERE NOT EXISTS (
SELECT 1
FROM ${tableName}
WHERE MaterialCode = ${dialect.param(0)}
)
`
await trackDuration(async () => await dbService.query(sqlString, [code, manager]), {
operationName: 'MaterialsToBeDeletedDAO.upsertMaterial',
context: { tableName, operationType: 'INSERT' }
})
const insertResult = await trackDuration(
async () => await dbService.query(insertSql, [code, manager]),
{
operationName: 'MaterialsToBeDeletedDAO.upsertMaterial',
context: { tableName, operationType: 'UPSERT_INSERT_FALLBACK' }
}
)
if (insertResult.result.rowCount > 0) {
return true
}
const retryUpdateResult = await trackDuration(
async () => await dbService.query(updateSql, [manager, code]),
{
operationName: 'MaterialsToBeDeletedDAO.upsertMaterial',
context: { tableName, operationType: 'UPSERT_UPDATE_RETRY' }
}
)
return retryUpdateResult.result.rowCount > 0
}
const { sql: sqlString } = dialect.upsert({
table: tableName,
keyColumns: ['MaterialCode'],
allColumns: ['MaterialCode', 'ManagerName'],
startParamIndex: 0
})
await trackDuration(async () => await dbService.query(sqlString, [code, manager]), {
operationName: 'MaterialsToBeDeletedDAO.upsertMaterial',
context: { tableName, operationType: 'UPSERT' }
})
return true
} catch (error) {
log.error('Upsert material error', {
@@ -176,7 +209,7 @@ export class MaterialsToBeDeletedDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
log.info('Batch upsert started', {
tableName,
@@ -196,39 +229,12 @@ export class MaterialsToBeDeletedDAO {
}
try {
if (isSqlServer) {
const sqlString = `
MERGE ${tableName} AS target
USING (VALUES (@p0, @p1)) AS source (MaterialCode, ManagerName)
ON target.MaterialCode = source.MaterialCode
WHEN MATCHED THEN UPDATE SET ManagerName = source.ManagerName
WHEN NOT MATCHED THEN INSERT (MaterialCode, ManagerName) VALUES (source.MaterialCode, source.ManagerName);
`
await trackDuration(
async () => await dbService.query(sqlString, [materialCode, managerName || null]),
{
operationName: 'MaterialsToBeDeletedDAO.upsertBatch',
context: { tableName, operationType: 'MERGE', batchId }
}
)
const success = await this.upsertMaterial(materialCode, managerName)
if (success) {
stats.success++
} else {
const sqlString = `
INSERT INTO ${tableName} (MaterialCode, ManagerName)
VALUES (?, ?)
ON DUPLICATE KEY UPDATE ManagerName = VALUES(ManagerName)
`
await trackDuration(
async () => await dbService.query(sqlString, [materialCode, managerName || null]),
{
operationName: 'MaterialsToBeDeletedDAO.upsertBatch',
context: { tableName, operationType: 'INSERT', batchId }
}
)
stats.failed++
}
stats.success++
} catch (error) {
log.error('Error upserting material', {
tableName,
@@ -237,7 +243,6 @@ export class MaterialsToBeDeletedDAO {
materialCode,
error: error instanceof Error ? error.message : String(error)
})
stats.failed++
}
}
@@ -274,29 +279,8 @@ export class MaterialsToBeDeletedDAO {
managerName: string
): Promise<{ success: boolean; error?: string }> {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
if (isSqlServer) {
const sqlString = `
MERGE ${tableName} AS target
USING (VALUES (@p0, @p1)) AS source (MaterialCode, ManagerName)
ON target.MaterialCode = source.MaterialCode
WHEN MATCHED THEN UPDATE SET ManagerName = source.ManagerName
WHEN NOT MATCHED THEN INSERT (MaterialCode, ManagerName) VALUES (source.MaterialCode, source.ManagerName);
`
await dbService.query(sqlString, [materialCode, managerName || null])
} else {
const sqlString = `
INSERT INTO ${tableName} (MaterialCode, ManagerName)
VALUES (?, ?)
ON DUPLICATE KEY UPDATE ManagerName = VALUES(ManagerName)
`
await dbService.query(sqlString, [materialCode, managerName || null])
}
return { success: true }
const success = await this.upsertMaterial(materialCode, managerName)
return { success }
} catch (error) {
log.error('Update manager error', {
materialCode,
@@ -387,9 +371,9 @@ export class MaterialsToBeDeletedDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const placeholder = isSqlServer ? '@p0' : '?'
const placeholder = dialect.param(0)
const sqlString = `
SELECT ID, MaterialCode, ManagerName
FROM ${tableName}
@@ -462,9 +446,9 @@ export class MaterialsToBeDeletedDAO {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const code = materialCode.trim()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const placeholder = isSqlServer ? '@p0' : '?'
const placeholder = dialect.param(0)
const sqlString = `
SELECT ID, MaterialCode, ManagerName
FROM ${tableName}
@@ -509,9 +493,9 @@ export class MaterialsToBeDeletedDAO {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const code = materialCode.trim()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const placeholder = isSqlServer ? '@p0' : '?'
const placeholder = dialect.param(0)
const sqlString = `
DELETE FROM ${tableName}
WHERE MaterialCode = ${placeholder}
@@ -542,9 +526,9 @@ export class MaterialsToBeDeletedDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const placeholder = isSqlServer ? '@p0' : '?'
const placeholder = dialect.param(0)
const sqlString = `
DELETE FROM ${tableName}
WHERE ManagerName = ${placeholder}
@@ -612,7 +596,7 @@ export class MaterialsToBeDeletedDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const totalBatches = Math.ceil(materialCodes.length / batchSize)
log.info('Batch delete started', {
@@ -627,7 +611,7 @@ export class MaterialsToBeDeletedDAO {
for (let i = 0; i < materialCodes.length; i += batchSize) {
const batch = materialCodes.slice(i, i + batchSize)
const batchNumber = Math.floor(i / batchSize) + 1
const placeholders = this.buildPlaceholders(batch.length, isSqlServer)
const placeholders = dialect.params(batch.length)
const sqlString = `
DELETE FROM ${tableName}
@@ -688,9 +672,9 @@ export class MaterialsToBeDeletedDAO {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const code = materialCode.trim()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const placeholder = isSqlServer ? '@p0' : '?'
const placeholder = dialect.param(0)
const sqlString = `
SELECT COUNT(*) as count
FROM ${tableName}
@@ -749,9 +733,9 @@ export class MaterialsToBeDeletedDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const placeholder = isSqlServer ? '@p0' : '?'
const placeholder = dialect.param(0)
const sqlString = `
SELECT COUNT(*) as count
FROM ${tableName}
@@ -845,6 +829,7 @@ export class MaterialsToBeDeletedDAO {
if (this.dbService) {
await this.dbService.disconnect()
this.dbService = null
this.dialect = null
}
}
}

View File

@@ -6,7 +6,8 @@
*/
import { create, type IDatabaseService } from './index'
import { createLogger, run, getRequestId, trackDuration } from '../logger'
import { createDialect, type SqlDialect } from './dialects'
import { createLogger, getRequestId, trackDuration } from '../logger'
const log = createLogger('MaterialsTypeToBeDeletedDAO')
@@ -32,8 +33,6 @@ export interface MaterialTypeBatchRequest {
* Configuration for MaterialsTypeToBeDeleted table
*/
export const MATERIALS_TYPE_TO_BE_DELETED_CONFIG = {
TABLE_NAME_SQLSERVER: '[dbo].[MaterialsTypeToBeDeleted]',
TABLE_NAME_MYSQL: 'dbo_MaterialsTypeToBeDeleted',
COLUMNS: {
ID: 'ID',
MATERIAL_NAME: 'MaterialName',
@@ -46,15 +45,20 @@ export const MATERIALS_TYPE_TO_BE_DELETED_CONFIG = {
*/
export class MaterialsTypeToBeDeletedDAO {
private dbService: IDatabaseService | null = null
private dialect: SqlDialect | null = null
private getDialect(): SqlDialect {
if (!this.dialect) {
this.dialect = createDialect(this.dbService!.type)
}
return this.dialect
}
/**
* Get the appropriate table name based on database type
*/
private getTableName(): string {
const isSqlServer = this.dbService?.type === 'sqlserver'
return isSqlServer
? MATERIALS_TYPE_TO_BE_DELETED_CONFIG.TABLE_NAME_SQLSERVER
: MATERIALS_TYPE_TO_BE_DELETED_CONFIG.TABLE_NAME_MYSQL
return this.getDialect().quoteTableName('dbo', 'MaterialsTypeToBeDeleted')
}
/**
@@ -116,9 +120,9 @@ export class MaterialsTypeToBeDeletedDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
const placeholder = isSqlServer ? '@p0' : '?'
const placeholder = dialect.param(0)
const sqlString = `
SELECT ID, MaterialName, ManagerName
FROM ${tableName}
@@ -204,34 +208,72 @@ export class MaterialsTypeToBeDeletedDAO {
const tableName = this.getTableName()
const name = materialName.trim()
const manager = managerName?.trim() || null
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
if (isSqlServer) {
const sqlString = `
MERGE ${tableName} AS target
USING (VALUES (@p0, @p1)) AS source (MaterialName, ManagerName)
ON target.MaterialName = source.MaterialName
WHEN MATCHED THEN UPDATE SET ManagerName = source.ManagerName
WHEN NOT MATCHED THEN INSERT (MaterialName, ManagerName) VALUES (source.MaterialName, source.ManagerName);
if (dbService.type === 'postgresql') {
const updateSql = `
UPDATE ${tableName}
SET ManagerName = ${dialect.param(0)}
WHERE MaterialName = ${dialect.param(1)}
`
await trackDuration(async () => await dbService.query(sqlString, [name, manager]), {
operationName: 'MaterialsTypeToBeDeletedDAO.upsertMaterial',
context: { tableName, operationType: 'MERGE' }
})
} else {
const sqlString = `
const updateResult = await trackDuration(
async () => await dbService.query(updateSql, [manager, name]),
{
operationName: 'MaterialsTypeToBeDeletedDAO.upsertMaterial',
context: { tableName, operationType: 'UPSERT_UPDATE_FIRST' }
}
)
if (updateResult.result.rowCount > 0) {
return true
}
const insertSql = `
INSERT INTO ${tableName} (MaterialName, ManagerName)
VALUES (?, ?)
ON DUPLICATE KEY UPDATE ManagerName = VALUES(ManagerName)
SELECT ${dialect.param(0)}, ${dialect.param(1)}
WHERE NOT EXISTS (
SELECT 1
FROM ${tableName}
WHERE MaterialName = ${dialect.param(0)}
)
`
await trackDuration(async () => await dbService.query(sqlString, [name, manager]), {
operationName: 'MaterialsTypeToBeDeletedDAO.upsertMaterial',
context: { tableName, operationType: 'INSERT' }
})
const insertResult = await trackDuration(
async () => await dbService.query(insertSql, [name, manager]),
{
operationName: 'MaterialsTypeToBeDeletedDAO.upsertMaterial',
context: { tableName, operationType: 'UPSERT_INSERT_FALLBACK' }
}
)
if (insertResult.result.rowCount > 0) {
return true
}
const retryUpdateResult = await trackDuration(
async () => await dbService.query(updateSql, [manager, name]),
{
operationName: 'MaterialsTypeToBeDeletedDAO.upsertMaterial',
context: { tableName, operationType: 'UPSERT_UPDATE_RETRY' }
}
)
return retryUpdateResult.result.rowCount > 0
}
const { sql: sqlString } = dialect.upsert({
table: tableName,
keyColumns: ['MaterialName'],
allColumns: ['MaterialName', 'ManagerName'],
startParamIndex: 0
})
await trackDuration(async () => await dbService.query(sqlString, [name, manager]), {
operationName: 'MaterialsTypeToBeDeletedDAO.upsertMaterial',
context: { tableName, operationType: 'UPSERT' }
})
return true
} catch (error) {
log.error('Upsert material error', {
@@ -257,25 +299,16 @@ export class MaterialsTypeToBeDeletedDAO {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const name = materialName.trim()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
let sqlString: string
let params: (string | null)[]
if (managerName) {
const placeholder1 = isSqlServer ? '@p0' : '?'
const placeholder2 = isSqlServer ? '@p1' : '?'
sqlString = `
DELETE FROM ${tableName}
WHERE MaterialName = ${placeholder1} AND ManagerName = ${placeholder2}
`
sqlString = `DELETE FROM ${tableName} WHERE MaterialName = ${dialect.param(0)} AND ManagerName = ${dialect.param(1)}`
params = [name, managerName.trim()]
} else {
const placeholder = isSqlServer ? '@p0' : '?'
sqlString = `
DELETE FROM ${tableName}
WHERE MaterialName = ${placeholder}
`
sqlString = `DELETE FROM ${tableName} WHERE MaterialName = ${dialect.param(0)}`
params = [name]
}
@@ -314,49 +347,27 @@ export class MaterialsTypeToBeDeletedDAO {
try {
const dbService = await this.getDatabaseService()
const tableName = this.getTableName()
const isSqlServer = dbService.type === 'sqlserver'
const dialect = this.getDialect()
if (isSqlServer) {
const sqlString = `
UPDATE ${tableName}
SET MaterialName = @p0, ManagerName = @p1
WHERE MaterialName = @p2 AND ManagerName = @p3
`
const result = await trackDuration(
async () =>
await dbService.query(sqlString, [
newName.trim(),
newManager.trim(),
oldName.trim(),
oldManager.trim()
]),
{
operationName: 'MaterialsTypeToBeDeletedDAO.updateMaterial',
context: { tableName, operationType: 'UPDATE' }
}
)
return result.result.rowCount > 0
} else {
const sqlString = `
UPDATE ${tableName}
SET MaterialName = ?, ManagerName = ?
WHERE MaterialName = ? AND ManagerName = ?
`
const result = await trackDuration(
async () =>
await dbService.query(sqlString, [
newName.trim(),
newManager.trim(),
oldName.trim(),
oldManager.trim()
]),
{
operationName: 'MaterialsTypeToBeDeletedDAO.updateMaterial',
context: { tableName, operationType: 'UPDATE' }
}
)
return result.result.rowCount > 0
}
const sqlString = `
UPDATE ${tableName}
SET MaterialName = ${dialect.param(0)}, ManagerName = ${dialect.param(1)}
WHERE MaterialName = ${dialect.param(2)} AND ManagerName = ${dialect.param(3)}
`
const result = await trackDuration(
async () =>
await dbService.query(sqlString, [
newName.trim(),
newManager.trim(),
oldName.trim(),
oldManager.trim()
]),
{
operationName: 'MaterialsTypeToBeDeletedDAO.updateMaterial',
context: { tableName, operationType: 'UPDATE' }
}
)
return result.result.rowCount > 0
} catch (error) {
log.error('Update material error', {
tableName: this.getTableName(),
@@ -456,6 +467,7 @@ export class MaterialsTypeToBeDeletedDAO {
if (this.dbService) {
await this.dbService.disconnect()
this.dbService = null
this.dialect = null
}
}
}

View File

@@ -0,0 +1,645 @@
import { Pool } from 'pg'
import type {
IDatabaseService,
DatabaseType,
QueryResult,
PostgreSqlConfig
} from '../../types/database.types'
import { createLogger, trackDuration } from '../logger'
const log = createLogger('PostgreSqlService')
export type { PostgreSqlConfig } from '../../types/database.types'
/**
* SQL keywords that should NOT be double-quoted during identifier preprocessing.
* PostgreSQL lowercases unquoted identifiers, but SSMA-migrated databases
* have uppercase column names that require double-quoting to preserve case.
*
* This list covers PostgreSQL reserved words across multiple categories:
* - DML (Data Manipulation Language)
* - DDL (Data Definition Language)
* - Window functions
* - CTEs (Common Table Expressions)
* - Advanced GROUP BY clauses
* - JSON operations
* - Type system
* - Table sampling
* - Transaction control
*/
const SQL_KEYWORDS = new Set([
// ==================== DML (Data Manipulation Language) ====================
'SELECT',
'FROM',
'WHERE',
'AND',
'OR',
'NOT',
'IN',
'IS',
'NULL',
'INSERT',
'INTO',
'VALUES',
'UPDATE',
'SET',
'DELETE',
// ==================== Ordering & Limiting ====================
'ORDER',
'BY',
'ASC',
'DESC',
'LIMIT',
'OFFSET',
'FETCH',
'NEXT',
'ROWS',
'ONLY',
// ==================== Joins ====================
'JOIN',
'LEFT',
'RIGHT',
'INNER',
'OUTER',
'CROSS',
'FULL',
'ON',
'NATURAL',
'LATERAL',
// ==================== Set Operations ====================
'UNION',
'ALL',
'INTERSECT',
'EXCEPT',
// ==================== Grouping & Aggregation ====================
'GROUP',
'HAVING',
'DISTINCT',
'GROUPING',
'SETS',
'ROLLUP',
'CUBE',
'FILTER',
'WITHIN',
// ==================== Window Functions ====================
'OVER',
'PARTITION',
'WINDOW',
'RANGE',
'UNBOUNDED',
'PRECEDING',
'FOLLOWING',
'CURRENT',
'ROW',
'GROUPS',
'EXCLUDE',
'TIES',
'RANK',
'DENSE_RANK',
'ROW_NUMBER',
'NTILE',
'LAG',
'LEAD',
'FIRST_VALUE',
'LAST_VALUE',
'NTH_VALUE',
// ==================== CTE (Common Table Expressions) ====================
'WITH',
'RECURSIVE',
'MATERIALIZED',
'SEARCH',
'CYCLE',
'PATH',
'ROOT',
'SIBLINGS',
// ==================== CASE Expressions ====================
'CASE',
'WHEN',
'THEN',
'ELSE',
'END',
// ==================== DDL (Data Definition Language) ====================
'CREATE',
'ALTER',
'DROP',
'TABLE',
'INDEX',
'COLUMN',
'ADD',
'MODIFY',
'RENAME',
'TO',
'GENERATED',
'ALWAYS',
'IDENTITY',
'INCLUDE',
'TEMP',
'TEMPORARY',
'UNLOGGED',
// ==================== PostgreSQL Specific - UPSERT/MERGE ====================
'CONFLICT',
'DO',
'NOTHING',
'EXCLUDED',
'RETURNING',
'MERGE',
'USING',
'MATCHED',
'TARGET',
'SOURCE',
// ==================== Aggregate Functions ====================
'COUNT',
'SUM',
'AVG',
'MIN',
'MAX',
'EXISTS',
'COALESCE',
'NULLIF',
'CAST',
'AS',
// ==================== JSON Operations ====================
'JSON',
'JSONB',
'JSON_ARRAY',
'JSON_OBJECT',
'JSON_AGG',
'JSONB_AGG',
'JSONB_OBJECT_AGG',
// ==================== Types & Casting ====================
'DECIMAL',
'NUMERIC',
'BOOLEAN',
'CHARACTER',
'VARYING',
'PRECISION',
'REAL',
'DOUBLE',
'FLOAT',
'TEXT',
'INTEGER',
'SERIAL',
'BIGINT',
'SMALLINT',
'DATE',
'TIME',
'TIMESTAMP',
'TIMESTAMPTZ',
'TIMEZONE',
'INTERVAL',
'BIGSERIAL',
'SMALLSERIAL',
// ==================== Table Sampling ====================
'TABLESAMPLE',
'BERNOULLI',
'SYSTEM',
'REPEATABLE',
'SEED',
// ==================== Transaction Control ====================
'BEGIN',
'COMMIT',
'ROLLBACK',
'SAVEPOINT',
'WORK',
'ISOLATION',
'LEVEL',
'READ',
'WRITE',
'COMMITTED',
'REPEATABLE',
'SERIALIZABLE',
// ==================== Types & Values ====================
'TRUE',
'FALSE',
'DEFAULT',
'PRIMARY',
'KEY',
'REFERENCES',
'FOREIGN',
'CONSTRAINT',
'UNIQUE',
'CHECK',
'NULLS',
'FIRST',
'LAST',
// ==================== Scalar & String Functions ====================
'UPPER',
'LOWER',
'TRIM',
'LTRIM',
'RTRIM',
'BTRIM',
'SUBSTRING',
'CONCAT',
'LENGTH',
'CHAR_LENGTH',
'CHARACTER_LENGTH',
'REPLACE',
'POSITION',
'OVERLAY',
'LPAD',
'RPAD',
'REPEAT',
'REVERSE',
'SPLIT_PART',
'INITCAP',
'NORMALIZE',
'CHR',
'ASCII',
'FORMAT',
// ==================== Numeric Functions ====================
'ABS',
'CEIL',
'CEILING',
'FLOOR',
'ROUND',
'POWER',
'SQRT',
'MOD',
'SIGN',
'TRUNC',
// ==================== Date/Time Functions ====================
'EXTRACT',
'DATE_TRUNC',
'TO_CHAR',
'TO_DATE',
'TO_TIMESTAMP',
'TO_NUMBER',
'AGE',
// ==================== Pattern Matching ====================
'BETWEEN',
'LIKE',
'ILIKE',
'SIMILAR',
'ESCAPE',
'ANY',
'SOME',
// ==================== Functions & Procedures ====================
'AFTER',
'BEFORE',
'EACH',
'STATEMENT',
'TRIGGER',
'FUNCTION',
'PROCEDURE',
'LANGUAGE',
'SQL',
'PLPGSQL',
'RETURNS',
'CALLED',
'STRICT',
'SECURITY',
'INVOKER',
'DEFINER',
'VOLATILE',
'STABLE',
'IMMUTABLE',
'PARALLEL',
'SAFE',
'RESTRICTED',
'UNSAFE',
// ==================== Utility Commands ====================
'CONCURRENTLY',
'REINDEX',
'VACUUM',
'ANALYZE',
'EXPLAIN',
'LOCAL',
'GLOBAL',
'ORDINALITY',
'FREEZE',
'VERBOSE',
'BUFFERS',
'FORMAT',
'XML',
'YAML',
// ==================== Additional Reserved Words ====================
'IF',
'CURRENT_TIMESTAMP',
'NOW',
'GETDATE',
// ==================== Timezone Expression ====================
'AT',
'ZONE'
])
/**
* Prepare SQL for PostgreSQL execution by quoting unquoted identifiers.
*
* PostgreSQL lowercases unquoted identifiers, but SSMA-migrated tables
* have uppercase column names (e.g., "UserName", "ID") that require
* double-quoting to preserve case.
*
* This function:
* - Preserves string literals ('...')
* - Preserves already-quoted identifiers ("...")
* - Preserves parameter placeholders ($1, $2, ...)
* - Preserves SQL keywords
* - Double-quotes remaining identifiers
*/
export function prepareSql(sql: string): string {
const result: string[] = []
let i = 0
const len = sql.length
while (i < len) {
const ch = sql[i]
// Skip whitespace
if (/\s/.test(ch)) {
result.push(ch)
i++
continue
}
// Skip single-line comments (--)
if (ch === '-' && i + 1 < len && sql[i + 1] === '-') {
while (i < len && sql[i] !== '\n') {
result.push(sql[i++])
}
continue
}
// Preserve string literals ('...')
if (ch === "'") {
result.push(ch)
i++
while (i < len) {
if (sql[i] === "'") {
result.push(sql[i++])
// Handle escaped quotes ('')
if (i < len && sql[i] === "'") {
result.push(sql[i++])
} else {
break
}
} else {
result.push(sql[i++])
}
}
continue
}
// Preserve already-quoted identifiers ("...")
if (ch === '"') {
result.push(ch)
i++
while (i < len && sql[i] !== '"') {
result.push(sql[i++])
}
if (i < len) {
result.push(sql[i++])
}
continue
}
// Preserve parameter placeholders ($N)
if (ch === '$') {
result.push(ch)
i++
while (i < len && /\d/.test(sql[i])) {
result.push(sql[i++])
}
continue
}
// Preserve @param placeholders
if (ch === '@') {
result.push(ch)
i++
while (i < len && /\w/.test(sql[i])) {
result.push(sql[i++])
}
continue
}
// Preserve ? placeholders
if (ch === '?') {
result.push(ch)
i++
continue
}
// Collect word tokens (identifiers and keywords)
if (/[a-zA-Z_]/.test(ch)) {
let word = ''
while (i < len && /\w/.test(sql[i])) {
word += sql[i++]
}
// Check if it's a SQL keyword (case-insensitive)
if (SQL_KEYWORDS.has(word.toUpperCase())) {
result.push(word)
} else {
// Quote the identifier to preserve case
result.push(`"${word}"`)
}
continue
}
// Everything else (operators, punctuation, numbers): pass through
result.push(ch)
i++
}
return result.join('')
}
export class PostgreSqlService implements IDatabaseService {
/** Database type identifier */
readonly type: DatabaseType = 'postgresql'
private pool: Pool | null = null
private config: PostgreSqlConfig
constructor(config: PostgreSqlConfig) {
this.config = config
}
/**
* Connect to PostgreSQL database
*/
async connect(): Promise<void> {
if (this.pool) {
log.warn('Already connected to PostgreSQL')
throw new Error('Already connected to PostgreSQL')
}
try {
this.pool = new Pool({
host: this.config.host,
port: this.config.port,
user: this.config.user,
password: this.config.password,
database: this.config.database,
max: this.config.maxPoolSize ?? 10,
/**
* Connection timeout in milliseconds.
* Time to wait when connecting to PostgreSQL before failing.
* Prevents hanging during network issues or server overload.
*/
connectionTimeoutMillis: 10000,
/**
* PostgreSQL statement timeout in milliseconds.
* Limits execution time for individual SQL statements.
* Prevents long-running queries from blocking the connection pool.
*/
statement_timeout: 30000,
/**
* Idle connection timeout in milliseconds.
* Closes connections that have been idle for this duration.
* Frees up pool resources and prevents stale connections.
*/
idleTimeoutMillis: 30000,
/**
* Query timeout in milliseconds (pg driver level).
* Fallback protection to abort queries that exceed this duration.
* Should be longer than statement_timeout to allow PG to handle first.
*/
query_timeout: 60000
})
// Test connection
const client = await this.pool.connect()
client.release()
log.info('Connected to PostgreSQL', {
host: this.config.host,
port: this.config.port,
database: this.config.database
})
} catch (error) {
this.pool = null
log.error('Failed to connect to PostgreSQL', {
host: this.config.host,
port: this.config.port,
database: this.config.database,
error
})
throw new Error(`Failed to connect to PostgreSQL: ${(error as Error).message}`)
}
}
/**
* Disconnect from PostgreSQL database
*/
async disconnect(): Promise<void> {
if (!this.pool) {
return
}
try {
await this.pool.end()
this.pool = null
log.info('Disconnected from PostgreSQL')
} catch (error) {
log.error('Failed to disconnect from PostgreSQL', { error })
throw new Error(`Failed to disconnect from PostgreSQL: ${(error as Error).message}`)
}
}
/**
* Check if connected to PostgreSQL database
*/
isConnected(): boolean {
return this.pool !== null
}
/**
* Execute a query and return results
*/
async query(sql: string, params?: any[]): Promise<QueryResult> {
if (!this.pool) {
throw new Error('Not connected to PostgreSQL. Call connect() first.')
}
// Quote unquoted identifiers to preserve case for SSMA-migrated columns
const preparedSql = prepareSql(sql)
const sqlPreview = preparedSql.substring(0, 100)
const paramCount = params?.length ?? 0
try {
const { result: queryResult } = await trackDuration(
async () => {
const result = await this.pool!.query(preparedSql, params)
// Extract column names from fields
const columns = result.fields ? result.fields.map((field) => field.name) : []
// Result rows
const rows = (result.rows as Record<string, unknown>[]) || []
const rowCount = result.rowCount ?? rows.length
return { rows, columns, rowCount }
},
{ operationName: 'PostgreSqlService.query' }
)
log.debug('Query executed', { sqlPreview, rowCount: queryResult.rowCount, paramCount })
return queryResult
} catch (error) {
log.error('PostgreSQL query failed', { sqlPreview, paramCount, error })
throw new Error(`PostgreSQL query failed: ${(error as Error).message}`)
}
}
/**
* Execute multiple queries in a transaction
*/
async transaction(queries: { sql: string; params?: any[] }[]): Promise<void> {
if (!this.pool) {
throw new Error('Not connected to PostgreSQL. Call connect() first.')
}
const queryCount = queries.length
log.info('Transaction started', { queryCount })
const client = await this.pool.connect()
try {
await client.query('BEGIN')
for (let i = 0; i < queries.length; i++) {
const { sql, params } = queries[i]
const preparedSql = prepareSql(sql)
await client.query(preparedSql, params)
log.debug('Transaction query executed', {
index: i,
sqlPreview: preparedSql.substring(0, 100)
})
}
await client.query('COMMIT')
log.info('Transaction committed', { queryCount })
} catch (error) {
await client.query('ROLLBACK')
log.warn('Transaction rolled back', { queryCount, error })
throw new Error(`PostgreSQL transaction failed: ${(error as Error).message}`)
} finally {
client.release()
}
}
}

View File

@@ -38,6 +38,8 @@ export class SqlServerService implements IDatabaseService {
user: this.config.user,
password: this.config.password,
database: this.config.database,
requestTimeout: 60000,
connectionTimeout: 15000,
options: {
encrypt: this.config.options?.encrypt ?? false,
trustServerCertificate: this.config.options?.trustServerCertificate ?? false

File diff suppressed because it is too large Load Diff

View File

@@ -3,6 +3,7 @@ import type { ErpConfig, ErpSession } from '../../types/erp.types'
import { createLogger } from '../logger'
import { capturePageContext } from './erp-error-context'
import { attachPageDiagnostics, attachContextDiagnostics } from './page-diagnostics'
import { capturePageState } from './page-state'
const log = createLogger('ErpAuthService')
@@ -64,6 +65,10 @@ export class ErpAuthService {
await page.goto(loginUrl)
log.debug('已导航到登录页面')
log.info('[PAGE_STATE] 页面状态快照', {
step: 'auth.login_page_loaded',
...(await capturePageState(page, context))
})
// Wait for page to load
await page.waitForLoadState('domcontentloaded', { timeout: PAGE_LOAD_TIMEOUT })
@@ -82,7 +87,14 @@ export class ErpAuthService {
if (!contentFrame) {
log.error('Failed to access forwardFrame content frame', {
...(await capturePageContext(page))
...(await capturePageContext(
page,
undefined,
'auth.forwardFrame',
undefined,
undefined,
'auth_forward_frame'
))
})
throw new Error('Failed to access forwardFrame content frame')
}
@@ -96,7 +108,14 @@ export class ErpAuthService {
} catch (e) {
log.error('Failed to find username input', {
error: e instanceof Error ? e.message : String(e),
...(await capturePageContext(page, undefined, 'login.username'))
...(await capturePageContext(
page,
undefined,
'login.username',
undefined,
undefined,
'login_username'
))
})
throw new Error(`Failed to find username input: ${e}`)
}
@@ -107,7 +126,14 @@ export class ErpAuthService {
} catch (e) {
log.error('Failed to find password input', {
error: e instanceof Error ? e.message : String(e),
...(await capturePageContext(page, undefined, 'login.password'))
...(await capturePageContext(
page,
undefined,
'login.password',
undefined,
undefined,
'login_password'
))
})
throw new Error(`Failed to find password input: ${e}`)
}
@@ -118,7 +144,14 @@ export class ErpAuthService {
} catch (e) {
log.error('Failed to click login button', {
error: e instanceof Error ? e.message : String(e),
...(await capturePageContext(page, undefined, 'login.button'))
...(await capturePageContext(
page,
undefined,
'login.button',
undefined,
undefined,
'login_button'
))
})
throw new Error(`Failed to click login button: ${e}`)
}
@@ -128,6 +161,10 @@ export class ErpAuthService {
})
await this.waitForLoginResult(mainFrame as unknown as import('playwright').Frame)
log.info('[PAGE_STATE] 页面状态快照', {
step: 'auth.login_result_confirmed',
...(await capturePageState(page, context, { includeFrameHierarchy: true }))
})
// Create session with mainFrame (Python returns main_frame as part of login result)
this.session = {

View File

@@ -16,6 +16,9 @@ export interface ErpErrorContext {
targetSelector?: string
step?: string
screenshotPath?: string
orderId?: string
materialCode?: string
errorStage?: string
}
/**
@@ -69,11 +72,18 @@ async function captureScreenshot(page: Page, step?: string): Promise<string | un
*
* @param page - The Playwright page to inspect
* @param targetSelector - Optional selector that was being targeted
* @param step - Optional step name for context
* @param orderId - Optional order ID for error correlation
* @param materialCode - Optional material code for error correlation
* @param stage - Optional stage name (defaults to 'unknown')
*/
export async function capturePageContext(
page: Page,
targetSelector?: string,
step?: string
step?: string,
orderId?: string,
materialCode?: string,
stage: string = 'unknown'
): Promise<ErpErrorContext> {
const ctx: ErpErrorContext = {}
@@ -98,6 +108,16 @@ export async function capturePageContext(
ctx.step = step
}
if (orderId) {
ctx.orderId = orderId
}
if (materialCode) {
ctx.materialCode = materialCode
}
ctx.errorStage = stage
ctx.screenshotPath = await captureScreenshot(page, step)
return ctx

View File

@@ -116,7 +116,14 @@ export class ExtractorCore {
if (!fFrame) {
log.error('Failed to access popup forward frame', {
...(await capturePageContext(popupPage, undefined, 'navigate.forwardFrame'))
...(await capturePageContext(
popupPage,
undefined,
'navigate.forwardFrame',
undefined,
undefined,
'navigate_forward_frame'
))
})
throw new Error('Failed to access popup forward frame')
}
@@ -128,7 +135,14 @@ export class ExtractorCore {
if (!workFrame) {
log.error('Failed to access inner work frame', {
...(await capturePageContext(popupPage, undefined, 'navigate.innerFrame'))
...(await capturePageContext(
popupPage,
undefined,
'navigate.innerFrame',
undefined,
undefined,
'navigate_inner_frame'
))
})
throw new Error('Failed to access inner work frame')
}

View File

@@ -10,6 +10,7 @@ import type {
LogLevel
} from '../../types/extractor.types'
import { DataImportService } from '../database/data-importer'
import type { MaterialPlanRecord } from '../database/discrete-material-plan-dao'
import { createLogger, withRequestContext, getRequestId } from '../logger'
import { trackDuration } from '../logger/performance-monitor'
@@ -112,16 +113,18 @@ export class ExtractorService {
// Always clean up temporary files regardless of merge success
await this.cleanupTempFiles(result.downloadedFiles, input.orderNumbers)
// Auto-import to database if merge was successful
if (result.mergedFile) {
// Auto-import parsed records directly. The merged Excel file is an archive artifact,
// not the source for persistence.
if (mergeResult.records.length > 0) {
const importProgress = (1 + totalBatches + 1) * progressPerPoint
input.onProgress?.('正在写入数据库...', importProgress, {
phase: 'importing',
totalBatches
})
const importResult = await this.importToDatabaseWithLogging(
result.mergedFile,
input.onLog
const importResult = await this.importRecordsToDatabaseWithLogging(
mergeResult.records,
input.onLog,
result.mergedFile
)
result.importResult = importResult
@@ -168,9 +171,10 @@ export class ExtractorService {
recordCount: number
error?: string
orderRecordCounts: Array<{ orderNumber: string; recordCount: number }>
records: MaterialPlanRecord[]
}> {
if (filePaths.length === 0) {
return { mergedFile: null, recordCount: 0, orderRecordCounts: [] }
return { mergedFile: null, recordCount: 0, orderRecordCounts: [], records: [] }
}
log.info('Starting merge', { fileCount: filePaths.length, orderCount: orderNumbers.length })
@@ -219,10 +223,11 @@ export class ExtractorService {
}
log.info('Merge summary', { orderCount: allOrders.length, recordCount })
const records = this.buildMaterialPlanRecords(allOrders)
if (recordCount === 0) {
log.warn('No records found in any downloaded files', { orderNumbers })
return { mergedFile: null, recordCount: 0, orderRecordCounts }
return { mergedFile: null, recordCount: 0, orderRecordCounts, records }
}
// Generate output filename with timestamp
@@ -238,7 +243,7 @@ export class ExtractorService {
log.info('Saving merged file', { outputPath })
await this.saveMergedOrders(allOrders, outputPath)
log.info('Merged file saved successfully', { recordCount })
return { mergedFile: outputPath, recordCount, orderRecordCounts }
return { mergedFile: outputPath, recordCount, orderRecordCounts, records }
} catch (error) {
const errorMsg = error instanceof Error ? error.message : String(error)
const errorStack = error instanceof Error ? error.stack : ''
@@ -253,6 +258,7 @@ export class ExtractorService {
mergedFile: null,
recordCount,
orderRecordCounts,
records,
error: `保存合并文件失败:${errorMsg}`
}
}
@@ -370,6 +376,78 @@ export class ExtractorService {
log.debug('File saved successfully', { outputPath })
}
private buildMaterialPlanRecords(
orders: Array<{ orderInfo: any; materials: any[] }>
): MaterialPlanRecord[] {
const records: MaterialPlanRecord[] = []
for (const order of orders) {
const { orderInfo, materials } = order
for (const material of materials) {
records.push({
factory: this.toText(orderInfo.factory),
materialStatus: this.toText(orderInfo.materialStatus),
planNumber: this.toText(orderInfo.planNumber),
sourceNumber: this.toText(orderInfo.productionOrder),
materialType: this.toText(orderInfo.materialType),
productCode: this.toText(orderInfo.productCode),
productName: this.toText(orderInfo.productName),
productPlanQuantity: this.toNumber(orderInfo.plannedQuantity),
productUnit: this.toText(orderInfo.unit),
useDepartment: this.toText(orderInfo.department),
remark: this.toText(orderInfo.remark),
creator: this.toText(orderInfo.creator),
createDate: this.toDate(orderInfo.createDate),
approver: this.toText(orderInfo.approver),
approveDate: this.toDate(orderInfo.approveDate),
sequenceNumber: this.toNumber(material.sequence),
materialCode: this.toText(material.materialCode),
materialName: this.toText(material.materialName),
specification: this.toText(material.specification),
model: this.toText(material.model),
drawingNumber: this.toText(material.drawingNumber),
materialQuality: this.toText(material.material),
planQuantity: this.toNumber(material.quantity),
unit: this.toText(material.unit),
requiredDate: this.toDate(material.requiredDate),
warehouse: this.toText(material.warehouse),
unitUsage: this.toNumber(material.unitUsage),
cumulativeOutputQuantity: this.toNumber(material.cumulativeOutboundQty),
bomVersion: ''
})
}
}
return records
}
private toText(value: unknown): string {
if (value === null || value === undefined) {
return ''
}
return String(value).trim()
}
private toNumber(value: unknown): number {
if (value === null || value === undefined || value === '') {
return 0
}
const parsed = Number(value)
return Number.isFinite(parsed) ? parsed : 0
}
private toDate(value: unknown): Date {
if (value instanceof Date) {
return value
}
if (value === null || value === undefined || value === '') {
return new Date(NaN)
}
const parsed = new Date(String(value))
return parsed
}
/**
* Clean up temporary batch files after merging
* @param filePaths - Array of temporary file paths to delete
@@ -458,4 +536,70 @@ export class ExtractorService {
return trackedResult.result
}
private async importRecordsToDatabaseWithLogging(
records: MaterialPlanRecord[],
onLog?: (level: LogLevel, message: string) => void,
archiveFilePath?: string | null
): Promise<ImportResult> {
log.info('Starting database import from parsed records', {
recordCount: records.length,
archiveFilePath
})
onLog?.('info', `开始导入数据到数据库...`)
const trackedResult = await trackDuration(
async () => {
const importService = new DataImportService()
try {
const result = await importService.importFromRecords(records, 1000)
log.info('Import completed', {
success: result.success,
recordsRead: result.recordsRead,
recordsDeleted: result.recordsDeleted,
recordsImported: result.recordsImported
})
if (result.success) {
onLog?.(
'success',
`导入完成:读取 ${result.recordsRead} 条,删除 ${result.recordsDeleted} 条,导入 ${result.recordsImported}`
)
} else if (result.errors.length > 0) {
result.errors.forEach((err) => onLog?.('error', err))
}
return result
} catch (error) {
const errorMsg = error instanceof Error ? error.message : String(error)
log.error('Import failed', {
error: errorMsg,
archiveFilePath,
downloadDir: this.downloadDir
})
onLog?.('error', `导入失败:${errorMsg}`)
return {
success: false,
recordsRead: 0,
recordsDeleted: 0,
recordsImported: 0,
uniqueSourceNumbers: 0,
errors: [errorMsg]
}
}
},
{
operationName: 'Database Import',
context: {
recordCount: records.length,
archiveFilePath
}
}
)
return trackedResult.result
}
}

View File

@@ -32,6 +32,8 @@ const PRODUCTION_ID_PATTERN = /^\d{2}[A-Z]\d{1,6}$/i
*/
const ORDER_NUMBER_PATTERN = /^SC\d{14}$/i
const RESOLUTION_QUERY_BATCH_SIZE = 1000
/**
* Database table and field names
* Loaded from config.yaml via ConfigManager
@@ -40,7 +42,7 @@ export function getDbConfig() {
const configManager = ConfigManager.getInstance()
const config = configManager.getConfig()
return {
TABLE_NAME: config.orderResolution.tableName || 'productionContractData_26 年压力表合同数据',
TABLE_NAME: config.orderResolution.tableName || 'ERPAuto.vw_productionContractData',
FIELD_PRODUCTION_ID: config.orderResolution.productionIdField || '总排号',
FIELD_ORDER_NUMBER: config.orderResolution.orderNumberField || '生产订单号'
}
@@ -56,26 +58,38 @@ export class OrderNumberResolver {
this.dbService = dbService
}
private chunk<T>(items: T[], size: number): T[][] {
const chunks: T[][] = []
for (let index = 0; index < items.length; index += size) {
chunks.push(items.slice(index, index + size))
}
return chunks
}
/**
* Get table name based on database type
* Converts MySQL schema_tablename format to SQL Server [schema].[tablename] format
* e.g., productionContractData_26年压力表合同数据 -> [productionContractData].[26年压力表合同数据]
* dbo_MaterialsToBeDeleted -> [dbo].[MaterialsToBeDeleted]
* Converts schema.tablename format to database-specific quoting:
* - SQL Server: [schema].[tablename]
* - PostgreSQL: "schema"."tablename"
* e.g., ERPAuto.vw_productionContractData ->
* SQL Server: [ERPAuto].[vw_productionContractData]
* PostgreSQL: "ERPAuto"."vw_productionContractData"
*/
private getTableName(tableName: string): string {
if (this.dbService.type === 'sqlserver') {
// Find the FIRST underscore to split schema and table name
// This handles patterns like: schema_tablename
const firstUnderscoreIndex = tableName.indexOf('_')
if (firstUnderscoreIndex > 0) {
const schema = tableName.substring(0, firstUnderscoreIndex)
const actualTableName = tableName.substring(firstUnderscoreIndex + 1)
const dotIndex = tableName.indexOf('.')
if (dotIndex > 0) {
const schema = tableName.substring(0, dotIndex)
const actualTableName = tableName.substring(dotIndex + 1)
if (this.dbService.type === 'sqlserver') {
return `[${schema}].[${actualTableName}]`
}
// If no underscore found, default to dbo schema
return `"${schema}"."${actualTableName}"`
}
// No dot found — use default schema
if (this.dbService.type === 'sqlserver') {
return `[dbo].[${tableName}]`
}
return tableName
return `"public"."${tableName}"`
}
/**
@@ -107,6 +121,12 @@ export class OrderNumberResolver {
// 使用 COLLATE 指定不区分大小写的排序规则
sql = `SELECT TOP 1 [${dbConfig.FIELD_ORDER_NUMBER}] FROM ${tableName} WHERE [${dbConfig.FIELD_PRODUCTION_ID}] COLLATE SQL_Latin1_General_CP1_CI_AS = @p0`
params = [productionId]
} else if (this.dbService.type === 'postgresql') {
// PostgreSQL: 使用双引号保护中文标识符UPPER 实现不区分大小写
// prepareSql() 会保留已双引号包裹的标识符
// 注意getTableName() 已返回带双引号的表名,不应再加引号
sql = `SELECT DISTINCT "${dbConfig.FIELD_PRODUCTION_ID}", "${dbConfig.FIELD_ORDER_NUMBER}" FROM ${tableName} WHERE UPPER("${dbConfig.FIELD_PRODUCTION_ID}") = UPPER($1) LIMIT 1`
params = [productionId]
} else {
// MySQL 默认不区分大小写,但显式使用 UPPER 确保一致性
sql = `SELECT \`${dbConfig.FIELD_ORDER_NUMBER}\` FROM \`${tableName}\` WHERE UPPER(\`${dbConfig.FIELD_PRODUCTION_ID}\`) = UPPER(?) LIMIT 1`
@@ -146,31 +166,36 @@ export class OrderNumberResolver {
// P1: Deduplicate input productionIds to avoid redundant queries
const uniqueProductionIds = [...new Set(productionIds)]
// Use parameterized query to prevent SQL injection
const placeholders = uniqueProductionIds.map((_, i) => `@p${i}`).join(', ')
const params = uniqueProductionIds
let sql: string
if (this.dbService.type === 'sqlserver') {
// P0: Use DISTINCT to prevent duplicates from one-to-many relationships
// 使用 COLLATE 指定不区分大小写的排序规则
sql = `SELECT DISTINCT [${dbConfig.FIELD_PRODUCTION_ID}], [${dbConfig.FIELD_ORDER_NUMBER}] FROM ${tableName} WHERE [${dbConfig.FIELD_PRODUCTION_ID}] COLLATE SQL_Latin1_General_CP1_CI_AS IN (${placeholders})`
} else {
const idPlaceholders = uniqueProductionIds.map(() => 'UPPER(?)').join(', ')
// P0: Use DISTINCT to prevent duplicates from one-to-many relationships
// MySQL: 使用 UPPER 确保不区分大小写
sql = `SELECT DISTINCT \`${dbConfig.FIELD_PRODUCTION_ID}\`, \`${dbConfig.FIELD_ORDER_NUMBER}\` FROM \`${tableName}\` WHERE UPPER(\`${dbConfig.FIELD_PRODUCTION_ID}\`) IN (${idPlaceholders})`
}
const result = await this.dbService.query(sql, params)
const mappings = new Map<string, string>()
for (const row of result.rows) {
const keys = Object.keys(row)
const prodId = row[keys[0]] as string
const orderNum = row[keys[1]] as string
if (prodId && orderNum) {
mappings.set(prodId, orderNum)
const batches = this.chunk(uniqueProductionIds, RESOLUTION_QUERY_BATCH_SIZE)
for (const batch of batches) {
let sql: string
const params = batch
if (this.dbService.type === 'sqlserver') {
const placeholders = batch.map((_, i) => `@p${i}`).join(', ')
// P0: Use DISTINCT to prevent duplicates from one-to-many relationships.
sql = `SELECT DISTINCT [${dbConfig.FIELD_PRODUCTION_ID}], [${dbConfig.FIELD_ORDER_NUMBER}] FROM ${tableName} WHERE [${dbConfig.FIELD_PRODUCTION_ID}] COLLATE SQL_Latin1_General_CP1_CI_AS IN (${placeholders})`
} else if (this.dbService.type === 'postgresql') {
// PostgreSQL: 使用双引号保护中文标识符UPPER 实现不区分大小写。
const pgPlaceholders = batch.map((_, i) => `UPPER($${i + 1})`).join(', ')
sql = `SELECT DISTINCT "${dbConfig.FIELD_PRODUCTION_ID}", "${dbConfig.FIELD_ORDER_NUMBER}" FROM ${tableName} WHERE UPPER("${dbConfig.FIELD_PRODUCTION_ID}") IN (${pgPlaceholders})`
} else {
const idPlaceholders = batch.map(() => 'UPPER(?)').join(', ')
// MySQL: 使用 UPPER 确保不区分大小写。
sql = `SELECT DISTINCT \`${dbConfig.FIELD_PRODUCTION_ID}\`, \`${dbConfig.FIELD_ORDER_NUMBER}\` FROM \`${tableName}\` WHERE UPPER(\`${dbConfig.FIELD_PRODUCTION_ID}\`) IN (${idPlaceholders})`
}
const result = await this.dbService.query(sql, params)
for (const row of result.rows) {
const keys = Object.keys(row)
const prodId = row[keys[0]] as string
const orderNum = row[keys[1]] as string
if (prodId && orderNum) {
mappings.set(prodId, orderNum)
}
}
}
@@ -220,13 +245,14 @@ export class OrderNumberResolver {
// Build results while preserving original input order
// Note: Multiple productionIDs mapping to the same order number is VALID (not an error)
const results: OrderMapping[] = []
const processedInputs = new Set<string>()
for (const input of inputs) {
// Skip if this exact input was already processed
const alreadyProcessed = results.some((r) => r.input === input)
if (alreadyProcessed) {
if (processedInputs.has(input)) {
continue
}
processedInputs.add(input)
const mapping: OrderMapping = { input, resolved: false }

View File

@@ -0,0 +1,214 @@
import type { BrowserContext, Page } from 'playwright'
export type ErpPageKind = 'login' | 'home' | 'query' | 'detail' | 'cas_login' | 'unknown'
export interface CapturePageStateOptions {
includeFrameHierarchy?: boolean
includeBodyTextPreview?: boolean
bodyTextPreviewLength?: number
}
export interface ErpPageState {
pageUrl?: string
pageTitle?: string
pageKind: ErpPageKind
hasForwardFrame: boolean
hasMainIframe: boolean
hasLoginForm: boolean
hasWorkbenchMarker: boolean
hasQueryMarker: boolean
hasDetailHeader: boolean
isCasLoginRedirect: boolean
frameCount?: number
popupCount?: number
visibleMarkers?: string[]
frameHierarchy?: Array<{ name: string; url: string }>
bodyTextPreview?: string
}
async function safePageUrl(page: Page): Promise<string | undefined> {
try {
return page.url()
} catch {
return undefined
}
}
async function safePageTitle(page: Page): Promise<string | undefined> {
try {
const title = await page.title()
return title.slice(0, 200)
} catch {
return undefined
}
}
async function safeFrameCount(page: Page): Promise<number | undefined> {
try {
return page.frames().length
} catch {
return undefined
}
}
async function safePopupCount(context?: BrowserContext): Promise<number | undefined> {
try {
return context?.pages().length
} catch {
return undefined
}
}
async function safeLocatorExists(locator: ReturnType<Page['locator']>): Promise<boolean> {
try {
return (await locator.count()) > 0
} catch {
return false
}
}
async function safeBodyPreview(page: Page, maxLength: number): Promise<string | undefined> {
try {
const text = await page.locator('body').innerText({ timeout: 1000 })
return text.replace(/\s+/g, ' ').trim().slice(0, maxLength)
} catch {
return undefined
}
}
export async function capturePageState(
page: Page,
context?: BrowserContext,
options: CapturePageStateOptions = {}
): Promise<ErpPageState> {
const pageUrl = await safePageUrl(page)
const pageTitle = await safePageTitle(page)
const isCasLoginRedirect = !!pageUrl?.includes('euc.yonyoucloud.com/cas/login')
let hasForwardFrame = false
let hasMainIframe = false
let hasLoginForm = false
let hasWorkbenchMarker = false
let hasQueryMarker = false
let hasDetailHeader = false
try {
hasForwardFrame = await safeLocatorExists(page.locator('#forwardFrame'))
} catch {
hasForwardFrame = false
}
let forwardFrame: Awaited<ReturnType<ReturnType<Page['locator']>['contentFrame']>> | null = null
if (hasForwardFrame) {
try {
forwardFrame = await page.locator('#forwardFrame').contentFrame()
} catch {
forwardFrame = null
}
}
if (forwardFrame) {
try {
hasMainIframe = (await forwardFrame.locator('#mainiframe').count()) > 0
} catch {
hasMainIframe = false
}
try {
hasWorkbenchMarker = (await forwardFrame.locator('.nc-workbench-icon').count()) > 0
} catch {
hasWorkbenchMarker = false
}
try {
hasLoginForm =
(await forwardFrame.getByRole('textbox', { name: '用户名' }).count()) > 0 ||
(await forwardFrame.getByRole('textbox', { name: '密码' }).count()) > 0
} catch {
hasLoginForm = false
}
}
let innerFrame: Awaited<
ReturnType<ReturnType<NonNullable<typeof forwardFrame>['locator']>['contentFrame']>
> | null = null
if (forwardFrame && hasMainIframe) {
try {
innerFrame = await forwardFrame.locator('#mainiframe').contentFrame()
} catch {
innerFrame = null
}
}
if (innerFrame) {
try {
hasQueryMarker =
(await innerFrame.getByText('订单号查询').count()) > 0 ||
(await innerFrame.locator('#rc_select_0').count()) > 0
} catch {
hasQueryMarker = false
}
try {
hasDetailHeader = (await innerFrame.getByText(/^离散备料计划维护:/).count()) > 0
} catch {
hasDetailHeader = false
}
}
if (!hasLoginForm) {
try {
hasLoginForm =
(await page.getByRole('textbox', { name: '用户名' }).count()) > 0 ||
(await page.getByRole('textbox', { name: '密码' }).count()) > 0
} catch {
hasLoginForm = false
}
}
const visibleMarkers: string[] = []
if (isCasLoginRedirect) visibleMarkers.push('cas_login_url')
if (hasLoginForm) visibleMarkers.push('login_form')
if (hasWorkbenchMarker) visibleMarkers.push('workbench_icon')
if (hasQueryMarker) visibleMarkers.push('query_marker')
if (hasDetailHeader) visibleMarkers.push('detail_header')
if (hasForwardFrame) visibleMarkers.push('forwardFrame')
if (hasMainIframe) visibleMarkers.push('mainiframe')
let pageKind: ErpPageKind = 'unknown'
if (isCasLoginRedirect) pageKind = 'cas_login'
else if (hasLoginForm) pageKind = 'login'
else if (hasDetailHeader) pageKind = 'detail'
else if (hasQueryMarker) pageKind = 'query'
else if (hasWorkbenchMarker) pageKind = 'home'
const state: ErpPageState = {
pageUrl,
pageTitle,
pageKind,
hasForwardFrame,
hasMainIframe,
hasLoginForm,
hasWorkbenchMarker,
hasQueryMarker,
hasDetailHeader,
isCasLoginRedirect,
frameCount: await safeFrameCount(page),
popupCount: await safePopupCount(context),
visibleMarkers
}
if (options.includeFrameHierarchy) {
try {
state.frameHierarchy = page.frames().map((frame) => ({ name: frame.name(), url: frame.url() }))
} catch {
state.frameHierarchy = undefined
}
}
if (options.includeBodyTextPreview) {
state.bodyTextPreview = await safeBodyPreview(page, options.bodyTextPreviewLength ?? 500)
}
return state
}

View File

@@ -3,6 +3,8 @@ import path from 'path'
import { app } from 'electron'
import fs from 'fs'
import { createLogger } from '../logger'
import { logAuditWithCurrentUser } from '../logger/audit-logger'
import { AuditAction, AuditStatus } from '../../types/audit.types'
import type { ExportResultItem, ExportResultResponse } from '../../types/cleaner.types'
const log = createLogger('ResultExporter')
@@ -37,8 +39,8 @@ export class ResultExporter {
* @returns Export result with file path or error
*/
async exportValidationResults(items: ExportResultItem[]): Promise<ExportResultResponse> {
const filePath = path.join(this.exportDir, this.fileName)
try {
const filePath = path.join(this.exportDir, this.fileName)
log.info('Exporting validation results', { count: items.length, path: filePath })
const workbook = new ExcelJS.Workbook()
@@ -97,6 +99,12 @@ export class ResultExporter {
await workbook.xlsx.writeFile(filePath)
log.info('Export completed', { path: filePath, rows: items.length })
// Audit log: RESULT_EXPORT success
logAuditWithCurrentUser(AuditAction.RESULT_EXPORT, 'VALIDATION_RESULT', AuditStatus.SUCCESS, {
itemCount: items.length,
filePath
})
return {
success: true,
filePath
@@ -104,6 +112,14 @@ export class ResultExporter {
} catch (error) {
const errorMessage = error instanceof Error ? error.message : String(error)
log.error('Export failed', { error: errorMessage })
// Audit log: RESULT_EXPORT failure
logAuditWithCurrentUser(AuditAction.RESULT_EXPORT, 'VALIDATION_RESULT', AuditStatus.FAILURE, {
itemCount: items.length,
filePath,
error: errorMessage
})
return {
success: false,
error: errorMessage

View File

@@ -6,33 +6,12 @@
import winston from 'winston'
import DailyRotateFile from 'winston-daily-rotate-file'
import path from 'path'
import { hostname } from 'os'
import { app } from 'electron'
import { getLogDir } from './shared'
/**
* Audit log entry structure
* All 8 required fields for comprehensive audit tracking
*/
export interface AuditEntry {
/** ISO 8601 timestamp of the audit event */
timestamp: string
/** The action that was performed (e.g., 'LOGIN', 'EXTRACT', 'DELETE') */
action: string
/** User ID who performed the action */
userId: string
/** Username of the user who performed the action */
username: string
/** Computer name from which the action was performed */
computerName: string
/** Application version when the action was performed */
appVersion: string
/** The resource that was affected (e.g., table name, file path) */
resource: string
/** Status of the action: 'success' | 'failure' | 'partial' */
status: 'success' | 'failure' | 'partial'
/** Additional metadata about the audit event */
metadata: Record<string, unknown>
}
import { SessionManager } from '../user/session-manager'
import type { AuditEntry } from '../../types/audit.types'
import { AuditAction, AuditStatus } from '../../types/audit.types'
/**
* JSONL formatter - outputs one JSON object per line
@@ -91,31 +70,58 @@ export function applyAuditConfig(retentionDays: number): void {
* @param details - Additional details including username, computerName, resource, status, and optional metadata
*/
export function logAudit(
action: string,
action: AuditAction,
userId: string,
details: {
username: string
computerName: string
resource: string
status: 'success' | 'failure' | 'partial'
status: AuditStatus
metadata?: Record<string, unknown>
}
): void {
const entry: AuditEntry = {
timestamp: new Date().toISOString(),
action,
userId,
username: details.username,
computerName: details.computerName,
appVersion: app.getVersion(),
resource: details.resource,
status: details.status,
metadata: details.metadata || {}
}
try {
const entry: AuditEntry = {
timestamp: new Date().toISOString(),
action,
userId,
username: details.username,
computerName: details.computerName,
appVersion: app.getVersion(),
resource: details.resource,
status: details.status,
metadata: details.metadata ?? {}
}
// Write as JSONL - one JSON object per line
// Using info level with the entry stringified as the message
auditLogger.info(JSON.stringify(entry))
// Write as JSONL - one JSON object per line
// Using info level with the entry stringified as the message
auditLogger.info(JSON.stringify(entry))
} catch (error) {
console.error('Audit logging failed:', error)
}
}
/** Cached hostname — invariant for the app lifecycle */
export const cachedHostname = hostname()
/**
* Audit log shortcut that auto-resolves the current user context.
* Falls back to 'anonymous' if no user is logged in, so the record is always written.
*/
export function logAuditWithCurrentUser(
action: AuditAction,
resource: string,
status: AuditStatus,
metadata?: Record<string, unknown>
): void {
const user = SessionManager.getInstance().getUserInfo()
logAudit(action, user ? String(user.id) : 'anonymous', {
username: user?.username ?? 'anonymous',
computerName: cachedHostname,
resource,
status,
metadata: metadata ?? {}
})
}
/**

View File

@@ -119,7 +119,6 @@ export async function trackDuration<T>(
return { result, durationMs, isSlow }
} catch (error) {
const durationMs = performance.now() - startTime
const isSlow = durationMs > slowThresholdMs
// Log the error with duration
logger.error(`${message} failed after ${durationMs.toFixed(2)}ms`, {

View File

@@ -1,299 +0,0 @@
import path from 'path'
import fs from 'fs'
import { app } from 'electron'
import { createLogger } from '../logger'
import type { CleanerResult, OrderCleanDetail, SkippedMaterial } from '../../types/cleaner.types'
const log = createLogger('CleanerReportGenerator')
export interface ReportOptions {
dryRun: boolean
username: string
startTime: number
endTime: number
}
interface OrderStats {
successCount: number
failureCount: number
successRate: number
}
export class CleanerReportGenerator {
private readonly reportDir: string
constructor() {
const logDir = app.isReady() ? app.getPath('logs') : path.join(process.cwd(), 'logs')
this.reportDir = path.join(logDir, 'reports')
this.ensureReportDir()
}
private ensureReportDir(): void {
if (!fs.existsSync(this.reportDir)) {
fs.mkdirSync(this.reportDir, { recursive: true })
log.info('Created report directory', { path: this.reportDir })
}
}
async generateReport(result: CleanerResult, options: ReportOptions): Promise<string> {
const filePath = this.getReportFilePath()
log.info('Generating cleaner report', { path: filePath })
const stats = this.calculateOrderStats(result)
const content = this.buildReportContent(result, options, stats)
await fs.promises.writeFile(filePath, content, 'utf-8')
log.info('Report generated successfully', { path: filePath })
return filePath
}
private getReportFilePath(): string {
const now = new Date()
const timestamp = now.toISOString().replace(/[:.]/g, '-').slice(0, -5).replace('T', '-')
const fileName = `cleaner-report-${timestamp}.md`
return path.join(this.reportDir, fileName)
}
private calculateOrderStats(result: CleanerResult): OrderStats {
const totalOrders = result.details.length
const failureCount = result.details.filter((d) => d.errors.length > 0).length
const successCount = totalOrders - failureCount
const successRate = totalOrders > 0 ? (successCount / totalOrders) * 100 : 0
return {
successCount,
failureCount,
successRate
}
}
private buildReportContent(
result: CleanerResult,
options: ReportOptions,
stats: OrderStats
): string {
const lines: string[] = []
lines.push('# ERP 物料清理执行报告')
lines.push('')
lines.push('## 执行摘要')
lines.push('')
lines.push('| 项目 | 值 |')
lines.push('| -------------- | --------------------------------- |')
lines.push(`| **执行时间** | \`${this.formatDateTime(options.endTime)}\``)
lines.push(`| **执行模式** | \`${options.dryRun ? '模拟运行 (Dry Run)' : '正式执行'}\``)
lines.push(`| **操作用户** | \`${options.username}\``)
lines.push(`| **处理订单数** | \`${result.ordersProcessed}\``)
lines.push(`| **删除物料数** | \`${result.materialsDeleted}\``)
lines.push(`| **跳过物料数** | \`${result.materialsSkipped}\``)
lines.push(`| **错误数量** | \`${result.errors.length}\``)
if (result.retriedOrders > 0) {
lines.push(`| **重试订单数** | \`${result.retriedOrders}\``)
lines.push(`| **成功重试数** | \`${result.successfulRetries}\``)
}
lines.push(`| **执行耗时** | \`${this.formatDuration(options.startTime, options.endTime)}\``)
lines.push('')
lines.push('---')
lines.push('')
lines.push('## 执行状态')
lines.push('')
lines.push('| 状态 | 数量 | 百分比 |')
lines.push('| ----------- | ---- | ------ |')
lines.push(`| ✅ 成功订单 | ${stats.successCount} | ${stats.successRate.toFixed(1)}% |`)
lines.push(`| ❌ 失败订单 | ${stats.failureCount} | ${(100 - stats.successRate).toFixed(1)}% |`)
if (result.retriedOrders > 0) {
const retrySuccessRate =
result.retriedOrders > 0 ? (result.successfulRetries / result.retriedOrders) * 100 : 0
lines.push(`| 🔄 重试订单 | ${result.retriedOrders} | 100% |`)
lines.push(`| ✅ 成功重试 | ${result.successfulRetries} | ${retrySuccessRate.toFixed(1)}% |`)
}
lines.push('')
lines.push('---')
lines.push('')
lines.push('## 订单处理详情')
lines.push('')
lines.push('| # | 订单号 | 删除数 | 跳过数 | 状态 | 错误信息 |')
lines.push('| --- | -------- | ------ | ------ | ------- | ------------------------ |')
result.details.forEach((detail, index) => {
const orderNum = index + 1
let status = detail.errors.length > 0 ? '❌ 失败' : '✅ 成功'
// Override status if retry was successful
if (detail.retrySuccess) {
status = '✅ 重试成功'
} else if (detail.retryCount > 0 && !detail.retrySuccess) {
status = '❌ 重试失败'
}
const errorMsg = detail.errors.length > 0 ? detail.errors[0] : '-'
const retryInfo = detail.retryCount > 0 ? ` [重试${detail.retryCount}次]` : ''
lines.push(
`| ${orderNum} | \`${detail.orderNumber}\` | ${detail.materialsDeleted} | ${detail.materialsSkipped} | ${status}${retryInfo} | \`${errorMsg}\` |`
)
})
lines.push('')
lines.push('---')
lines.push('')
const allSkippedMaterials = this.collectAllSkippedMaterials(result.details)
if (allSkippedMaterials.length > 0) {
lines.push('## 跳过的物料原因说明')
lines.push('')
lines.push('| 订单号 | 物料代码 | 物料名称 | 行号 | 跳过原因 |')
lines.push('| -------- | -------- | -------- | ---- | --------------------------------- |')
allSkippedMaterials.forEach((skipped) => {
lines.push(
`| \`${skipped.orderNumber}\` | \`${skipped.materialCode}\` | \`${skipped.materialName}\` | ${skipped.rowNumber} | ${skipped.reason} |`
)
})
lines.push('')
lines.push('---')
lines.push('')
}
if (result.errors.length > 0) {
lines.push('## 错误详情')
lines.push('')
lines.push(`**错误总数**: \`${result.errors.length}\``)
lines.push('')
lines.push('### 错误订单列表')
lines.push('')
const errorOrders = this.extractErrorOrders(result.details)
errorOrders.forEach((order) => {
lines.push(`- \`${order}\``)
})
lines.push('')
lines.push('### 错误详细信息')
lines.push('')
result.details
.filter((d) => d.errors.length > 0)
.forEach((detail) => {
lines.push(`#### \`${detail.orderNumber}\``)
lines.push('')
lines.push('```')
lines.push(`订单号:${detail.orderNumber}`)
detail.errors.forEach((error) => {
lines.push(`错误:${error}`)
})
lines.push('```')
lines.push('')
})
lines.push('---')
lines.push('')
}
// Add retry details section
if (result.retriedOrders > 0) {
lines.push('## 重试执行详情')
lines.push('')
lines.push(
`**重试订单总数**: \`${result.retriedOrders}\` | **成功**: \`${result.successfulRetries}\` | **失败**: \`${result.retriedOrders - result.successfulRetries}\``
)
lines.push('')
const retriedDetails = result.details.filter((d) => d.retryCount > 0)
if (retriedDetails.length > 0) {
lines.push('### 重试订单列表')
lines.push('')
lines.push('| 订单号 | 重试次数 | 重试结果 | 重试时间 |')
lines.push('| -------- | -------- | -------- | ------------ |')
retriedDetails.forEach((detail) => {
const retryStatus = detail.retrySuccess ? '✅ 成功' : '❌ 失败'
const retryTime = detail.retriedAt ? this.formatDateTime(detail.retriedAt) : '-'
lines.push(
`| \`${detail.orderNumber}\` | ${detail.retryCount} | ${retryStatus} | ${retryTime} |`
)
})
lines.push('')
lines.push('### 重试尝试详细记录')
lines.push('')
retriedDetails.forEach((detail) => {
lines.push(`#### \`${detail.orderNumber}\``)
lines.push('')
lines.push(`- **重试次数**: ${detail.retryCount}`)
lines.push(`- **最终结果**: ${detail.retrySuccess ? '✅ 成功' : '❌ 失败'}`)
if (detail.retryAttempts && detail.retryAttempts.length > 0) {
lines.push('')
lines.push('**重试尝试记录**:')
lines.push('')
detail.retryAttempts.forEach((attempt, idx) => {
lines.push(
`${idx + 1}. **第${attempt.attempt}次尝试** - ${this.formatDateTime(attempt.timestamp)}`
)
lines.push(` - 错误:${attempt.error}`)
})
lines.push('')
}
lines.push('---')
lines.push('')
})
}
lines.push('')
}
lines.push(`**报告生成时间**: \`${this.formatDateTime(options.endTime)}\``)
lines.push('**报表版本**: `v1.0`')
return lines.join('\n')
}
private collectAllSkippedMaterials(
details: OrderCleanDetail[]
): Array<SkippedMaterial & { orderNumber: string }> {
const result: Array<SkippedMaterial & { orderNumber: string }> = []
details.forEach((detail) => {
if (detail.skippedMaterials && detail.skippedMaterials.length > 0) {
detail.skippedMaterials.forEach((skipped) => {
result.push({
...skipped,
orderNumber: detail.orderNumber
})
})
}
})
return result
}
private extractErrorOrders(details: OrderCleanDetail[]): string[] {
return details.filter((d) => d.errors.length > 0).map((d) => d.orderNumber)
}
private formatDateTime(timestamp: number): string {
const date = new Date(timestamp)
const year = date.getFullYear()
const month = String(date.getMonth() + 1).padStart(2, '0')
const day = String(date.getDate()).padStart(2, '0')
const hours = String(date.getHours()).padStart(2, '0')
const minutes = String(date.getMinutes()).padStart(2, '0')
const seconds = String(date.getSeconds()).padStart(2, '0')
return `${year}-${month}-${day} ${hours}:${minutes}:${seconds}`
}
private formatDuration(startTime: number, endTime: number): string {
const durationMs = endTime - startTime
const minutes = Math.floor(durationMs / 60000)
const seconds = Math.floor((durationMs % 60000) / 1000)
return `${minutes}${seconds}`
}
}

View File

@@ -15,7 +15,7 @@ import {
type GetObjectCommandInput,
type DeleteObjectCommandInput
} from '@aws-sdk/client-s3'
import { createLogger, run, trackDuration, PerformanceTracker } from '../logger'
import { createLogger } from '../logger'
import type { RustfsConfig } from '../../types/config.schema'
import * as fs from 'fs'
import * as path from 'path'

View File

@@ -1,6 +1,8 @@
import * as fs from 'fs'
import { ConfigManager } from '../config/config-manager'
import { createLogger, run, trackDuration, PerformanceTracker } from '../logger'
import { createLogger } from '../logger'
import { logAuditWithCurrentUser } from '../logger/audit-logger'
import { AuditAction, AuditStatus } from '../../types/audit.types'
import type { UpdateConfig } from '../../types/config.schema'
import type { UserType } from '../../types/user.types'
import type {
@@ -281,10 +283,21 @@ export class UpdateService {
log.error('Update package hash mismatch', {
version: request.version,
channel: request.channel,
expectedHash: request.sha256,
actualHash: hash
expectedHash: request.sha256.substring(0, 16),
actualHash: hash.substring(0, 16)
})
await fs.promises.rm(downloadPath, { force: true })
// Audit log: APP_UPDATE download hash mismatch
logAuditWithCurrentUser(AuditAction.APP_UPDATE, 'UPDATE_PACKAGE', AuditStatus.FAILURE, {
version: request.version,
channel: request.channel,
phase: 'download',
error: 'Hash mismatch',
expectedHash: request.sha256.substring(0, 16),
actualHash: hash.substring(0, 16)
})
throw new Error('更新包校验失败,文件哈希不匹配')
}
@@ -294,6 +307,13 @@ export class UpdateService {
downloadPath
})
// Audit log: APP_UPDATE download success
logAuditWithCurrentUser(AuditAction.APP_UPDATE, 'UPDATE_PACKAGE', AuditStatus.SUCCESS, {
version: request.version,
channel: request.channel,
phase: 'download'
})
this.publishStatus({
phase: 'downloaded',
progress: 100,
@@ -336,11 +356,29 @@ export class UpdateService {
error: undefined
})
await this.installer.installDownloadedRelease(downloaded)
log.info('Update installation completed', {
version: downloaded.version,
channel: downloaded.channel
})
try {
await this.installer.installDownloadedRelease(downloaded)
log.info('Update installation completed', {
version: downloaded.version,
channel: downloaded.channel
})
// Audit log: APP_UPDATE install success
logAuditWithCurrentUser(AuditAction.APP_UPDATE, 'UPDATE_PACKAGE', AuditStatus.SUCCESS, {
version: downloaded.version,
channel: downloaded.channel,
phase: 'install'
})
} catch (installError) {
const msg = installError instanceof Error ? installError.message : String(installError)
logAuditWithCurrentUser(AuditAction.APP_UPDATE, 'UPDATE_PACKAGE', AuditStatus.FAILURE, {
version: downloaded.version,
channel: downloaded.channel,
phase: 'install',
error: msg
})
throw installError
}
}
private ensureInitialized(): void {

Some files were not shown because too many files have changed in this diff Show More