Replace the Extract → Write Excel → Read Excel → Import DB flow with
direct record-to-database persistence. The extractor now builds
MaterialPlanRecord[] from parsed orders and imports them without the
round-trip through a merged Excel file.
Key changes:
- Add importFromRecords() to DataImportService for record-based import
- Add SQL Server OPENJSON batch insert and atomic replace operations
in DiscreteMaterialPlanDAO for efficient bulk writes
- Extract common import logic into private importRecords() method
- Configure explicit request/connection timeouts for SQL Server
- Add unit tests for direct record import path
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace underscore-based table name splitting with dot-based splitting
to match the standard schema.tablename format, removing MySQL compatibility.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Record ALL input orders in history, including resolution failures (not_found)
and ERP query misses (erp_not_found). Add ProductionId column to track original
总排号 input. Add 总排号 column and new status styles to the history UI. Fix
empty result caching that prevented retry on transient query failures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Record materials not in the deletion list as "skipped" with reason
instead of just logging them. Skip inserting material details to
database during dry runs to avoid phantom records.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously only skipped and failed materials were persisted. Now every
material (deleted, uncertain, skipped, failed) is recorded in
CleanerMaterialDetail for full audit traceability.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When CleanerService hits a fatal error (browser crash, timeout), the
outer catch now sets result.crashed=true. CleanerApplicationService
detects this, closes the dead browser session, re-logs into ERP, and
re-runs all orders once. An execution ID (CLN-yyyyMMddHHmmss-XXXX)
generated at startup ensures report files are deduplicated across
retries. Reports now display execution ID and app version.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
OrderNumberResolver, validation, and cleaner services had incomplete
PostgreSQL support - they only handled SQL Server and MySQL, causing
PostgreSQL to fall through to MySQL code paths with invalid syntax
(backticks, ? placeholders) and missing schema.table name splitting.
Changes:
- Add PostgreSQL SQL generation ($N params, double-quoted identifiers)
in OrderNumberResolver, validation-application-service,
production-input-service, and validation-database
- Add PostgreSQL to database factory functions in validation-database
and cleaner-application-service
- Add UPPER, LOWER, and 40+ common SQL functions to SQL_KEYWORDS to
prevent prepareSql() from quoting them as identifiers
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add detailed step-by-step logging in navigation phase with elapsed time tracking
- Enhance query interface setup with individual step logging and timing
- Improve order query and result collection with validation logging
- Add comprehensive processDetailPage logging with 8 tracked steps
- Detail material processing loop with decision tracking (delete/skip reasons)
- Enhance retry mechanism with per-attempt logging and success rate tracking
- Add performance monitoring with slow operation detection (isSlow flags)
- All logs use consistent Chinese labeling with [Phase] prefix format
Total: +437 lines of logging instrumentation across cleaner.ts
Enhance ERP automation error diagnostics by capturing PNG screenshots
on every error and forwarding browser console warnings/errors to the
structured logger. Includes automatic cleanup of old screenshots
aligned with the configured log retention period.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add ~45 structured log calls across extractor-core, cleaner, and erp-auth
to cover all automation steps (navigation, query, download, material processing).
Enhance capturePageContext with a step parameter for precise failure localization,
and fix missing capturePageContext calls in error handlers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Eliminate console.* remnants in bootstrap, session-manager, migrations, and app entry
so startup and login failures are captured in log files. Add log.error before all 14
throw sites in ERP services (auth, extractor, cleaner, browser manager) to ensure
critical automation failures are traceable. Introduce capturePageContext utility for
defensive Playwright page state capture during error logging.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1. Extract shared module: consolidate getLogDir() and isProduction()
into shared.ts, eliminate duplication across logger modules
2. Make retention config effective: delay file transport creation
until config is loaded, apply appRetention/auditRetention from config.yaml
3. Add before-quit log flush: close logger and audit logger on
app exit to prevent log loss
4. Unify logError entry point: remove duplicate logError from index.ts,
re-export from error-utils.ts with richer error context
5. Renderer log level filtering: add client-side level check in preload
to skip IPC for filtered-out messages
6. Child logger cache + audit cleanup: cache child loggers in IPC
handler for performance, remove redundant timestamp format in audit logger
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously updateBatchStatus wrote the batch-level total recordCount to
every row, causing the detail view to show misleading identical counts.
Now mergeFiles collects per-order material counts, the handler writes
each order's count individually via updateRecordStatus, and batch
aggregation uses SUM instead of MAX for accurate totals.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Change the protected row number range from 7000-7999 to 2000-7999 to prevent deletion of materials in this broader range.
- Updated isMaterialDeletable() method logic
- Updated getSkipReason() error messages
- Updated test cases to reflect new range boundaries
- Updated documentation templates and error collection guide
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add QueryResultRow interface to represent query results with order numbers
- Add collectQueryResultRows() method to extract order numbers upfront before processing
- Add extractOrderNumberFromQueryRow() helper to parse order numbers from query result cells
- Process rows with order number context instead of just row indexes
- Use actual order numbers in error details instead of BATCH_ROW_X placeholders
- Pass expected order number to detail processing for better validation
- Fix retry success handling to properly update statistics when retries succeed
This change provides better error context by associating each processed row with its actual order number from the query results, improving traceability and debugging.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add case-insensitive comparison for production ID database queries:
- SQL Server: use COLLATE SQL_Latin1_General_CP1_CI_AS
- MySQL: use UPPER() function for both field and input
- Map lookup: store keys in lowercase for consistent matching
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add *.yaml text eol=lf rule to .gitattributes for consistent line endings
- Format cleaner.ts with Prettier (parameter and chain formatting)
- Format CleanerPage.tsx (JSX formatting)
- Format cleaner.test.ts (array formatting)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add retry logic with max 2 attempts per failed order
- Track retry statistics (retriedOrders, successfulRetries)
- Generate detailed retry report section in execution reports
- Display retry metrics in ExecutionReportDialog UI
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix getTableName to properly split schema_tablename format
- Convert productionContractData_26年压力表合同数据 to [productionContractData].[26年压力表合同数据]
- Extend productionId pattern to support 1-6 digit serial numbers
Co-Authored-By: Claude (glm-5) <noreply@anthropic.com>
Fix all TS6133 errors (unused variables) across service layer:
- Remove unused imports (path, ExtractionProgress type)
- Prefix unused parameters with underscore (_session, _totalBatches, etc.)
- Remove unused _verbose field and constructor from ExcelParser
- Remove unused _importToDatabase method from ExtractorService
- Remove unused _importProgress variable
This ensures clean type checking and eliminates dead code.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit integrates the logging infrastructure across the entire application:
IPC Layer:
- Add logger-handler.ts with centralized IPC logging channels
- Integrate audit logging into auth, cleaner, extractor handlers
- Add structured logging for IPC operations and data flow
Service Layer:
- Add logger integration to ERP services (extractor, cleaner)
- Integrate logging into excel-parser and user DAO
- Add operation tracking and error logging
Renderer Layer:
- Add useLogger hook for component-level logging
- Update App.tsx with session and user activity logging
- Enable frontend audit trail for critical actions
Testing:
- Add comprehensive IPC logging integration tests
- Enhance unit test coverage for logger and audit-logger
- Add end-to-end logging flow validation
Types:
- Update preload type definitions for logging APIs
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add waitForLoginResult() method using Promise.race to detect:
- Success: .nc-workbench-icon element visible
- Failure: '名称或密码错误' error text visible
- Force login: click confirm button and re-detect
- Extract timeout constants (PAGE_LOAD_TIMEOUT, LOGIN_RESULT_TIMEOUT, FORCE_LOGIN_TIMEOUT)
- Improve error handling with clear error messages
- Add unit tests for class structure verification
- Fix test setup for Electron app mock
Fixes: ERP login success/failure detection was ambiguous
Add waitForLoginResult() method with Promise.race to detect three login outcomes:
- Success: detects .nc-workbench-icon element
- Failure: detects '名称或密码错误' error text
- Force login: clicks confirm button and re-detects
Improves login reliability by properly handling all authentication scenarios.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
- Removed ERP_HEADLESS, ERP_IGNORE_HTTPS_ERRORS, ERP_AUTO_CLOSE_BROWSER from ConfigManager .env operations.
- Hardcoded these values in ConfigManager's default settings and getAllSettings() return.
- Hardcoded ignoreHTTPSErrors to true inside ErpBrowserManager when creating a new browser context.
Co-authored-by: luwamgere15-crypto <255338376+luwamgere15-crypto@users.noreply.github.com>
- Add CleanerReportGenerator service to create execution reports
- Extend OrderCleanDetail with skippedMaterials for detailed tracking
- Record skip reasons for each material (protected range, pending qty, etc.)
- Generate timestamped markdown reports in logs/reports/ directory
- Reports include execution summary, order details, skip reasons, and errors
- Implement IPC event system for pushing progress updates from main to renderer
- Add Zustand store for centralized extractor state management
- Refactor useExtractor hook to use store pattern
- Add chromium-bidi dependency and externalize Playwright for build compatibility
- Show detailed logs during extraction (DB connection, order resolution, ERP login, data import)
Apply consistent code formatting using Prettier to improve code readability
and maintain style consistency throughout the project.
Co-Authored-By: Claude (glm-5) <noreply@anthropic.com>
- Add experimentalDecorators support in tsconfig.node.json for TypeORM entities
- Fix mssql module import in order-resolver.ts (static vs dynamic import)
- Extend ISqlType parameter types in sql-server.ts for NVarChar compatibility
- Fix variable naming and type assertions in bip-users-dao.ts
- Add proper type assertions for IPC call results in renderer hooks
(useAuth, useCleaner, useExtractor, useValidation)
- Add definite assignment assertions in config-manager.ts
- SQL Server has a maximum of 2100 parameters per query
- Each record has 28 columns, so max batch is ~71 records (2000/28)
- Fixed syntax error: removed extra closing brace in extractor.ts
- Added debug logging for batch insert parameters
- Add DataImportService for reading Excel and importing to database
- Extend DiscreteMaterialPlanDAO with deleteBySourceNumbers and batchInsert
- Auto-trigger database write after successful Excel merge
- Support batch delete by SourceNumber and batch insert (1000/batch)
- Update ExtractorPage UI to show import results
- Fix SQL Server query to handle undefined recordset for DELETE/INSERT
Co-Authored-By: Claude (glm-5) <noreply@anthropic.com>
- Add getTableName() method to convert MySQL table names to SQL Server format
- Use queryWithParams with sql.NVarChar for proper SQL Server parameter handling
- Implement case-insensitive matching for production IDs (e.g., 26b10433 vs 26B10433)
- Align resolver logic with validation-handler.ts for consistent database queries
Co-Authored-By: Claude (glm-5) <noreply@anthropic.com>
The product unit column header was incorrectly showing "单位" instead of
"产品单位", causing confusion with the material unit column which also
uses "单位". Fixed in both extractor.ts and excel-parser.ts.
Co-Authored-By: Claude (glm-5) <noreply@anthropic.com>
- Fix "Workbook is not a constructor" error by handling ESM/CommonJS module format
- Add try-catch around saveMergedOrders to capture and report errors
- Return parsed recordCount even when save fails so users see actual data count
- Add detailed logging throughout merge process for debugging
- Clean up temporary batch files after merge completion
Co-Authored-By: Claude (glm-5) <noreply@anthropic.com>
- Add mergeFiles() method in ExtractorService to combine downloaded batch files
- Add saveMergedOrders() method to output full 31-column Excel format
- Update recordCount to return actual material record count
- Add missing field mappings in OrderHeader type and ExcelParser:
- factory, materialStatus, planNumber, materialType
- department, remark, createDate, approveDate
- Output file named with timestamp: merged_YYYYMMDDHHMMSS.xlsx
Co-Authored-By: Claude (glm-5) <noreply@anthropic.com>
- Add IDatabaseService interface with unified query(transaction) methods
- Create DatabaseFactory for centralized database service creation
- Modify MySqlService and SqlServerService to implement IDatabaseService
- Unify SqlServerService.query() to accept array params (internally converts to @p0, @p1...)
- Refactor OrderNumberResolver to use IDatabaseService
- Refactor DiscreteMaterialPlanDAO to use DatabaseFactory
- Refactor MaterialsToBeDeletedDAO to use DatabaseFactory
- Update IPC handlers to use DatabaseFactory.create()
- Add database.types.ts with shared type definitions
This enables switching between MySQL and SQL Server via DB_TYPE env variable.
Co-Authored-By: Claude (glm-5) <noreply@anthropic.com>
