feat(cleaner): add per-order video recording retention
This commit is contained in:
273
docs/plans/2026-04-17-cleaner-video-recording-design.md
Normal file
273
docs/plans/2026-04-17-cleaner-video-recording-design.md
Normal file
@@ -0,0 +1,273 @@
|
|||||||
|
# Cleaner Video Recording Design
|
||||||
|
|
||||||
|
Date: 2026-04-17
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Add an optional video recording capability to the Cleaner execution flow. When enabled, the system records the ERP detail-page handling process for each order and stores videos under `logs/clearner-video-records/<batchId>/`. Videos are retained only for orders whose final persisted status is not `success`; videos for successful orders, including orders that failed initially but succeeded after retry, are deleted automatically.
|
||||||
|
|
||||||
|
This design is based on the current Cleaner architecture:
|
||||||
|
|
||||||
|
- The execution setting panel already exposes Cleaner-only settings in `CleanerExecutionBar`.
|
||||||
|
- `processConcurrency` is already persisted through `window.electron.config.updateCleaner(...)` into `config.yaml`.
|
||||||
|
- Cleaner history is already persisted per `batchId + attemptNumber + orderNumber` in `CleanerOrderHistory`.
|
||||||
|
- Each order detail is handled in a separate Playwright popup page, which matches the requirement of "one order, one video file" at the page level.
|
||||||
|
|
||||||
|
## Goals
|
||||||
|
|
||||||
|
- Add a new execution setting to control whether Cleaner records videos.
|
||||||
|
- Persist that setting using the same config path as `processConcurrency`.
|
||||||
|
- Save videos under `logs/clearner-video-records/<batchId>/`.
|
||||||
|
- Use order number as the logical file name.
|
||||||
|
- Keep all videos for orders whose final order history status is non-`success`.
|
||||||
|
- Delete videos for orders whose final order history status is `success`.
|
||||||
|
- Preserve multiple videos for the same order when retry attempts also fail, using an incrementing suffix.
|
||||||
|
|
||||||
|
## Non-Goals
|
||||||
|
|
||||||
|
- No video playback UI in the current PRD scope.
|
||||||
|
- No database schema change is required if file retention is derived from existing Cleaner history state.
|
||||||
|
- No cross-run deduplication is required because `batchId` already isolates one execution.
|
||||||
|
|
||||||
|
## Current Code Facts
|
||||||
|
|
||||||
|
## Execution Settings
|
||||||
|
|
||||||
|
- `CleanerPage.tsx` reads execution settings from `useCleaner()`.
|
||||||
|
- `CleanerExecutionBar.tsx` already renders:
|
||||||
|
- `dryRun`
|
||||||
|
- `headless`
|
||||||
|
- `processConcurrency`
|
||||||
|
- `useCleaner.ts` persists `processConcurrency` through `window.electron.config.updateCleaner({ processConcurrency })`.
|
||||||
|
- `config:getCleaner` / `config:updateCleaner` are already exposed through IPC and stored in `config.yaml`.
|
||||||
|
|
||||||
|
## Cleaner Runtime
|
||||||
|
|
||||||
|
- `CleanerApplicationService.runCleaner(...)` creates a `batchId`, inserts execution history, runs Cleaner, then calls `saveAttemptToDatabase(...)`.
|
||||||
|
- `saveAttemptToDatabase(...)` writes per-order final state through `historyDao.updateOrderStatus(...)`.
|
||||||
|
- Order status currently resolves to:
|
||||||
|
- `success` when `detail.errors.length === 0`
|
||||||
|
- `failed` when `detail.errors.length > 0`
|
||||||
|
- `erp_not_found` when `detail.notFound === true`
|
||||||
|
- Retry success is recorded via `RetrySuccess = 1`, but the order `Status` still becomes `success`.
|
||||||
|
|
||||||
|
## Retry Model
|
||||||
|
|
||||||
|
- Inner retry: failed orders are retried inside the same execution attempt by `CleanerService.retryFailedOrders(...)`.
|
||||||
|
- Outer retry: when the whole run crashes, `CleanerApplicationService` creates attempt 2 and writes a second set of `CleanerExecution` and `CleanerOrderHistory` records.
|
||||||
|
- Therefore the natural identity for one recorded video is:
|
||||||
|
- `batchId`
|
||||||
|
- `attemptNumber`
|
||||||
|
- `orderNumber`
|
||||||
|
- `retry sequence within the same attempt`, if multiple failed recordings must be kept
|
||||||
|
|
||||||
|
## Product Rules
|
||||||
|
|
||||||
|
## User-Facing Behavior
|
||||||
|
|
||||||
|
- Add a new toggle in `执行设置`: `录制处理视频`.
|
||||||
|
- Default value: `false`.
|
||||||
|
- Persist the value in Cleaner config, same as `processConcurrency`.
|
||||||
|
- When disabled, Cleaner behavior is unchanged.
|
||||||
|
- When enabled, the system records the per-order ERP detail-page process.
|
||||||
|
|
||||||
|
## Storage Rules
|
||||||
|
|
||||||
|
- Root path: `logs/clearner-video-records/<batchId>/`
|
||||||
|
- One execution batch corresponds to one directory.
|
||||||
|
- The logical base name is the order number.
|
||||||
|
- If the same order has multiple retained failed runs in the same batch, append a numeric suffix:
|
||||||
|
- `SC20260101000123.webm`
|
||||||
|
- `SC20260101000123__2.webm`
|
||||||
|
- `SC20260101000123__3.webm`
|
||||||
|
- File extension should follow the actual Playwright video output format.
|
||||||
|
|
||||||
|
## Retention Rules
|
||||||
|
|
||||||
|
- If final persisted order status is `success`, delete all videos for that order in the current batch and current attempt chain.
|
||||||
|
- If an order fails once and then succeeds on retry, do not keep any video for that order.
|
||||||
|
- If an order fails and all retries fail, keep every recorded video for that order.
|
||||||
|
- If an order resolves to a non-success terminal status such as `failed` or `erp_not_found`, keep the video.
|
||||||
|
- If recording setup fails, Cleaner execution must continue; recording is diagnostic, not blocking.
|
||||||
|
|
||||||
|
## Recommended Technical Design
|
||||||
|
|
||||||
|
## Config Changes
|
||||||
|
|
||||||
|
Add a new Cleaner config field:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
cleaner: {
|
||||||
|
queryBatchSize: number
|
||||||
|
processConcurrency: number
|
||||||
|
recordVideo: boolean
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Required touch points:
|
||||||
|
|
||||||
|
- `src/main/types/config.schema.ts`
|
||||||
|
- `src/main/services/config/config-manager.ts` default config
|
||||||
|
- `src/preload/index.d.ts`
|
||||||
|
- `src/preload/api/materials.ts`
|
||||||
|
- renderer Cleaner config loader / updater
|
||||||
|
|
||||||
|
## Input Flow
|
||||||
|
|
||||||
|
Pass `recordVideo` from renderer to main:
|
||||||
|
|
||||||
|
```text
|
||||||
|
CleanerExecutionBar
|
||||||
|
-> useCleaner
|
||||||
|
-> runCleanerExecution(...)
|
||||||
|
-> window.electron.cleaner.runCleaner(...)
|
||||||
|
-> CleanerApplicationService.runCleaner(...)
|
||||||
|
-> CleanerService.clean(...)
|
||||||
|
```
|
||||||
|
|
||||||
|
This allows the run to decide per batch whether recording is enabled.
|
||||||
|
|
||||||
|
## Recording Strategy
|
||||||
|
|
||||||
|
Recommended approach:
|
||||||
|
|
||||||
|
1. Enable Playwright context-level video recording only when `recordVideo === true`.
|
||||||
|
2. Record all popup detail pages created during Cleaner execution.
|
||||||
|
3. After each detail page closes, resolve the generated Playwright video file path.
|
||||||
|
4. Move or rename that file into `logs/<batchId>/` using the order-based naming convention.
|
||||||
|
5. Defer final retention cleanup until the order's final persisted result is known.
|
||||||
|
|
||||||
|
Reasoning:
|
||||||
|
|
||||||
|
- Current code creates one `BrowserContext` in `ErpAuthService.login()`.
|
||||||
|
- Each order detail is processed in a separate popup `Page`.
|
||||||
|
- Playwright video is page-based under a recorded context, so this aligns with the current "one detail page, one video" architecture.
|
||||||
|
|
||||||
|
Inference:
|
||||||
|
This should allow one raw video file per detail popup page without forcing a redesign to one context per order. The implementation still needs a small proof-of-concept to confirm popup-page video behavior in this ERP flow.
|
||||||
|
|
||||||
|
## Video File Registry
|
||||||
|
|
||||||
|
Add an in-memory registry for the current batch, for example:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
type CleanerVideoArtifact = {
|
||||||
|
batchId: string
|
||||||
|
attemptNumber: number
|
||||||
|
orderNumber: string
|
||||||
|
sequence: number
|
||||||
|
tempPath: string
|
||||||
|
finalPath: string
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The registry should support:
|
||||||
|
|
||||||
|
- tracking all videos produced for one order
|
||||||
|
- deleting all videos for a finally successful order
|
||||||
|
- preserving all videos for a finally failed order
|
||||||
|
- handling both inner retry and outer retry attempts cleanly
|
||||||
|
|
||||||
|
## Retention Timing
|
||||||
|
|
||||||
|
Recommended retention point:
|
||||||
|
|
||||||
|
- perform final keep/delete cleanup after `saveAttemptToDatabase(...)` finishes for the attempt
|
||||||
|
- use the same final status logic as `historyDao.updateOrderStatus(...)`
|
||||||
|
|
||||||
|
Why this is safer:
|
||||||
|
|
||||||
|
- it matches the persisted truth used by history UI
|
||||||
|
- it avoids deleting a video too early before retry outcome is known
|
||||||
|
- it keeps product behavior aligned with your rule: final `success` means no video retained
|
||||||
|
|
||||||
|
## Edge Cases
|
||||||
|
|
||||||
|
## Retry Success
|
||||||
|
|
||||||
|
Case:
|
||||||
|
|
||||||
|
- first processing fails and produces video A
|
||||||
|
- retry succeeds and produces video B
|
||||||
|
- final order status is `success`
|
||||||
|
|
||||||
|
Expected behavior:
|
||||||
|
|
||||||
|
- delete A
|
||||||
|
- delete B
|
||||||
|
|
||||||
|
## Retry Failure
|
||||||
|
|
||||||
|
Case:
|
||||||
|
|
||||||
|
- first processing fails and produces video A
|
||||||
|
- retry 1 fails and produces video B
|
||||||
|
- retry 2 fails and produces video C
|
||||||
|
- final order status is `failed`
|
||||||
|
|
||||||
|
Expected behavior:
|
||||||
|
|
||||||
|
- keep A, B, C
|
||||||
|
|
||||||
|
## ERP Not Found
|
||||||
|
|
||||||
|
Case:
|
||||||
|
|
||||||
|
- order query returns no detail page / cannot enter ERP detail page
|
||||||
|
|
||||||
|
Expected behavior:
|
||||||
|
|
||||||
|
- if no page exists, there may be no video file to keep
|
||||||
|
- final order status remains non-success
|
||||||
|
- PRD should accept "no video generated" as valid when the detail page never opened
|
||||||
|
|
||||||
|
## Outer Retry
|
||||||
|
|
||||||
|
Case:
|
||||||
|
|
||||||
|
- attempt 1 crashes mid-run
|
||||||
|
- attempt 2 reruns orders
|
||||||
|
|
||||||
|
Expected behavior:
|
||||||
|
|
||||||
|
- videos must stay isolated by `attemptNumber`
|
||||||
|
- cleanup must never delete attempt 2 videos because of attempt 1 status, or vice versa
|
||||||
|
|
||||||
|
## UI Changes
|
||||||
|
|
||||||
|
- In `CleanerExecutionBar.tsx`, add a new toggle block below `后台模式 (Headless)` and above `并行处理数量`.
|
||||||
|
- Suggested label: `录制处理视频`
|
||||||
|
- Suggested help text: `为每个订单详情页生成视频,仅保留最终失败订单的视频`
|
||||||
|
|
||||||
|
## Implementation Breakdown
|
||||||
|
|
||||||
|
| Area | Change |
|
||||||
|
|------|--------|
|
||||||
|
| Renderer | Add `recordVideo` state, config loading, config persistence, execution payload field |
|
||||||
|
| Preload | Extend `CleanerConfig` IPC typing and `configApi` usage |
|
||||||
|
| Main config | Add `cleaner.recordVideo` schema and default value |
|
||||||
|
| Cleaner input types | Extend `CleanerInput` with `recordVideo?: boolean` |
|
||||||
|
| ERP auth / browser context | Conditionally enable Playwright video recording |
|
||||||
|
| Cleaner runtime | Capture per-detail-page video artifacts and map them to `orderNumber` |
|
||||||
|
| Application service | Run final video retention cleanup after order statuses are persisted |
|
||||||
|
| Logging | Add diagnostic logs for recording enabled, file move, delete, retention outcome |
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
1. Should dry-run mode also support video recording?
|
||||||
|
Current recommendation: yes, because dry-run is often used for diagnosis.
|
||||||
|
2. Should login page and top-level query popup videos be retained?
|
||||||
|
Current recommendation: no, only order detail page videos participate in naming and retention.
|
||||||
|
3. Should we expose retained video paths in operation history UI later?
|
||||||
|
Current recommendation: not in this scope, but keep naming stable to support a future enhancement.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- The execution settings panel contains a persistent `录制处理视频` switch.
|
||||||
|
- `config.yaml` stores `cleaner.recordVideo`.
|
||||||
|
- When the switch is off, no video files are generated.
|
||||||
|
- When the switch is on, videos are generated under `logs/clearner-video-records/<batchId>/`.
|
||||||
|
- Final successful orders leave no retained video files.
|
||||||
|
- Orders that remain failed retain all their recorded videos.
|
||||||
|
- Multiple retained videos for the same order are distinguishable by suffix.
|
||||||
|
- Recording failures do not interrupt Cleaner execution.
|
||||||
@@ -1,4 +1,6 @@
|
|||||||
import type { WebContents } from 'electron'
|
import type { WebContents } from 'electron'
|
||||||
|
import * as fs from 'fs/promises'
|
||||||
|
import * as path from 'path'
|
||||||
import type { IDatabaseService } from '../../types/database.types'
|
import type { IDatabaseService } from '../../types/database.types'
|
||||||
import { ErpAuthService } from '../erp/erp-auth'
|
import { ErpAuthService } from '../erp/erp-auth'
|
||||||
import { CleanerService } from '../erp/cleaner'
|
import { CleanerService } from '../erp/cleaner'
|
||||||
@@ -11,6 +13,7 @@ import { ResultExporter } from '../excel/result-exporter'
|
|||||||
import { SessionManager } from '../user/session-manager'
|
import { SessionManager } from '../user/session-manager'
|
||||||
import { UserErpConfigService } from '../user/user-erp-config-service'
|
import { UserErpConfigService } from '../user/user-erp-config-service'
|
||||||
import { createLogger } from '../logger'
|
import { createLogger } from '../logger'
|
||||||
|
import { getLogDir } from '../logger/shared'
|
||||||
import { logAuditWithCurrentUser } from '../logger/audit-logger'
|
import { logAuditWithCurrentUser } from '../logger/audit-logger'
|
||||||
import { AuditAction, AuditStatus } from '../../types/audit.types'
|
import { AuditAction, AuditStatus } from '../../types/audit.types'
|
||||||
import { IPC_CHANNELS } from '../../../shared/ipc-channels'
|
import { IPC_CHANNELS } from '../../../shared/ipc-channels'
|
||||||
@@ -132,7 +135,8 @@ export class CleanerApplicationService {
|
|||||||
url: erpConfig.url,
|
url: erpConfig.url,
|
||||||
username: erpConfig.username,
|
username: erpConfig.username,
|
||||||
password: erpConfig.password,
|
password: erpConfig.password,
|
||||||
headless: input.headless ?? true
|
headless: input.headless ?? true,
|
||||||
|
recordVideoDir: input.recordVideo ? await this.prepareVideoTempDir(batchId, 1) : undefined
|
||||||
})
|
})
|
||||||
|
|
||||||
log.info('Logging in to ERP...')
|
log.info('Logging in to ERP...')
|
||||||
@@ -159,6 +163,8 @@ export class CleanerApplicationService {
|
|||||||
const modifiedInput: CleanerInput = {
|
const modifiedInput: CleanerInput = {
|
||||||
...input,
|
...input,
|
||||||
orderNumbers: validOrderNumbers,
|
orderNumbers: validOrderNumbers,
|
||||||
|
videoBatchId: batchId,
|
||||||
|
videoAttemptNumber: 1,
|
||||||
onProgress: (message, progress, extra) => {
|
onProgress: (message, progress, extra) => {
|
||||||
this.sendProgress(eventSender, message, progress ?? 0, extra)
|
this.sendProgress(eventSender, message, progress ?? 0, extra)
|
||||||
}
|
}
|
||||||
@@ -180,6 +186,7 @@ export class CleanerApplicationService {
|
|||||||
|
|
||||||
// Save attempt 1 result as crashed
|
// Save attempt 1 result as crashed
|
||||||
await this.saveAttemptToDatabase(historyDao, batchId, 1, result)
|
await this.saveAttemptToDatabase(historyDao, batchId, 1, result)
|
||||||
|
await cleaner.finalizeVideoArtifacts(result.details)
|
||||||
|
|
||||||
this.sendProgress(eventSender, '流程崩溃,正在重新登录并重试...', 0, {
|
this.sendProgress(eventSender, '流程崩溃,正在重新登录并重试...', 0, {
|
||||||
phase: 'retry',
|
phase: 'retry',
|
||||||
@@ -200,7 +207,8 @@ export class CleanerApplicationService {
|
|||||||
url: erpConfig.url,
|
url: erpConfig.url,
|
||||||
username: erpConfig.username,
|
username: erpConfig.username,
|
||||||
password: erpConfig.password,
|
password: erpConfig.password,
|
||||||
headless: input.headless ?? true
|
headless: input.headless ?? true,
|
||||||
|
recordVideoDir: input.recordVideo ? await this.prepareVideoTempDir(batchId, 2) : undefined
|
||||||
})
|
})
|
||||||
|
|
||||||
try {
|
try {
|
||||||
@@ -238,10 +246,14 @@ export class CleanerApplicationService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
cleaner = new CleanerService(authService)
|
cleaner = new CleanerService(authService)
|
||||||
result = await cleaner.clean(modifiedInput)
|
result = await cleaner.clean({
|
||||||
|
...modifiedInput,
|
||||||
|
videoAttemptNumber: 2
|
||||||
|
})
|
||||||
|
|
||||||
// Save attempt 2 result
|
// Save attempt 2 result
|
||||||
await this.saveAttemptToDatabase(historyDao, batchId, 2, result)
|
await this.saveAttemptToDatabase(historyDao, batchId, 2, result)
|
||||||
|
await cleaner.finalizeVideoArtifacts(result.details)
|
||||||
|
|
||||||
log.info('Outer retry completed', {
|
log.info('Outer retry completed', {
|
||||||
batchId,
|
batchId,
|
||||||
@@ -252,6 +264,7 @@ export class CleanerApplicationService {
|
|||||||
} else {
|
} else {
|
||||||
// No crash — save attempt 1 result
|
// No crash — save attempt 1 result
|
||||||
await this.saveAttemptToDatabase(historyDao, batchId, 1, result)
|
await this.saveAttemptToDatabase(historyDao, batchId, 1, result)
|
||||||
|
await cleaner.finalizeVideoArtifacts(result.details)
|
||||||
}
|
}
|
||||||
|
|
||||||
if (warnings.length > 0) {
|
if (warnings.length > 0) {
|
||||||
@@ -300,6 +313,18 @@ export class CleanerApplicationService {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private async prepareVideoTempDir(batchId: string, attemptNumber: number): Promise<string> {
|
||||||
|
const tempDir = path.join(
|
||||||
|
getLogDir(),
|
||||||
|
'clearner-video-records',
|
||||||
|
batchId,
|
||||||
|
'.tmp',
|
||||||
|
`attempt-${attemptNumber}`
|
||||||
|
)
|
||||||
|
await fs.mkdir(tempDir, { recursive: true })
|
||||||
|
return tempDir
|
||||||
|
}
|
||||||
|
|
||||||
async exportResults(items: ExportResultItem[]): Promise<ExportResultResponse> {
|
async exportResults(items: ExportResultItem[]): Promise<ExportResultResponse> {
|
||||||
log.info('Exporting validation results', { count: items.length })
|
log.info('Exporting validation results', { count: items.length })
|
||||||
if (!items || items.length === 0) {
|
if (!items || items.length === 0) {
|
||||||
|
|||||||
@@ -99,7 +99,8 @@ const DEFAULT_CONFIG: FullConfig = {
|
|||||||
},
|
},
|
||||||
cleaner: {
|
cleaner: {
|
||||||
queryBatchSize: 100,
|
queryBatchSize: 100,
|
||||||
processConcurrency: 1
|
processConcurrency: 1,
|
||||||
|
recordVideo: false
|
||||||
},
|
},
|
||||||
orderResolution: {
|
orderResolution: {
|
||||||
tableName: '',
|
tableName: '',
|
||||||
|
|||||||
@@ -9,7 +9,10 @@ import type {
|
|||||||
} from '../../types/cleaner.types'
|
} from '../../types/cleaner.types'
|
||||||
import type { ErpSession } from '../../types/erp.types'
|
import type { ErpSession } from '../../types/erp.types'
|
||||||
import type { FrameLocator, Locator, Page } from 'playwright'
|
import type { FrameLocator, Locator, Page } from 'playwright'
|
||||||
|
import * as fs from 'fs/promises'
|
||||||
|
import * as path from 'path'
|
||||||
import { createLogger, run, trackDuration } from '../logger'
|
import { createLogger, run, trackDuration } from '../logger'
|
||||||
|
import { getLogDir } from '../logger/shared'
|
||||||
import { capturePageContext } from './erp-error-context'
|
import { capturePageContext } from './erp-error-context'
|
||||||
|
|
||||||
const log = createLogger('CleanerService')
|
const log = createLogger('CleanerService')
|
||||||
@@ -39,6 +42,12 @@ interface QueryResultRow {
|
|||||||
orderNumber: string
|
orderNumber: string
|
||||||
}
|
}
|
||||||
|
|
||||||
|
interface CleanerVideoArtifact {
|
||||||
|
orderNumber: string
|
||||||
|
sequence: number
|
||||||
|
finalPath: string
|
||||||
|
}
|
||||||
|
|
||||||
class AsyncMutex {
|
class AsyncMutex {
|
||||||
private queue: Promise<void> = Promise.resolve()
|
private queue: Promise<void> = Promise.resolve()
|
||||||
|
|
||||||
@@ -176,6 +185,8 @@ export async function runWithConcurrency<T, R>(
|
|||||||
export class CleanerService {
|
export class CleanerService {
|
||||||
private authService: ErpAuthService
|
private authService: ErpAuthService
|
||||||
private dryRun: boolean
|
private dryRun: boolean
|
||||||
|
private readonly recordedVideos = new Map<string, CleanerVideoArtifact[]>()
|
||||||
|
private readonly videoSequenceCounters = new Map<string, number>()
|
||||||
|
|
||||||
constructor(authService: ErpAuthService, options: CleanerOptions = {}) {
|
constructor(authService: ErpAuthService, options: CleanerOptions = {}) {
|
||||||
this.authService = authService
|
this.authService = authService
|
||||||
@@ -374,7 +385,10 @@ export class CleanerService {
|
|||||||
return page
|
return page
|
||||||
})
|
})
|
||||||
|
|
||||||
let detail: OrderCleanDetail
|
let detail: OrderCleanDetail = this.createErrorDetail(
|
||||||
|
orderNumber,
|
||||||
|
'详情页处理未返回结果'
|
||||||
|
)
|
||||||
try {
|
try {
|
||||||
detail = await this.processDetailPage({
|
detail = await this.processDetailPage({
|
||||||
detailPage: openedDetailPage,
|
detailPage: openedDetailPage,
|
||||||
@@ -388,12 +402,14 @@ export class CleanerService {
|
|||||||
const message = error instanceof Error ? error.message : 'Unknown error'
|
const message = error instanceof Error ? error.message : 'Unknown error'
|
||||||
detail = this.createErrorDetail(orderNumber, message)
|
detail = this.createErrorDetail(orderNumber, message)
|
||||||
} finally {
|
} finally {
|
||||||
progressState.ordersStarted += 1
|
await this.captureRecordedVideo(openedDetailPage, input, detail?.orderNumber || orderNumber)
|
||||||
progressState.ordersCompleted += 1
|
|
||||||
progressState.lastCompletedOrder = orderNumber
|
|
||||||
lastActivityTime = Date.now() // [新增] 健康检查:更新活动时间
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
progressState.ordersStarted += 1
|
||||||
|
progressState.ordersCompleted += 1
|
||||||
|
progressState.lastCompletedOrder = orderNumber
|
||||||
|
lastActivityTime = Date.now() // [新增] 健康检查:更新活动时间
|
||||||
|
|
||||||
result.details.push(detail)
|
result.details.push(detail)
|
||||||
|
|
||||||
if (detail.errors.length > 0) {
|
if (detail.errors.length > 0) {
|
||||||
@@ -456,6 +472,7 @@ export class CleanerService {
|
|||||||
),
|
),
|
||||||
deleteSet,
|
deleteSet,
|
||||||
dryRun,
|
dryRun,
|
||||||
|
input,
|
||||||
onProgress: input.onProgress
|
onProgress: input.onProgress
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -1678,13 +1695,14 @@ export class CleanerService {
|
|||||||
failedDetails: OrderCleanDetail[]
|
failedDetails: OrderCleanDetail[]
|
||||||
deleteSet: Set<string>
|
deleteSet: Set<string>
|
||||||
dryRun: boolean
|
dryRun: boolean
|
||||||
|
input: CleanerInput
|
||||||
onProgress?: (
|
onProgress?: (
|
||||||
message: string,
|
message: string,
|
||||||
progress?: number,
|
progress?: number,
|
||||||
extra?: Partial<import('../../types/cleaner.types').CleanerProgress>
|
extra?: Partial<import('../../types/cleaner.types').CleanerProgress>
|
||||||
) => void
|
) => void
|
||||||
}): Promise<RetryResult> {
|
}): Promise<RetryResult> {
|
||||||
const { workFrame, popupPage, failedDetails, deleteSet, dryRun, onProgress } = params
|
const { workFrame, popupPage, failedDetails, deleteSet, dryRun, input, onProgress } = params
|
||||||
|
|
||||||
const result: RetryResult = {
|
const result: RetryResult = {
|
||||||
retriedOrders: 0,
|
retriedOrders: 0,
|
||||||
@@ -1802,6 +1820,7 @@ export class CleanerService {
|
|||||||
skipped: retryDetail.materialsSkipped,
|
skipped: retryDetail.materialsSkipped,
|
||||||
elapsedMs: Date.now() - attemptStartTime
|
elapsedMs: Date.now() - attemptStartTime
|
||||||
})
|
})
|
||||||
|
await this.captureRecordedVideo(detailPage, input, retryDetail.orderNumber)
|
||||||
|
|
||||||
// Success - update counters
|
// Success - update counters
|
||||||
retryResult.successfulRetries += 1
|
retryResult.successfulRetries += 1
|
||||||
@@ -1897,4 +1916,97 @@ export class CleanerService {
|
|||||||
|
|
||||||
return trackedResult.result
|
return trackedResult.result
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public async finalizeVideoArtifacts(details: OrderCleanDetail[]): Promise<void> {
|
||||||
|
if (this.recordedVideos.size === 0) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const detail of details) {
|
||||||
|
const artifacts = this.recordedVideos.get(detail.orderNumber)
|
||||||
|
if (!artifacts || artifacts.length === 0) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
if (detail.errors.length === 0) {
|
||||||
|
for (const artifact of artifacts) {
|
||||||
|
try {
|
||||||
|
await fs.unlink(artifact.finalPath)
|
||||||
|
} catch (error) {
|
||||||
|
log.warn('删除成功订单视频失败', {
|
||||||
|
orderNumber: detail.orderNumber,
|
||||||
|
filePath: artifact.finalPath,
|
||||||
|
error: error instanceof Error ? error.message : String(error)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
this.recordedVideos.delete(detail.orderNumber)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
log.info('保留失败订单视频', {
|
||||||
|
orderNumber: detail.orderNumber,
|
||||||
|
videoCount: artifacts.length,
|
||||||
|
files: artifacts.map((artifact) => artifact.finalPath)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async captureRecordedVideo(
|
||||||
|
page: Page,
|
||||||
|
input: CleanerInput,
|
||||||
|
orderNumber: string
|
||||||
|
): Promise<void> {
|
||||||
|
if (!input.recordVideo || !input.videoBatchId) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
const video = page.video()
|
||||||
|
if (!video) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
const finalDir = this.getVideoBatchDir(input.videoBatchId)
|
||||||
|
const normalizedOrderNumber = this.sanitizeVideoFileName(orderNumber || 'UNKNOWN_ORDER')
|
||||||
|
const sequence = (this.videoSequenceCounters.get(normalizedOrderNumber) ?? 0) + 1
|
||||||
|
this.videoSequenceCounters.set(normalizedOrderNumber, sequence)
|
||||||
|
|
||||||
|
const fileName =
|
||||||
|
sequence === 1 ? `${normalizedOrderNumber}.webm` : `${normalizedOrderNumber}__${sequence}.webm`
|
||||||
|
const finalPath = path.join(finalDir, fileName)
|
||||||
|
|
||||||
|
try {
|
||||||
|
await fs.mkdir(finalDir, { recursive: true })
|
||||||
|
await video.saveAs(finalPath)
|
||||||
|
|
||||||
|
const artifacts = this.recordedVideos.get(orderNumber) ?? []
|
||||||
|
artifacts.push({
|
||||||
|
orderNumber,
|
||||||
|
sequence,
|
||||||
|
finalPath
|
||||||
|
})
|
||||||
|
this.recordedVideos.set(orderNumber, artifacts)
|
||||||
|
|
||||||
|
log.info('订单视频已归档', {
|
||||||
|
orderNumber,
|
||||||
|
filePath: finalPath,
|
||||||
|
sequence
|
||||||
|
})
|
||||||
|
} catch (error) {
|
||||||
|
log.warn('订单视频归档失败', {
|
||||||
|
orderNumber,
|
||||||
|
filePath: finalPath,
|
||||||
|
error: error instanceof Error ? error.message : String(error)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private getVideoBatchDir(batchId: string): string {
|
||||||
|
return path.join(getLogDir(), 'clearner-video-records', batchId)
|
||||||
|
}
|
||||||
|
|
||||||
|
private sanitizeVideoFileName(value: string): string {
|
||||||
|
return value.replace(/[<>:"/\\|?*\x00-\x1F]/g, '_')
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { chromium } from 'playwright'
|
import { chromium } from 'playwright'
|
||||||
|
import * as fs from 'fs/promises'
|
||||||
import type { ErpConfig, ErpSession } from '../../types/erp.types'
|
import type { ErpConfig, ErpSession } from '../../types/erp.types'
|
||||||
import { createLogger } from '../logger'
|
import { createLogger } from '../logger'
|
||||||
import { capturePageContext } from './erp-error-context'
|
import { capturePageContext } from './erp-error-context'
|
||||||
@@ -52,7 +53,15 @@ export class ErpAuthService {
|
|||||||
viewport: { width: 1920, height: 1080 },
|
viewport: { width: 1920, height: 1080 },
|
||||||
ignoreHTTPSErrors: true, // Ignore SSL certificate errors
|
ignoreHTTPSErrors: true, // Ignore SSL certificate errors
|
||||||
// Disable web security for internal VPN
|
// Disable web security for internal VPN
|
||||||
javaScriptEnabled: true
|
javaScriptEnabled: true,
|
||||||
|
...(this.config.recordVideoDir
|
||||||
|
? {
|
||||||
|
recordVideo: {
|
||||||
|
dir: this.config.recordVideoDir,
|
||||||
|
size: { width: 1920, height: 1080 }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
: {})
|
||||||
})
|
})
|
||||||
|
|
||||||
const page = await context.newPage()
|
const page = await context.newPage()
|
||||||
@@ -223,6 +232,17 @@ export class ErpAuthService {
|
|||||||
await this.session.context.close()
|
await this.session.context.close()
|
||||||
await this.session.browser.close()
|
await this.session.browser.close()
|
||||||
this.session = null
|
this.session = null
|
||||||
|
|
||||||
|
if (this.config.recordVideoDir) {
|
||||||
|
try {
|
||||||
|
await fs.rm(this.config.recordVideoDir, { recursive: true, force: true })
|
||||||
|
} catch (error) {
|
||||||
|
log.warn('清理视频临时目录失败', {
|
||||||
|
recordVideoDir: this.config.recordVideoDir,
|
||||||
|
error: error instanceof Error ? error.message : String(error)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -18,6 +18,9 @@ export interface CleanerInput {
|
|||||||
headless?: boolean
|
headless?: boolean
|
||||||
queryBatchSize?: number
|
queryBatchSize?: number
|
||||||
processConcurrency?: number
|
processConcurrency?: number
|
||||||
|
recordVideo?: boolean
|
||||||
|
videoBatchId?: string
|
||||||
|
videoAttemptNumber?: number
|
||||||
onProgress?: (message: string, progress?: number, extra?: Partial<CleanerProgress>) => void
|
onProgress?: (message: string, progress?: number, extra?: Partial<CleanerProgress>) => void
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -117,7 +117,8 @@ export const validationConfigSchema = z.object({
|
|||||||
*/
|
*/
|
||||||
export const cleanerConfigSchema = z.object({
|
export const cleanerConfigSchema = z.object({
|
||||||
queryBatchSize: z.number().int().min(1).max(100).default(100),
|
queryBatchSize: z.number().int().min(1).max(100).default(100),
|
||||||
processConcurrency: z.number().int().min(1).max(20).default(1)
|
processConcurrency: z.number().int().min(1).max(20).default(1),
|
||||||
|
recordVideo: z.boolean().default(false)
|
||||||
})
|
})
|
||||||
export type CleanerConfig = z.infer<typeof cleanerConfigSchema>
|
export type CleanerConfig = z.infer<typeof cleanerConfigSchema>
|
||||||
|
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ export interface ErpConfig {
|
|||||||
username: string
|
username: string
|
||||||
password: string
|
password: string
|
||||||
headless?: boolean // Optional: override default headless setting
|
headless?: boolean // Optional: override default headless setting
|
||||||
|
recordVideoDir?: string
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface ErpSession {
|
export interface ErpSession {
|
||||||
|
|||||||
@@ -10,6 +10,8 @@ interface CleanerExecutionBarProps {
|
|||||||
setHeadless: (value: boolean) => void
|
setHeadless: (value: boolean) => void
|
||||||
processConcurrency: number
|
processConcurrency: number
|
||||||
updateProcessConcurrency: (value: number) => void
|
updateProcessConcurrency: (value: number) => void
|
||||||
|
recordVideo: boolean
|
||||||
|
updateRecordVideo: (value: boolean) => void
|
||||||
showSettingsMenu: boolean
|
showSettingsMenu: boolean
|
||||||
setShowSettingsMenu: (open: boolean) => void
|
setShowSettingsMenu: (open: boolean) => void
|
||||||
handleExecuteDeletion: () => Promise<void>
|
handleExecuteDeletion: () => Promise<void>
|
||||||
@@ -26,6 +28,8 @@ export function CleanerExecutionBar({
|
|||||||
setHeadless,
|
setHeadless,
|
||||||
processConcurrency,
|
processConcurrency,
|
||||||
updateProcessConcurrency,
|
updateProcessConcurrency,
|
||||||
|
recordVideo,
|
||||||
|
updateRecordVideo,
|
||||||
showSettingsMenu,
|
showSettingsMenu,
|
||||||
setShowSettingsMenu,
|
setShowSettingsMenu,
|
||||||
handleExecuteDeletion,
|
handleExecuteDeletion,
|
||||||
@@ -82,6 +86,22 @@ export function CleanerExecutionBar({
|
|||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
<div className="border-t border-slate-100 pt-3 space-y-3">
|
||||||
|
<div className="flex items-center justify-between">
|
||||||
|
<div>
|
||||||
|
<div className="text-sm font-medium text-slate-800">录制处理视频</div>
|
||||||
|
<div className="text-xs text-slate-500 mt-0.5">
|
||||||
|
为每个订单详情页生成视频,仅保留最终失败订单的视频
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<button
|
||||||
|
onClick={() => void updateRecordVideo(!recordVideo)}
|
||||||
|
className={`transition-colors flex-shrink-0 ml-4 ${recordVideo ? 'text-emerald-500' : 'text-slate-300'}`}
|
||||||
|
>
|
||||||
|
{recordVideo ? <ToggleRight size={32} /> : <ToggleLeft size={32} />}
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
<div className="border-t border-slate-100 pt-3 space-y-3">
|
<div className="border-t border-slate-100 pt-3 space-y-3">
|
||||||
<div>
|
<div>
|
||||||
<div className="text-sm font-medium text-slate-800">并行处理数量</div>
|
<div className="text-sm font-medium text-slate-800">并行处理数量</div>
|
||||||
|
|||||||
@@ -61,7 +61,8 @@ export async function loadCleanerConfig(): Promise<CleanerConfigResult | null> {
|
|||||||
|
|
||||||
return {
|
return {
|
||||||
queryBatchSize: result.data.queryBatchSize,
|
queryBatchSize: result.data.queryBatchSize,
|
||||||
processConcurrency: result.data.processConcurrency
|
processConcurrency: result.data.processConcurrency,
|
||||||
|
recordVideo: result.data.recordVideo
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -120,6 +121,7 @@ export async function runCleanerExecution(params: {
|
|||||||
headless: boolean
|
headless: boolean
|
||||||
queryBatchSize: number
|
queryBatchSize: number
|
||||||
processConcurrency: number
|
processConcurrency: number
|
||||||
|
recordVideo: boolean
|
||||||
selectedManagers: string[]
|
selectedManagers: string[]
|
||||||
}): Promise<CleanerReportData> {
|
}): Promise<CleanerReportData> {
|
||||||
const cleanerDataResult = await window.electron.validation.getCleanerData({
|
const cleanerDataResult = await window.electron.validation.getCleanerData({
|
||||||
@@ -149,7 +151,8 @@ export async function runCleanerExecution(params: {
|
|||||||
dryRun: params.dryRun,
|
dryRun: params.dryRun,
|
||||||
headless: params.headless,
|
headless: params.headless,
|
||||||
queryBatchSize: params.queryBatchSize,
|
queryBatchSize: params.queryBatchSize,
|
||||||
processConcurrency: params.processConcurrency
|
processConcurrency: params.processConcurrency,
|
||||||
|
recordVideo: params.recordVideo
|
||||||
})
|
})
|
||||||
|
|
||||||
const cleanerRunData = response.success ? (response.data as CleanerRunPayload | null) : null
|
const cleanerRunData = response.success ? (response.data as CleanerRunPayload | null) : null
|
||||||
|
|||||||
@@ -60,6 +60,7 @@ export interface CleanerInitializationResult {
|
|||||||
export interface CleanerConfigResult {
|
export interface CleanerConfigResult {
|
||||||
queryBatchSize: number
|
queryBatchSize: number
|
||||||
processConcurrency: number
|
processConcurrency: number
|
||||||
|
recordVideo: boolean
|
||||||
}
|
}
|
||||||
|
|
||||||
// Cleaner operation history types (mirrors preload/index.d.ts)
|
// Cleaner operation history types (mirrors preload/index.d.ts)
|
||||||
|
|||||||
@@ -62,6 +62,7 @@ export function useCleaner() {
|
|||||||
const [headless, setHeadless] = useState(() => getStoredBoolean('cleaner_headless', true))
|
const [headless, setHeadless] = useState(() => getStoredBoolean('cleaner_headless', true))
|
||||||
const [queryBatchSize, setQueryBatchSize] = useState(100)
|
const [queryBatchSize, setQueryBatchSize] = useState(100)
|
||||||
const [processConcurrency, setProcessConcurrency] = useState(1)
|
const [processConcurrency, setProcessConcurrency] = useState(1)
|
||||||
|
const [recordVideo, setRecordVideo] = useState(false)
|
||||||
const [showSettingsMenu, setShowSettingsMenu] = useState(false)
|
const [showSettingsMenu, setShowSettingsMenu] = useState(false)
|
||||||
|
|
||||||
// Inline editing state for manager field (Admin only)
|
// Inline editing state for manager field (Admin only)
|
||||||
@@ -138,6 +139,7 @@ export function useCleaner() {
|
|||||||
if (result) {
|
if (result) {
|
||||||
setQueryBatchSize(result.queryBatchSize)
|
setQueryBatchSize(result.queryBatchSize)
|
||||||
setProcessConcurrency(result.processConcurrency)
|
setProcessConcurrency(result.processConcurrency)
|
||||||
|
setRecordVideo(result.recordVideo)
|
||||||
}
|
}
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
logger.error('Failed to load cleaner config', {
|
logger.error('Failed to load cleaner config', {
|
||||||
@@ -169,6 +171,18 @@ export function useCleaner() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const updateRecordVideo = async (value: boolean) => {
|
||||||
|
setRecordVideo(value)
|
||||||
|
try {
|
||||||
|
await window.electron.config.updateCleaner({ recordVideo: value })
|
||||||
|
} catch (err) {
|
||||||
|
logger.error('Failed to update record video setting', {
|
||||||
|
error: err instanceof Error ? err.message : String(err),
|
||||||
|
value
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
sessionStorage.setItem('cleaner_validationMode', valMode)
|
sessionStorage.setItem('cleaner_validationMode', valMode)
|
||||||
}, [valMode])
|
}, [valMode])
|
||||||
@@ -377,6 +391,7 @@ export function useCleaner() {
|
|||||||
headless,
|
headless,
|
||||||
queryBatchSize,
|
queryBatchSize,
|
||||||
processConcurrency,
|
processConcurrency,
|
||||||
|
recordVideo,
|
||||||
selectedManagers: Array.from(selectedManagers)
|
selectedManagers: Array.from(selectedManagers)
|
||||||
})
|
})
|
||||||
setReportData(result)
|
setReportData(result)
|
||||||
@@ -441,6 +456,8 @@ export function useCleaner() {
|
|||||||
processConcurrency,
|
processConcurrency,
|
||||||
setProcessConcurrency,
|
setProcessConcurrency,
|
||||||
updateProcessConcurrency,
|
updateProcessConcurrency,
|
||||||
|
recordVideo,
|
||||||
|
updateRecordVideo,
|
||||||
showSettingsMenu,
|
showSettingsMenu,
|
||||||
setShowSettingsMenu,
|
setShowSettingsMenu,
|
||||||
filteredResults,
|
filteredResults,
|
||||||
|
|||||||
@@ -42,6 +42,8 @@ const CleanerPage: React.FC = () => {
|
|||||||
setHeadless,
|
setHeadless,
|
||||||
processConcurrency,
|
processConcurrency,
|
||||||
updateProcessConcurrency,
|
updateProcessConcurrency,
|
||||||
|
recordVideo,
|
||||||
|
updateRecordVideo,
|
||||||
showSettingsMenu,
|
showSettingsMenu,
|
||||||
setShowSettingsMenu,
|
setShowSettingsMenu,
|
||||||
filteredResults,
|
filteredResults,
|
||||||
@@ -126,6 +128,8 @@ const CleanerPage: React.FC = () => {
|
|||||||
setHeadless={setHeadless}
|
setHeadless={setHeadless}
|
||||||
processConcurrency={processConcurrency}
|
processConcurrency={processConcurrency}
|
||||||
updateProcessConcurrency={updateProcessConcurrency}
|
updateProcessConcurrency={updateProcessConcurrency}
|
||||||
|
recordVideo={recordVideo}
|
||||||
|
updateRecordVideo={updateRecordVideo}
|
||||||
showSettingsMenu={showSettingsMenu}
|
showSettingsMenu={showSettingsMenu}
|
||||||
setShowSettingsMenu={setShowSettingsMenu}
|
setShowSettingsMenu={setShowSettingsMenu}
|
||||||
handleExecuteDeletion={handleExecuteDeletion}
|
handleExecuteDeletion={handleExecuteDeletion}
|
||||||
|
|||||||
Reference in New Issue
Block a user