refactor(logging): Add optional logging support throughout codebase

Add centralized logging utility and optional logger parameters to all core functions for better observability and debugging capabilities. New modules: - utils/logging.py: Centralized logger configuration with console and optional file handlers Enhanced features: - Added optional logger parameter to all extractor_core functions - Added logger support to extractor, excel_converter, and auth modules - Functions remain silent when logger=None (backward compatible) - Improved environment variable validation in test files Documentation: - Added discrete_material_plan_extractor_core.md with complete API reference and usage patterns Benefits: - Consistent logging format across all components - Optional debug output for troubleshooting - No breaking changes - fully backward compatible - Better error messages and validation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 16:08:21 +08:00
parent c3bbc919a5
commit e7bbbbc194
10 changed files with 790 additions and 138 deletions
--- a/docs/discrete_material_plan_extractor_core.md
+++ b/docs/discrete_material_plan_extractor_core.md
@@ -0,0 +1,426 @@
+# extractor_core.py - API Reference
+
+> **Core Web Operations for Yonyou BIP**
+>
+> Low-level pure functions for interacting with the discrete material plan maintenance page.
+
+---
+
+## Overview
+
+The `extractor_core.py` module provides low-level pure functions for web operations on the Yonyou BIP ERP system. All functions are **stateless** and accept an optional `logger` parameter for observability.
+
+**Design Principles:**
+- ✅ **Silent by default**: Functions produce no output when `logger=None`
+- ✅ **Optional logging**: Pass a logger to get debug/info messages
+- ✅ **Stateless**: No internal state, all dependencies explicit
+- ✅ **Pure functions**: Same inputs → same outputs, no side effects
+
+**Module Location:** `utils/discrete_material_plan/extractor_core.py`
+
+---
+
+## Functions
+
+### `navigate_to_discrete_material_page(main_frame, page, logger=None)`
+
+Navigate to the discrete material plan maintenance page.
+
+**Parameters:**
+| Name | Type | Default | Description |
+|------|------|---------|-------------|
+| `main_frame` | `Frame` | - | The main forwardFrame iframe |
+| `page` | `Page` | - | The Playwright page object |
+| `logger` | `Optional[logging.Logger]` | `None` | Optional logger for debug output |
+
+**Returns:** `tuple[FrameLocator, Page]` - (work_frame, page1)
+
+**Behavior:**
+- Clicks menu icon to open navigation
+- Opens popup and clicks "离散备料计划维护" menu item
+- Navigates through nested iframes to return work frame
+
+**Logging (when logger provided):**
+- `DEBUG`: "Navigated to discrete material plan page"
+
+**Example:**
+```python
+from utils.discrete_material_plan.extractor_core import navigate_to_discrete_material_page
+import logging
+
+# Silent mode (default)
+work_frame, page1 = navigate_to_discrete_material_page(main_frame, page)
+
+# With logging
+logging.basicConfig(level=logging.DEBUG)
+logger = logging.getLogger(__name__)
+work_frame, page1 = navigate_to_discrete_material_page(main_frame, page, logger)
+```
+
+---
+
+### `setup_query_interface(work_frame, logger=None)`
+
+Initialize the query interface by selecting order number query tab.
+
+**Parameters:**
+| Name | Type | Default | Description |
+|------|------|---------|-------------|
+| `work_frame` | `FrameLocator` | - | The inner work iframe |
+| `logger` | `Optional[logging.Logger]` | `None` | Optional logger for debug output |
+
+**Returns:** `None`
+
+**Behavior:**
+- Opens search panel
+- Selects "订单号查询" (order number query) tab
+- Selects "全部" (All) tab
+- Sets page size to 5000
+
+**Logging (when logger provided):**
+- `DEBUG`: "Query interface setup complete"
+
+**Example:**
+```python
+from utils.discrete_material_plan.extractor_core import setup_query_interface
+
+# Silent mode
+setup_query_interface(work_frame)
+
+# With logging
+setup_query_interface(work_frame, logger)
+```
+
+---
+
+### `fill_and_search_orders(work_frame, order_ids, logger=None)`
+
+Fill order IDs into the search textbox and trigger search.
+
+**Parameters:**
+| Name | Type | Default | Description |
+|------|------|---------|-------------|
+| `work_frame` | `FrameLocator` | - | The work iframe containing search form |
+| `order_ids` | `List[str]` | - | List of order IDs to search for |
+| `logger` | `Optional[logging.Logger]` | `None` | Optional logger for debug output |
+
+**Returns:** `None`
+
+**Behavior:**
+- Clears and fills order IDs into "来源生产订单号" textbox
+- Clicks search button
+- Waits for loading indicator (3s timeout, continues if not found)
+
+**Logging (when logger provided):**
+- `DEBUG`: "Loading indicator timeout - continuing anyway" (if timeout occurs)
+- `DEBUG`: "Searched for {n} order IDs" (after search)
+
+**Example:**
+```python
+from utils.discrete_material_plan.extractor_core import fill_and_search_orders
+
+# Silent mode
+fill_and_search_orders(work_frame, ["SC70202603240001", "SC70202603240002"])
+
+# With logging
+fill_and_search_orders(work_frame, order_ids, logger)
+```
+
+---
+
+### `download_batch_data(work_frame, page, order_ids, batch_index, download_dir, logger=None)`
+
+Execute the download workflow for a single batch of order IDs.
+
+**Parameters:**
+| Name | Type | Default | Description |
+|------|------|---------|-------------|
+| `work_frame` | `FrameLocator` | - | The work iframe containing data grid |
+| `page` | `Page` | - | Playwright page for download handling |
+| `order_ids` | `List[str]` | - | List of order IDs (for context) |
+| `batch_index` | `int` | - | Zero-based batch index |
+| `download_dir` | `str` | - | Directory to save downloaded file |
+| `logger` | `Optional[logging.Logger]` | `None` | Optional logger for info output |
+
+**Returns:** `str` - Full path to downloaded file
+
+**Behavior:**
+1. Selects first row in grid
+2. Hovers over "更多" (More) button
+3. Clicks "输出" (Export)
+4. Sets row threshold to 300000
+5. Triggers download and saves as `temp_batch_{n}.xlsx`
+
+**Logging (when logger provided):**
+- `INFO`: "Downloaded batch {n} to {path}"
+
+**Raises:**
+- `TimeoutError`: If UI elements not found or operations timeout
+
+**Example:**
+```python
+from utils.discrete_material_plan.extractor_core import download_batch_data
+
+# Silent mode
+file_path = download_batch_data(
+    work_frame, page, order_ids, batch_index=0, download_dir="./downloads"
+)
+
+# With logging
+file_path = download_batch_data(
+    work_frame, page, order_ids, 0, "./downloads", logger
+)
+# INFO: Downloaded batch 1 to ./downloads/temp_batch_1.xlsx
+```
+
+---
+
+### `execute_batch_download_workflow(work_frame, page, order_ids, batch_index, download_dir, logger=None)`
+
+Complete workflow: fill orders, search, and download for a single batch.
+
+**Parameters:**
+| Name | Type | Default | Description |
+|------|------|---------|-------------|
+| `work_frame` | `FrameLocator` | - | Work iframe with search form and data grid |
+| `page` | `Page` | - | Playwright page for download |
+| `order_ids` | `List[str]` | - | List of order IDs for this batch |
+| `batch_index` | `int` | - | Zero-based batch index |
+| `download_dir` | `str` | - | Directory to save downloaded file |
+| `logger` | `Optional[logging.Logger]` | `None` | Optional logger for debug output |
+
+**Returns:** `str` - Full path to downloaded file
+
+**Behavior:**
+- Calls `fill_and_search_orders()` then `download_batch_data()`
+- Propagates logger to both inner calls
+
+**Logging (when logger provided):**
+- All logs from `fill_and_search_orders()` and `download_batch_data()`
+
+**Example:**
+```python
+from utils.discrete_material_plan.extractor_core import execute_batch_download_workflow
+
+# Silent mode
+file_path = execute_batch_download_workflow(
+    work_frame, page, order_ids, batch_index=0, download_dir="./downloads"
+)
+
+# With logging
+file_path = execute_batch_download_workflow(
+    work_frame, page, order_ids, 0, "./downloads", logger
+)
+```
+
+---
+
+## Usage Patterns
+
+### Pattern 1: Silent Mode (Default)
+
+```python
+from utils.discrete_material_plan.extractor_core import (
+    navigate_to_discrete_material_page,
+    setup_query_interface,
+    execute_batch_download_workflow,
+)
+
+# No output, completely silent
+work_frame, page1 = navigate_to_discrete_material_page(main_frame, page)
+setup_query_interface(work_frame)
+file_path = execute_batch_download_workflow(
+    work_frame, page, order_ids, 0, "./downloads"
+)
+```
+
+### Pattern 2: With Logging
+
+```python
+import logging
+from utils.discrete_material_plan.extractor_core import (
+    navigate_to_discrete_material_page,
+    setup_query_interface,
+    execute_batch_download_workflow,
+)
+
+# Configure logging
+logging.basicConfig(
+    level=logging.DEBUG,
+    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
+)
+logger = logging.getLogger(__name__)
+
+# Verbose output
+work_frame, page1 = navigate_to_discrete_material_page(main_frame, page, logger)
+setup_query_interface(work_frame, logger)
+file_path = execute_batch_download_workflow(
+    work_frame, page, order_ids, 0, "./downloads", logger
+)
+
+# Output:
+# DEBUG:__main__:Navigated to discrete material plan page
+# DEBUG:__main__:Query interface setup complete
+# DEBUG:__main__:Searched for 10 order IDs
+# INFO:__main__:Downloaded batch 1 to ./downloads/temp_batch_1.xlsx
+```
+
+### Pattern 3: Complete Workflow
+
+```python
+from playwright.sync_api import sync_playwright
+from utils.auth import login
+from utils.discrete_material_plan.extractor_core import (
+    navigate_to_discrete_material_page,
+    setup_query_interface,
+    execute_batch_download_workflow,
+)
+import logging
+
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+with sync_playwright() as playwright:
+    browser, context, page, main_frame = login(
+        playwright, "user", "pass",
+        url="https://erp.example.com/...",
+        headless=True,
+        ignore_https_errors=True,
+    )
+    
+    try:
+        # Navigate and setup
+        work_frame, page1 = navigate_to_discrete_material_page(main_frame, page, logger)
+        setup_query_interface(work_frame, logger)
+        
+        # Download batches
+        order_ids = ["SC70202603240001", "SC70202603240002", ...]
+        file_path = execute_batch_download_workflow(
+            work_frame, page1, order_ids, 0, "./downloads", logger
+        )
+        
+    finally:
+        context.close()
+        browser.close()
+```
+
+---
+
+## Error Handling
+
+### TimeoutError on Loading Indicator
+
+The `fill_and_search_orders()` function handles loading indicator timeouts gracefully:
+
+```python
+try:
+    loading_locator.wait_for(state="visible", timeout=3000)
+    loading_locator.wait_for(state="hidden", timeout=0)
+except TimeoutError:
+    if logger:
+        logger.debug("Loading indicator timeout - continuing anyway")
+    # Continues execution - not a fatal error
+```
+
+### When to Raise vs Log
+
+- **Timeout on loading indicator**: Log at DEBUG, continue (non-fatal)
+- **Missing UI elements**: Raise TimeoutError (fatal)
+- **Download failures**: Raise exception (fatal)
+
+---
+
+## Architecture
+
+### Function Call Flow
+
+```mermaid
+graph TD
+    A[User Script] --> B[navigate_to_discrete_material_page]
+    A --> C[setup_query_interface]
+    A --> D[execute_batch_download_workflow]
+    
+    D --> E[fill_and_search_orders]
+    D --> F[download_batch_data]
+    
+    E --> G[fill_and_search_orders implementation]
+    F --> H[download_batch_data implementation]
+    
+    style A fill:#e1f5ff
+    style B fill:#fff4e1
+    style C fill:#fff4e1
+    style D fill:#e8f5e9
+    style E fill:#fce4ec
+    style F fill:#fce4ec
+```
+
+### Type Hierarchy
+
+```
+Page (from playwright)
+  └─> main_frame: Frame
+       └─> navigate_to_discrete_material_page()
+            └─> Returns: FrameLocator (work_frame)
+                 └─> setup_query_interface(work_frame: FrameLocator)
+                 └─> fill_and_search_orders(work_frame: FrameLocator)
+                 └─> download_batch_data(work_frame: FrameLocator)
+```
+
+---
+
+## Testing
+
+### Import Test
+
+```bash
+python -c "from utils.discrete_material_plan.extractor_core import fill_and_search_orders; print('[OK] Import works')"
+```
+
+### Silent Mode Test
+
+```python
+from utils.discrete_material_plan.extractor_core import fill_and_search_orders
+
+# Should produce ZERO output
+fill_and_search_orders(mock_frame, ["ID1", "ID2"])
+print("Silent mode: OK")
+```
+
+### With Logger Test
+
+```python
+import logging
+from utils.discrete_material_plan.extractor_core import fill_and_search_orders
+
+logging.basicConfig(level=logging.DEBUG)
+logger = logging.getLogger(__name__)
+
+# Should produce debug output
+fill_and_search_orders(mock_frame, ["ID1", "ID2"], logger)
+```
+
+---
+
+## Related Documentation
+
+- [`discrete_material_plan_extractor_api.md`](discrete_material_plan_extractor_api.md) - High-level extractor API
+- [`extractor_post_processing.md`](extractor_post_processing.md) - Excel conversion
+- [`authentication.md`](authentication.md) - Auth and session management
+
+---
+
+## Version
+
+**Current:** v2.1.0 (added optional logger parameter)
+
+**Changes in v2.1.0:**
+- Added optional `logger` parameter to all functions
+- Functions remain silent when `logger=None`
+- TimeoutError handling now logs at DEBUG level
+- No breaking changes - backward compatible
+
+---
+
+## License
+
+Part of BIPAuto project - see project root for license information.