142 lines
4.8 KiB
Markdown
142 lines
4.8 KiB
Markdown
# Queue Retry Feature Documentation
|
|
|
|
## Overview
|
|
Added smart retry logic for batch queue processing to handle temporary source availability issues. When a batch run completes with 2+ "Folder not found" errors (indicating the source is down), the system will automatically retry at regular intervals for a configurable timeout period.
|
|
|
|
## New CLI Parameters
|
|
|
|
### `--retry-minutes` (int, default: from config.xml)
|
|
Minutes to wait between retry attempts when batch fails with 2+ "Folder not found" errors.
|
|
- Default: 10 minutes (configurable in config.xml)
|
|
- Example: `python main.py --paths-file paths.txt --retry-minutes 5`
|
|
|
|
### `--retry-timeout` (int, default: from config.xml)
|
|
Total minutes to keep retrying before giving up.
|
|
- Default: 60 minutes (configurable in config.xml)
|
|
- If source becomes reachable during retry window, queue immediately restarts
|
|
- Example: `python main.py --paths-file paths.txt --retry-timeout 120`
|
|
|
|
## Configuration (config.xml)
|
|
|
|
Added new section `<queue_retry>` in config.xml with defaults:
|
|
|
|
```xml
|
|
<queue_retry>
|
|
<!-- Minutes to wait between retry attempts -->
|
|
<retry_minutes>10</retry_minutes>
|
|
|
|
<!-- Total minutes to keep retrying before giving up -->
|
|
<retry_timeout>60</retry_timeout>
|
|
</queue_retry>
|
|
```
|
|
|
|
## How It Works
|
|
|
|
### Trigger Conditions
|
|
Retry logic activates when ALL of the following are true:
|
|
1. Queue processing completes
|
|
2. Failed count > 2
|
|
3. All failures are "Folder not found" type (indicates source path is unreachable)
|
|
|
|
### Retry Loop Behavior
|
|
Once activated, the system enters a retry loop that:
|
|
|
|
1. **Checks source reachability** every N minutes (--retry-minutes)
|
|
2. **If source becomes reachable:**
|
|
- Immediately restarts batch queue processing
|
|
- Processes all previously failed items
|
|
- Displays detailed progress with timestamps
|
|
3. **If source stays unreachable:**
|
|
- Displays countdown showing elapsed/remaining time
|
|
- Continues checking at configured interval
|
|
4. **If timeout exceeded:**
|
|
- Exits retry loop
|
|
- Displays final summary showing total attempts
|
|
|
|
### Example Output
|
|
|
|
```
|
|
================================================================================
|
|
✓ BATCH PROCESSING COMPLETE
|
|
Total items processed: 33
|
|
✓ Succeeded: 8
|
|
❌ Failed: 25
|
|
================================================================================
|
|
|
|
================================================================================
|
|
⚠️ RETRY LOGIC TRIGGERED
|
|
Failed items: 25 | Folder not found: 25
|
|
This suggests the source location (P:\tv\Show\Season 3) may be temporarily unreachable
|
|
================================================================================
|
|
|
|
🔄 Retry Configuration:
|
|
- Retry interval: 10 minute(s)
|
|
- Total retry timeout: 60 minute(s)
|
|
- Will check source availability and retry if it becomes reachable
|
|
|
|
⏳ Attempt 1: Source not yet reachable
|
|
Elapsed: 0.0m | Remaining: 60.0m
|
|
Waiting 10 minute(s) before next check...
|
|
|
|
⏳ Attempt 2: Source not yet reachable
|
|
Elapsed: 10.1m | Remaining: 49.9m
|
|
Waiting 10 minute(s) before next check...
|
|
|
|
✅ Source is now reachable! (P:\tv\Show\Season 3)
|
|
Restarting queue processing...
|
|
|
|
================================================================================
|
|
🔃 RESTARTING BATCH QUEUE
|
|
================================================================================
|
|
|
|
📋 Found 25 item(s) to retry
|
|
[Processing continues with failed items...]
|
|
```
|
|
|
|
## Usage Examples
|
|
|
|
### Using defaults (10 min interval, 60 min timeout)
|
|
```bash
|
|
python main.py --paths-file paths.txt
|
|
```
|
|
|
|
### Custom retry interval (5 minute checks, 120 minute timeout)
|
|
```bash
|
|
python main.py --paths-file paths.txt --retry-minutes 5 --retry-timeout 120
|
|
```
|
|
|
|
### Disable retry by setting timeout to 0
|
|
```bash
|
|
python main.py --paths-file paths.txt --retry-timeout 0
|
|
```
|
|
|
|
### Quick test (1 minute interval, 5 minute timeout)
|
|
```bash
|
|
python main.py --paths-file paths.txt --retry-minutes 1 --retry-timeout 5
|
|
```
|
|
|
|
## Implementation Details
|
|
|
|
### New Functions
|
|
- `is_path_reachable(path: Path) -> bool`: Checks if a path (network share, etc.) is accessible
|
|
|
|
### Modified Components
|
|
- **main.py:**
|
|
- Added import: `import time`
|
|
- Added `--retry-minutes` and `--retry-timeout` CLI arguments
|
|
- Added `is_path_reachable()` helper function
|
|
- Added retry loop logic after batch queue completion
|
|
- Detects folder-not-found failures and triggers retry logic
|
|
|
|
- **config.xml:**
|
|
- Added `<queue_retry>` section with default values
|
|
|
|
## Notes
|
|
|
|
- Retry logic **only activates** for queue mode (--paths-file flag)
|
|
- Single-file processing is not affected
|
|
- Retry attempts are logged with timestamps for audit trail
|
|
- Network paths (e.g., P:\tv) are checked using `Path.exists()` which properly handles network timeouts
|
|
- Each retry attempt shows elapsed and remaining time for transparency
|
|
- If source becomes available, all failed items are re-processed with original parameters
|