185 lines
7.2 KiB
Markdown
185 lines
7.2 KiB
Markdown
# AV1 Batch Video Transcoder - Project Structure
|
|
|
|
## Overview
|
|
A modular batch AV1 video transcoding system using NVIDIA's av1_nvenc codec (10-bit p010le) with intelligent audio/video processing, subtitle embedding, and optional audio language tagging.
|
|
|
|
## Recent Changes (Latest Session)
|
|
|
|
### Removed
|
|
- ❌ **Sonarr/Radarr integration** - Removed helper module, cache loading, and config sections (simplified to basic tagging)
|
|
- ❌ **Auto-rename functionality** - No longer renames based on Sonarr metadata
|
|
- ❌ **Web UI** - Removed `/webui` folder (can be added back if needed)
|
|
- ❌ **Rename tool** - Moved to separate `/rename` folder
|
|
|
|
### Added
|
|
- ✅ **Subtitle detection & embedding** - Auto-finds .vtt, .srt, .ass, .ssa, .sub files (including language-prefixed like .en.vtt)
|
|
- ✅ **Subtitle cleanup** - Deletes embedded subtitle files after successful encoding
|
|
- ✅ **Test mode** (`--test`) - Encodes first file, shows compression ratio, doesn't move files
|
|
- ✅ **Optional language tagging** (`--language`) - Only tags audio if explicitly provided (default: no tagging)
|
|
- ✅ **Always output MKV** - Changed from using source extension to always outputting .mkv
|
|
- ✅ **Improved subtitle matching** - Finds both exact matches (video.vtt) and language-prefixed (video.en.vtt)
|
|
|
|
### Refactored
|
|
- 🔧 **File structure reorganization**: Moved path_manager GUI, rename tool, and cache to separate folders
|
|
- 🔧 **Config simplification**: Removed Sonarr/Radarr sections, cleaner general settings
|
|
- 🔧 **Suffix handling**: Applied once during encoding, moved directly without re-tagging
|
|
- 🔧 **Audio language**: Changed from config-based default to CLI-only optional flag
|
|
|
|
## Architecture
|
|
|
|
### Entry Point
|
|
- **main.py** - CLI with argparse
|
|
- Arguments: `folder`, `--cq`, `--m {cq,bitrate}`, `--r {480,720,1080}`, `--test`, `--language`
|
|
- Loads config.xml, initializes logging
|
|
- Calls `process_folder()` from process_manager
|
|
|
|
### Core Modules
|
|
|
|
#### `core/config_helper.py`
|
|
- **`load_config_xml(config_path)`** - Parses XML configuration
|
|
- Returns dict with keys:
|
|
- `general`: processing_folder, suffix (" - [EHX]"), extensions, reduction_ratio_threshold, subtitles config
|
|
- `encode.cq`: CQ values per content type (tv_1080, tv_720, movie_1080, movie_720)
|
|
- `encode.fallback`: Bitrate fallback (Phase 2 retry)
|
|
- `audio`: Bitrate buckets for stereo/multichannel
|
|
- `path_mappings`: Windows ↔ Linux path conversion
|
|
|
|
#### `core/logger_helper.py`
|
|
- Sets up logging to `logs/conversion.log` (INFO+) and console (DEBUG+)
|
|
- Separate failure logger for `logs/conversion_failures.log`
|
|
- Captures encoding decisions, bitrates, resolutions, timings
|
|
|
|
#### `core/process_manager.py`
|
|
- **`process_folder(folder, cq, transcode_mode, resolution, config, tracker_file, test_mode, audio_language)`**
|
|
- Scans folder for video files
|
|
- **Per file**: Copy to temp, detect subtitles, analyze streams, encode, move, cleanup
|
|
- **Subtitle detection**: Looks for exact match + glob pattern (filename.*.ext)
|
|
- **Phase 1 (CQ)**: Try CQ-based encoding, check size threshold
|
|
- **Phase 2 (Bitrate)**: Retry failed files with bitrate mode
|
|
- **Cleanup**: Delete original + subtitle + temp copies on success
|
|
- **`_save_successful_encoding(...)`** - Moves file from temp → original folder
|
|
- File already has ` - [EHX]` suffix from temp_output filename
|
|
- Deletes original file, subtitle file, and temp copies
|
|
- Logs to CSV tracker
|
|
|
|
#### `core/encode_engine.py`
|
|
- **`run_ffmpeg(input_file, output_file, cq, scale_width, scale_height, src_width, src_height, filter_flags, audio_config, method, bitrate_config, subtitle_file, audio_language)`**
|
|
- Builds FFmpeg command with av1_nvenc codec (preset p1, pix_fmt p010le)
|
|
- Per-stream audio codec/bitrate decisions
|
|
- Conditional subtitle input mapping (if subtitle_file provided)
|
|
- Optional audio language metadata (only if audio_language not None)
|
|
- Returns: (orig_size, out_size, reduction_ratio)
|
|
|
|
#### `core/audio_handler.py`
|
|
- **`get_audio_streams(input_file)`** - Detects all audio streams with bitrate info
|
|
- **`choose_audio_bitrate(channels, avg_bitrate, audio_config, is_1080_class)`** - Returns (codec, target_bitrate) tuple
|
|
- Stereo 1080p: >192k → encode to 192k, ≤192k → copy
|
|
- Stereo 720p: >160k → encode to 160k, ≤160k → copy
|
|
- Multichannel: Encode to 384k (low) or 448k (medium)
|
|
|
|
#### `core/video_handler.py`
|
|
- **`get_source_resolution(input_file)`** - ffprobe detection
|
|
- **`determine_target_resolution(src_width, src_height, explicit_resolution)`** - Smart scaling
|
|
- If >1080p → scale to 1080p
|
|
- Else → preserve source
|
|
- Override with `--r {480,720,1080}`
|
|
|
|
## Workflow Example
|
|
|
|
```bash
|
|
python main.py "P:\tv\Supernatural\Season 7" --language eng
|
|
```
|
|
|
|
**Processing:**
|
|
1. Scan folder for .mkv/.mp4 files
|
|
2. For each file:
|
|
- Copy to `processing/Supernatural - S07E01 - Pilot.mkv`
|
|
- Look for subtitle: `Supernatural - S07E01 - Pilot.en.vtt` ✓ found
|
|
- Detect source: 1920x1080 (1080p) ✓
|
|
- Get audio streams: [AAC 2ch @ 192k, AC3 6ch @ 448k]
|
|
- Determine CQ: tv_1080 → CQ 28
|
|
- Build FFmpeg command:
|
|
- Video: av1_nvenc (CQ 28)
|
|
- Audio 0: Copy AAC (≤192k already good)
|
|
- Audio 1: Re-encode AC3 to AAC 6ch @ 448k
|
|
- Subtitles: Input subtitle, map as srt stream, language=eng
|
|
- Output: `processing/Supernatural - S07E01 - Pilot - [EHX].mkv`
|
|
- FFmpeg runs, outputs ~400MB (original 1.2GB)
|
|
- Check size: 400/1200 = 33.3% < 75% ✓ SUCCESS
|
|
- Move: `processing/... - [EHX].mkv` → `P:\tv\Supernatural\Season 7/... - [EHX].mkv`
|
|
- Cleanup: Delete original + subtitle + temp copy
|
|
- Log to CSV
|
|
|
|
**Result:**
|
|
- Original files gone
|
|
- New `Supernatural - S07E01 - Pilot - [EHX].mkv` (subtitle embedded, audio tagged with language=eng)
|
|
|
|
## Configuration
|
|
|
|
### config.xml Key Sections
|
|
|
|
```xml
|
|
<general>
|
|
<processing_folder>processing</processing_folder>
|
|
<suffix> - [EHX]</suffix>
|
|
<extensions>.mkv,.mp4</extensions>
|
|
<reduction_ratio_threshold>0.75</reduction_ratio_threshold>
|
|
<subtitles>
|
|
<enabled>true</enabled>
|
|
<extensions>.vtt,.srt,.ass,.ssa,.sub</extensions>
|
|
<codec>srt</codec>
|
|
</subtitles>
|
|
</general>
|
|
|
|
<encode>
|
|
<cq>
|
|
<tv_1080>28</tv_1080>
|
|
<tv_720>32</tv_720>
|
|
<movie_1080>32</movie_1080>
|
|
<movie_720>34</movie_720>
|
|
</cq>
|
|
</encode>
|
|
|
|
<audio>
|
|
<stereo>
|
|
<high>192000</high>
|
|
<medium>160000</medium>
|
|
</stereo>
|
|
<multi_channel>
|
|
<medium>448000</medium>
|
|
<low>384000</low>
|
|
</multi_channel>
|
|
</audio>
|
|
```
|
|
|
|
## File Movements
|
|
|
|
```
|
|
Original:
|
|
P:\tv\Show\Episode.mkv (1.2GB)
|
|
P:\tv\Show\Episode.en.vtt
|
|
|
|
During Encoding:
|
|
processing/Episode.mkv (temp copy)
|
|
processing/Episode - [EHX].mkv (encoding output)
|
|
|
|
After Success:
|
|
P:\tv\Show\Episode - [EHX].mkv (1.2GB → 400MB)
|
|
(original .mkv deleted)
|
|
(original .en.vtt deleted)
|
|
(temp folder cleaned)
|
|
```
|
|
|
|
## Validation Checklist
|
|
|
|
- ✅ All core modules import correctly
|
|
- ✅ Config loads without Sonarr/Radarr references
|
|
- ✅ Subtitle detection finds exact matches + language-prefixed files
|
|
- ✅ Audio language tagging only applied with --language flag
|
|
- ✅ Output always MKV regardless of source format
|
|
- ✅ Suffix applied once (in temp output filename)
|
|
- ✅ Subtitle files deleted with original files
|
|
- ✅ Test mode shows compression ratio and stops
|
|
- ✅ Phase 1 (CQ) and Phase 2 (Bitrate) retry logic works
|
|
- ✅ CSV tracking logs all conversions
|