# FFmpeg 8.x Features Used by FFmpeg Media


This module targets FFmpeg 8.x and relies on features that are available in modern FFmpeg builds. Not every feature is required; availability depends on how FFmpeg was compiled on your system.

---

## 1. Codecs

Supported via codec plugins (if the encoder exists in your FFmpeg build):

- **H.264** (`libx264`, plus hardware encoders)
- **HEVC** (`libx265`, plus hardware encoders)
- **VP9** (`libvpx-vp9`)
- **AV1** (`libsvtav1`, `libaom-av1`, plus hardware encoders)
- **VVC/H.266** (`libvvenc`) - experimental
- **FFV1** (lossless archival)
- **ProRes** (`prores_ks`)
- **JPEG XL** (`libjxl`) - still-image extraction from video frames

---

## 2. Hardware Acceleration

Hardware encoders are used when detected and enabled:

- **NVENC** (NVIDIA)
- **CUDA** (NVIDIA compute framework)
- **VAAPI** (Intel/AMD on Linux)
- **QSV** (Intel Quick Sync)
- **VideoToolbox** (macOS)
- **AMF** (AMD)
- **Vulkan** (cross-platform GPU processing)
- **OpenHarmony / MediaCodec NDK** (HarmonyOS and Android NDK)

The module includes a hardware pipeline builder (init devices, hwupload, filter chains) and a circuit breaker to fall back to software when hardware repeatedly fails.

---

## 3. Streaming

The streaming submodule provides HLS and DASH output plugins with:

- Adaptive representation ladders
- Configurable segment duration and naming patterns
- Per-title encoding (BitrateAnalyzer) when video metadata is available
- External subtitle tracks (WebVTT) and multi-audio track mapping

---

## 4. Whisper Transcription

The `ffmpeg_media_whisper` submodule generates subtitles from audio using the [whisper.cpp](https://github.com/ggerganov/whisper.cpp) CLI tool. It uses a two-step pipeline: FFmpeg extracts audio to 16kHz mono WAV, then `whisper-cli` transcribes it to SRT/VTT.

### Prerequisites

1. **Install whisper.cpp** (CLI binary + shared library):

```bash
git clone --branch v1.7.5 https://github.com/ggerganov/whisper.cpp.git
cd whisper.cpp
cmake -B build -DCMAKE_INSTALL_PREFIX=/usr/local -DBUILD_SHARED_LIBS=ON
cmake --build build -j$(nproc)
sudo cmake --install build
sudo ldconfig
```

2. **Download a GGML model**:

```bash
# Available: tiny, base, small, medium, large-v2, large-v3 (+ .en variants)
mkdir -p /usr/local/share/whisper/models
wget https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.bin \
  -O /usr/local/share/whisper/models/ggml-base.bin
```

3. **Verify** the CLI is available:

```bash
whisper-cli --help
```

### Activation in Drupal

1. Enable the submodule: `drush en ffmpeg_media_whisper`
2. Configure CLI path and model directory at `/admin/config/media/ffmpeg-media/whisper`
3. Enable the Whisper processor in your conversion profile at `/admin/config/media/ffmpeg-media/profiles`
4. Configure transcript output in the FFmpeg formatter settings on your media bundle's Manage Display page
5. Trigger transcription via the "Generate Transcript" button on any media status page, or run `drush ffmpeg:transcribe <media_id>`

### Generated Commands

```bash
# Step 1: Extract audio (FFmpeg)
ffmpeg -y -i input.mp4 -ar 16000 -ac 1 -f wav /tmp/whisper_XXXX.wav

# Step 2: Transcribe (whisper-cli)
whisper-cli -m /usr/local/share/whisper/models/ggml-base.bin -l auto --beam-size 5 --max-context 224 -osrt -of output /tmp/whisper_XXXX.wav
```

---

## 5. Filters Used by Processors

Depending on enabled processors, the module may require:

- `scale` and `fps` (thumbnails / GIFs)
- `zscale` (HDR tone mapping)
- `loudnorm` (audio normalization)

---

## 6. Build Recommendations (Optional)

For the best experience, build FFmpeg with:

- `--enable-libx264 --enable-libx265 --enable-libvpx --enable-libsvtav1`
- `--enable-libvvenc` (VVC) if needed
- `--enable-libjxl` if using JPEG XL output
- whisper.cpp CLI for AI transcription (see section 4)
- GPU acceleration flags for your platform (NVENC / VAAPI / QSV / Vulkan)
