# Video transcoding

Source: https://frameleaf.app/docs/admin/video-transcoding/

Your videos play smoothly on every device, and your originals stay exactly as you shot them. Frameleaf makes a **playback copy** when the original won't play well everywhere, for example because of its format, file type or size. Playback copies are stored in `UPLOAD_LOCATION/encoded-video` and can always be made again. They're made by the **Converting videos** background work.

Edits you render in the editor and Studio exports use the same settings, including the graphics card. When an edit needs filters that only run on the processor, rendering switches to the processor by itself.

## Settings

Find these in **Settings**, then **Editing & playback**, then **Video playback copies**. [System settings](https://frameleaf.app/docs/admin/system-settings/#video-playback-copies) lists every control on that screen.

<span id="transcode-policy"></span>

### Which videos get a copy

The **Rule** under **Advanced: which videos need a copy** decides which videos get a playback copy. Called the transcode policy. HDR videos, and videos whose colour layout is not the common one (YUV 4:2:0, called the pixel format), always get a copy unless copies are turned off.

| Rule | Gets a copy |
| --- | --- |
| **Only videos that need one** (default) | Only videos not in a format, sound format and file type played as they are. Called `required` |
| **All videos** | Every video. Called `all` |
| **Also videos larger than the playback size** | Those, plus videos above the playback size. Called `optimal` |
| **Also videos above the largest file rate** | Those, plus videos above the largest file rate. Called `bitrate` |
| **Never make copies** | No videos. Some videos won't play on some devices. Called `disabled` |

### How copies are made

| Setting | What it does | Default |
| --- | --- | --- |
| **Video format** | Called the video codec. H.264 plays almost everywhere and is quick, but makes much larger files. HEVC and VP9 are more efficient; VP9 plays better on the web but takes longer. AV1 is the most efficient but isn't supported on older devices | H.264 |
| **Sound format** | Called the audio codec. AAC, MP3 or Opus. Opus is the highest quality but less compatible with old devices | AAC |
| **Video formats played as they are** | Called accepted video codecs. Formats that don't need a copy (for the rules that use them) | H.264 |
| **Sound formats played as they are** | Called accepted audio codecs. Sound formats that don't need converting | AAC, MP3, Opus |
| **File types played as they are** | Called accepted containers. File types that don't need repackaging as MP4 | MOV, Ogg, WebM |
| **Playback size** | Called the target resolution. Higher keeps more detail but takes longer, makes larger files and can make the apps less responsive | 1080p |
| **Quality number** | Called the constant rate factor (`-crf`). Lower looks better but is larger. Typical values: 23 for H.264, 28 for HEVC, 31 for VP9, 35 for AV1 | 23 |
| **Largest file rate** | Called the maximum bitrate. Makes file sizes more predictable at a small cost to quality. `0` turns it off. At 720p, typical values are 2600 kbit/s for VP9 or HEVC and 4500 kbit/s for H.264. `5000`, `5000k` and `5M` mean the same | 0 |
| **Conversion speed** | Called the preset (`-preset`). Slower speeds make smaller files and improve quality at a given file rate. VP9 ignores speeds above `faster` | `ultrafast` |
| **Processor cores for video** | Called threads. More cores convert faster but leave less room for other work. Don't set more than your processor has; `0` lets Frameleaf decide | 0 |
| **HDR method** | Called tone-mapping. Keeps HDR videos looking right on regular screens. Hable preserves detail, Mobius colour and Reinhard brightness | Hable |
| **Convert in two passes** | Called two-pass encoding. Converts twice for a better result. With H.264 and HEVC it needs a largest file rate and ignores the quality number. Only NVIDIA (NVENC) supports it among the graphics cards | Off |

The options under **Advanced: graphics card** (**B-frames**, **Reference frames**, **Constant-quality mode**, **Smarter quality over time**, **Which graphics card**) and **Keyframe spacing** (called the maximum keyframe interval, or GOP size) have sensible defaults, so you rarely need to change them. **Smarter quality over time** (called temporal AQ) applies only to NVIDIA cards; **Which graphics card** (called the preferred hardware device) applies only to VAAPI and Quick Sync and picks the `/dev/dri` node to use.

<span id="real-time-transcoding"></span>

### Adjust quality to the connection

**Adjust quality to the connection (early feature)** converts a video while it streams, so the player can switch quality as the connection changes. Called real-time transcoding. On a server that can't keep up, it can add delay and stuttering. It's off by default. When on, **Formats to stream** offers H.264, HEVC and AV1, and **Quality steps** go from 480p up to 4K, never larger than the original. If you use a graphics card, choose only formats it can make.

## Log footage and shared videos

- **Log footage plays in normal colour.** Cameras that record in a Log profile (Apple Log, S-Log3, V-Log, Canon Log, F-Log, N-Log, DJI D-Log and DJI D-Log M) make flat, grey-looking files meant for grading. Frameleaf recognises them from what the camera wrote into the file, or for DJI into the `.SRT` file beside the clip, and converts the playback copy, thumbnails and previews to normal colour. The original is never changed. HDR videos keep using the **HDR method** above.
- **A video shared with a partner is converted once.** A partner's copy uses the playback copy made for the original, the same way it shares the original's thumbnails, so sharing a library doesn't double the converting.

## Seek-bar previews

**Converting videos** also prepares every video's seek-bar previews: one sheet of 120 small frames, made from the version of the video that plays. The seek bar shows them as you point at it. The previews follow this work's concurrency and the overnight schedule, take about 2 MB per video, and can always be made again. See [Video scrubbing and previews](https://frameleaf.app/docs/using/video-previews/), including how much space they need.

When [hardware transcoding](#hardware-transcoding) is on and **Read videos with the graphics card too** is checked, the graphics card also reads the video to cut these preview frames, so the processor stays free. Videos the card doesn't read for playback copies either, such as ProRes or Dolby Vision, stay with the processor. If the card fails on a video, Frameleaf makes that video's previews with the processor instead.

## Hardware transcoding

A graphics card takes most of the converting off the processor, so videos are ready sooner and the server stays free for everything else.

### Supported hardware

| Option | Hardware | Compose service |
| --- | --- | --- |
| NVENC | NVIDIA GPUs | `nvenc` |
| Quick Sync | Intel CPUs with integrated graphics, 7th generation or later | `quicksync` |
| RKMPP | Rockchip SoCs | `rkmpp` |
| VAAPI | AMD, NVIDIA and Intel | `vaapi` (`vaapi-wsl` on WSL2) |

Not supported: Raspberry Pi, and Quick Sync under WSL2. VAAPI works on NVIDIA and Intel too, but the specific options are better optimised for their own hardware.

Which video formats work depends on the hardware. H.264 and HEVC usually work; NVIDIA and AMD GPUs can't encode VP9. Newer devices tend to give better quality.

:::note
Copies made on a graphics card are noticeably larger than copies made on the processor at similar settings, usually at lower quality. A slower conversion speed and a more efficient video format narrow the gap. If the card fails for a video, Frameleaf converts it on the processor instead, and **Video playback copies** says the graphics card stopped helping.
:::

### Before you start

**NVENC**

- The official NVIDIA driver.
- On Linux (not WSL2), the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).

**Quick Sync**

- VP9 needs a 9th generation Intel CPU or newer.
- 11th generation and older CPUs may need low-power encoding turned on in the host's graphics driver for VP9.
- An 11th generation CPU on kernel 5.15 (as shipped with Ubuntu 22.04 LTS) needs a newer kernel.

**RKMPP**

- A supported Rockchip SoC. Only RK3588 does tone-mapping in hardware; others tone-map in software but still encode in hardware.
- Hardware tone-mapping needs `/usr/lib/aarch64-linux-gnu/libmali.so.1` on the host. Install the `libmali` release for your Mali GPU (`libmali-valhall-g610-g13p0-gbm` on RK3588), then in `hwaccel.transcoding.yml`, under `rkmpp`, uncomment these three lines:

```yaml
- /dev/mali0:/dev/mali0
- /etc/OpenCL:/etc/OpenCL:ro
- /usr/lib/aarch64-linux-gnu/libmali.so.1:/usr/lib/aarch64-linux-gnu/libmali.so.1:ro
```

### Set it up

These steps are for manual Docker Compose installations. `hwaccel.transcoding.yml` is Frameleaf's **hardware-acceleration helper**: it supplies the GPU device mappings and runtime options for the supported backends. It comes with the release assets; use the one from the same release as your Compose file.

If you prefer a single Compose file, use the [inline configuration below](#single-compose-file) instead of downloading the helper.

1. If the helper isn't already in your release bundle, download [`hwaccel.transcoding.yml`](https://github.com/Frameleaf/frameleaf-app/releases/latest/download/hwaccel.transcoding.yml) into the same folder as `docker-compose.yml`. The link follows the latest published release; for an older installation, use its matching release asset.
2. In `docker-compose.yml`, under `services` → `frameleaf-server`, uncomment the `extends` section and change `cpu` to `nvenc`, `quicksync`, `rkmpp`, `vaapi` or `vaapi-wsl`. Keep the service's existing image, volumes, environment and ports.
3. Run `docker compose up -d frameleaf-server` to recreate the server with GPU access.
4. In **Video playback copies**, under **Advanced: graphics card**, set **Kind of graphics card** to the same option and save. On Jasper Lake and Elkhart Lake CPUs, also set **Constant-quality mode** to `CQP`.
5. Check **Read videos with the graphics card too** (called hardware decoding). It's on by default, so the card reads videos as well as converting them; turn it off if some videos fail to convert.

```yaml
services:
  frameleaf-server:
    # Keep your other service settings; add or enable this section.
    extends:
      file: hwaccel.transcoding.yml
      service: quicksync
```

If you use a [config file](https://frameleaf.app/docs/admin/config-file/), set `accel` (for example `qsv` for Intel or `nvenc` for NVIDIA) and `accelDecode`:

```json
{
  "ffmpeg": {
    "accel": "qsv",
    "accelDecode": true
  }
}
```

There's nothing to redo afterwards: every video converted from then on uses the graphics card.

To confirm it's working, choose **Test it** under **Speed** on **Video playback copies**: it makes a short test video on the card and says how much faster than the processor it was. For a fuller check, open **Settings**, then **Computers & background work**, then **Graphics card check** and run its speed test (see [Hardware acceleration](https://frameleaf.app/docs/admin/hardware-acceleration/#check-your-hardware)), or watch GPU use with `nvtop` (NVIDIA) or `intel_gpu_top` (Intel) while a video converts. Clean logs while a video converts are a good sign too.

<span id="unraid-portainer-and-single-compose-files"></span>

### Single Compose file

For a single-file setup, including a NAS or container editor that can't resolve `extends`, add the GPU settings directly to your existing `frameleaf-server` service. For Intel Quick Sync, add this device mapping and leave out `extends`; no helper download is needed:

```yaml
services:
  frameleaf-server:
    # Keep your existing image, volumes, environment and ports.
    devices:
      - /dev/dri:/dev/dri
```

Then continue from step 3 above.

For a native NAS template, follow its release instructions for GPU device access. Then check that the server container itself, not just the host, can use the GPU.

### Tips

- Choose a slower **Conversion speed** than you would on the processor, to keep quality and file size in check.
- Prefer the specific option for your hardware (NVENC, Quick Sync) over VAAPI.
