Stage models for offline inference

Model staging lets your machine-learning worker run with no internet access at all. A separate downloader fetches and verifies the models into a cache, without ever seeing your library. The inference worker then reads those verified models from the cache and never downloads anything itself. It’s an advanced option for locked-down installations; ordinary online and trusted remote workers work as before.

Use matching release files

Use your release’s docker-compose.ml-isolated.yml and its matching staging command. Your host needs to support the Compose overrides and network isolation that file uses; on a NAS, use an equivalent host firewall instead. Keep the isolation settings in place and check that your host enforces them, because that isolation is the point of this mode.

Prepare the worker

1. Keep your existing Compose project, model cache and hardware settings.
2. Use the same release and hardware variant for the downloader and inference worker.
3. Put machine-learning runtime settings in .env.ml and model-source settings in model-staging.env. Neither file needs database, backup or Cloud provisioning credentials, or media folders.
4. Validate the complete Compose configuration on the target host before changing the running worker.
5. Stop inference before acquiring, refreshing or rolling back its models. Your other server services can keep running.

See Hardware acceleration and Workers and where jobs run for choosing the worker and its device.

Download and verify a model

From the installation directory, using the release’s regular Compose file:

docker compose -f docker-compose.yml -f docker-compose.ml-isolated.yml config
docker compose -f docker-compose.yml -f docker-compose.ml-isolated.yml stop immich-machine-learning
docker compose -f docker-compose.yml -f docker-compose.ml-isolated.yml run --rm immich-model-staging --task image-description-tagging --type visual --model microsoft/Florence-2-base-ft --options '{"acceleration":"cuda"}'
docker compose -f docker-compose.yml -f docker-compose.ml-isolated.yml up -d immich-machine-learning

The example stages a model for a CUDA description worker. Use the task, model and acceleration configured for your own installation. Rootless installations use their matching Compose file and cache-owner settings.

Stage each configured model, including fallback models. Smart search needs both visual and textual models; faces and text recognition each need detection and recognition. Semantic masks and restoration have their own provisioning steps in the matching release.

Staging verifies model files without processing photos or loading custom model code. Each model gets a verification receipt that catches any later change to its files, on top of the release’s trusted origin and pinned model information.

Offline jobs and errors

Once staged, the verified cache serves local inference with no internet connection. Run your configured jobs once to confirm each one has its model. The worker stays strict about what it loads:

• A missing or changed file produces a clear error instead of an automatic download.
• A mirror that lacks the pinned files is refused rather than swapped for another source.
• Custom models need an explicit trust decision on the worker itself; choosing one in the web interface isn’t enough.
• Existing unverified caches need restaging. The old directory is kept, but isn’t used as a fallback.

If a job can’t load its model, check the recorded error, cache permissions, selected model and worker version. Keep your working cache while you troubleshoot.

Refresh and recovery

To update a model, stop inference and rerun the same acquisition command with --refresh. It downloads and verifies the update; if the download fails, your active cache stays as it was. A successful refresh keeps one previous staged copy, so you can go back with --rollback and the same task, model and options.

A new pinned revision can use a different cache directory, so keep the previous matching release and cache until you’ve checked the new one. Rolling back a model only affects that model, not your library database, application release or edits.