Remote machine learning

If your server is low on memory, such as a Raspberry Pi, you can run Frameleaf’s machine learning container on a more powerful computer, like a desktop or laptop. The server sends image previews to that computer for processing. The machine learning container doesn’t keep them or link them to a user.

Smart search and face detection use the remote container. Grouping faces into people doesn’t, because it works from results already saved in the database.

Danger

Image previews are sent to the remote computer, so think carefully before using a shared computer or a paid cloud service. The machine learning container has no security of its own; make sure only your server can reach it.

Set it up

1. Install Docker on the remote computer.
2. Save this as docker-compose.yml on the remote computer:

name: frameleaf_remote_ml

services:
  immich-machine-learning:
    container_name: frameleaf_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, rocm, openvino, rknn] to the image tag.
    # Example tag: ${FRAMELEAF_VERSION:-${IMMICH_VERSION:-release}}-cuda
    image: ghcr.io/frameleaf/frameleaf-machine-learning:${FRAMELEAF_VERSION:-${IMMICH_VERSION:-release}}
    # extends:
    #   file: hwaccel.ml.yml
    #   service: # set to one of [armnn, cuda, rocm, openvino, openvino-wsl, rknn]
    volumes:
      - model-cache:/cache
    restart: always
    ports:
      - 3003:3003

volumes:
  model-cache:

3. Start it with docker compose up -d.
4. On your Frameleaf server, go to Administration, then Settings, then Machine Learning Settings.
5. Click Add URL and enter the remote container’s address, for example http://192.168.1.50:3003.

You can also manage the list in Administration, then Processing destinations, where each address becomes a destination you can route work to and check. See Workers and where jobs run.

To use a GPU on the remote computer, also add hwaccel.ml.yml and follow Hardware acceleration.

Caution

Keep both computers on the same Frameleaf version. Update the remote container whenever you upgrade the server.

Route work to the remote computer

Each address in the list becomes a destination. Library work that has no route yet goes to the first address in the list; the order sets nothing else. To send work to the remote computer, route each kind of work (faces, smart search, text recognition, descriptions) to its destination in Administration, then Processing destinations. See Workers and where jobs run.

No automatic fallback

Frameleaf doesn’t fall back to another address. If the remote computer is off, jobs routed to it are refused and say why; they aren’t quietly sent to the local container. Smart search and face detection then don’t run for new photos, and neither do duplicate detection and face grouping, which depend on them. When the remote computer is back, run the missing jobs for Visual search and Face detection in Settings, then Compute & jobs to catch up.

If everything runs on the remote computer, you can remove the local address and the immich-machine-learning service from your server’s Compose file to save resources. At least one address must stay in the list.

Several machine learning computers

Each kind of work goes to one destination, so Frameleaf doesn’t spread one kind of work between several computers. To share the load, put a load balancer in front of them and use only its address for that work. That also lets you mix configurations on one computer, for example an OpenVINO container next to a CUDA one, or a CPU container alongside a GPU one to use both.

One machine learning container can serve several Frameleaf servers, even if they use different models, though that raises peak memory use.