# Monitoring and logs

Source: https://frameleaf.app/docs/admin/monitoring/

Frameleaf gives you everything you need to keep an eye on your server locally: logs, container health checks, job progress and version history. It all stays on your server.

## No telemetry

Frameleaf collects no telemetry, ever: nothing about your server or library is sent to Frameleaf or anyone else. The server contains no OpenTelemetry code, exporters or usage collectors, and legacy telemetry variables have no effect.

If you want graphs of your own, you can turn on [local metrics](#local-metrics-with-prometheus-and-grafana) for a Prometheus you run. They're off by default, and they stay on your server unless your own Prometheus collects them.

Update checks are optional and only talk to Frameleaf. When you turn them on, the server asks Frameleaf's release feed whether there's a newer version, and Frameleaf's GitHub releases if the feed can't answer.

## Where to look

| What | Where |
| --- | --- |
| Server health, storage, backups and what's processing | The **Command Center** overview. See [Command Center](https://frameleaf.app/docs/admin/command-center/) |
| Job queues, failures and errors | **Settings**, then **Computers & background work**, then **Background work**. See [Jobs and queues](https://frameleaf.app/docs/admin/jobs/) |
| Machine learning and other workers | **Settings**, then **Computers & background work**, then **Computers**. See [Workers and where jobs run](https://frameleaf.app/docs/admin/workers/) |
| GPU set-up | **Settings**, then **Computers & background work**, then **Graphics card check** |
| Missing, unknown and changed files | **Settings**, then **Library care**, then **File checks**. See [System integrity](https://frameleaf.app/docs/admin/system-integrity/) |
| Container health | `docker compose ps` |
| Logs | **Settings**, then **Server & updates**, then **Advanced**, then **Server logs**. Or `docker compose logs` |
| Metrics for your own Prometheus and Grafana | Off until you turn them on. See [Local metrics](#local-metrics-with-prometheus-and-grafana) |

## Read the logs

Administrators can read the server's logs in the browser: open **Settings**, then **Server & updates**, then **Advanced**, then **Server logs**. You can also get there from **Read the logs yourself** on the **Troubleshooting** page, or by searching settings for "logs".

- The newest lines are at the bottom, and new lines appear there as they happen. **Pause** holds them back until you choose **Resume**, and **Show them** adds the lines that arrived meanwhile. **Show older lines** goes further back.
- **Search the logs** matches the text of a line and the photo, request and job it's about. **From** picks a part of the server: **Web**, **Background work**, **Machine learning** or **Maintenance**. **Show** picks how much: **Problems only**, **Warnings and problems**, **Everything normal** or **Everything, with details**. The filters work together, and **Clear filters** resets them. Detail lines appear only when the [log level](#log-level) records them.
- Select a line to see when it happened, its level, which part of the server wrote it, and the request, background job or photo it belongs to, with **Open the photo** and **Copy line**.
- **Download…** saves the **Last day**, **Last 3 days** or **Last 7 days** as a .zip file you can send to whoever is helping you. Open it before you send it: it can mention file names and who signed in.

At home the logs open straight away. Away from home, through remote access, reading them needs a Frameleaf sign-in from the last 15 minutes, and the page asks you to **Sign in again**. Members can't open the logs.

The server keeps these logs in a `logs` folder in your library's storage: 7 days by default and at most 200 MB in all, removing the oldest first. Passwords, keys, sign-in codes and email addresses are removed before anything is written. To keep them for longer or shorter, set `FRAMELEAF_LOG_RETENTION_DAYS` in `.env` to a number of days from 1 to 30 and restart. The logs never leave the server unless you download them.

From your Compose folder:

```bash
docker logs -f frameleaf_server            # the server
docker logs -f frameleaf_machine_learning  # machine learning
docker compose logs -f database                 # PostgreSQL
```

These examples use the default container names. For a Manager installation, select **Manage** on the Overview's **Services** card, then **Logs** beside a service. The **Recent server logs** panel has **Refresh logs** and **Download logs**.

## Health checks

Every service in the provided Compose file has a health check, and `docker compose ps` shows whether each one is healthy. The server's check uses the `frameleaf-healthcheck` command in the server image, and the database image has its own check built in, including a check for data checksum failures.

## Log level

Set how much the server logs in **Settings**, then **Server & updates**, then **Troubleshooting**: turn logging on or off and choose a level. You can also set it with `FRAMELEAF_LOG_LEVEL` in `.env`.

| Level | What's logged |
| --- | --- |
| `verbose` | Everything, including very detailed tracing |
| `debug` | Detail useful when troubleshooting |
| `log` | Normal activity (the default) |
| `warn` | Warnings and errors only |
| `error` | Errors only |
| `fatal` | Only errors that stop the server |

Raise the level while you troubleshoot, then set it back: `verbose` and `debug` produce a lot of output.

## Structured JSON logs

Frameleaf logs human-readable lines by default. To feed logs into your own log collector, switch to JSON with `FRAMELEAF_LOG_FORMAT` in `.env` and restart:

```bash
FRAMELEAF_LOG_FORMAT=json
```

| Value | Format |
| --- | --- |
| `console` | Human-readable lines with colours (the default) |
| `json` | One JSON object per line |

Each JSON line looks like this:

```json
{"level":"log","pid":36,"timestamp":1766533331507,"message":"Initialized websocket server","context":"WebsocketRepository"}
{"level":"warn","pid":48,"timestamp":1766533331629,"message":"Unable to open /build/www/index.html, skipping SSR.","context":"ApiService"}
```

| Field | Meaning |
| --- | --- |
| `level` | Log level, such as `log`, `warn` or `error` |
| `pid` | Process ID |
| `timestamp` | Unix time in milliseconds |
| `message` | The log message |
| `context` | The part of the server that wrote it |

Your logs stay on your server: Frameleaf never forwards them, so you choose which tools collect them.

## Local metrics with Prometheus and Grafana

The server can offer live measurements for your own [Prometheus](https://prometheus.io/) to collect, and Frameleaf supplies a Grafana dashboard to show them. It's off until you turn it on, and Frameleaf doesn't send the metrics anywhere: they're there only for a Prometheus you run to collect.

| Part | Turn it on | Where Prometheus collects it |
| --- | --- | --- |
| The server (web requests) | `FRAMELEAF_METRICS_ENABLED=true` on the server | port `8081`, path `/metrics` |
| Background work (jobs) | the same setting | the next port, `8082` |
| Machine learning | `FRAMELEAF_ML_METRICS_ENABLED=true` on the machine learning container | its usual port, `3003`, path `/metrics` |

`FRAMELEAF_METRICS_PORT` moves the server's port from `8081`; background work always uses the port after it. To require a password-style token on the server's two ports, set `FRAMELEAF_METRICS_TOKEN` and give Prometheus the same value as a bearer token. A machine learning container that uses `FRAMELEAF_ML_AUTH_TOKEN` needs that token for its metrics too.

What's measured: how long web requests take, background jobs and how many are waiting, how many photos and videos the library holds, used and free space, the database's connections, and machine learning requests and model timings. Requests are counted by kind, never by their full address, and your own model names are grouped together, so nothing about your photos or people appears in the measurements.

The ports answer only the same computer and private-network addresses. They refuse anything that arrives through a reverse proxy, so keep ports `8081`, `8082` and `3003` off the internet and out of your proxy rules. Other services on the same private network can read metrics without a token, so set one if you don't trust them. On a non-Docker installation the server's metrics ports listen only on the same computer.

To set it up with Docker Compose:

1. From the `docker` folder of Frameleaf's source for the release you run, copy `prometheus.yml` and `grafana/frameleaf-dashboard.json` beside your `docker-compose.yml`, keeping the `grafana` folder.
2. Save an override file that turns metrics on and adds Prometheus and Grafana, then start the stack with both Compose files:

   ```yaml
   services:
     frameleaf-server:
       environment:
         FRAMELEAF_METRICS_ENABLED: 'true'
     frameleaf-machine-learning:
       environment:
         FRAMELEAF_ML_METRICS_ENABLED: 'true'
     prometheus:
       image: prom/prometheus:latest
       volumes:
         - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
         - prometheus-data:/prometheus
     grafana:
       image: grafana/grafana:latest
       ports:
         - '127.0.0.1:3001:3000'
       volumes:
         - grafana-data:/var/lib/grafana

   volumes:
     prometheus-data:
     grafana-data:
   ```

3. In Grafana, add a Prometheus data source at `http://prometheus:9090`, then import `frameleaf-dashboard.json` and choose that data source.

The supplied `prometheus.yml` collects from `frameleaf-server:8081`, `frameleaf-server:8082` and `frameleaf-machine-learning:3003`. If you change the port, update its targets. If you keep machine learning on its own isolated network, put Prometheus on both the default network and that network. The machine learning panels fill in once processing starts.
