# AI descriptions

Source: https://frameleaf.app/docs/using/ai-descriptions/

Find any photo by describing it. Frameleaf writes plain-sentence descriptions of your photos and videos and tags what's in them, all on your own server. Descriptions name the people you've recognised, make your library searchable by what's actually in each shot, and feed [smart albums](https://frameleaf.app/docs/using/smart-albums/) that gather your travel, food and pet photos for you.

Descriptions are on unless your administrator turns them off, and your administrator chooses how they're written. See [Set up AI descriptions](https://frameleaf.app/docs/using/ai-descriptions-setup/). In the app's settings they're called captions.

## What you get

- **An AI description** added to each photo's description field, in an `AI description:` block. Anything you wrote yourself is kept: the generated block is added once, after your text, and never replaces it.
- **Searchable tags** for visible objects, scenes, text and context, all lowercase, without duplicates and without filler words. They're ordinary [tags](https://frameleaf.app/docs/using/tags/), so you can browse and filter by them.
- **Names, not "a family".** With **Mention people I have named** on, a description says "Kelly and Connor playing baseball" instead of "a family playing baseball".
- **Video descriptions** that cover the whole clip, not just its first frame.

For example, a photo of a car might be described as "A red Tesla Model 3 parked in a driveway at dusk", with tags such as `car`, `driveway` and `dusk`.

## Search with descriptions

Descriptions make your library easier to search in three ways:

- **Smart search** gives a small boost to photos whose description matches your words. The boost is capped, so a description never outranks a photo that plainly looks like what you typed.
- The **Description** search mode finds words in descriptions, yours and generated ones.
- **Tags** from descriptions work with the `tag:` filter and the **Tags** filter.

See [Search](https://frameleaf.app/docs/using/search/) and [Search syntax](https://frameleaf.app/docs/using/search-syntax/).

## Names in descriptions

When your administrator has **Mention people I have named** on, Frameleaf tells the model who it recognised in the photo, using the names you've given in [People](https://frameleaf.app/docs/using/people/), and where they are in the frame. The model has to use each name at least once and mustn't fall back on words such as "a family", "a group" or "everyone".

Frameleaf then checks the result:

- **Only your names are used.** Any other name the model writes is replaced with "Someone". Days, months, holidays and well-known places, such as "Monday", "December", "Christmas" or "Paris", are allowed.
- **One person, named.** If one known person is in the photo and the model writes "the woman" or "the boy", Frameleaf puts the name in.
- **Several people.** If several known people are in the photo and the model uses a general word, Frameleaf leaves the sentence as written, since the word could mean any of them.

Frameleaf passes only named people who aren't hidden. Faces that were detected but not named are skipped.

### Keeping names up to date

Descriptions keep up with your corrections. Fix a face or a name in People, and descriptions that used the old name are redone the next time descriptions are processed. Naming someone new applies to descriptions written from then on; your administrator can redo older ones.

See "Someone" where you expected a name? That person isn't named on that photo yet. Add them in the photo's People list and ask your administrator to rerun the description, or edit the description yourself.

## Videos

Video descriptions tell the story of the whole clip. Frameleaf cuts six evenly spaced frames from each video, ranks them, and keeps them as the video's moment frames. To describe the video, it lays the frames out as one grid, in time order, and tells the model how long the video is and when each frame was taken. The model then describes the video as a whole, including what changes from frame to frame.

A video description might read: "A short clip of Connor swinging a bat in a backyard; he begins facing the camera, swings the bat to his right, and walks off-frame at the end."

The faces Frameleaf found in the video's thumbnail are used to name the people in it.

A video that's shorter than a fraction of a second, longer than three hours, or can't be read gets no description. Open its **Moments** section and choose **Find moments** to try again.

If the part of a video you care about falls between the six frames, add your own moment at that time. Your moments are searchable and are never replaced.

## Video moments

Jump straight to the part of a video you want. A video's **Moments** section in the info panel shows its frames ranked best first, the cover frame and its timestamped moments. Choose a frame or moment to play from there.

- **Cover:** pick any frame as the cover. It's kept as a time in the video, so it survives the frames being cut again. **Use the best frame** goes back to the best-ranked one.
- **Your moments:** choose **Add moment** to add one at any time, with a **Time**, a **Title** and an optional typed **Transcript**. Transcripts here are typed by you and never generated; to search what was said in your videos, turn on [spoken words](https://frameleaf.app/docs/using/spoken-words/). Your moments are never removed by a refresh, a replaced original or a face correction.
- **Find moments** or **Refresh moments** cuts the frames and builds the moment search index. **Add captions** describes each frame on its own, which takes one extra model request per frame.
- **Find similar** looks for moments like the one you're watching in your other videos.
- **Search:** searches show **Moments in your videos** above the results, one tile for each video, matching frames by meaning and your moments by their words. Only your own videos are searched. See [Video moments in search](https://frameleaf.app/docs/using/search/#video-moments).

## When results go out of date

Each generated result remembers what it was made from, so Frameleaf refreshes only what needs it:

| Change | What happens |
| --- | --- |
| The original file is replaced | Frames, their search data and generated moments are removed. The description is marked out of date and is never shown against the new file. |
| A face correction changes the names in a photo | Generated descriptions and captions that used the old names are marked out of date and redone next time. |
| Your administrator changes how captions are written | Generated descriptions are marked out of date, and stay as they are until your administrator updates them. |
| Your administrator changes the smart search model | Only the frames' search data is cleared. Frames and moments stay. |

Descriptions you wrote, your own moments, typed transcripts and your cover choice always stay exactly as you left them.

## Edit or remove a description

The description is an ordinary field: open a photo's **Info** and edit it as you like. The panel says who wrote it: **Written by AI**, with how confident Frameleaf was, or **Written by you**.

On your own photos, a description written by AI also offers:

| Action | What it does |
| --- | --- |
| **Accept** | Keeps the description Frameleaf suggested |
| **Rewrite with AI** | Writes the description and tags for that photo again |
| **Clear tags** | Takes the generated tags off the photo, without deleting the tags themselves |
| **Clear** | Removes the description |

When Frameleaf writes captions again for the whole library, it only replaces what it wrote itself. A description you typed is kept, unless your administrator has turned off **Keep what I wrote myself**.

## Smart albums

Tags from descriptions feed six ready-made albums: Travel, Documents & Receipts, Screenshots, Food, Pets and Nature. They fill themselves as descriptions finish, and anything you take out stays out. See [Built-in smart albums](https://frameleaf.app/docs/using/smart-albums/).

## Privacy

- Descriptions are made by your own server, or by a computer on your home network that your administrator has added. Your photos only go to Frameleaf Cloud if your administrator sends descriptions there, and each person chooses whether their own photos may use Frameleaf Cloud.
- [Locked](https://frameleaf.app/docs/using/private-content/) photos are described like any other, but their descriptions and tags only appear in your unlocked session.
- Tags are visible to anyone you share a photo with. To keep something private, use [Locked](https://frameleaf.app/docs/using/private-content/).

## For administrators

Descriptions are set up in **Settings**, then **Search & intelligence**, then **Descriptions & tags**. The full guide is [Set up AI descriptions](https://frameleaf.app/docs/using/ai-descriptions-setup/).

### A good setup order

1. Pick the **Caption model**. Florence-2 writes short captions only and ignores your wording.
2. Name your most-photographed people in [People](https://frameleaf.app/docs/using/people/).
3. Use **Try it on a few photos**. Nothing is saved.
4. Tune **Caption length**, the word lists and **Extra instructions**.
5. Choose **Update captions now**.
6. Turn on [smart albums](https://frameleaf.app/docs/using/smart-albums/), then update them once captions finish.

Doing it in this order means you only describe your library once. Each step is explained in [Set up AI descriptions](https://frameleaf.app/docs/using/ai-descriptions-setup/#setup-order).
