Skip to content

AI descriptions

Find any photo by describing it. Frameleaf writes plain-sentence descriptions of your photos and videos and tags what’s in them, all on your own server. Descriptions name the people you’ve recognised, make your library searchable by what’s actually in each shot, and feed smart albums that gather your travel, food and pet photos for you.

Descriptions are on unless your administrator turns them off, and your administrator chooses how they’re written. See Set up AI descriptions. In the app’s settings they’re called captions.

  • An AI description added to each photo’s description field, in an AI description: block. Anything you wrote yourself is kept: the generated block is added once, after your text, and never replaces it.
  • Searchable tags for visible objects, scenes, text and context, all lowercase, without duplicates and without filler words. They’re ordinary tags, so you can browse and filter by them.
  • Names, not “a family”. With Mention people I have named on, a description says “Kelly and Connor playing baseball” instead of “a family playing baseball”.
  • Video descriptions that cover the whole clip, not just its first frame.

For example, a photo of a car might be described as “A red Tesla Model 3 parked in a driveway at dusk”, with tags such as car, driveway and dusk.

Descriptions make your library easier to search in three ways:

  • Smart search gives a small boost to photos whose description matches your words. The boost is capped, so a description never outranks a photo that plainly looks like what you typed.
  • The Description search mode finds words in descriptions, yours and generated ones.
  • Tags from descriptions work with the tag: filter and the Tags filter.

See Search and Search syntax.

When your administrator has Mention people I have named on, Frameleaf tells the model who it recognised in the photo, using the names you’ve given in People, and where they are in the frame. The model has to use each name at least once and mustn’t fall back on words such as “a family”, “a group” or “everyone”.

Frameleaf then checks the result:

  • Only your names are used. Any other name the model writes is replaced with “Someone”. Days, months, holidays and well-known places, such as “Monday”, “December”, “Christmas” or “Paris”, are allowed.
  • One person, named. If one known person is in the photo and the model writes “the woman” or “the boy”, Frameleaf puts the name in.
  • Several people. If several known people are in the photo and the model uses a general word, Frameleaf leaves the sentence as written, since the word could mean any of them.

Frameleaf passes only named people who aren’t hidden. Faces that were detected but not named are skipped.

Descriptions keep up with your corrections. Fix a face or a name in People, and descriptions that used the old name are redone the next time descriptions are processed. Naming someone new applies to descriptions written from then on; your administrator can redo older ones.

See “Someone” where you expected a name? That person isn’t named on that photo yet. Add them in the photo’s People list and ask your administrator to rerun the description, or edit the description yourself.

Video descriptions tell the story of the whole clip. Frameleaf cuts six evenly spaced frames from each video, ranks them, and keeps them as the video’s moment frames. To describe the video, it lays the frames out as one grid, in time order, and tells the model how long the video is and when each frame was taken. The model then describes the video as a whole, including what changes from frame to frame.

A video description might read: “A short clip of Connor swinging a bat in a backyard; he begins facing the camera, swings the bat to his right, and walks off-frame at the end.”

The faces Frameleaf found in the video’s thumbnail are used to name the people in it.

A video that’s shorter than a fraction of a second, longer than three hours, or can’t be read gets no description. Open its Moments section and choose Find moments to try again.

If the part of a video you care about falls between the six frames, add your own moment at that time. Your moments are searchable and are never replaced.

Jump straight to the part of a video you want. A video’s Moments section in the info panel shows its frames ranked best first, the cover frame and its timestamped moments. Choose a frame or moment to play from there.

  • Cover: pick any frame as the cover. It’s kept as a time in the video, so it survives the frames being cut again. Use the best frame goes back to the best-ranked one.
  • Your moments: choose Add moment to add one at any time, with a Time, a Title and an optional typed Transcript. Transcripts here are typed by you and never generated; to search what was said in your videos, turn on spoken words. Your moments are never removed by a refresh, a replaced original or a face correction.
  • Find moments or Refresh moments cuts the frames and builds the moment search index. Add captions describes each frame on its own, which takes one extra model request per frame.
  • Find similar looks for moments like the one you’re watching in your other videos.
  • Search: searches show Moments in your videos above the results, one tile for each video, matching frames by meaning and your moments by their words. Only your own videos are searched. See Video moments in search.

Each generated result remembers what it was made from, so Frameleaf refreshes only what needs it:

Change What happens
The original file is replaced Frames, their search data and generated moments are removed. The description is marked out of date and is never shown against the new file.
A face correction changes the names in a photo Generated descriptions and captions that used the old names are marked out of date and redone next time.
Your administrator changes how captions are written Generated descriptions are marked out of date, and stay as they are until your administrator updates them.
Your administrator changes the smart search model Only the frames’ search data is cleared. Frames and moments stay.

Descriptions you wrote, your own moments, typed transcripts and your cover choice always stay exactly as you left them.

The description is an ordinary field: open a photo’s Info and edit it as you like. The panel says who wrote it: Written by AI, with how confident Frameleaf was, or Written by you.

On your own photos, a description written by AI also offers:

Action What it does
Accept Keeps the description Frameleaf suggested
Rewrite with AI Writes the description and tags for that photo again
Clear tags Takes the generated tags off the photo, without deleting the tags themselves
Clear Removes the description

When Frameleaf writes captions again for the whole library, it only replaces what it wrote itself. A description you typed is kept, unless your administrator has turned off Keep what I wrote myself.

Tags from descriptions feed six ready-made albums: Travel, Documents & Receipts, Screenshots, Food, Pets and Nature. They fill themselves as descriptions finish, and anything you take out stays out. See Built-in smart albums.

  • Descriptions are made by your own server, or by a computer on your home network that your administrator has added. Your photos only go to Frameleaf Cloud if your administrator sends descriptions there, and each person chooses whether their own photos may use Frameleaf Cloud.
  • Locked photos are described like any other, but their descriptions and tags only appear in your unlocked session.
  • Tags are visible to anyone you share a photo with. To keep something private, use Locked.

Descriptions are set up in Settings, then Search & intelligence, then Descriptions & tags. The full guide is Set up AI descriptions.

  1. Pick the Caption model. Florence-2 writes short captions only and ignores your wording.
  2. Name your most-photographed people in People.
  3. Use Try it on a few photos. Nothing is saved.
  4. Tune Caption length, the word lists and Extra instructions.
  5. Choose Update captions now.
  6. Turn on smart albums, then update them once captions finish.

Doing it in this order means you only describe your library once. Each step is explained in Set up AI descriptions.