Set up AI descriptions
Give everyone on your server searchable captions that name the people in their photos. This guide is for administrators: it shows you how to set up AI descriptions, pick the right model, tune the wording and caption your existing library in one pass. The settings are in Settings, then Search & intelligence, then Descriptions & tags. Every control on that page is listed in System settings.
Before you start
Section titled “Before you start”- The computer that does this work is answering, ideally with a graphics card. See Hardware acceleration.
- For names in captions, Faces & identity is on and you’ve named some people. See People.
- Videos need nothing extra. Each video’s moment frames are cut the first time it’s described.
Setup order
Section titled “Setup order”| Step | Do | Why now |
|---|---|---|
| 1 | Pick the caption model | Florence-2 ignores most other choices on this page. Get the model right before you tune anything. |
| 2 | Name your most-photographed people | Captions only use names you’ve given. Doing this first means the first run already says “Kelly” instead of “Someone”. |
| 3 | Try it on a few photos | Nothing is written, so you see the result before a library-wide run. |
| 4 | Tune the wording | Tuning before the big run means you don’t caption everything twice. |
| 5 | Update captions now | One library-wide pass with everything set the way you want. |
| 6 | Turn on smart albums | The built-in smart albums use the tags captions add, and what visual search has read. |
| 7 | Update albums now once captions are done | A one-time catch-up for older photos. |
Already started in a different order? No problem. You can update captions at any time, and update smart albums one album at a time.
Quick setup
Section titled “Quick setup”- Go to Settings, then Search & intelligence, then Descriptions & tags.
- Make sure Write captions for photos is on. It is on a new server.
- Under Where and when captions are written, pick the Caption model. The default, Qwen2.5-VL 3B, suits most servers.
- Make sure Mention people I have named is on.
- Choose Try it beside Try it on a few photos, and read the Sample captions.
- Save your changes.
- Choose Update captions now at the top of the page, check the estimate, and choose Re-queue now.
- Turn on smart albums if you want them.
New photos are captioned soon after they arrive, using your current settings. Rewriting many captions waits for the overnight schedule.
Choose a model
Section titled “Choose a model”The Caption model picker shows the models that suit the computer doing the work, lightest first. A bigger model writes better captions and needs more graphics memory.
| Model | Graphics memory | Notes |
|---|---|---|
| Florence-2 base | About 1 GB | Short captions, tags and text; not for long descriptions. The usual backup model. |
| Florence-2 large | About 3 GB | Short captions, tags and text; not for long descriptions |
| Qwen2.5-VL 3B (default) | About 6 GB | Natural descriptions on small graphics cards or the processor. On Intel it runs as llmware/qwen2.5-vl-3b-ov. |
| Qwen2.5-VL 7B | About 16 GB | More detailed descriptions and better text reading. On Intel it runs as llmware/qwen2.5-vl-7b-ov. |
| Qwen3-VL 30B-A3B | About 60 GB | Large-model quality at small-model speed. NVIDIA only. |
| Qwen2.5-VL 32B | About 64 GB | Very detailed descriptions. NVIDIA only. |
| Qwen2.5-VL 72B | About 144 GB | The most detailed descriptions; needs several graphics cards. NVIDIA only. |
A model that doesn’t fit your hardware is shown as unavailable, with the reason, such as how much graphics memory it needs and how much your card has. When this server is linked to a Frameleaf account, the picker also shows the models Frameleaf Cloud offers, in blue, with the cost shown first.
To use a model that isn’t in the picker, open Advanced: caption model and hardware and type its full name, like organisation/model-name, in Hugging Face model. It’s downloaded the first time captions are written. Empty the box to go back to a published model.
Backup model
Section titled “Backup model”The Backup model, under Advanced: caption model and hardware, keeps captions flowing when the main model isn’t available. Florence-2 base is the default because it’s small enough to share a graphics card with other models. It must be installed too. Choose Another model, by its Hugging Face name… to type a different one.
Hardware
Section titled “Hardware”Kind of graphics card is Automatic by default, and says what it found. Choose NVIDIA (CUDA) or Intel (OpenVINO) to set it yourself; that also picks the caption and private-photo models that suit the card. It doesn’t install anything. Leave Which graphics card as AUTO unless the computer has more than one.
Models come from your configured model source. See Where models come from. Captions are written by your own server, or by a computer at home, unless you send that work to Frameleaf Cloud in Where work runs.
Preview before you commit
Section titled “Preview before you commit”See what you’ll get before you run anything across the library.
Try it on a few photos writes Sample captions for three of your newest photos, with the choices on the page, saved or not. Each sample shows its caption and tags. Change something and choose Try again with these choices. Nothing in your library changes.
More photos and choices… opens Try an enrichment change, for a closer look:
- Choose sample. Pick up to six of your own photos or videos, and the computer to run them on.
- Compare. The model and wording you’re editing run on each sample. The Current description and the Candidate appear side by side, with the tags, any names that were removed because they aren’t confirmed on the photo and, for a video, how many frames it saw. Nothing is saved, and existing descriptions are unchanged.
- Scope. Choose what a background plan should do to the samples you chose, such as descriptions and tags for photos, or frames, the moment search index and moment captions for videos. Moment captions add one model request per frame, and the dialog says how many.
A plan uses your saved model and wording, so save first to run the version you previewed. It runs in the background, shows in Activity, and continues after you close the page. You can Pause, Resume or Cancel it, and Retry runs the items that failed once more. Descriptions written by hand and people’s own moments are kept.
Prompt & Vocabulary
Section titled “Prompt & Vocabulary”The everyday choices are at the top of the page:
| Setting | What it does | Default |
|---|---|---|
| Caption length | Short (one sentence), Medium (two or three sentences) or Detailed (up to six sentences, with colours, weather and what people are doing). Longer captions are not always more accurate. | Medium |
| Mention people I have named | Captions say “Jamie at the lake”. People you haven’t named stay unnamed. | On |
| Things to look out for | Things captions mention when they can be seen in the photo | brands, signage, screens, documents, uniforms, tools, vehicles, animals, food, landmarks |
| Never guess at | Things captions leave out, even when the photo suggests them. Check a few captions after changing this list: it can’t be guaranteed. | diagnoses, medication names, procedures, pregnancy, disability |
Advanced: wording lists and extra instructions has the rest:
| Setting | What it does | Default |
|---|---|---|
| Sentences in a caption | From 1 to 6. Set by Caption length. The model sometimes goes one over. | 3 |
| Writing style | Plain and brief, Balanced or Rich in detail. Set by Caption length. | Balanced |
| Preferred tag words | One per line. Captions and tags use these words when they fit. | Empty |
| Words allowed for medical things in view | One per line. Visible things only, such as a cast or a wheelchair. Captions never name a condition. | A built-in list |
| Words allowed in captions of Locked photos | One per line. This doesn’t change who can see those photos. | A built-in list |
| Extra instructions | Anything else captions should follow, in full sentences, up to 2,000 characters | Empty |
Change Sentences in a caption or Writing style by hand and Caption length shows Custom.
Settings by library type
Section titled “Settings by library type”| Library | Caption length | Names | Add to Things to look out for | Add to Preferred tag words |
|---|---|---|---|---|
| Family | Medium or Detailed | On | birthday cake, sparklers, candles, prom, graduation, recital, soccer ball, baseball glove | golden hour, overcast, dappled light, backyard, park, beach |
| Documents and receipts | Short | Off | total amount, store name, line items, payment method, invoice number, due date, expiry date, signature line | Keep it short |
| Travel | Medium | On | mountain range, beach, hotel lobby, train station, food market, street scene, landmark | The landmarks and cities you visit often |
| Pets | Short or Medium | Either | breed, leash, food bowl, harness, fetch, asleep, swimming, kennel | Your pets’ breeds, such as golden retriever or tabby |
For a documents library, also add account numbers verbatim, full credit card numbers, social security numbers to Never guess at.
For a hobby, teach the model its words through Preferred tag words: long exposure, leading lines, bokeh for photography, latte art, sourdough, plating for cooking, or gravel, derailleur, peloton for cycling.
Extra instructions
Section titled “Extra instructions”Extra instructions, under Advanced: wording lists and extra instructions, is the easiest way to change how captions are written without replacing the built-in instructions. Treat it like a short brief to a careful assistant: write full sentences, say what to do and when not to, and keep it to a couple of paragraphs. The text is added to the prompt before the output rules, so the model reads it first.
Some examples that work well:
Vehicles
If you can clearly see a car, truck, or motorcycle, identify the make and model inthe description (e.g. "a red Tesla Model 3", not "a red car"). If you are uncertainabout either the make or the model, just say "a red car" rather than guessing.Sports
When people are clearly playing a sport, name the sport in the description (soccer,baseball, basketball, tennis, etc.) and mention the visible equipment they're using.Do not guess the sport from clothing alone, only from visible play, equipment, orfield markings.Travel
When the photo is clearly outdoors and looks like travel, identify the location type:beach, mountain, forest, city street, market, train station, airport, museum, cathedral,temple, ruins, hotel lobby. If a recognizable landmark is visible, name it. Do notinvent landmarks if you are not sure.Documents and receipts
When the photo is a document, receipt, or screenshot of text, transcribe the mostimportant visible identifiers: store name, total amount, date, invoice number, anddocument type. Do not transcribe full account numbers, full credit card numbers,social security numbers, or any other sensitive personal data even if visible; referto them generically (e.g. "account number redacted").Food
For food photos: identify the dish by name if recognizable, name the cuisine if youcan tell, and note the plating style (rustic, fine-dining, casual, street-food). Donot name a dish you are not confident about; say "pasta dish" rather than guessing.Pets
For pets: identify the breed when you can recognize it. For dogs in particular, notethe activity (sleeping, playing fetch, swimming, on a leash, in a car). Do not guessa breed from coat color alone.Tone
Keep descriptions plain and factual. Do not use poetic language ("a tapestry ofcolors", "bathed in golden light"). Do not editorialize about the people or events("a joyful family", "an intimate moment"). Stick to what is visible.A family library, all in one
Always name every recognized person and avoid generic group terms like "the family"or "everyone".
If you see a car, name the make and model when recognizable.If people are playing a sport, name the sport and any visible equipment.For pets, identify the breed when recognizable.For travel scenes, name the landmark if you are certain.
Keep descriptions factual and short. Do not use poetic language.What extra instructions can’t do
Section titled “What extra instructions can’t do”- They can’t override the safety rules. The handling of private and medical content, and the check that removes invented names, still apply.
- They can’t add names. A name that isn’t confirmed on the photo is still replaced with “Someone”.
- They can’t change the output format. Use Write my own instructions for that.
- They can’t make the model remember anything between photos. Each one is described on its own.
- They’re ignored while Write my own instructions is on.
Names in captions
Section titled “Names in captions”Mention people I have named passes the named people in each photo to the model. Two numbers under Advanced: names, and writing your own instructions tune it:
| Setting | What it does | Default |
|---|---|---|
| Most names in one caption | How many people to name in one photo, from 1 to 20 | 5 |
| How sure before naming someone | From 0 to 1. Higher names fewer people. | 0.7 |
Lower Most names in one caption to 1 or 2 for crowd photos where only the main people matter. Raise it to 10 to 15 for teams, weddings and other big groups. If a photo has more named people than that, some are left out. Fixing a face updates its captions.
Raw prompt editor
Section titled “Raw prompt editor”The choices above build the instructions for you, with safe defaults, and cover most needs. Reach for Write my own instructions, under Advanced: names, and writing your own instructions, when you want something they can’t express, such as a different output format. It’s for experts: it replaces the built-in caption instructions with your own text.
- Turn it on and Your instructions holds the text to edit.
- Start again from the built-in text replaces your text with the built-in instructions, discarding your edits.
- Turning it off goes back to the choices on the page.
These placeholders are filled in for each photo:
| Placeholder | Becomes |
|---|---|
{names} |
The recognised people. Empty when names are off or nobody is recognised. |
{schema} |
The output format the model must return. Keep it, or captions can’t be read back. |
{vocabulary} |
Your preferred tag words |
{style_hint} |
The length and tone cue for the chosen caption length |
If ‘{schema}’ is missing chooses what happens when you save without it: Don’t save, the default, or Warn me.
Extra instructions aren’t added to your own text; paste them where you want them. If your text is wrong, captions come out empty or broken. Turn Write my own instructions off to recover.
Describe your library
Section titled “Describe your library”The status at the top of the page says how captions stand: Every photo has a caption, or how many photos have one and how many are waiting.
Update captions now opens a window with:
| Field | Shows |
|---|---|
| Total eligible assets | Everything captions can be written for |
| Already have a description | Items with a description |
| Missing a description | Items without one |
| Estimated seconds per asset | The recent pace |
| Estimated total time | How long the whole run should take |
| Active backend and Active model | Where captions will be written, and with what |
- Cancel closes it.
- Re-queue later keeps a reminder, so you can make several changes and run once at the end. The page then shows Caption choices changed since the captions were written, with Update captions now to start.
- Re-queue now starts straight away. If a run is already going, Frameleaf tells you instead of starting a second one.
When captions are sent to Frameleaf Cloud, nothing is queued here: Frameleaf Cloud describes photos in batches, and you start them from Frameleaf Cloud processing, where the estimate comes first.
The first estimate is a starting point. Model size, hardware, other work and video length all change the speed, so try a sample first and plan the full library from what you measure.
What happens to descriptions people wrote
Section titled “What happens to descriptions people wrote”Descriptions written by hand are kept says whether a run can replace someone’s own words. It follows Keep what I wrote myself, in Settings, then Library care, then Search and recognition progress, which is on by default: writing captions again only replaces what Frameleaf wrote. With it off, a description written by hand can be replaced the next time captions are written.
Describe moments in videos
Section titled “Describe moments in videos”Describe moments in videos is off by default. When it’s on, each video also gets captions for a few moments in it, one extra model request per moment. It takes longer, and speech is not written down. For speech, see Spoken words.
Locked-content detection
Section titled “Locked-content detection”If Locked-content detection is also on, Frameleaf checks each photo for private content first and passes the result to the caption model, so the caption stays factual. See Private content.
A worked example
Section titled “A worked example”Here’s how to caption an existing library from start to finish. Let each stage finish before you start the next one.
- Prepare faces. Turn on Faces & identity and let it finish. Name your 20 or so most-photographed people; the People page shows them first.
- Try captions. Choose a model that runs on your hardware. Set Caption length to Medium, add a few things to Things to look out for and Preferred tag words, paste the family example into Extra instructions, and set Most names in one caption to 8. Try it on a few photos, then save.
- Caption the library. Choose Update captions now. Turn on smart albums, add any extra words, and choose Update albums now when captions finish.
- Review results. Open ten random photos and videos. Are names used? Are your instructions followed? Do smart albums make sense? Adjust, rewrite a few from their Info panels with Rewrite with AI, and repeat until you’re happy.
- After that. New photos are captioned automatically. New names appear in new captions straight away; rewrite older ones if you want them updated.
Troubleshooting
Section titled “Troubleshooting”Captions are still generic with names turned on
Section titled “Captions are still generic with names turned on”- Check the photo has at least one named face in its People list.
- Check the model is a Qwen model, not Florence-2.
- Check Most names in one caption is at least the number of named people in the photo.
- Open the photo’s Info and choose Rewrite with AI. Older captions keep the wording they were written with.
Extra instructions aren’t working
Section titled “Extra instructions aren’t working”- Check Write my own instructions is off.
- Check you saved.
- Rewrite some photos. Existing captions don’t change by themselves.
- Try plainer, more direct wording: “If you see a car, identify the make and model” works better than “try to be more specific about vehicles”.
- Check you’re under 2,000 characters.
A video gets no description
Section titled “A video gets no description”No frames could be cut. The video is very short, longer than three hours, or unreadable. Choose Find moments in its Moments section to try again. If that fails, check that video processing works for other files and that the file isn’t damaged.
The sample captions could not be written
Section titled “The sample captions could not be written”Check that the computer doing this work is on, then choose Try it again. Open Computers on the Search & intelligence page shows whether it’s answering.
A real name was replaced with “Someone”
Section titled “A real name was replaced with “Someone””The person isn’t named on that photo. Name them in its People list and rewrite the caption, or edit the description by hand. There’s no separate list of allowed names.
© 2026 Frameleaf Inc. All rights reserved.
Frameleaf and the Frameleaf logo are Trademarks of Frameleaf Inc.