How accurate is AI image tagging, really?

“Is AI image tagging accurate?” is the wrong question, and it is the one everybody asks first. It invites a number — 87%! — and the number is meaningless without knowing what kind of wrong the other 13% is.

The useful question is: what kind of mistakes does it make, and can I design around them?

What these models actually do

An image classification model has a fixed vocabulary — a list of categories it was trained on, commonly around a thousand for a general-purpose model. It looks at an image and returns the categories it thinks are present, each with a score.

Two consequences follow immediately, and they explain most of the disappointment people feel:

It can only return words it knows. If sourdough is not in the vocabulary, the model will never say sourdough — it will say bread, or French loaf, and it will be sure. There is no “I do not have a word for this” output.

It has no idea what your image is for. It sees pixels. It does not know the photo is from your spring campaign, that the building is your head office, or that the man in the picture is the client. Every layer of meaning that lives in context rather than in pixels is invisible to it.

So the model is very good at “what is visibly in this frame, expressed in common nouns” and structurally incapable of anything else. Almost every complaint about AI tagging is a complaint about asking it to do the second thing.

What a confidence score actually means

The score is the model’s internal certainty, not a probability that it is right. This distinction matters, and it is routinely misunderstood.

Two things are worth internalising:

High confidence is usually reliable. When a general-purpose model says 95%, it is right the overwhelming majority of the time. This is the part you can lean on.

Confident wrongness happens, and it is weird. The failures are not gentle. A model will not say “40% husky, 35% wolf” — it will say “94% wolf” about a photo of a husky in snow, because it learned that snow means wolf. The errors are not random noise around the truth; they are systematic, and they look exactly like the successes. There is no signal in the output that says “this one is odd”.

That is the single most important fact about working with these models. You cannot tell a confident right answer from a confident wrong one by looking at the score. You can only tell by looking at the image.

So where do you set the threshold?

Given the above, a confidence threshold is not a truth filter. It is a workload filter — a way of deciding which images you look at, not which tags are correct.

The trade-off is real and it is yours to make:

• A high threshold (90%+) accepts few tags. Almost everything it accepts is right, and you are left reviewing most of the library by hand. You have automated very little.
• A low threshold (50%) accepts a lot of tags, including a meaningful number of confidently-wrong ones, which then sit in your catalogue looking exactly like the correct ones.
• Around 75–85% is where most people land, and there is no deep theory behind it: it is where the accepted tags are right often enough that spot-checking is sufficient, and enough work has been removed to be worth doing.

The right answer depends entirely on what a wrong tag costs you:

If a wrong tag means…	Then…
A photo shows up in a search it shouldn’t	Set it low. Nobody dies. Extra recall is worth the noise.
A product is mislabelled on a live storefront	Set it high, and review. A wrong tag is a customer complaint.
An asset is licensed under the wrong rights	Do not automate this field at all. Rights are not a tagging problem.

That table is more useful than any accuracy percentage, because it is about your consequences, not the model’s benchmark.

Designing a workflow around the errors

The models are not going to stop being confidently wrong, so the workflow has to absorb it. Three principles that hold up in practice:

Never let a tag reach the catalogue unreviewed. Not “never review” — rather, nothing gets written that a human did not at least glance at, in bulk or individually. In PicsTag, only accepted tags are exported; rejected and un-reviewed ones are dropped. That default is not a UX opinion, it is a guardrail.

Review by image, not by tag. Twenty tags spread over four images are judged instantly if you can see the image at a decent size, and are impossible to judge in a spreadsheet. This is why the review interface shows one image large, with its tags next to it.

Treat AI output as candidates, not values. Especially against a controlled vocabulary. The model’s word is sports car; your taxonomy’s term is vehicle — passenger. No amount of confidence changes that mismatch, and any tool claiming to auto-populate a controlled vocabulary is overselling. See keywording a photo library.

When a bigger model is worth it

Cloud vision models (GPT-4-class and similar) are genuinely better: they are not limited to a fixed vocabulary, they read text inside images, and they understand context far better. They also cost money per image and require you to send them the image.

Worth it when: the image contains text you need; the subject is unusual; you need conceptual tags (“this photo is about loneliness”) rather than literal ones; the volume is small enough that the bill is irrelevant.

Not worth it when: you are tagging forty thousand images of common objects; the assets cannot legally be sent to a third party; or the local model was already going to be right and you would have reviewed it anyway.

More on that trade-off in local vs cloud AI image tagging.

The honest summary

AI image tagging is a very good first pass and a very bad final answer. It will get you from an untagged library — which is unusable — to an adequately tagged one, in hours instead of weeks. It will also, occasionally and with total confidence, call a husky a wolf.

Build the review step in, and it is the most valuable hour you will spend on your library. Skip the review step, and you have automated the production of plausible nonsense.