AI Tools9 min read

I Benchmarked GPT Image 2 on 12 Faces and 69 Haircuts. Here Is What It Gets Wrong

Identity preservation hit 100%. Honesty about density, curl and hairlines did not. The full prompt experiment log, the measured cost per image, and the two measurement bugs that nearly sent me tuning the wrong thing.

I built an AI hairstyle try-on and benchmarked the model properly before shipping it. Not "these look good to me" — a twelve-subject matrix, an eight dimension rubric, and before-and-after runs on every prompt change.

Here are the numbers, the failures, and the two times my own measurement lied to me.

The setup#

Model: gpt-image-2 at quality: "medium", 4:5 at 1K (1024x1280).

Cohort: twelve synthetic adults, generated text-to-image so no real person's face was used for testing. Chosen to break the thing rather than flatter it — coily and curly hair, fine and thinning hair, receding and mature hairlines, grey and white hair, deep skin tones, ages 24 to 62.

The fixtures are candid phone selfies on purpose: flat indoor light, plain wall, slightly wrong white balance. A studio portrait would make the edit job artificially easy.

Rubric: eight dimensions, each 0 to 5 — identity preservation, hair-only edit, style fidelity, texture honesty, density honesty, hairline honesty, realism, and whether the image would actually help a haircut decision. A dimension "passes" at 4 or above.

Result 1: identity was solved on the first try#

Identity preservation scored 5.00 out of 5, a 100% pass rate, on the very first prompt version, and it never moved again.

The reason is one framing decision. My existing AI Headshots tool asks the model to build a new photograph — new studio, new outfit, and an explicit "relight from scratch" instruction to fight the edit endpoint's pixel anchoring. That is right for a headshot and catastrophic for a hairstyle preview, because it makes the model synthesise a new canonical portrait that looks like your better- looking sibling.

So every prompt here is framed as a surgical edit: this is a photograph, change the hair, touch nothing else — with the preserved list enumerated item by item. Eye spacing. Ear shape. Skin texture. Every mole. Clothing. Background. Lighting direction. Image grain.

Models only preserve what you name. The list is long on purpose.

Result 2: honesty was the hard part#

Identity was easy. Getting the model to stop flattering people took four rounds.

Three failures showed up over and over:

1. Density inflation. The output hair was consistently fuller and more even than the source. Visible scalp got quietly filled in.
2. Curl invention. Straight hair came back with a wave. And the one that matters most: 4C coily hair came back as looser, larger, smoother spirals. That is exactly what this category gets criticised for, and it lands hardest on the people already worst served.
3. Hairline restoration. Receded temples grew hair back. The hairline came forward. The model made people look younger than the photo it was handed.

Every one of these produces an attractive image. That is what makes them dangerous. In a product whose whole promise is an honest haircut decision, a flattering lie is worse than a plain truth.

What fixed it#

Pattern-specific guards, not generic ones. "Keep the same texture" did nothing. Naming the exact direction the model drifts in worked:

Keep the coils exactly as tight and as small in diameter as they are in the photograph. Do not loosen, elongate, stretch, smooth or enlarge the coil pattern into looser spirals or waves. Coily hair rendered as looser curly hair is a failure even if it looks attractive.

That single change took texture honesty for the 4C subject from 2/5 to 5/5.

Then I found the real culprit. One subject kept failing badly no matter what I did to the prompt file. The cause was not in the prompt file at all — it was in my own style catalog. An audit found 26 of 69 style descriptions asserting hair quality: "full and defined", "dense", "springy", "healthy", "voluminous".

So every generation carried two contradictory instructions. "Curls fully defined and springy" from the style, and "this person's hair is thinning, keep it thinning" from the honesty block. The model consistently took the flattering reading.

I rewrote 21 of them to describe shape, length, weight and direction only, and added an explicit precedence rule:

PRECEDENCE: if anything in the haircut description conflicts with the section headed STAY HONEST TO THIS PERSON'S ACTUAL HAIR, the honest version wins. A haircut description describes a SHAPE that can be cut into anyone's hair. It never describes how much hair this person has, how tight their curl is, or where their hairline sits.

Texture honesty went to 100% across every cohort.

Result 3: my measurement lied to me twice#

This is the part I would most want to read in someone else's write-up.

Bug one: the rubric conflated two meanings of "texture". An early run scored a subject 1/5 on texture honesty with the verdict "invents texture". But the cut requested was a textured fringe — which is produced by point-cutting straight hair so the ends separate. The judge was penalising the haircut for being the haircut.

Rescoring the identical images under a corrected rubric moved the overall mean from 4.46 to 4.72. Nothing about the product changed. If I had skipped that, I would have spent a round "fixing" a prompt that was already right.

Bug two: single-sample judging was pure noise. Comparing two prompt versions produced a five-point swing on one subject across two outputs that were, on inspection, visually equivalent. One scored 5/5/5 on texture, density and hairline. The other scored 0/1/1.

Every image is now judged three times and the per-dimension median taken. That costs about a cent per image. Rescoring an earlier run under median judging dropped it from 4.72 to 4.53 — the original number had been inflated by lucky draws.

Two of the three "regressions" I chased in this project were measurement artefacts. Fix the instrument before you tune against it.

Result 4: one prompt change made things worse#

Worth recording, because negative results usually go unpublished.

Identity preservation scores a flat 5.00. So I tried declaring the forehead and temple corners to be facial skin, copied unchanged — the theory being that the hairline could ride on the mechanism the model is already perfect at.

It made everything worse. Overall 4.81 → 4.68. Hairline honesty 4.50 → 4.25. Style fidelity 4.58 → 4.42.

The likely reason is over-constraint: declaring the forehead untouchable fights the fringe placement that several haircuts legitimately require. Reverted, and recorded in the source so nobody re-runs it.

The final numbers#

Two independent full runs of the shipped prompt, twelve subjects each, every image judged three times:

Identity passed 100% in both runs. Style fidelity passed 92% and 100%.

Cost and speed, measured#

Not estimated. Taken from the usage object on every call.

• 6.1c per preview at 4:5 1K medium with one reference
• 38.6 seconds mean latency, 49s worst case
• 0.24c for the vision analysis that reads someone's hair
• Zero content-policy refusals across 60+ generations

One number surprised me: a reference image bills a flat ~760 tokens regardless of its pixel dimensions. I measured this across eleven different reference counts and it is dead linear — about $0.0061 each. At seven references that is roughly 40% of an image's total cost. Reference slots are not free headroom.

A bonus finding about reference limits#

The docs say gpt-image-2 accepts 16 input images on the edits endpoint. A probe I ran in July found that at eight references the API returned 200 OK with image_tokens: 0 — the entire reference array silently discarded, after billing. The only symptom in production is "the AI ignored my photos".

Before encoding that limit anywhere, I re-probed. It is fixed. Counts from 5 to 17 all ingest now, perfectly linearly.

I kept the cap at 7 anyway, for reasons that are current rather than superstitious: the flat per-reference cost above, and a separate finding from my headshot tool that past about five identity references the model starts averaging incidental signals and muddying the face. More references make output worse, not better.

The lesson is the boring one: re-verify recorded API behaviour before you build on it. Mine was six weeks old and wrong.

What is still weak#

One cell in the matrix remains stubborn: coily hair, cut very short, with a mature hairline, asked for a style that needs grow-out. Density and hairline hover at 2 to 4 there where everything else sits at 5.

That turned out to be as much a product problem as a prompt problem. The style genuinely cannot be cut from that length without growing it first, so the honest answer is a "needs growing out" label and a lower recommendation rank — not a better render. The image is doing something the words are responsible for qualifying.

See it#

The tool is live: Hairstyle Try-On. Two free previews, no account. The style library and the stylist plan are free too — only generation volume is metered, because generation is the only part that costs money to produce.

If you want the practical side rather than the model side, I wrote up how to tell your barber what you actually want. And if you are comparing image models more broadly, my realism comparison covers the same ground for general image work.