Data report: Ecommerce virtual try-on benchmark — measuring garment fidelity, model consistency, and brand accuracy across 100 catalog images

What did this 100-image ecommerce virtual try-on benchmark actually measure?

This benchmark checks whether virtual try-on outputs retain the garment, keep the virtual model internally consistent, and stay safe for retailer brand presentation. It runs 100 catalog garments, evenly divided across five garment classes, and generates two outputs per SKU against the same fixed virtual-model specification. The result is 200 paired images, not a handpicked vendor-demo reel.

The prompt direction is the only variable under test. Variant A anchors on the garment with explicit preservation constraints; Variant B uses a looser styled-retail prompt. Source-image order is randomized, and reviewers never see the variant label, keeping the comparison on prompt discipline rather than rater expectation.

There is a hard limit here: no human scores or outcome measurements were supplied. The benchmark can report operational measurements now. It cannot honestly name a winner for garment fidelity, brand accuracy, model consistency, occlusion handling, or scene realism.

What results did the paired prompt test actually measure?

There is one measured result: operations. Both prompts cost $0.04 per generated asset, while the garment-grounded prompt averaged about 24 seconds and the styled-retail prompt about 27 seconds. Across hundreds of SKUs, that roughly three-second edge gives the constrained condition a little more iteration room at the same generation spend.

That is not the cost of a published asset. Human review, rejected generations, revisions, creative-direction time, and any media spend sit outside it. Nor is this a future-latency promise: input complexity, queue conditions, and the requested output treatment can all move production time.

Do not treat the faster run as proof that Variant A holds a logo, hem, print, or silhouette more faithfully. The stated hypothesis predicts that result. A hypothesis is not a scorecard.

Why score virtual try-on quality across separate dimensions?

Virtual try-on needs separate scores because a plausible-looking image can still contain the wrong SKU, a distorted neckline, or an inconsistent person. VTBench frames real-world evaluation across image quality, texture preservation, background consistency, cross-category size adaptability, and hand occlusion, with human-preference annotations alongside those cases.

Use three commercial pillars. Garment fidelity asks whether the product facts survived generation. Model consistency covers whether the person, anatomy, pose, hair, hands, and scene stay intact; brand accuracy asks whether the result reads as the actual item and is fit for a catalog. VTEdit-Bench likewise separates model consistency, cloth consistency, and overall image quality.

One generic image metric cannot settle this. VTON-IQA research notes that real try-on work often has no ground-truth photo of the same person wearing the target garment, and dataset-level FID and KID cannot express the perceived quality of one unpaired output. Put blinded human review at the center of the call.

Which launch gates should an ecommerce team set before publishing virtual try-on images?

Set launch gates before generation starts: no critical wrong-SKU or brand failures in priority categories, a predeclared garment-fidelity mean and top-box rate, a separate model-consistency threshold, and an acceptable p95 generation time. The retailer sets the exact threshold. Choosing it after seeing the outputs invites a result-shaped standard.

Put brand-critical hero images into a tighter review lane; do not abandon generation over them. Strong briefs can direct on-model styling, material detail, product preservation, and campaign context at a fraction of a traditional shoot's turnaround, while a human art director still approves what represents the brand. Weak instructions produce weak evidence.

Once the quality gate clears, test the business case in the storefront. Visitor-level randomization keeps each shopper in one experimental arm for the test window, making conversion, add-to-cart, and return-rate measurement cleaner. An image benchmark establishes visual readiness. It does not establish commercial impact.

What can this benchmark let you conclude today?

This benchmark shows that, in one paired 100-image run, the garment-grounded prompt was modestly faster at the same measured generation cost. It cannot support a claim that either prompt produced better garment fidelity, model consistency, or brand accuracy, because no rater outcomes were recorded.

That restraint is the whole point. A useful ecommerce benchmark publishes its test set, fixed conditions, scoring dimensions, failure definition, and operational limits, so a merchandising team can rerun it on its own assortment. A visually convincing sample does not become a performance ranking without the underlying review data.