Case study
AI-Powered Gesture-Controlled Virtual Try-On System Delivering 95%+ Pose Tracking Accuracy for Touchless Retail Experiences
MoreYeahs built a virtual try-on smart mirror system that uses a large display and cameras to capture body pose and hand gestures, overlaying garments and accessories in real time.
- Industry
- Retail & E-Commerce
- Engagement
- AI-Powered Virtual Try-On Experience
- Focus
- Gesture-Controlled Smart Mirror
Results at a glance
- 0195%+ pose tracking accuracy achieved for major body joints.
- 0290%+ gesture recognition accuracy achieved in test scenarios.
- 03Overlay rendering latency under 100 ms, sustaining smooth real-time try-on at 25+ FPS.
01The client
Customer
Retail customers often face inconvenience trying on clothes and accessories in-store due to limited trial rooms, hygiene concerns, wait times, and repeated inventory handling.
MoreYeahs built an AI-powered gesture-controlled virtual try-on system for retailers looking to modernize the in-store experience, giving shoppers a realistic, touchless way to try on items directly at a smart mirror display.
02Challenge
Existing try-on options fell short of what modern, high-footfall retail environments needed:
- 01Physical Trial Friction
Limited trial rooms and wait times slowed the shopping experience during busy periods.
- 02Hygiene Concerns
Repeated physical handling of garments and accessories raised hygiene considerations for shoppers.
- 03Unrealistic Digital Alternatives
Conventional e-commerce try-on tools lacked realism and interactive control.
- 04Static In-Store Experience
Most in-store digital displays were touch-driven rather than immersive, limiting engagement.
03Solution
Solution
MoreYeahs built a virtual try-on smart mirror system that uses a large display and cameras to capture body pose and hand gestures, overlaying garments and accessories in real time.
Computer vision and pose estimation models capture the user's body pose in real time.
Clothes, bags, and eyewear are overlaid and aligned to the user's body as they move.
Users browse, select, switch, resize, and compare items using hand gestures, with no touch required.
Shoppers can capture a snapshot or comparison view of items they've tried.
04How we delivered
The experience runs as a continuous loop of pose capture, gesture recognition, and overlay rendering while the shopper stands at the mirror.
- Step 1Body and Gesture Capture
Cameras capture body and hand movement, and pose estimation extracts body keypoints in real time.
- Step 2Gesture-Driven Menu
A gesture recognition model detects control gestures to navigate product categories and switch items.
- Step 3Keypoint-Aligned Overlay
Selected garments and accessories are scaled and aligned to the user's pose keypoints.
- Step 4Graceful Recovery
Unrecognized gestures trigger a hint overlay, and lost pose tracking prompts the user to re-align in frame.
05Technology
The solution pairs RGB/depth cameras with pose estimation, gesture recognition, and an AR overlay rendering engine.
- Pose Estimation & Gesture Recognition
- Models for real-time body and hand tracking
- AR Overlay Rendering
- Engine for aligning and scaling garment and accessory assets to the user's pose
- GPU-Enabled Processing
- Hardware sized for real-timelow-latency rendering
- Smart Mirror Display & UI Framework
- Large-format display with a touchless interaction framework
06Results
In validation testing, the system met its target benchmarks for tracking accuracy and responsiveness.
- 95%+ pose tracking accuracy achieved…
95%+ pose tracking accuracy achieved for major body joints.
- 90%+ gesture recognition accuracy achieved…
90%+ gesture recognition accuracy achieved in test scenarios.
- Overlay rendering latency under 100…
Overlay rendering latency under 100 ms, sustaining smooth real-time try-on at 25+ FPS.
- Users able to browse, switch,…
Users able to browse, switch, and compare items entirely through gesture control, with no touch input.
07Business impact
A touchless, immersive try-on experience gives retailers a differentiated way to engage shoppers at the point of decision.
An interactive, gesture-driven experience draws and holds shopper attention longer than static displays.
Realistic overlays help shoppers visualize fit and style before buying.
Fewer physical try-ons ease congestion and hygiene concerns during peak footfall.
The smart mirror experience positions the retailer as an innovator in physical retail.
More projects like this
All case studiesFacing a similar challenge?
Tell us what you need. A technical lead will reply within one business day with how we would approach it.
- Reply within one business day
- Free 30-minute call
- NDA before you share data