Design AR Try-On Experience

Hard45 min
1 / 30
understanding•6 min read

Problem Statement & Context

What AR try-on means in retail interviews

Design an E-Commerce AR Try-On Experience

Retailers like Warby Parker (glasses), Sephora (makeup), and Nike (footwear/apparel) use augmented reality so shoppers preview products on their own face or body before buying. The interview question is not "draw ARKit on a whiteboard" — it is design the platform that serves 3D assets, tracks the user, respects privacy, and ties try-on to catalog, inventory, and conversion analytics.

What "try-on" includes in scope

  • Face try-on: glasses, hats, makeup overlays with landmark alignment
  • Body try-on: shirts, shoes (often 2.5D or simplified mesh warping)
  • Session lifecycle: open camera → load asset → render overlay → capture/share → add to cart
  • Merchandising hooks: recommend size, surface similar SKUs, A/B creative variants

Why interviews focus here

AR try-on sits at the intersection of mobile performance, 3D content pipelines, computer vision, and e-commerce core services. Weak answers treat rendering as a black box; strong answers separate the on-device real-time path from the cloud asset and catalog path, state explicit latency budgets, and discuss biometric-adjacent data carefully.

Scale anchors (stated up front)

Assume a global beauty/fashion retailer with 80M MAU, 12% opening try-on on supported PDPs, and 3 try-on sessions/user/month on average. Peak events (product drops) push concurrent sessions 10× normal. These numbers drive CDN egress, asset storage, and telemetry volume later.

Interview arc you will follow

  1. Clarify product surfaces (mobile app vs mobile web vs in-store kiosk)
  2. Lock functional and non-functional requirements (FPS, offline, consent)
  3. Estimate traffic, storage, bandwidth
  4. Model catalog ↔ 3D asset relationships
  5. Design APIs and high-level architecture with the device-edge vs cloud split
  6. Deep dive the tracking pipeline, asset versioning, and failure modes
  7. Close with trade-offs (on-device ML vs cloud), cost, and compliance

Key Highlights

  • •AR try-on is a platform problem spanning 3D assets, CV, and commerce
  • •Separate real-time on-device rendering from cloud catalog/asset paths
  • •Privacy and biometric-adjacent data are first-class requirements
  • •State scale assumptions early to anchor later estimations
Name the split
Say “on-device render path” vs “cloud asset path” in the first two minutes.
Companies matter
Warby = precise face mesh; Sephora = makeup segmentation; Nike = body scale — pick one primary vertical.

Section Rescue Kit

Buzzwords to use:

Try-On SessionAsset Manifest

Safe statements:

  • "I'll separate real-time rendering from catalog/asset distribution."
  • "Privacy constraints shape what never leaves the device."
Design AR Try-On Experience - System Design | WinJob | WinJob