Problem Statement: A Roaming Camera You Drive Over the Internet
Frames telepresence robot control as a real-time media system fused with a safety-critical control loop, not a chat app with wheels.
Problem statement
Design a telepresence robot control platform. Users connect from a laptop or tablet to a physical robot that carries a camera, microphone, speaker, screen, and motorized base. They drive it around an office, hospital wing, campus, or event venue, see live video, talk to the people nearby, and expect the robot to respond to joystick input quickly enough that driving feels natural. The platform must deliver live video from the robot camera, carry remote movement commands forward, backward, and through turns, prevent collisions when the driver makes a mistake, run a two-way audio channel, survive unstable WiFi, and stop an attacker from hijacking the robot or watching its camera.
This is not a video call that happens to have wheels, and it is not an autonomous delivery robot that happens to have a camera. It is both problems at once, coupled by one hard constraint: the human driver sees the world through a 150-300 ms video pipeline, so any latency, jitter, or loss in the network becomes a physical risk. When the joystick says forward and the video lags, the robot may be closer to the wall than the picture shows. Therefore the architecture must separate three concerns that generic RTC answers merge into one: the media plane that carries audio and video, the control plane that carries drive commands, and the safety plane that owns the physical authority to move.
Why this problem is distinctive
A video conference can tolerate a frozen frame. A command queue can tolerate a delayed message. A telepresence robot tolerates neither in the same way, because motion is continuous and the operator is physically absent. Three properties define the design space. First, the control loop is closed over a network: joystick sample, encode, transmit, decode, motor command, physical motion, and the next video frame showing the result. Any segment of that loop adds latency the driver experiences as mushiness or danger. Second, the camera is a privacy surface: a remotely driven camera rolling through a hospital or office is a roving surveillance device unless the platform treats consent, recording, and bystander visibility as first-class features. Third, the robot must be safe when the network is not: WiFi dead zones, congested conference rooms, and cellular handoffs are ordinary events, and the correct response to a lost control channel is a local, immediate stop, not a queued retry.
The four architectural planes
- Media plane: camera and microphone capture, encoding, RTP transport, selective forwarding, jitter buffering, and rendering. Measured in glass-to-glass latency and quality.
- Control plane: drive vectors, head and elevation commands, dock requests, session handover. Measured in command acknowledgement latency and delivery guarantees.
- Safety plane: on-device obstacle detection, velocity capping, dead-man heartbeat, emergency stop, minimal-risk state. Runs without the cloud and can veto any command from any source.
- Fleet plane: robot registry, site maps, session scheduling, charging, firmware, telemetry, audit, and billing. Tolerant of seconds-level latency.
A strong interview answer keeps these planes visibly separate. The fleet plane may degrade for minutes without harm. The media plane may drop to audio-only for seconds. The control plane must acknowledge within hundreds of milliseconds. The safety plane must never depend on any of the other three. This ordering is the spine of the entire design.
Public operating baseline versus design assumptions
The category is real and shipped. Double Robotics has sold iPad-mounted telepresence robots since 2012 and its Double 3 generation added lidar-based spatial sensing and an autonomous follow mode. Suitable Technologies ran the Beam fleet in enterprise offices and was acquired by BlueJeans in 2019. Beam Technologies placed thousands of telepresence robots in hospitals, a business later absorbed into Cerner and then Oracle Health. OhmniLabs ships self-docking telepresence robots for telehealth. These anchors tell us the market exists; they do not tell us how to build it. For capacity planning, this answer states its own assumptions explicitly: a mature fleet of 50,000 registered robots, 40,000 online at any hour, and 12,000 concurrent drive sessions at the global peak. Unless a number is attributed to a public source, treat it as a stated design assumption, target, or budget.
Key Highlights
- •A telepresence robot couples an RTC media pipeline with a networked control loop and a local safety authority.
- •The four planes are media, control, safety, and fleet; they have different latency budgets and different failure semantics.
- •Motion safety is a local invariant; the cloud can coordinate sessions but never owns the brake.
- •The camera is a privacy surface, so consent, recording, and bystander visibility are architecture, not policy footnotes.
- •All uncited scale figures in this answer are explicit design assumptions, starting from 50,000 registered robots and 12,000 concurrent sessions.
Section Rescue Kit
Buzzwords to use:
Safe statements:
- "I will separate session success from motion safety: sessions can retry and degrade, while motion permission must fail closed locally."
- "Before drawing services, let me name which decisions happen on-device, which in the cloud, and which only with local safety approval."