Design a Live Translation Chat App

Problem statement

Design a chat application where every participant writes in their own language and reads every message in their preferred language. A user in Osaka sends Japanese; a user in São Paulo sees Portuguese within about a second, with the original available one tap away. The system must detect the source language, translate the message, deliver both forms, respect per-user and per-chat preferences, survive provider outages, and stay affordable at global chat scale.

This is not a chat app with an API bolted on. It is a durable, ordered messaging system coupled to a stochastic ML pipeline with three properties that fight the chat contract: translation is slow relative to messaging (hundreds of milliseconds versus tens), it is expensive per character when bought from third parties, and it is probabilistic — detection can be wrong and translation quality varies wildly by language pair. The architecture must therefore separate message delivery, which is a correctness problem, from translation attachment, which is an eventually-consistent enrichment problem.

Why the problem is distinctive

A normal chat backend retries a failed notification. Here, a failed or slow translation must never block or reorder the underlying message. Delivery is an invariant: once accepted, a message reaches every recipient in order exactly once. Translation is a best-effort projection: it may arrive 300 ms later, 5 s later with a fallback engine, or not at all with a manual translate affordance. If you model translation as part of the send path, every provider hiccup becomes a chat outage.

The second distinctive tension is privacy. Translation requires reading message content. End-to-end encrypted chats make server-side translation impossible by construction, which is why Apple's iOS 26 Messages translation runs on-device and why WhatsApp's 2024 in-chat translation applies to conversations where Meta processes content. The design must state explicitly which encryption mode it serves and how consent flows.

Public operating baseline

The category is real and large. WhatsApp, which Meta reported at roughly 140 billion messages per day in 2024 with about 3 billion users, began rolling out in-chat translation in December 2024 with six languages on Android and more than fifteen on iOS. Telegram, reporting over 900 million monthly active users, has offered whole-chat translation since 2021. Google Translate, the dominant provider substrate, reached 249 languages after its October 2024 expansion and has reported figures around 100 billion words translated per day. These are cited public figures for context; every capacity number later in this answer is an explicit design assumption.

The four architectural planes

1. Chat plane: connections, ordering, fanout, delivery receipts, offline inbox, durable history.
2. Translation plane: language detection, provider routing, batching, caching, fallback chains, quality telemetry.
3. Identity and preference plane: user languages, per-chat overrides, consent, locale rendering.
4. Learning and governance plane: quality metrics, cost accounting, safety classification on both original and translated text, incident evidence.

A strong answer keeps these planes separable: the chat plane degrades without the translation plane, the translation plane swaps providers without the chat plane noticing, and governance observes both without sitting in the hot path.