Design IoT Platform

IoT Platform Problem Framing and Device Lifecycle

Design a multi-tenant IoT platform that registers tens of millions of heterogeneous devices, ingests their telemetry, issues commands back, and runs rules — without ever treating firmware blobs as stream messages. The defining insight is that an IoT platform is really three systems with different consistency needs glued together, and the senior move is to separate them up front.

The three planes:

Device identity & lifecycle (control plane, CP). Every device walks a lifecycle: provision → active → suspended → decommissioned. Each authenticates with a per-device X.509 certificate (ideally TPM-backed) minted at provisioning, and brokers enforce per-tenant topic ACLs so a compromised sensor can never publish into another tenant's namespace.
Telemetry hot path (AP, high-throughput). Device → gateway → ingest → time-series store. This is the firehose — at fleet scale ~50M devices and ~2M messages/s — and it tolerates eventual visibility with at-least-once, idempotent writes.
Command & twin path (CP where it matters). Commands to devices, and the device twin (desired vs reported state), are correctness-critical: a command ACK commits on a CP ledger even though the telemetry around it is eventually consistent.

Numbers worth stating: 50M devices, 2M msg/s ingest, p99 command delivery < 5s — devices may be asleep on LTE, so commands queue with a TTL.

The failure story that proves you get it: a buggy firmware build spams telemetry and melts a TSDB shard — per-device rate limits shed that load before a hot shard forms. And the cardinal rule: never route OTA firmware bytes through the MQTT/Kafka telemetry topic — firmware ships as objects in storage behind signed manifests, keeping brokers lean. Merge the two and one OTA campaign takes down ingest for the whole fleet.

Key Highlights

•50M devices, 2M msg/s ingest
•Per-device X.509 identity + tenant topic ACLs
•Three planes: identity (CP), telemetry (AP), command/twin (CP)
•OTA via signed manifests, never the telemetry topic

1 # Telemetry partitions by (tenant_id, device_id): per-device ordering for rule 2 # state machines, and tenant isolation so one noisy tenant cannot starve others. 3 4 from dataclasses import dataclass 5 6 @dataclass(frozen=True) 7 class DeviceShardKey: 8 tenant_id: str 9 device_id: str 10 11 def partition(self, partition_count: int) -> int: 12 if not self.tenant_id or not self.device_id: 13 raise ValueError('tenant_id and device_id required') 14 # stable hash -> a device's stream always lands on one partition (ordered) 15 return hash((self.tenant_id, self.device_id)) % partition_count

IoT platform problem framing and device lifecycle

IoT Platform Problem Framing and Device Lifecycle

Key Highlights

Device shard key — per-device ordering, per-tenant isolation

Section Rescue Kit

1	# Telemetry partitions by (tenant_id, device_id): per-device ordering for rule
2	# state machines, and tenant isolation so one noisy tenant cannot starve others.
3
4	from dataclasses import dataclass
5
6	@dataclass(frozen=True)
7	class DeviceShardKey:
8	tenant_id: str
9	device_id: str
10
11	def partition(self, partition_count: int) -> int:
12	if not self.tenant_id or not self.device_id:
13	raise ValueError('tenant_id and device_id required')
14	# stable hash -> a device's stream always lands on one partition (ordered)
15	return hash((self.tenant_id, self.device_id)) % partition_count