Design a Log Analytics Platform (Splunk-like)

Problem statement

Design a log analytics platform that ingests structured and unstructured machine data from thousands of services, parses and indexes it in near real-time, and lets engineers run flexible full-text and field-based searches, build dashboards, and configure anomaly alerts. The brief calls this out explicitly: ingestion of structured and unstructured logs, field extraction or schema-on-the-fly, full-text search across large volumes, and alerting on error thresholds. It also names the non-functional bar: horizontal scaling of indexing nodes, low-latency ingestion with near-real-time query availability, high availability with replication for durability, and efficient query performance via index merges and caching.

The single most important framing decision is this: a log analytics platform is a write-heavy, append-only search engine. It is not an OLTP database and it is not a data warehouse batch job. Logs arrive continuously at millions of events per second, are never updated and never deleted in place, and must become searchable within seconds to minutes. Reads are interactive, ad-hoc, and dominated by time-range filters plus keyword and field predicates. Every architecture choice below flows from that asymmetry.

Why this problem is distinctive

A web backend can afford to normalize rows and serve point lookups. A warehouse can afford to load in hourly batches. A log platform can do neither. It must absorb a firehose of semi-structured text, extract meaning without a fixed schema, and make that meaning queryable almost immediately while the data is still arriving. Three properties dominate the design:

First, the write path is a one-way append stream. Events are immutable once ingested. There are no updates, no row-level deletes, no transactions across events. This is what makes horizontal scaling tractable at all: you shard by time and source and never worry about concurrent mutation.

Second, the read path is exploratory. An on-call engineer types something like status:500 service:checkout | timechart count and expects an answer over billions of events in a few seconds. You cannot precompute every possible query, so you need an index structure (the inverted index) that makes arbitrary keyword and field predicates cheap, plus aggressive pruning by time.

Third, schema is discovered, not declared. Sources emit JSON, key=value pairs, CSV, and free-form syslog lines. The platform must extract fields at index time or query time and must tolerate sources that change shape without warning.

The four planes

I will structure the architecture into four planes and keep them separate, exactly as you would separate concerns in an interview:

1. Ingestion plane: collection, buffering, parsing, enrichment, and durable handoff. It must never lose data and must never block the producer.
2. Indexing plane: sharding, inverted-index construction, segment creation, and near-real-time visibility.
3. Query plane: parsing the search language, pruning partitions, scatter-gather execution, and result merging and caching.
4. Control plane: alert evaluation, dashboards, saved searches, tenants, quotas, retention, and schema/metadata management.

A strong answer keeps these planes decoupled so that a query spike does not stall ingestion, and an ingestion burst does not break alert evaluation. The briefing's competitor note is a useful compass: most shallow treatments skip inverted indexing, field extraction, and large-scale sharding. This design goes deep on exactly those three.

Public operating baseline versus design assumptions

Public evidence shows the category is real and enormous. LinkedIn has described running Elasticsearch clusters holding well over a hundred petabytes and has published that Apache Kafka carries trillions of messages per day through its data pipeline. Elastic documents hot-warm-cold tiers and index lifecycle management as the standard way to manage log cost. Grafana Loki documents the opposite cost strategy: index only labels, keep compressed chunks in object storage. Splunk documents indexer clusters, buckets, and the SPL pipeline model. These are cited directional facts, not requirements for our fictional system.

For capacity planning, this answer explicitly assumes a mature platform with 2 million events per second average ingest, a 5x peak of 10 million events per second, 500 bytes average event size, 1,000 queries per second average with a 5,000 QPS peak, and 5,000 concurrent analysts. Unless a number is tied to a citation, it is a stated design assumption, target, or budget, not a claim about any company's private system.