Problem Statement: Turning Idle Cloud Capacity Into Measured Savings
Frames the product as a data-heavy analytics and workflow engine, not merely a billing dashboard.
Problem statement
Design an infrastructure cost optimization engine that continuously analyzes resource usage across one or more cloud providers and environments, compares that usage against the price of the instance type or resource actually provisioned, and recommends concrete actions: right-size an over-provisioned virtual machine, shut down a dev server that runs all weekend, move a workload to cheaper hardware such as spot or ARM instances, or buy a commitment that matches observed steady usage. The engine must gather CPU, memory, storage, and network metrics per environment, join them with pricing and billing data, produce ranked recommendations with an estimated monthly saving, and optionally apply the change automatically when the owner consents.
This is not a read-only billing report. A bill tells you what you spent last month; it arrives too late to act on and is aggregated to a level that hides the individual idle instance. The optimization engine must instead correlate two very different datasets: high-frequency resource telemetry that proves a machine is underused, and pricing metadata that proves a smaller or cheaper machine would still satisfy the workload. The output is an action with a confidence score, a projected saving, and a rollback path.
Why the problem is distinctive
A monitoring system answers "is it healthy right now." A cost engine answers "is it worth this much money over time." The distinction matters for the whole architecture. Health checks need second-level freshness and can tolerate loss; cost decisions need week-level statistical confidence and cannot tolerate silently dropped samples, because a missing week of CPU data can turn a genuinely busy database into a "safe to shrink" recommendation.
Public industry context establishes that the problem is real and large. Flexera's State of the Cloud report has repeatedly estimated that roughly a quarter to a third of enterprise cloud spend is waste, and Gartner has projected that through 2025 most organizations will eliminate unnecessary cloud spend by at least 20 percent. Netflix has spoken publicly about managing well over a billion dollars of annual cloud spend and about building internal rightsizing tooling; Pinterest published engineering work describing how rightsizing and commitment purchases saved millions of dollars; Intuit has described a FinOps transformation that embedded cost visibility into engineering workflows. These are cited company signals, not requirements for our fictional system.
For capacity planning, this answer explicitly assumes a mature multi-account organization with 500 AWS accounts, 300 GCP projects, and 120 Azure subscriptions, roughly 500,000 active compute resources, and a FinOps and engineering audience of 10,000 weekly dashboard users. Unless a number is tied to a citation, it is a stated design assumption, target, or illustrative threshold, not a claim about any company's private architecture.
The four architectural planes
- Ingestion plane: collect resource telemetry, cloud billing exports, pricing catalogs, and inventory snapshots from every account and provider.
- Analysis plane: aggregate utilization into statistically meaningful windows, attribute cost to workloads and teams, and detect waste and anomalies.
- Recommendation plane: score right-sizing, scheduling, commitment, and hardware-swap actions, and rank them by saving, risk, and effort.
- Execution plane: obtain consent, apply changes through provider APIs or infrastructure-as-code, observe the result, and roll back on regression.
A strong interview answer keeps these planes separate. It allows the dashboard to degrade without stopping data ingestion, and it permits the execution plane to be switched off entirely while the recommendation plane still delivers value. That separation is what makes the engine safe enough to trust with automatic changes.
Key Highlights
- •The engine joins two datasets that no bill alone provides: utilization telemetry that proves underuse and pricing metadata that proves a cheaper fit.
- •Cost decisions need week-level statistical confidence, so dropped telemetry samples are a correctness bug, not just a monitoring gap.
- •Public FinOps benchmarks put wasted cloud spend at roughly 25-30 percent; every uncited scale number here is an explicit design assumption.
- •The architecture has four planes: ingestion, analysis, recommendation, and execution.
- •Execution is optional and consent-gated; the engine must deliver value even with automation fully disabled.
Section Rescue Kit
Buzzwords to use:
Safe statements:
- "I will separate cost analysis from health monitoring: the former needs week-level statistical confidence while the latter needs second-level freshness."
- "Before selecting services, let me define which plane owns ingestion, analysis, recommendation, and execution, because each has a different failure and consistency model."