Design a Self-Learning Bot for E-Commerce Chat (User Feedback Loop)

Problem statement

Design an e-commerce chat assistant that helps shoppers find products, compare options, answer catalog questions, and resolve simple purchase queries. The distinguishing requirement is not the chat itself but the loop: the system gathers feedback such as recommendation clicks, add-to-cart events, purchases, star ratings, and thumbs up or down on answers, and uses that data to refine both what it suggests and how it converses. Updates happen on a cadence that ranges from near-real-time bandit parameter refreshes to daily or weekly model fine-tuning.

This is not a stateless LLM wrapper. A static bot recommends a product that went out of stock an hour ago, repeats an answer users consistently downvote, and cannot tell which of its phrasings actually drive purchases. A self-learning bot treats every session as labeled data: the conversation is the policy, the shopper's downstream behavior is the reward, and the platform must close the loop without degrading the experience while it learns.

Why this problem is distinctive

Three properties separate this from a standard recommendation system or a standard chatbot. First, the action space is compositional: the bot chooses retrieval queries, product rankings, answer phrasing, clarification questions, and whether to hand off to a human. Credit assignment across a multi-turn conversation is genuinely hard. Second, rewards are delayed and censored: a purchase may occur hours after the session, on another device, or not at all, and the absence of a purchase is ambiguous. Third, the feedback loop is self-referential: the data the bot learns from was generated by the bot's own behavior, which creates exposure bias, popularity spirals, and distribution shift the moment the policy changes.

The problem requires an LLM-based chat interface with product catalog knowledge, feedback signals from clicks, purchases, and ratings, a model update or partial reinforcement-learning approach, integration with inventory and personalization, scalability for many concurrent sessions, reliability so partial feedback does not break updates, bounded retraining cost, and a business dashboard that tracks improvement.

Public operating baseline versus design assumptions

Public evidence shows this category is operationally real and economically material. Klarna announced in February 2024 that its OpenAI-powered assistant handled 2.3 million conversations in its first month, roughly two-thirds of the company's customer service chats, doing the work it estimated at 700 full-time agents and cutting average resolution time from 11 minutes to 2. Alibaba published that its AliMe assistant handles millions of customer inquiries per day and absorbed roughly ninety percent of service volume during peak sale events. Amazon rolled its Rufus shopping assistant out to all US customers in 2024 after training it on the product catalog. These are cited company figures, not requirements for our system.

For capacity planning this answer assumes a marketplace with 25 million monthly active users, 5 million daily active users, 1.5 million bot sessions per day, 9 turns per session on average, a 5 million SKU catalog with 50,000 daily price or inventory changes, a 4x evening peak, and a 10x mega-sale peak. Unless tied to a citation, every number is an explicitly stated design assumption.

The four architectural planes

1. Serving plane: session gateway, dialog orchestration, retrieval over the catalog, LLM generation, guardrails, and answer assembly under a hard latency budget.
2. Feedback plane: clickstream join, explicit rating capture, session reward assembly, attribution windows, and an append-only, replayable event log.
3. Learning plane: near-real-time contextual bandit updates, daily batch feature and reward materialization, weekly fine-tuning or preference optimization, offline counterfactual evaluation, and online experimentation.
4. Governance plane: model registry, canary promotion gates, guardrail metrics, rollback, audit, and the business dashboard that proves improvement to owners.

A strong answer keeps these planes separate. The serving plane must degrade without poisoning the feedback plane, the learning plane must never push an unvalidated model into serving, and the governance plane must be able to freeze learning without taking the bot offline.