The New Architecture for Generative Search and Recommendations

Search and recommendation systems have traditionally retrieved candidates from an index and scored them independently. Generative systems introduce new capabilities: generating catalog entities directly from intent, reasoning over longer interaction histories, and producing an ordered slate as one decision. Each capability requires different representations, training data, serving infrastructure, evaluation, and controls.

In this talk, I will present the components behind production-scale generative search and recommendation systems. We will begin with the shared foundations that make generation possible: semantic item tokens learned from catalog content and behavior, persistent consumer memory, and ordered event sequences. We will then separate three architectures that are often conflated: generative retrieval, sequence-aware ranking, and end-to-end slate generation.

The session will follow these components into production, covering constrained decoding over a changing catalog, dynamic availability and policy enforcement, latency-aware serving, source attribution, outcome-based learning, and deterministic fallback. We will also examine where these systems fail: item representations that become more unique but less learnable, generated experiences limited by weak retrieval, and memory evaluations that preserve preferences users have already abandoned.

Attendees will leave with an architecture map for deciding which generative capability to build, which foundations it requires, and when an existing retrieval-and-ranking system remains the better design.

Key Takeaways:

1. Generative retrieval, sequence ranking, and slate generation are different system designs. Teams should choose among them based on the decision being modeled, rather than treating them as stages of one inevitable migration.
2. Shared representations and state are the foundation. Semantic item tokens, persistent memory, and ordered event sequences determine what generative models can learn and transfer across search and recommendation surfaces.
3. Grounding requires more than catalog-valid decoding. Availability, geography, policy, inventory, and marketplace constraints remain dynamic and authoritative.
4. Evaluation must follow candidates through the complete system. Standalone recall is insufficient; teams need source attribution, merge and ranker survival, final-slate contribution, online outcomes, latency, and fallback behavior.