Building Self-Healing, Hallucination-Aware AI Systems with Runtime Configuration

Building Self-Healing, Hallucination-Aware AI Systems with Runtime Configuration
Accuracy failures in production AI aren’t hypothetical—they’re operational incidents. Using a RAG-based health-insurance chatbot as the example, this session focuses on the patterns needed to detect hallucinations, observe model behavior, and automate recovery in live systems.We’ll show how runtime configuration and evaluation tooling make LLM deployments observable and tunable. A live demo illustrates hallucination detection at the decision boundary—catching fabricated answers before they reach users—and how to instrument latency, cost, confidence, and accuracy signals.We’ll then cover self-healing techniques: circuit breakers for degraded confidence, fallback workflows for misbehaving models, progressive rollouts for prompt or model changes, and configuration-driven guardrails that improve safety without risky redeploys.What you’ll learn:
 

• Patterns for detecting and mitigating hallucinations in production
• How to instrument GenAI pipelines with meaningful observability
• Runtime configuration strategies that make AI systems safer and more predictable

You'll leave with practical patterns you can apply to both new and scaled AI deployments.