With Great Tooling Comes Great Responsibilities

AI tooling for coding, productivity, and as a product is taking over the engineering world but with all this great tooling, some questions remain. How do you keep track of your spend and ensure that ROI is clear? How do you secure your system and your data? How do you govern your agents?

In this talk I'll discuss how governance, compliance, and cost tracking can turn a good product into a profitable one. Drawing on my experience shipping AI tooling at Datadog, where we serve thousands of engineers and process millions of AI-assisted interactions a month, I'll share how we've navigated these challenges at scale, including the guardrails, cost controls, and model-routing strategies (open-weight and proprietary) that reduced our cost per interaction by 3X while keeping quality and latency intact.

From putting guardrails in place to using open-weight models, I'll cover the best practices I learned along the way, so you can leave with battle-tested, real-world patterns rather than theoretical advice.

Key Takeaways:

1. How to bake in Governance, compliance and cost tracking
2. Making trade off to reach profitability