See It, Govern It: The Operating Disciplines Behind Production-Grade AI
At ALIGN Chicago 2026, Dynatrace's Akash Kadakia explained why AI needs different operating disciplines than traditional software, and how tracing, scoring, audit, lineage and access controls let organizations scale AI with confidence.
By The AI Enterprise Magazine Team3 min read
Key takeaways
- AI fails silently. Confident wrong answers don't throw errors.
- Embed, don't bolt on. Value comes from integrating AI into real workflows.
- Trace every answer. Score and alert on 100 percent of prompts, not a sample.
- Measure cost per outcome. Tie token spend to business results.
- Govern in practice. Audit trails, lineage, guardrails and response access controls.
- Trust enables scale. Seeing and governing together let AI expand safely.
When traditional software fails, it fails loudly: an error page, a checkout that won't complete. AI fails silently, with a confident answer that may be completely wrong. That difference is why Akash Kadakia of Dynatrace argues that production AI needs two operating disciplines, seeing it and governing it. Kadakia spoke at the ALIGN AI Executive Summit Chicago 2026 on July 15, in a session sponsored by Dynatrace titled See It, Govern It: The Operating Disciplines Behind Production-Grade AI.
What separates AI projects that deliver value?
Kadakia cited MIT research finding that about 95 percent of generative AI projects had no measurable impact on profit and loss, and that MIT framed the cause as a learning gap rather than a technology gap. The model's performance isn't the main issue, he said. Operations are.
He pointed to three things the successful minority does differently:
- Workflow integration. Handing out AI licenses bolts AI onto existing work. The value comes from embedding it into each role's actual workflow and replacing specific steps.
- Context and memory. Employees who start a new chat for every question lose shared context. Organizations should make internal documentation and knowledge available consistently.
- A feedback loop. Teams need concrete technical and business metrics on how AI performs in production with real users.
What does "see it" mean?
Seeing AI means observing it in the context of the full application stack, not as an isolated workload, Kadakia said. Token usage is one metric, but teams also need to know how AI affects the application, user experience and underlying infrastructure.
He described a three-step loop for turning AI quality into an operational signal:
- Trace every answer. Capture 100 percent of prompts and responses, not a sample, because sampling lets problems slip through.
- Score it. Use evaluations and LLM-as-a-judge to check accuracy, PII exposure and toxicity.
- Alert on it. Flag a failing prompt or model in real time so it can be fixed immediately.
He also urged teams to move from cost per token to cost per outcome, tying AI spending to results such as a resolved case or additional completed checkouts.
What does "govern it" mean?
Governance is not just a written AI policy, Kadakia said. Every prompt needs to be evaluated to confirm it is controlled the way the policy intends, and someone must own that control, including monitoring for toxicity, PII leakage and misuse. He listed four practices to start with:
- Audit trail. Track every prompt and response going into the organization's models.
- Data lineage. Connect AI traces to the apps and users they touch, so a false answer can be traced to its business effect, such as a checkout completed on bad information.
- Guardrails. Detect sensitive data entering prompts and enforce compliance requirements such as GDPR.
- Access to answers. Apply identity and access controls to responses, so users only see information they are permitted to see.
How do the two disciplines connect?
Seeing and governing together create trust, Kadakia said, and trust is what allows AI to scale across teams. Without it, leaders can't make confident decisions about expanding AI.
He left the audience with three questions to take back to their teams:
- Can we see it? Are we observing and scoring AI in production?
- What does it cost? Not in tokens, but in business terms and ROI.
- Can we prove it's governed? Are audit, lineage, guardrails and access controls in place?
In a show of hands, nearly everyone in the room said they were piloting or deploying AI. When Kadakia asked who felt confident they had the trust needed to scale, he counted one or two hands.
Where to hear more in 2027
Enterprise leaders will keep comparing notes on running AI in production at the 2027 ALIGN AI Executive Summits, invitation-only gatherings in eight U.S. cities: Atlanta, New York City, Dallas, Chicago, San Francisco, Boston, DC and Los Angeles. Registration is coming soon. See the ALIGN AI Executive Summits page.
Frequently asked questions
Why is AI observability different from traditional monitoring?
AI can return confident wrong answers without errors, so every response must be traced, scored and connected to the rest of the application stack.
What is cost per outcome?
Measuring AI spending against business results, such as resolved cases or completed checkouts, rather than tokens alone.
What governance controls should production AI have?
Kadakia recommended audit trails, data lineage, guardrails for sensitive data and compliance, and access controls on responses.
This article is based on remarks by Akash Kadakia, Dynatrace at ALIGN AI Executive Summit Chicago 2026 on July 15, 2026. More coverage: ALIGN AI Executive Summits.
Filed under: Observability & Data Quality, AI Governance & Trust, AI Strategy & ROI
