As enterprises transfer from experimenting with generative AI to deploying agentic programs in manufacturing, the dialog is shifting. The query executives are asking is not “Can this mannequin purpose?” however “Can this technique be trusted?”
To discover what that shift actually means, I sat down with Maria Zervou, Chief AI Officer for EMEA at Databricks. Maria works intently with prospects throughout regulated and fast-moving industries and spends her time on the intersection of AI structure, governance, and real-world execution.
All through the dialog, Maria saved returning to the identical level: success with agentic AI isn’t concerning the mannequin. It’s concerning the programs round it—information, engineering self-discipline, and clear accountability.
Catherine Brown: Many executives I communicate with nonetheless equate AI high quality with how spectacular the mannequin appears. You’ve argued that’s the incorrect body. Why?
Maria Zervou: The most important misunderstanding I see is folks complicated a mannequin’s cleverness or perceived reasoning capacity with high quality. These are usually not the identical factor.
High quality, particularly in agentic programs, is about compounding reliability. You’re not evaluating a single response. You’re evaluating a system which may take lots of of steps—retrieving information, calling instruments, making selections, escalating points. Even small errors can compound in unpredictable methods.
So the questions change. Did the agent use the correct information? Did it discover the correct assets? Did it know when to cease or escalate? That’s the place high quality actually lives.
And importantly, high quality means various things to completely different stakeholders. Technical groups typically give attention to KPIs like value, latency, or throughput. Finish customers care about model compliance, tone, and authorized constraints. So, if these views aren’t aligned, you find yourself optimizing the incorrect factor.
Catherine: That’s attention-grabbing, particularly as a result of many leaders assume AI programs must be “excellent” to be usable, significantly in regulated environments. How ought to corporations in highly-regulated industries strategy AI initiatives?
Maria: In extremely regulated sectors, you do want very excessive accuracy, however the first benchmark must be human efficiency. People make errors in the present day, on a regular basis. If you happen to don’t anchor expectations in actuality, you’ll by no means transfer ahead.
What issues extra is traceability and accountability. When one thing goes incorrect, are you able to hint why a call was made? Who owns the result? What information was used? If you happen to can’t reply these questions, the system isn’t production-ready, no matter how spectacular the output seems.
Catherine: You speak loads about domain-specific brokers versus general-purpose fashions. How ought to executives take into consideration that distinction?
Maria: A general-purpose mannequin is basically a really succesful reasoning engine skilled on very massive and numerous datasets. Nevertheless it doesn’t perceive what you are promoting. A site-specific agent makes use of the identical base fashions, however it turns into extra highly effective by context. You pressure it right into a predefined use case. You restrict the house it will probably search. You train it what your KPIs imply, what your terminology means, and what actions it’s allowed to take.
That constraint is definitely what makes it higher. By narrowing the area, you scale back hallucinations and improve the reliability of outputs. A lot of the worth doesn’t come from the mannequin itself. It comes from the proprietary information it will probably securely entry, the semantic layer that defines that means, and the instruments it’s allowed to make use of. Basically, it will probably purpose in your information. That’s the place aggressive benefit lives.
Catherine: The place do you usually see AI agent workflows break when organizations attempt to transfer from prototype to manufacturing?
Maria: There are three foremost failure factors. The primary is tempo mismatch. The expertise strikes sooner than most organizations. Groups bounce into constructing brokers earlier than they’ve executed the foundational work on information entry, safety, and construction.
The second is tacit data. A variety of what makes staff efficient lives in folks’s heads or scattered paperwork. If that data isn’t codified in a type an agent can use, the system won’t ever behave the way in which the enterprise expects.
The third is infrastructure. Many groups don’t plan for scale or real-world utilization. They construct one thing that works as soon as, in a demo, however collapses underneath manufacturing load.
All three points have a tendency to indicate up collectively.
Catherine: You’ve mentioned earlier than that capturing enterprise data is as necessary as selecting the best mannequin. How do you see organizations doing that effectively?
Maria: It begins with recognizing that AI programs are usually not one-off initiatives. They’re residing programs. One sensible strategy is to file and transcribe conferences and deal with that as uncooked materials. You then construction, summarize, and tag that data so the system can retrieve it later. Over time, you’re constructing a data base that displays how the enterprise really thinks.
Equally necessary is the way you design evaluations. Early variations of an agent must be utilized by enterprise stakeholders, not simply engineers. Their suggestions—what feels proper, what doesn’t, why one thing is incorrect—turns into coaching information.
Constructing an efficient analysis system, customized to that agent’s particular goal, is essential to making sure high-quality outputs, which is finally essential for any AI initiatives in manufacturing. Our personal utilization information reveals that prospects who use AI analysis instruments get almost 6x extra AI initiatives into manufacturing than those that don’t.
In impact, you’re codifying the enterprise mind into analysis standards.
Catherine: That sounds costly and time-consuming. How do you stability rigor with pace?
Maria: That is the place I speak about minimal viable governance. You don’t remedy governance for your entire enterprise on day one. You remedy it for the precise area and use case you’re engaged on. You be sure that the info is managed, traceable, and auditable for that agent. Then, because the system proves helpful, you broaden.
What helps is having repeatable constructing blocks—patterns that already encode good engineering and governance practices. That’s the pondering behind approaches like Agent Bricks, the place groups can begin from refined foundations as a substitute of reinventing workflows, evaluations, and controls from scratch every time.
Executives ought to nonetheless insist on just a few non-negotiables up entrance: clear enterprise KPIs, a named govt sponsor, evaluations constructed with enterprise customers, and powerful software program engineering fundamentals. The primary challenge will probably be painful—however it units the sample for every little thing that follows and makes subsequent brokers a lot sooner to deploy.
If you happen to skip that step, you find yourself with what I name “demo put on”: spectacular prototypes that by no means fairly develop into actual.
Catherine: Are you able to share examples the place brokers have materially modified how work will get executed?
Maria: Internally at Databricks, we’ve seen this in just a few locations. In Skilled Providers, brokers are used to scan buyer environments throughout migrations. As an alternative of engineers manually reviewing each schema and system, the agent generates really helpful workflows primarily based on greatest practices. That dramatically reduces time spent on repetitive evaluation.
In Area Engineering, brokers mechanically generate demo environments tailor-made to a buyer’s business and use case. What used to take hours of guide prep now occurs a lot sooner, with larger consistency.
In each circumstances, the agent didn’t substitute experience—it amplified it.
Catherine: If you happen to needed to distill this for a CIO or CDO simply beginning down this path, what ought to they give attention to first?
Maria: Begin with the info. Trusted brokers require a unified, controllable, and auditable information basis. In case your information is fragmented or inaccessible, the agent will fail—regardless of how good the mannequin is. Second, be clear about possession. Who owns high quality? Who owns outcomes? Who decides when the agent is “ok”? And eventually, do not forget that agentic AI isn’t about exhibiting how sensible the system is. It’s about whether or not the system reliably helps the enterprise make higher selections, sooner, with out introducing new threat.
Closing Ideas
Agentic AI represents an actual shift—from instruments that help people to programs that act on their behalf. However as Maria makes clear, success relies upon far much less on mannequin sophistication than on self-discipline: in information, in governance, and in engineering.
For executives, the problem isn’t whether or not brokers are coming. It’s whether or not their organizations are able to construct programs that may be trusted as soon as they arrive.
To study extra about constructing an efficient working mannequin, obtain the Databricks AI Maturity Mannequin.
