Key takeaways
- Databricks Apps hosts data and AI applications.
- Unity Gateway and Unity Catalog control tool access.
- MLflow scores traces with judges and code checks.
- Production monitoring runs scorers on sampled traffic.
Databricks agentic AI is easier to assess when each production concern has a named owner. The useful question is not whether the platform has an agent feature. It is whether its components create the boundaries your system needs around data, tools, evaluation, observability, policy, identity, and state.
Agentic AI combines models, tools, and consequential actions
Agentic AI describes a software architecture, not just a model responding to a prompt. In these systems, large language models plan multi-step tasks, invoke external tools, and execute consequential actions with minimal human supervision. Those three mechanisms change the engineering problem.
Planning means the model can select and order work across several steps. Tool invocation gives that plan a path into documents, tables, APIs, or code. Consequential action means an output may change something outside the model session. A Databricks AI agent therefore has to be evaluated as a system with execution paths and controls, not as a model endpoint alone.
The three-part mechanism is a useful test for scope. A model that plans work creates a sequence that the surrounding system has to execute. Once tools enter that sequence, the system also has to represent what the agent can reach. When the tool can perform a consequential action, the architecture has to place a control around that action. The model is part of the system, but it is not the only place where behavior is determined.
The production review has to make each boundary legible before “agentic” becomes a useful engineering label.
Our method is to map each operating concern to the component that handles it, then record the mechanism and its documented scope. That keeps a broad platform label from hiding the boundary you actually need to inspect. The same distinction between model behavior and surrounding-system responsibility also shapes our guide to agentic AI versus LLMs.
Does Databricks have AI capabilities?
Yes. Databricks has several AI application, serving, and inference capabilities, but they sit in different parts of the platform. Databricks Apps lets developers build and deploy secure data and AI applications directly on Databricks. When an application chain is complete, it can be served as a Model Serving Endpoint.
The serving layer has concrete routing behavior. Databricks Model Serving uses token-aware and cache-aware routing, prefix caching, and a multi-layer cache hierarchy. It also uses Dicer for session-sticky routing. These are specific Model Serving mechanisms, not a description of every part of an agent application.
There is also a batch path. Databricks SQL provides high-throughput batch LLM inference through ai_query, while custom model serving with VLLM supports transformer-based architectures.
Place the execution surface before choosing the control
These capabilities answer different execution questions. Databricks Apps is the application surface. A Model Serving Endpoint is a serving destination for a completed application chain. Databricks SQL supplies the ai_query batch inference path, while VLLM supports custom serving for transformer-based architectures.
Model Serving also has behavior that matters at that surface. Token-aware and cache-aware routing considers request characteristics. Prefix caching and a multi-layer cache hierarchy address repeated prompt prefixes, while Dicer supplies session-sticky routing. Those mechanisms belong to serving. They do not describe how an agent gets permission to call a tool, how a trace is scored, or where a transactional write lands.
Databricks Apps keeps application deployment on the platform itself. That fact answers where the application can run, without deciding which model, tool path, scorer, or state layer it uses.
Write the chosen execution surface beside the application chain in the architecture. This prevents “served on Databricks” from obscuring whether the workload uses an application host, an online endpoint, a batch SQL function, or a custom model-serving path.
The answer to “Does Databricks have AI capabilities?” is therefore a clear yes. The harder decision is where the application gets authority to reach data and act through tools.
How do Databricks agents reach data and tools?
Start with the kind of access the agent needs. Databricks agent tools can search documents, query tables, call external APIs, and run custom code. Those actions are not one generic integration path. Databricks documents several ways to connect an agent, and each fits a different implementation need.
- Pre-configured MCP access: Databricks provides pre-configured MCP servers for immediate access to Genie Agents and SQL warehouses.
- Custom tools: Specialized workflows can use custom tools to reach structured data in Unity Catalog tables or external data stores.
- Document and API tools: Agent tools can search documents and call external APIs, as well as query tables or run custom code.
- Deterministic functions: Unity Catalog functions can run deterministic, repeatable queries.
MCP is an open-source standard that connects agents to tools, resources, and prompts. It is one of several ways to connect tools on Databricks. That boundary matters because a pre-configured MCP server, a custom tool, and a deterministic Unity Catalog function are different implementation choices. They should not be collapsed into a single “tool support” box during architecture review.
For structured retrieval, pre-configured MCP servers and custom tools offer separate paths. The former supplies immediate access to Genie Agents and SQL warehouses. The latter serves specialized workflows. Unity Catalog functions add a repeatable query path when that is the required mechanism.
Once the connection path is chosen, authority becomes the next boundary. In Azure Databricks, Unity Gateway controls and monitors agent tool access, while Unity Catalog enforces permissions and manages credentials. Connection and authority meet at runtime, but they are not the same design decision.
Production Boundary Scorecard for Databricks agentic AI
| Boundary | Databricks component or control | Documented capability | Documented scope or limit |
|---|---|---|---|
| Data context | Unity Catalog | Unifies discovery, access, lineage, monitoring, auditing, semantics, and sharing across data and AI in open formats. | Record the governed data and AI assets the agent can discover and access. Include their lineage, monitoring, audit, semantics, and sharing controls. |
| Tool authority | Unity Gateway and Unity Catalog | Unity Gateway controls and monitors tool access; Unity Catalog enforces permissions and manages credentials. | Record the tool, permission, credential, and monitoring owner for each access path. |
| Evaluation | MLflow | Scores agent traces with LLM judges and code-based checks. The same scorers can run in mlflow.genai.evaluate(). | Record the trace, scorer type, and repeatable evaluation loop used for each assessment. |
| Observability | Unity Gateway usage tracking | Records token counts, latency, requester identity, and model destination across Databricks-hosted and external providers. | Record token count, latency, requester identity, model destination, and the team, project, or cost-center tag. |
| Approval controls | Service policies and runtime policies | Service policies return ALLOW, DENY, or ASK. Runtime policies can allow, deny, or require approval. | During the service-policy beta, decisions do not transform request or response content. Record the target service, attached policy, expected ALLOW, DENY, or ASK outcome, and the separate guardrail used for content handling. |
| Transactional state | Lakebase, Agent Bricks, and Unity Catalog | Lakebase provides an operational Postgres layer for transactional agent writes. Agent Bricks, Lakebase, and Unity Catalog support writes to private company tables. | Warehouse tables remain analytical. Unity Catalog enforces existing row-level controls for agent integrations. Record which writes land in Lakebase, which analytical workloads remain in warehouse tables, and which existing row-level controls govern each agent integration. |
| Identity | Identity and access management | Requester identity can be recorded in Unity Gateway usage tracking. | Credential sharing, static tokens, overly broad access, and overused human approvals are poor practices. Record requester identity in usage tracking. Remove credential sharing and static tokens from the design. Narrow overly broad access. Avoid overusing human approvals. |
| Governance scope | Access, tool, data, and action-policy layers | Governance addresses access to agents and models, access to external tools and data, and policies that constrain actions. | LLM-based guardrails can block harmful outputs, redact PII, and enforce business rules. Record access to agents and models, access to external tools and data, and every policy that constrains an agent action. Record harmful-output blocking, PII redaction, and business rules separately. |
| Reversibility and recovery | Application harness and stateful execution layer; owner to be assigned | For each consequential action, record how the system restores prior state, replays work, recovers from a checkpoint, or issues a compensating action. Name the component that detects and owns failure recovery. | Required record: the action in scope; the state that can change; the rollback, replay, checkpoint, or compensating mechanism; the failure detector; the recovery owner; and the completion signal. Record the trigger, authority, state dependency, and expected result for every recovery path. Keep this boundary separate from data governance, transactional storage, evaluation, and approval controls. |
Recovery needs explicit ownership. Before deployment, write down the owner and mechanism for rollback, replay, checkpoint recovery, compensating actions, and failure recovery. Keep unresolved paths visible in the architecture record. Completed data, state, policy, and evaluation rows remain separate controls.
Service policies still carry beta limits
The service-policy boundary is useful, but its beta limits materially shape an implementation. Custom policy functions support only LANGUAGE SQL. Policy attachment is UI-only and scoped to an individual service, and the policy applies to all account users.
Three broader attachment and identity options are unavailable during the beta:
- catalog-level or schema-level policy attachment;
- attribute-based access control conditions;
- custom principals.
Service policies return ALLOW, DENY, or ASK. They do not transform request or response content during the beta. That means the decision outcome and content handling are separate mechanisms. Databricks also describes LLM-based guardrails that can block harmful outputs, redact personally identifiable information, and enforce business rules. Those guardrails should not be treated as if the service-policy decision itself rewrites content.
The individual-service attachment rule also fixes the current unit of policy. Attachment happens through the user interface, the target is one service, and the policy applies to all account users. Catalog-level and schema-level attachment cannot be used to widen that unit during the beta. Attribute-based conditions and custom principals cannot be added to make the beta attachment more selective.
An ASK result is an available policy outcome, but that fact does not make human approval the default answer to every access problem. Strong identity design still avoids shared credentials, static tokens, overly broad access, and excessive reliance on human-in-the-loop approvals. The policy decision and the identity design should each be explicit.
Keep this limitation beside the approval-control row in an architecture review. A design that requires broader attachment scope, different principal support, or a non-SQL custom policy function cannot assume those beta capabilities exist.
How does evaluation continue after deployment?
MLflow production monitoring can run evaluation scorers automatically on a sample of production traffic. This carries the scoring mechanism beyond an initial test set without requiring every production trace to be evaluated.
The cost controls are distinct. Databricks identifies five levers for managing agent-evaluation cost: sample production traffic, cache repeated evaluations, optimize judge usage, reduce unnecessary LLM inference, and prioritize deterministic scorers for structural checks. These choices change how often evaluation runs and which kind of scorer does the work.
Sampling limits the share of production traffic sent through scorers. Caching avoids repeating the same evaluation. Judge optimization and inference reduction address model calls used during evaluation. Deterministic scorers give structural checks a code-based path. None of those levers has to stand in for the others, so the evaluation plan can name each one separately.
The scorer itself can also be selected for the job. MLflow supports LLM judges and code-based checks. A judge can attach a quality assessment to a trace, while a deterministic scorer can handle a structural check. Moving the same scorer into mlflow.genai.evaluate() makes the evaluation repeatable before production monitoring applies it to sampled traffic.
Observability and budgets operate on retained request data
Consider a production trace after the agent runs. MLflow can write OpenTelemetry traces directly to Unity Catalog tables through a managed, serverless ingestion path. When those traces land in the Lakehouse, the resulting observability data is governed, ready for analytics, and available for long-term retention. The same retained data supports unified evaluation and monitoring workflows.
That path connects three otherwise separate concerns: trace ingestion, durable observability data, and evaluation workflows. It does not turn them into one control. Ingestion gets the trace into Unity Catalog tables. Lakehouse storage retains it in an analytics-ready form. Evaluation and monitoring can then operate on that retained data.
Unity Gateway adds request-level usage tracking. It records token counts, latency, requester identity, and model destination for requests across Databricks-hosted and external providers in one table. Teams can tag requests by team, project, or cost center. That gives the request record a consistent set of measures and grouping fields across both provider locations.
Budgets add a spend threshold to that operating picture. Unity Gateway administrators can set monthly thresholds per user or group. Alerts can fire as consumption approaches or crosses a threshold. This is a budget-alert mechanism tied to users or groups, alongside the trace and request data used for observability.
For an architecture review, record all three paths: where traces are ingested, where they are retained, and where monthly spending thresholds are set. Each answers a different production question.
What does Databricks teach about building agents?
Databricks offers an official AI Agent Fundamentals course. It introduces enterprise AI agents on Databricks, including the Mosaic AI platform and Agent Bricks. The curriculum also covers real-world agent use cases, agentic workflows, and multi-agent systems.
That makes the course a direct starting point for learning the Databricks vocabulary around agents. It connects the Mosaic AI and Agent Bricks names to agentic workflows and multi-agent systems without substituting for an architecture review. A course can explain available concepts and components. The scorecard still has to be applied to the authority, state, policy, evaluation, and observability boundaries in a specific system.
The curriculum spans both basic use cases and more complex coordination patterns. Real-world agent use cases ground the starting point. Agentic workflows introduce multi-step work, and multi-agent systems introduce several agents into the design. Those topics make the course relevant to builders who need the platform vocabulary before they decide how much orchestration their workload needs.
Keep that learning task separate from the platform decision. Knowing what Mosaic AI and Agent Bricks are does not fill in the scorecard automatically. The architecture still needs a named tool-access path, a trace-scoring loop, a monitoring sample, a policy scope, and a state layer. Course coverage supplies concepts; the boundary review turns those concepts into explicit engineering choices.
Use the AI Agent Fundamentals course to learn the documented platform concepts. Then apply the Production Boundary Scorecard to your own Databricks architecture. Mark the responsible component for every row, the mechanism you will use, and the current scope or limit.
If the remaining decision is about agent topology and the cost of coordinating several agents, read our AI agent orchestration framework and test whether another agent earns its operating cost.