The "Read-Only" Trap: How to Build Agents That Can Safely Write to ERPs
Most enterprise AI today is stuck in the “Read-Only” trap.
Teams have built chatbots that can read documents (RAG) and query databases (text-to-SQL). That’s real progress. But when you ask, “Update the inventory count,” or “Process a refund for Order #992,” the answer is usually the same:
“I cannot perform that action.”
That’s the ROI ceiling. A read-only AI behaves like a research assistant. A read-write AI starts to look like a digital employee.
So why aren’t more teams building agents that can take action?
Because the fear is justified.
Engineering leaders don’t want a probabilistic model touching systems like SAP, Salesforce, or Oracle. One bad loop could trigger 1,000 refunds. One hallucinated digit could change an invoice by an order of magnitude. The blast radius is simply too high.
Still, if we want real value, we have to move from “chat” to “work.”
Here’s the pattern that makes that shift possible on Databricks: Unity Catalog Functions + the “Human Brake.”
The “Undo” Problem: Why APIs Are Dangerous
The obvious question is: why not just give the LLM an API key?
Because direct API access puts the model in charge of irreversible actions. You’re relying on its “judgment” to execute correctly—every time.
Two common failure modes show up immediately:
- Parameter hallucination: the user asks for $50, the model sends $500.
- Retry loops: the API returns a 503, the agent retries 50 times in a second—duplicating transactions or tripping DDoS alarms.
Standard APIs are built for deterministic code. Agents aren’t deterministic.
So we need a safety layer between the model and the system of record.
The Solution: Governed Unity Catalog Functions
On Databricks, the agent should not talk to the ERP directly. It should call a Unity Catalog Function.
Think of a Unity Catalog Function as a secure proxy:
- it enforces validation rules,
- applies policy checks,
- logs what happened,
- and only allows safe, well-formed requests through.
The agent calls the function. The function decides whether the action is allowed.
This is how you turn “agent actions” into something your risk team can actually approve.
Defining a Safe Tool
Instead of exposing a raw Python function, we register a governed tool in prod.genai_tools.
Importantly, this tool does not execute a refund.
It only validates inputs and creates a draft for review.
from unitycatalog.ai.core.databricks import DatabricksFunctionClient
def draft_refund(order_id: str, amount: float, reason: str) -> str:
"""
Safely initiates a refund DRAFT for human approval.
Does NOT execute the money transfer.
"""
import json
# 1. HARD RULE: Validate Order ID format
if not order_id.startswith("ORD-"):
return json.dumps({"status": "REJECTED", "error": "Invalid Order ID."})
# 2. HARD RULE: Refunds > $500 require special handling
status = "DRAFT"
message = "Refund drafted. Awaiting human approval."
if amount > 500.00:
status = "REQUIRES_MANAGER_APPROVAL"
message = "High-value refund drafted. Manager review required."
return json.dumps({
"status": status,
"message": message,
"order_id": order_id,
"amount": amount
})
# Register as a Governed Tool
client.create_python_function(
func=draft_refund,
catalog="prod",
schema="genai_tools"
)
The agent only sees the function signature. It never sees API keys. And it cannot bypass your validation logic—no matter how the user prompts it.
The “Human Brake” Pattern
For high-stakes actions, validation is necessary—but not sufficient.
That’s where the Human Brake comes in.
In this architecture, the agent is not allowed to execute writes. It is only allowed to stage them. A human makes the final call.
The Workflow
- User: “Refund Order #123 for $200.”
- Agent: calls draft_refund(order_id="123", amount=200)
- System: stores the request in a pending_actions table
- UI: renders a “Review” button
- Human: clicks “Approve”
- Worker: a separate deterministic job executes the API call
This is undoable by design. The AI creates intent. A human authorizes impact.
The Staging Table (The Queue)
action_id STRING,
action_type STRING,
payload STRING,
status STRING, -- PENDING | APPROVED | REJECTED
requested_by STRING,
approved_by STRING
);
This table becomes your control point. It’s also your audit trail: who requested what, what was approved, and who approved it.
Managerial Takeaway: The “Submit” Button
To deploy transactional AI safely, you have to change the metaphor.
Don’t think of the AI as the driver. Think of it as the navigator. It plans the route, prepares the action, and drafts the paperwork. But a human still presses the button.
The golden rule of transactional agents:
Never give an LLM the keys to the database. Give it a “Submit for Approval” button.
When you combine Unity Catalog Functions (policy and control) with staging tables (oversight and audit), you can safely unlock write capabilities—without exposing the enterprise to operational chaos.
Contact Everstone AI to discuss safe, transactional agent workflows
Comments
Post a Comment