The hard part of putting an AI agent into a team’s chat is not the chat. It is encoding a complex workflow into the agent and then proving the agent does that workflow correctly. The market for AI teammates that live in a chat channel is crowded. A lot of vendors sell a persistent agent that works a channel and runs tasks async, and the chat platforms are now shipping native versions of the same idea. Building a cheaper one, backed by a model gateway for model choice, routing, load balancing, and cost telemetry, is useful infrastructure. The business sits above it.
The Work Above the Chat Window
That work gets done with the customer, item by item. It starts with the role the agent plays, the contract for what it does in each channel, the tools it can call, and the permissions on those tools. Next comes what the agent carries between conversations: its memory, the skills that encode the domain judgment, and the examples it learns from. Last are the dispatch rules for when it should speak up, the escalation rules for when it should hand off to a human, the telemetry it emits, and the eval suite that says whether any of this works. Then the agent runs in production, and real telemetry drives what changes next.
Trace to Eval to Improvement
The loop underneath it is trace to eval to improvement. The agent does real work. The system captures what happened: whether it should have responded at all, which skill ran, which tools were called, and whether the downstream action actually succeeded. It also captures whether a human corrected it, whether cost and latency were acceptable, and what changed between versions. Failures and human corrections become eval cases. Any change to a prompt, a tool, a skill, memory, the model, or the routing has to pass the eval suite before it ships, because otherwise there is no way to confirm that feature A still works after a fix to feature C. That is a developer workflow, and it should look like one: a dev and a prod version of the agent’s plugin, pull request checks that run as evals, a local CLI that emulates the bot so the iterating does not happen in a live channel.
What a Gateway Sees
A model gateway stays useful as the brain and routing layer. Model choice, cost visibility, provider routing, and guardrails are all genuinely useful. But a gateway only sees calls. It can tell you which model answered and what the answer cost. It cannot tell you whether the workflow worked, whether a human corrected the output afterward, or which skill to change next. The layer that knows all of that is the harness sitting above the gateway, which owns the role, the skills, the telemetry, and the evals. That is where I think the durable value sits.
When the Platform Ships the Thing You Were Building
There is a build-versus-buy inflection that makes this urgent. When the platform ships a native version of the agent you were assembling yourself, it takes away the undifferentiated heavy lifting: the container, the polling daemon, the hand-rolled memory. What I expect to survive is everything that encoded somebody’s judgment, the rubrics and the scoring and the routing logic and the skills, because none of that is generic enough for a platform to ship for you. Your value moves from infrastructure builder to the person deciding which workflows to hand the agent, which skills and rubrics encode the domain judgment, and what the release gates are. The native platforms become the substrate, and the encoded workflows and eval suites are what run on top.
Soft Enforcement Is the Gap
The AI-teammate tools aimed at business users offer soft enforcement: tell the agent to do better and hope. That holds up until the workflow has consequences. A skill can carry deterministic code, and where a rule is enforced by that code rather than left to the model’s discretion, it holds whether or not the model would have chosen to follow it. How that category handles workflow encoding, evals, and measuring whether the agent is succeeding is not something I have a clear read on yet. The gap, as far as I can see it, is developer-grade agent tooling: build, execute, maintain, with the evals and the observability included rather than sold as a later add-on. The spectrum runs from a spreadsheet to a no-code automation tool to a real workflow-automation platform, and the agent version of that runs from a single skill file to a hosted native agent to an on-prem deployment, with the same tooling needed across all of it. The native chat agents are worth studying for their primitives. I think a team’s chat is the right ambient interface for workflow agents, and channel-scoped memory plus channel-scoped skills are the right building blocks. Those primitives now come with the platform. The repeatable motion for encoding a customer’s workflow into them, and for proving on every change that it still works, is the part I think is worth building.