Brex Developed Its AI Agent Strategy by Observing Agent Behavior Rather Than Establishing Rules First

OpenClaw has gained significant traction as an agentic framework but has yet to demonstrate its effectiveness in large enterprises. For agents to operate optimally, they require genuine credentials such as API keys, OAuth tokens, and service accounts. However, Brex has identified that conventional security measures have not been able to manage the actions of these agents effectively.

In response to these challenges, Brex developed an internal system called CrabTrap. This open-source HTTP/HTTPS proxy intercepts network traffic, reviews policy guidelines, and employs an LLM-as-a-judge to determine the approval or denial of agent requests.

“We realized the network layer was an underutilized enforcement point,” remarked Pedro Franceschi, co-founder and CEO of Brex. “Each agent request presents an opportunity to intercept, evaluate, and make policy decisions.”

Franceschi emphasizes that IT leaders should transition agent governance from SDK-level permissions and model restrictions to a centralized network control system that adapts based on real-world agent actions.

Brex’s Focus on the Transport Layer

Initially, the simple solution to the security gaps concerning agents was to implement guardrails. Much of the foundational work revolved around scoped tools, specific permissions per action, and requiring human approvals. However, as agents develop, each added feature introduces the need for additional APIs to audit or modify, according to Franceschi.

“Creating an agent with numerous tools and internet access brings a challenge for developers: the more you enhance an agent’s abilities, the riskier it becomes; conversely, the more you restrict it, the less effective it is,” he explained.

Available solutions to this dilemma were inadequate. Fine-grained API tokens offer minimal assistance and may still be exploited, impacting functionality. Semantic guardrails, meant to guide interactions, can easily be circumvented through prompt injections, especially with internet-connected agents.

While restricting agents to read-only access can mitigate risks, it significantly hinders their utility. On the flip side, granting broad access can lead to inaccuracies and negative outcomes.

Model context protocol (MCP) gateways can enforce rules at the protocol level, but only for traffic using that protocol. Guardrails provided by LLM vendors are often tied to a specific model and can be difficult to align with enterprise-specific policies. Tools like Nvidia OpenShell provide what is basically “per-sandbox egress control.”

“When we started, we struggled to find a reliable method for safely deploying tools like OpenClaw,” Franceschi noted. “Instead of waiting for a broader solution, we decided to tackle the problem ourselves and create the necessary tools.”

They aimed to develop a platform that would function as an intermediary between every agent and network request, capable of making nuanced decisions regarding permissions.

This positioning of the transport layer as a key architectural element became evident, as it is framework-agnostic, language-agnostic, and API-agnostic. The users of CrabTrap simply set the HTTP_PROXY and HTTPS_PROXY variables in the agent’s environment, redirecting all outbound requests through the proxy.

Franceschi also pointed out that their approach did not stem from the belief that the transport layer was the singular solution; instead, they advocate for a multi-tiered security strategy.

“We viewed the transport layer as an overlooked area where meaningful enforcement could be integrated along with other layers,” he commented.

The Role of the LLM-as-a-Judge

CrabTrap integrates both static rules and an LLM-as-a-judge for requests that fall outside predefined norms, as explained by Franceschi. The judge is activated predominantly for rare requests or those that do not follow established patterns, typically accounting for less than 3% of all requests from mature agents.

A guiding question was determining whether a policy is indeed effective. For static rules, assessing accuracy is relatively simple; however, with an LLM, this becomes more complex, necessitating confidence that the policy makes correct approvals while blocking inappropriate requests.

Franceschi noted, “Our main insight was to derive policies from actual observed behavior rather than crafting them from scratch.” Utilizing genuine behaviors and refining them based on real-world analytics proved to be significantly more effective than creating guidelines from a blank slate.

The Brex team developed a policy creator, which operates in shadow mode, analyzes historical traffic, samples representative calls, and formulates a natural-language policy that aligns with actual agent activities. Once this policy is formulated, an evaluation system tests any policy changes before going into effect. CrabTrap compares past audit records against draft policies and identifies necessary adjustments. Results can be filtered by method, URL, original decision, and agreement status.

This entire process runs concurrently, allowing for the rapid testing of thousands of requests. Brex also implemented a responsive feedback loop, where audit records are stored in PostgreSQL and can be accessed via an administrative API. If an agent’s access is consistently denied, the platform can alert a human or another agent to suggest a policy revision.

“This mechanism connects observed denials with policy adjustments,” Franceschi noted.

Challenges and Hurdles Faced

The development process was not without its obstacles. One significant concern was latency: “Introducing an LLM to handle every outbound API request initially sounded unfeasible,” Franceschi mentioned.

Nevertheless, it was less problematic than anticipated for a couple of reasons. Firstly, the LLM judge activates only for a small number of requests (approximately 3%). Agents tend to establish predictable traffic patterns, which quickly become static once identified. Secondly, leveraging swift models such as Claude Haiku meant that even when the judge was engaged, any added latency was minimal and could be further reduced with local models and prompt caching.

A more challenging and subtle issue was prompt injection. The judge receives entire HTTP requests, which contain user-controlled content, so a carefully crafted request could potentially manipulate the judge’s decision.

Brex resolved this by restructuring requests as JSON objects before submitting them to the model, ensuring that all user-controlled elements are safely processed rather than treated as raw input, Franceschi explained.

Impact of CrabTrap and Future Directions

Brex evaluates CrabTrap’s impact based on factors like agent engagement, network traffic trends, and net promoter scores (NPS). Franceschi highlighted that the most profound effect of CrabTrap has been an increase in “organizational confidence.”

In the past, the team had significant reservations about deploying autonomous agents throughout the organization, as existing safeguards did not offer sufficient reliability.

“CrabTrap changed that dynamic,” said Franceschi. They now have a trusted enforcement structure, boosting confidence in increasing agent deployment across various business sectors and allowing users more control over agent configuration and management.

Franceschi described the policies derived from traffic as unexpectedly robust. The team initially believed the policy builder would generate a rough draft requiring significant manual refinement. However, data from just a few days of actual traffic produced policies that closely aligned with human judgment across most held-out requests.

Furthermore, CrabTrap illuminated the extent of noise generated by agents. “The audit trail made this evident for the first time,” Franceschi stated. They leveraged denial logs and traffic insights not only to fine-tune policies but also to enhance agent efficacy, eliminate redundant tools, and eliminate categories of requests that consumed unnecessary resources.

“The proxy evolved into a discovery mechanism, not merely an enforcement tool,” he noted.

Opportunities for Improvement and Community Collaboration

Brex intends for CrabTrap to continue evolving, especially after releasing it as an open-source project. “We hope the community contributes to its development,” Franceschi commented.

Areas identified for enhancement include advanced authentication capabilities like single sign-on (SSO), detailed role-based access control (RBAC), escalation workflows allowing agents to request additional access, and policy recommendations informed by rejection trends.

Implementing programmatic configuration—creating API endpoints for developing, modifying, and applying policies to agents—could fully automate the policy lifecycle instead of requiring manual input.

Concerning escalation, if an agent is persistently denied access to a resource or endpoint, it should be equipped to redirect requests for human or other AI agent reviews, substantiating its need for access.

“This would transform CrabTrap from a strict enforcement barrier to a more managed permission framework,” Franceschi stated.

Additionally, while policies were established from network traffic, there exists potential for incorporating additional signals tied to agent activities and contextual understanding of their objectives, which would yield more precise and intricate policies.

Finally, there is an ongoing philosophical debate about CrabTrap’s nature: Should it act as a fully transparent layer that agents are unaware of or function more like a “well-meaning manager” that agents recognize and can interact with?

The open-source community’s engagement will influence these developments, and CrabTrap can only improve with a broader user base, according to Franceschi. Brex’s agents cater to specific APIs; teams employing CrabTrap with varied agents, services, and policy needs will surface “edge cases and patterns we cannot uncover independently.”

“We have ambitious aspirations for its future, and we prefer to build this openly,” Franceschi stated.

Lessons for Other Developers from CrabTrap

The feedback has exceeded expectations, with CrabTrap garnering over 700 stars on GitHub. Brex has also received inquiries from industry leaders such as OpenAI, Y Combinator CEO Garry Tan, and programmer Pete Steinberger, expressing interest in implementing similar infrastructure.

The overarching lesson is clear: “Don’t allow infrastructure deficiencies to serve as excuses for inaction,” Franceschi advised. Significant obstacles exist for enterprises aiming to deploy AI agents effectively, such as security fears, a lack of adequate tools, or ambiguous guidelines.

“It’s easy to hesitate until the industry progresses,” he cautioned. “The important takeaway from CrabTrap is that you can assert control over these issues directly.”