Closing the Speed Gap: How LinkedIn, Walmart, and Zendesk Transformed Agent Efficiency Beyond Legacy Systems at VB Transform 2026
During the recent VB Transform 2026 event, top infrastructure executives from LinkedIn, Walmart, and Zendesk discussed the hindrances faced by AI agents in enterprise settings. The consensus was that outdated infrastructure, rather than the AI models themselves, is primarily responsible for slowdowns.
Animesh Singh from LinkedIn, Desiree Gosby from Walmart, and Sami Ghoche from Zendesk shared their experiences transitioning AI agents from pilot programs to full production. Despite approaching the challenges from different angles, they all arrived at a similar conclusion: the bottlenecks they encountered were not related to the AI models.
The common thread in their discussions was the notion that corporate infrastructure has traditionally been designed for human workflows, not for AI agents. This disparity creates significant challenges in engineering capabilities.
Gosby emphasized the importance of ensuring that engineering does not hinder progress: “We must prevent engineering from becoming the bottleneck in our endeavors.”
Identifying the Real Bottlenecks
Each organization faced unique challenges stemming from their infrastructure. LinkedIn’s initial slowdown arose from Kubernetes, which was designed for on-demand container provisioning—a process that was too slow for the needs of AI agents. To address this, LinkedIn shifted from on-demand provisioning to using pre-provisioned container pools, optimizing the container management for real-time workload adjustments.
Another challenge emerged when LinkedIn allowed agents to manage their orchestration directly. Although a five-point evaluation system seemed effective, issues such as hallucination persisted. Singh noted that when one language model evaluates another, it can inherit the same flaws. To correct this, LinkedIn created a custom control flow, allowing for about 80% of their workflows to be scripted and deterministic, reserving the use of language models only for tasks requiring reasoning.
Walmart faced a different type of challenge due to unexpected success. An internal agent framework became widely adopted, leading to what Gosby called “citizen developers” creating their own solutions without formal engineering oversight. While this led to innovative solutions, it also resulted in many redundant agents. Instead of limiting the citizen developers, Walmart focused on implementing governance to identify duplicate efforts and streamline the best agents into production efficiently.
Zendesk encountered its limitations from the data aspect. Ghoche, after joining through the acquisition of Forethought, described managing an extensive archive of 20 billion customer conversations. He noted that relying solely on existing data to power language models isn’t effective. Instead, investing in robust data pipelines and infrastructure became critical for effective AI implementation.
Embracing Open Source
When discussing open source tools, the three leaders exhibited similar perspectives: prioritize ownership where feasible and seek help from cutting-edge labs only when necessary. Ghoche expressed that many enterprises prefer to retain ownership over their models and infrastructure, which shapes Zendesk’s strategy. However, they do still utilize frontier models in select areas.
LinkedIn approached this by developing two distinct systems aimed at achieving independence. The AI gateway acts as a unified access point for all outbound calls to models, irrespective of the provider. Additionally, they developed a memory system to maintain contextual data regardless of the model used.
Walmart also created an internal gateway to maintain flexibility across different operational types, ensuring compliance and governance while evaluating models based on specific workloads rather than adhering to a strict policy.
Key Recommendations for Modernizing Infrastructure
Based on their experiences, the leaders offered three key recommendations:
1. Prioritize evaluations from the start. Ghoche highlighted that evaluations are fundamental to all use cases, enabling organizations to dissect problems effectively.
2. Empower employees with AI tools early. Gosby urged that giving employees direct access to AI harnesses, along with necessary oversight, can unlock significant innovation.
3. Design for independence in models and context. Singh stressed the importance of ensuring flexibility, advocating for the creation of infrastructure that allows the reuse of contextual data across evolving models and systems.
