Wall Street Weighs In on AI Development: Enterprises Reveal 86% of GPUs Function at Half Capacity or Less

Many enterprise organizations are leveraging AI agents without the necessary oversight and control mechanisms in place, and they are aware of this imbalance. This insight comes from a June study conducted by VentureBeat Research, which surveyed 573 technical leaders at companies with at least 100 employees across five distinct areas of the AI agent infrastructure.

As these enterprises revise their structures to align with emerging standards, budgeting for enhancements is underway. Approximately 60% of organizations plan to either replace or incorporate new vendors across five control categories within the next year. Furthermore, about a third intends to make such changes within the next quarter, according to the findings.

The five key areas being developed include: agent identity (determining which agent is permitted to do what), evaluation of agent output (assessing the quality of the results), cost tracking (monitoring expenses associated with each agent), the context layer (the business data and definitions that inform agents), and the orchestration control plane (software that manages complex, multi-step agent operations).

Companies are already facing consequences for deploying these agents without comprehensive control functions. Reports indicate that 54% of organizations experienced a security incident involving an agent or nearly faced one in the last year. Additionally, 27% only respond reactively to agent expenditures, discovering costs upon receiving invoices without setting per-agent budgets or limits.

The report identifies five notable findings, each representing one layer of the technology stack and offering practical steps based on the data.

Underutilized GPU Resources: 86% Report Low Usage Rates

A staggering 86% of enterprises utilizing their own GPUs report that these resources are running at less than 50% capacity. Financial analysts have been debating the potential overinvestment in AI infrastructure, and this reflects the concerns raised by organizations sourcing this expensive hardware. Most enterprises aren’t maximizing their resources effectively.

Furthermore, only 44% of companies closely track the actual costs and returns of their AI computing investments, with others relying on estimates. As enterprises continue to explore their procurement options, 45% indicate that they are likely to evaluate AI-specialized cloud services in the next year, although less than 2% are currently using such services. Additionally, about one-third of respondents are considering alternatives to Nvidia hardware.

Majority of Agents are Limited to Simple Tasks: 71% Operate as Basic Chatbots

Seventy-one percent of surveyed enterprises report that only a quarter or fewer of their deployed agents can conduct multi-step tasks autonomously, while most serve as single-query chatbots. Only 10% claimed that true agents dominated their deployments. The findings highlight a discrepancy between the industry’s claims of broad adoption and the actual capabilities of the technology in use, with many agents falling into the “agentwashing” phenomenon.

Two-Thirds of Enterprises Trust Automated Evaluations for Production Changes

Two-thirds of enterprises either already enable AI agents to implement code changes based on automated evaluations alone or are developing systems to facilitate this within a year. However, only 5% fully trust these evaluations. These findings come amid reports of a significant number of organizations experiencing failures due to agents that passed internal evaluations.

Most evaluation checks occur before deployment, with only 23% of companies conducting real-time quality assurance for agents in operation. The need for rigorous testing prior to removing human oversight from workflows is a critical takeaway.

CREDENTIAL SHARING AMONG AGENTS INCREASES SECURITY RISKS

Approximately 69% of organizations allow shared credentials among their agents during active operations, which correlates with a higher likelihood of security incidents. Businesses with such systems experienced incidents at a rate of 63.5%, compared to 40.9% for those with strictly individual agent identities.

Contextual Errors Lead to Misinformation: 57% Report Incorrect AI Responses

More than half of enterprises reported that inaccurate, confident outputs from agents could be traced back to poor or inconsistent business context, with many seeing repeated errors. A considerable number are already implementing solutions to govern business definitions in an effort to mitigate these issues.

Shifting Priorities and Vendor Independence in AI Solutions

Another noteworthy trend is the growing emphasis on the portability of AI technologies. Earlier in the year, security and permission limitations were prevalent concerns, but now, vendor lock-in has emerged as a dominant issue. This change may be linked to recent developments in the AI space, where accessibility to alternative AI models has seemingly increased competition.

Overall, the synthesis of these surveys indicates that there is a significant opportunity for enterprises to reassess their vendor relationships and introduce new controls within the next year. Across the five control layers, a substantial percentage of organizations are prepared to change or add vendors in the near future. The emphasis on existing providers’ default tools may shift as organizations evaluate the effectiveness of these offerings in light of their evolving needs.

As enterprises prepare for the upcoming fiscal year, it will be important to track whether they align with these strategies — particularly in the areas of agent identity, evaluation efficacy, GPU utilization, and semantic context governance.