Enterprises have spent the past two years writing large checks for AI models, compute capacity, software licenses and graphics processors. The investment has been historic, but a growing number of organizations are discovering a stubborn operational truth: they can build an impressive AI system and still fail to deliver it reliably to the people, applications and agents that need it. When an expensive AI workflow collapses, the model is often the first suspect. In many cases, the real weakness sits beneath the model, in the connectivity and data movement that carry requests between users, security controls, cloud regions, data sources and other systems.

That gap is no longer theoretical. Recent research shows 59% of organizations reporting bandwidth issues, up from 43% a year earlier. Latency challenges have nearly doubled, rising from 32% to 53%. Those numbers point to a broader problem: enterprises are underinvesting in the network conditions required by AI, then paying for the gap through poor performance, damaged user experience and unnecessary spending on the wrong parts of the technology stack.

Key facts

• 59% of organizations report bandwidth issues, up from 43% the previous year.
• Latency problems have risen from 32% to 53%.
• A typical AI agent can generate up to 450% more total network traffic per task than a human performing equivalent work.
• Local pilots can perform well when users are close to cloud regions and bandwidth is abundant, then deteriorate when deployments go global.
• Bandwidth is about volume; latency is about time, and latency is the harder challenge for chained AI workflows.
• Compute and GPU availability are rarely the primary bottleneck when end-to-end performance fails.

Why AI failures often start below the model

An AI conversation from two years ago and one today would be almost unrecognizable. The market has moved from asking whether the technology is real to declaring full commitment and demanding that it work at scale. During that shift, most of the attention has gone to model structures, model choice and model performance. But a model is only as effective as the infrastructure connecting it to users, applications, agents and the other models on which it depends.

Traditional enterprise networks were built around a predictable human transaction. A person searches for something, the system fetches it, the transaction ends. AI breaks that pattern. Requests are distributed, bursty and concurrent. They are generated at machine speed rather than human speed. An AI agent may call a model, retrieve enterprise data, invoke an API to another system and trigger further model calls in turn. Each step adds distance, delay and dependency.

That is why a successful pilot proves only that the model works in a controlled setting. It does not prove that the enterprise can run the model across its real geography, security architecture and production traffic patterns.

The data movement problem in numbers

The scale of the shift is easy to underestimate. One statistic remains striking: a typical AI agent can generate up to 450% more total network traffic per task than a human performing the equivalent work. This is not simply more traffic of the same kind. It is a different traffic profile: distributed, bursty, concurrent and generated at machine speed. Networks designed for human browsing patterns can struggle under that load even when raw bandwidth appears sufficient on paper.

The research on bandwidth and latency reinforces the point. Bandwidth issues are now reported by 59% of organizations, up from 43% a year earlier. Latency challenges have climbed from 32% to 53%. These are not marginal changes. They suggest that many AI rollouts are colliding with infrastructure limits that were never designed for agentic workloads, rich data types and global user bases.

The pilot trap: local success, global slowdown

A familiar pattern plays out across large enterprises. In the initial environment, the AI deployment performs exactly as expected. Users are close to the cloud region. Bandwidth is ample. Usage is still modest because the organization has not yet rolled the tool out broadly. Everyone declares success.

Then the deployment goes global. Employees in Latin America, parts of Asia or Africa see dramatically slower responses. The instinctive conclusion is that the AI platform is underperforming. The logical response seems to be more spending: more compute, more licenses, more capital expenditure aimed at expanding the platform. But that can be the wrong lever. When the full end-to-end transaction is examined, the model usually is not the problem.

Consider the path of a single AI request. It may move from a branch office to a centralized security stack, then to a cloud region, then to a data-hosted backend somewhere else. Add inconsistent local internet performance, packet loss and congestion, and the AI experience deteriorates. The model may still be functioning as designed, but the data movement around it is failing the user.

The challenge also varies widely by geography. In some regions, the central question is not which model to use but how to deploy it at all. In others, the dominant concern is latency, especially when local infrastructure routes traffic back to major cloud regions. Silicon Valley and São Paulo do not present the same infrastructure conversation. Treating them as though they do is exactly how well-intentioned rollouts stall.

Bandwidth is volume; latency is time

Bandwidth and latency are often treated as the same problem, but they are not. Bandwidth is volume. AI increases it because organizations are moving richer data: documents, images, video and telemetry. Latency is time, and it is where the harder problem lives.

AI workflows are chained. An agent calls a model, retrieves enterprise data, invokes an API to another system and that system may call another model in turn. The cumulative delay across that chain is not predictable, because no one knows in advance what will be asked or how far the request must travel. A bigger circuit does not solve that problem by itself. What matters is congestion routing, packet loss, jitter and where security inspection happens along the path.

Even hyperscale infrastructure does not eliminate the issue. The system’s speed depends on whether a request can traverse the full chain, from user to network to security to execution and back, without delay compounding at every hop. When teams are certain that a cloud provider must be the bottleneck, the infrastructure on their own end is rarely the issue. More often, that infrastructure is excellent because it is built to deliver exactly what it promises. The real question is whether the request can survive the full round trip without losing time at every handoff.

Why compute and GPUs are rarely the bottleneck

Organizations have become accustomed to solving AI performance problems by adding compute. If a model is slow, buy more processing power. If usage is growing, buy more licenses. If the pilot succeeded but production is uneven, expand the platform. That reflex is understandable, but it can direct money toward the wrong layer of the stack.

In many failing AI experiences, compute and GPU availability are not the primary constraint. The model may be healthy. The application may be well designed. The problem may be congestion, routing, packet loss, jitter or security inspection along the path. Leaders who rely on average network metrics can miss these problems because averages conceal trouble along specific routes or in particular locations. A network can look healthy overall while a critical region, office or user group experiences degraded performance.

Why geography changes the AI performance equation

AI performance is not uniform across a global enterprise. It depends on where users, models and data sit, and on the legal and technical ability to move traffic across regions. Data sovereignty rules can limit where information may travel. Local internet performance can vary sharply. Routing decisions can send traffic thousands of miles away from the nearest cloud region. Each of these factors changes the experience of an AI workflow.

That is why leaders should evaluate performance in their hardest geography, not their best one. A rollout that works near a major cloud hub tells little about how it will perform in a remote office, an emerging market or a region with constrained connectivity. The organizations that plan for the hardest cases will be better prepared to scale; those that extrapolate from the easiest cases will encounter surprises after the budget has already been committed.

Questions leaders should ask before spending more

There is no one-size-fits-all fix, but there are questions every leader should be able to answer before scaling an AI deployment:

• Where do AI users, models and data sit: locally, regionally or centrally?
• What is the data sovereignty picture, and can traffic legally and technically move across the regions the rollout depends on?
• Are teams assessing the actual end-to-end network conditions experienced by AI users, including latency, jitter, packet loss and routing, or relying on average metrics that conceal problems?
• Do the team running the AI strategy and the team running the network infrastructure communicate, or do they operate in separate silos?
• What does performance look like in the hardest geography, not the best one?

Until those questions can be answered, scaling will remain difficult regardless of how well the pilot performed. Enterprises risk spending billions on an elite racing engine and then trying to race it on a road never designed for that speed. They will not get the performance they paid for, not because the engine is wrong, but because the infrastructure underneath it cannot support what they are asking it to do.

AI strategy and network strategy can no longer be separate conversations. They must be part of the same conversation from the very beginning. The organizations that scale AI successfully will be those that treat the network as part of the product, not as an afterthought.

Source: Network World News