Select a country Global - English
Contact Us

Infrastructure is critical again: Here's what it means for every CIO

I have spent the better part of three decades advising CIOs of multinational corporations and public sector organizations with some of the world's most complex enterprise technology environments. I have watched enterprises navigate the internet era, the cloud era and the mobility era. Each wave changed how IT leaders think about infrastructure, but what's happening right now is different - more consequential, and more urgent than anything I've seen before.

We're in the AI execution era, and most organizations aren't ready for it.

That's not a criticism. It's an observation grounded in conversations I have every week with CIOs, CISOs and infrastructure leaders who are wrestling with the same hard truth: AI has outgrown the infrastructure built to support it.

The problem is not your AI strategy but what's underneath

I often hear about organizations that invested significantly in AI. The pilot projects worked, the board was excited and the business cases were approved. Then, somewhere between ambition and execution, things stalled. Models that performed brilliantly in controlled environments began to struggle in production at scale. Latency spiked, costs climbed and the security team started raising flags. The ROI that seemed so compelling on a slide failed to materialize.

The instinct is to blame the model, the data or the vendor. But in almost every case, the real culprit is the infrastructure beneath it.

AI workloads don't arrive in bursts; they create a continuous operational pulse. Inference runs constantly, data is always moving and latency is now a business outcome. But organizations' current infrastructure was designed for a completely different heartbeat - one built for the transactional, episodic demands of legacy systems.

According to NTT DATA's 2026 Global AI Report: A Playbook for Private and Sovereign AI, 96% of organizations agree that their current infrastructure is slowing AI adoption. That is a strategic emergency.

AI infrastructure debt: The hidden tax

Every organization carries what I call AI infrastructure debt. It's the cost of running modern AI on systems built for a different era, and it compounds quickly:

  • Aging platforms demand increasing effort just to stay stable.
  • Traditional infrastructure cannot scale with AI expanding the enterprise footprint.
  • Vendor lock-in inflates total cost of ownership.
  • Fragmented, multivendor environments create operational drag precisely where speed and control matter most.

What begins as manageable friction becomes a structural constraint. And as investment grows, impact does not. This problem can be solved, but it requires a fundamentally different approach to infrastructure than most organizations are taking.

The mandate has changed. Is your operating model keeping up?

I talk to technology leaders every day who understand the problem intellectually but are struggling with the execution, and I understand why. The mandate has changed dramatically. CIOs are now being asked to modernize fast, secure deeply, automate broadly and scale AI responsibly - all at once, and on budgets that have not tripled to match the ambition.

What makes this particularly challenging is that the barrier is both technical and operational. Most organizations are still running infrastructure the same way: reactive, ticket-driven and dependent on manual interpretation of signals across fragmented environments. At AI scale, that operating model becomes a liability. It creates drag, inflates costs and introduces risk precisely when speed and control matter most.

We need a foundational change in how infrastructure is planned, managed and measured.

3 things to get right in your organization

At NTT DATA, we've structured our thinking around three things that must work together - not as separate workstreams or as a checklist but as one integrated execution model.

1. Modernizing the AI foundation

This means rebuilding your infrastructure stack for continuous inference, not patching what exists. Engineer your network for sustained AI traffic, embed security by design so controls scale without constraining inference, and upgrade your data and storage pipelines to keep feeding accelerated computing.

2. Embedding AI into your IT operations

This is the one I'm most passionate about, because it's where AI stops being a project and starts being the way your organization actually runs. Our agentic Software-defined Infrastructure (SDI) Services platform moves you from reactive operations to an intelligent, always-on operating model. The SDI Services Agent delivers multivendor intelligence across complex environments through natural language. You transition from SLAs to value-level agreements, and from keeping the lights on to turning infrastructure performance into measurable business outcomes.

A secure, enterprise-grade infrastructure foundation combined with a conversational agentic service experience is now a strategic business differentiator.

3. Scaling with trust and control

Trust is the foundation on which autonomous AI can responsibly scale, and this is where I see most organizations making a critical mistake.

Governance, auditability and security cannot be bolted on after the fact. When AI scales, it touches more systems, runs in more places and moves to the edge. Every new deployment that happens without trust built in from the start creates a new point of risk - a new gap between what AI is doing and what you can see, verify and control.

This is why our Enterprise AI Factory is deployed on your premises, with sovereign and private AI requirements - data sovereignty, security by design and compliance by architecture - treated as first-order priorities. The word "private" is deliberate. It signals something specific to every CIO and CISO: Your data stays where it should. Your models run where your regulators say they must. Your controls are embedded, not assumed.

This is the moment

I started this article by saying we are in the AI execution era. Let me be more precise about what that means.
It means the window for deliberate, foundation-first infrastructure investment is open right now. It will not stay open indefinitely. As AI workloads scale, the cost and complexity of retrofitting the right foundation increases exponentially. If you make these decisions today, you are building a structural advantage.

Infrastructure Solutions at NTT DATA exists to help you build and run AI continuously and at scale, on a trusted core foundation. It's a commitment backed by 4,700 certified experts in 165 countries and trusted by 75% of the Fortune Global 100.

According to an IDC Market Note, "IDC is encouraged that the Infrastructure Solutions unit is now one of the centerpieces of the overall NTT DATA strategy. It reflects the impact that AI is having on the market and the importance of building a modern infrastructure foundation, the vitality of key partners in areas like networking and security, and the innovation investments by services partners like NTT DATA Infrastructure Services organization outlined previously."*

The leaders of the next decade will be defined by the foundations they build today. If you're ready to move from pilot projects to operating reality, I would like to show you exactly how.

WHAT TO DO NEXT

Download our guide, Architecting infrastructure for the secure AI enterprise, and book your AI Infrastructure Readiness Assessment today.

Dilip Kumar

President and Global Head: Infrastructure Solutions at NTT DATA

Dilip Kumar is the global head of Technology Solutions at NTT DATA and a seasoned IT services leader with over 25 years of experience in technology leadership, business transformation and digital transformation.

At NTT DATA, he has been instrumental in driving the One NTT Digital Transformation strategy, successfully integrating 30 diverse operating companies to enhance collaboration and operational efficiency. Known for his strategic vision, leadership and results-driven approach, he has consistently driven impactful technology solutions that fuel business growth.

Dilip holds a management degree from Kellogg School of Management and is deeply committed to mentoring leaders, leveraging technology to solve complex challenges and making a positive societal impact.