“Cloud-Native” Was a Best Practice Consequence
It’s been a month since the Cloud Native Computing Foundation (CNCF) Kubecon in Amsterdam, and I have had a chance to process the wide variety of signals from the presentations and from the expo floor. My conclusion: whilst cloud-native architectures and stacks are underpinning the tidal wave of (generative) AI, they remain a work in progress. This is understandable, perhaps, but still needs to be taken into account as organizations look to adopt.
Before I continue, a caveat: while we now speak about “cloud-native” as though it were a singular theme, it didn’t arrive as a finished architecture. Given the attraction of cloud-based services to spin up workloads in an instant, developer teams were quick to adopt Docker’s container models for their distributed applications. Google’s open-sourcing of Kubernetes offered a way of orchestrating and managing them.
Other areas of tooling and best practices emerged in response to the challenges of running distributed applications at scale; for example, observability emerged as fault diagnosis notoriously became a whodunnit across a swarm of small systems, and platform engineering formed as a discipline to move infrastructure knowledge into the developer space.
And so on and so on until we arrive at the current situation, which is one of a mature stack for cloud-native applications, together with reasonably well-understood practices. As Chris Aniszczyk, CNCF CTO put it:
“Kubernetes has become stable and boring in some ways … it consistently ships, and people expect it to work. Generally, that’s standardized into a core team kind of responsible for that, mostly called a platform engineering team, that’s become a popular pattern.”
Given where much of the AI stack has come from (hello Google, with another open sourcing—Tensorflow this time), it is unsurprising that many AI workloads (I can’t find the exact number, but I remember two-thirds being mentioned) now run on cloud-native architectures. But, once again, this isn’t due to some master plan; we can see it more as serendipity on a global scale.
Several forces conspire to make cloud-native architectures a work in progress when it comes to AI workloads. First, the workloads themselves, and then the tooling and best practices.
The Nature of AI Workloads is Changing.
From billions of dollars of resources being thrown at training new, massive-scale “frontier” models, word on the street is that processing will shift increasingly toward inference. To quote Jonathan Bryce, Executive Director, CNCF:
“Inference is going to be two-thirds of all AI compute by the end of this year, and that’s completely flipped in just three years. There’s a huge footprint, a massive amount of investment, and we do see a lot of inference workloads on Kubernetes already.”
You don’t have to be a data scientist or AI platform engineer to understand that the architecture required for AI model training (requiring vast pools of data, processed in whatever time it takes) will be different from that required for inference (needing processor and memory resources to be focused on efficient maths at scale). Understandably, tooling is a work in progress. Says Allan Naim, director of product management at Google Cloud:
“We’re early in this journey. We’ve transitioned from kind of the world of training to now inference to agents, so it’s a step-by-step process where you really have to pay close attention to performance, scale, to your security, to all those various aspects of getting something into production.”
It would be unfair to suggest that a standards organization such as the CNCF should have a clear view of even three months’ time, when nobody else has. Nonetheless, the fact remains that the architecture and stack we land on this time next year may differ from what we have now. The lesson: don’t be too quick to back any one horse (prompt idea for your next LLM session: “Why didn’t every organization standardize on OpenStack?”)
The Missing Pieces: Governance, Operations, Security
The best practices and tooling required for “building things” are different from those for “running things.” At GigaOm, we are seeing no end of new and existing vendors looking to bridge the platform gap—everyone is now an agentic stack provider, with a similar, yet different ‘marchitecture’ diagram. From CNCF’s perspective, the emphasis is on supporting platform engineers believes Ellis Tarn (Principal Engineer at AWS):
“The requirements that platform engineering teams have been working towards—to build guardrails for safe software—are only stronger now that we have more software that we trust less, generated by agents or developers working with agents.”
Next, we revisit operations and observability, as the wonderful new principle of non-determinism is coming into play. In simple terms, given the same inputs, traditional IT systems would produce the same outputs. This is not the case with LLM-based decision-making, meaning that a problem may manifest differently over time, and today’s standard tools are not yet capable of responding. Said one vendor I spoke to:
“We spend so much time thinking about agentic AI and automation and all of that. But when you talk to people, they’re like, ‘I'm stuck with, you know, Grafana and Prometheus.’ They really want to progress, but they're still stuck with existing tools.”
Finally, we have security, with its double whammy of giving the bad guys the same tools as the good guys. Supply chain security is already a huge issue for open source software, even as LLMs open up new forms of exploits. Said Jim Zemlin, CEO, of the Linux Foundation:
“In the last few years we’ve seen very real vulnerabilities, zero days that can be exploited very, very quickly, and this is placing a different burden on already overworked maintainers of these open source projects.”
Agreed Brian Stevens, SVP and AI CTO at Red Hat:
“We need to relook at trust in addition to security, as we look at how artificial intelligence is impacting the open source community.”
Again, completely understandable, but no less real.
The Burning Platform: Developers are Moving Faster Than the Architecture
I remember the first demo I saw of Microsoft Copilot generating a website, about 18 months ago. Since then, progress has been astonishing: today, Claude Code (likely with Sonnet 4.6) and other model-based environments can stand up and deploy entire applications. The blessing and the curse is that you can have something up and running in your lunch hour: we are all developers now.
Why a curse? Because coding was only ever one part of the end-to-end lifetime of an application. Software is ephemeral (how many times have I written that), and as a result, it can grow and grow. Mickey Mouse, as the Sorcerer’s Apprentice, discovered the spell to create a broom; it wasn’t long before thousands of brooms were running amok. As with sorcery so vibe-coded software, believes Chris Aniszczyk:
“What we’re seeing is people deploying a bunch of crazy tools outside of their core infrastructure platform teams, and a lot of these guys are still trying to figure out, how do I consolidate?”
The consequences are felt by operations teams and security teams, who find themselves faced with the consequences (and associated risks) of new software packages popping up left, right, and center. Developers, too, end up with multiple new code repositories, half-finished projects, and incomplete pipelines, all of which sap their time:
“These are the same people who are trying and testing new things and going into the whole agentic world with this DIY setup, but they’re also now thinking about how we bring that back under some form of control.”
Add that the architecture is itself evolving, and we have a heady mix. A conversation I had with several vendors was on the relationship between platform teams (tuning the stack based on general assumptions of need) and developer teams (assuming the platform just works). The result is inefficiency and poor performance, said one:
“Application people will just build things on top of it and then wonder why it’s running badly because it’s not tuned right for the platform that they’re running on.”
Without coordination, today’s developer agility is tomorrow’s inertia. Given a core tenet of AI, that it will take any resources you have available, inefficiency in platform or practice is not tenable.
So What Can Enterprises Do?
Great question, I am glad you asked. It possibly doesn’t need to be said that AI-native architectures of tomorrow will look remarkably similar to cloud-native architectures of today. This means loosely coupled, scalable, and environment-agnostic. Sure, we may see new capabilities added to the de facto stack—my money is on non-deterministic observability for a start.
While nobody has a monopoly on where things are going, I would suggest two parallel paths: build for the future and accept the present. Avoid investing too heavily in any product that claims to solve for agentic architectures or similar. Nothing can, or at least, if it does, it may do so in a proprietary way.
At the same time, keep an eye on emerging standards and best practices. Follow the actions (and blogs) of cloud-first organizations, go to the forums, and learn from experiences across the industry. We may be emerging from the pilot phase of AI adoption, but we are a long way away from the “it just works” period of future calm.
And follow the pharmacist's unspoken rule, "If a bottle doesn’t have a label, throw it away." From a vibe coding perspective, this means: if a new piece of software hasn’t been built according to baseline norms and guardrails, then it isn’t credible. The good news is, LLMs are also super at bringing poorly designed and documented chunks of code up to scratch. If the software is functionally useful, it can be seen as an input.
Overall, let's not doubt the huge progress that has been made with LLMs, but also be aware of the distance still to go. Just as cloud-native architectures evolved into the more stable base they offer today, so AI-native platforms and practices will build out and mature. Don’t be in a rush to back any particular horses, but be prepared to join in the race.