The atmosphere at this year's industry conference has changed

For the last few years, attending software conferences has been like feverishly bathing in a cocktail of fear and hopium. The main stage has been selling "magic AI-beans" to an audience desperate for a miracle. It was a collective fantasy with critical thinking dialled down to a murmur. On the one hand there was the hope of receiving a letter to Hogwarts to deliver the audience to a magical new AI world. On the other hand there was the fear of being left behind - the corporate equivalent of a life under the staircase while a platform shift occurred in the world outside.

To be kind - missing a platform shift of this dimension can spell extinction for a business. The pressure for legacy software builders to be considered one of the AI "cool kids" has been immense. The pressure on non-tech executives to seek the strategic advantage in AI software products has been equally intense.

So with that in mind, I was pleasantly surprised to discover a major shift this year. It's not main-stage evangelists. They are still doing their best impression of Steve Jobs and remain steadfastly committed to their airbrushed stage-ware deliveries, but the crowd is no longer buying in blindly. The polite applause is there, but the blind faith is gone.

The B2B software buyer has turned the corner. They’ve realized that AI isn't a miracle; it’s software. There is a shift underway for certain and the nature of software and software developement is changing but it is not quite the alien presence that grabbed the public consciousness in 2022. This is still software: it costs money to run, requires active management, and offers no guarantees.

The Rise of the Pragmatic AI Buyer: Business Value Over Hype

At the executive level, even within the software-buying community, the will to be "wowed" lingers as the fear of missing the platform shift remains present. Directly below them, at the operational level, there is a new, dominant persona: the methodical operative. These aren't dreamers; they are project managers tasked with implementing agentic software for clear, well-defined benefits in the near term. The difference is the potential for a future threat at the executive eye level and the clear and present reality of AI underperformance at the operational level.

Operators have completely rejected the "buy it and hope" strategy. Instead, they're arriving with a cold, cost-conscious approach rooted in rigorous benchmarks and precise expectations.

They aren't looking to be dazzled. They are scouting for toolkits that fulfill measurable business outcomes that their bosses can package however the hype-cycle demands.

The Same Tech, Different Results

As these pragmatists dig in, they’ve stumbled upon a glaring paradox: The projects that thrive and the ones that collapse often rely on the exact same underlying AI.

There are foundational questions being asked. If everyone is using the same LLMs, why do some companies see a return on investment while others face resounding failure?

Increasingly the conclusion is being reached: the variable isn't the AI. The variable is the reliability of substance the AI is being fed by the company looking to deploy it.

The Skeletons in the Data Closet

For two years, the business community has hoped for a miracle. We’ve seen a series of useful developments to enable AI consumption of existing knowledge repositories: RAG, GraphRAG, prompt-tuning, and complex chunking strategies. These are useful deconstructors of legacy data stores. But as they melt the substance they reveal a nasty truth buried in the ice: AI does not magically turn poor data into good data. Consequently bad knowledge leads to bad AI responses. There is no chunking strategy that can transform bad knowledge into good.

For a while the business comunity clung to the hope that the "magic box" of AI could somehow transcend the abysmal quality of the source material. Or maybe there was some hope that the data was not all that bad to begin with. Perhaps the ones that got lucky and stumbled on good quality data correlates to the percentage of AI projects that were successful.

For most organizations, however, it was a vain hope. The old adage remains an absolute truth: garbage in, garbage out. You cannot prompt-tune your way out of a data swamp. Any attempt to do so is a betrayal of the technology's potential and a recipe for failure.

The New Frontier: Refining Knowledge Substance

Adopters of AI are waking up to a basic reality: retrieval is not the same as quality. The question has shifted from "Which AI should I use?" to "What do I have to do to get my knowledge AI-ready?"

The only viable path forward is the aggressive refinement of knowledge substance. Stop treating the AI as the solution and start treating it as a scalpel. Use AI-assisted tooling to excise the knowledge garbage that is the visceral fat that the enterprise has been storing for decades and leave behind a leaner and trimmer dataset.

This is the only way to move toward a predictable, cost-effective model. Focus on the substance rather than the retrieval mechanism. Stop gambling and start delivering real value.

TL;DR:

The hype cycle is collapsing. The "magic beans" of the main stage are pure fiction. If you continue to ignore your data rot and rely on RAG to fix fundamentally broken information, your AI ROI will remain abysmal. Stop buying magic and start cleaning your data.