Why Business Managers May Be Steering AI Programmes Toward Failure Without Realising It

The artificial intelligence industry is experiencing one of the fastest technology adoption cycles in history.

Models are becoming cheaper. Inference costs continue to fall. Context windows are expanding. Agent frameworks appear almost weekly. Organisations are under increasing pressure to demonstrate AI adoption, while vendors and analysts continue to predict transformative gains in productivity and automation.

At first glance, this appears entirely logical. More capability, lower costs, faster delivery, and increased automation should naturally create better outcomes.

Yet many experienced project and programme managers are beginning to notice something uncomfortable.

Projects appear busier than ever, but outcomes are not always improving at the same rate.

Teams are producing more outputs, yet delivery confidence is becoming harder to validate.

Automation is increasing, while dependency visibility is often decreasing.

The problem is not that AI is failing.

The problem may be that organisations are measuring the wrong things.

A combination of four powerful forces—Jevons Paradox, Goodhart's Law, the Barnum Effect, and what I describe as Agentic Emergence Delusion Delta (AEDD)—is creating conditions where AI adoption can increase rapidly while reality alignment quietly declines.

For project and programme managers, this matters because successful delivery has never been about activity. It has always been about outcomes. And outcomes require a constant connection to reality.

When Efficiency Increases Consumption

In 1865, economist William Stanley Jevons observed that improvements in steam engine efficiency led to increased coal consumption rather than reduced consumption.

The explanation was simple. As coal became more efficient to use, people found more ways to use it.

The same pattern is emerging across artificial intelligence.

Every reduction in model cost encourages greater adoption. Every increase in capability encourages additional use cases. Every improvement in accessibility expands deployment across the organisation.

The industry often interprets this as evidence of success.

In many respects, it is.

However, there is an important distinction between increased consumption and increased value.

More prompts do not automatically create better decisions.

More agents do not automatically create better outcomes.

More automation does not automatically create better businesses.

The danger begins when organisations assume that increased AI activity is itself evidence of progress.

When the Metric Becomes the Objective

This is where Goodhart's Law enters the picture.

Economist Charles Goodhart famously observed that when a measure becomes a target, it ceases to be a good measure.

Many organisations are now tracking AI adoption metrics as indicators of innovation and progress. They measure AI usage, automation rates, model interactions, generated content, and agent deployments.

Initially these metrics provide useful visibility.

The challenge emerges when they become objectives in their own right.

Employees are encouraged to use AI because usage is measured. Teams are rewarded for automation because automation is reported. Products are judged by AI engagement because AI engagement appears on executive dashboards.

Gradually the question changes.

Instead of asking whether AI is creating value, organisations begin asking whether enough AI is being used.

The distinction may seem subtle, but it fundamentally alters programme behaviour.

Project managers have seen this before. Whenever activity becomes the target, outcomes often become secondary.

Confidence Is Not Evidence

A third force quietly amplifies the problem.

Psychologists refer to it as the Barnum Effect—the tendency to accept information as accurate because it sounds convincing and authoritative.

Modern AI systems are exceptionally effective at producing authoritative language.

They confidently report that features have been implemented, integrations completed, workflows repaired, analyses performed, and recommendations generated.

The language sounds definitive.

Humans naturally interpret confidence as competence.

Yet confidence is not evidence.

A generated report may look professional while containing flawed assumptions. A coding agent may declare success while introducing new defects. A planning agent may create an elegant programme plan while overlooking critical dependencies.

The risk is not that the system deliberately misleads.

The risk is that persuasive outputs can become accepted as reality without independent verification.

For project professionals, this should trigger immediate concern.

Delivery has always depended upon the distinction between what is reported and what is proven.

The Reality Gap

This leads to what may become one of the defining risks of enterprise AI.

The gap between what a system believes is true and what is actually true.

This gap can be described as Agentic Emergence Delusion Delta, or AEDD.

At its simplest:

AEDD = Perceived Reality − Actual Reality

A small AEDD indicates strong alignment with reality.

A large AEDD indicates growing divergence.

The challenge is that many modern AI systems are optimised for capability rather than reality alignment.

They become increasingly effective at producing outputs while remaining relatively disconnected from the environments in which those outputs must operate.

As capability increases, confidence often increases with it.

Reality alignment does not necessarily follow.

The Dependency Blind Spot

One reason AEDD is difficult to detect is that most AI systems have limited awareness of dependency structures.

A coding agent can create functions, pages, APIs, and workflows.

A planning agent can generate schedules, workstreams, and milestones.

A reporting agent can generate summaries, dashboards, and recommendations.

What many systems do not truly understand is why these artefacts exist, what depends upon them, what outcomes they support, and whether those outcomes have actually been achieved.

The artefact exists.

The intended result may not.

This distinction is critical.

Project managers understand that successful delivery is not measured by completed tasks. It is measured by realised outcomes.

AI often excels at the former while remaining largely blind to the latter.

The Time Hierarchy Problem

Perhaps the most significant blind spot of all is what might be called the Time Hierarchy Problem.

Projects do not exist within a single timeframe.

They operate simultaneously across multiple layers of time.

At the lowest level sit individual tasks. Above those sit workstreams. Above workstreams sit programme objectives. Above programme objectives sit strategic business outcomes.

Each layer depends upon the layers beneath it.

However, the consequences of decisions made at one level often emerge much later and at a completely different level of the hierarchy.

Experienced programme managers understand this instinctively.

They know that a seemingly minor technical decision today can become a major operational issue six months later.

They understand that small compromises accumulate.

They recognise that local optimisation does not always create global optimisation.

Current AI systems generally struggle with this.

Most systems optimise the immediate problem presented to them. They improve the current prompt, refactor the current code, enhance the current workflow, or accelerate the current task.

Viewed locally, each change appears beneficial.

Viewed across an entire programme, the picture may be very different.

A coding agent may improve a component while weakening architectural consistency.

A planning agent may accelerate one workstream while creating delays elsewhere.

A reporting agent may optimise delivery metrics while reducing visibility of emerging risks.

Each individual decision appears reasonable.

Collectively they can move a programme further away from its intended destination.

This is project drift.

Not because the system is malfunctioning.

Because it cannot see the full hierarchy of consequences.

Death by a Thousand Optimisations

Imagine driving from London to Edinburgh.

Every mile travelled north feels like progress.

Yet if the road gradually curves east, small navigation errors accumulate.

No single correction appears significant.

No individual decision seems wrong.

Eventually the destination itself is missed.

Many AI initiatives face exactly this challenge.

Each recommendation appears sensible.

Each optimisation appears justified.

Each generated output appears useful.

The problem only becomes visible when viewed across time.

By then, substantial investment may already have been committed to a trajectory that no longer aligns with the original objective.

This is why experienced programme managers often sense drift before they can formally prove it.

They are seeing the consequences of time hierarchy breakdown.

Why Traditional Programme Governance Matters More Than Ever

Ironically, the rise of AI may increase the importance of disciplines that many organisations have spent years attempting to minimise.

Verification.

Traceability.

Dependency management.

Acceptance criteria.

Outcome measurement.

Independent validation.

Change control.

These practices are frequently dismissed as administrative overhead.

In reality, they exist for one reason.

They keep programmes connected to reality.

As AI systems generate more outputs, reality alignment becomes increasingly important rather than less.

The faster the system moves, the stronger the reality-coupling mechanisms must become.

The Coming Divide

Over the next decade we may see a separation emerge between two distinct classes of AI systems.

The first will optimise for capability, speed, cost reduction, adoption, and output generation.

These systems will dominate headlines and marketing campaigns.

The second will optimise for dependency awareness, verification, continuity, traceability, outcome validation, and reality alignment.

These systems may attract less attention.

Yet they may become the systems trusted with mission-critical operations.

Because organisations do not ultimately buy intelligence.

They buy outcomes.

And outcomes depend upon remaining connected to reality.

The Question Every Programme Board Should Ask

Much of the industry is currently asking:

"How intelligent are the agents?"

A far more important question may be:

"How connected to reality are the agents?"

Because intelligence alone is not enough.

A highly capable system that misunderstands reality remains dangerous.

A moderately capable system with strong dependency awareness and reality alignment may prove far more valuable.

This is not a new lesson.

Human history is full of examples where confidence became detached from reality and failure followed shortly afterwards.

Artificial intelligence may simply be rediscovering the same principle at machine speed.

Final Thoughts

The greatest risk facing enterprise AI may not be hallucination.

It may be the accumulation of thousands of seemingly correct local decisions that gradually disconnect a programme from its intended destination.

Jevons Paradox encourages more AI consumption.

Goodhart's Law encourages AI adoption as an objective.

The Barnum Effect encourages trust in persuasive outputs.

AEDD quietly measures the growing distance between perceived and actual reality.

The Time Hierarchy Problem then compounds the issue by allowing countless local optimisations to drift away from long-term outcomes.

None of these forces are inherently dangerous on their own.

Together, however, they create an environment where confidence can rise while certainty falls.

For project and programme managers, this creates a new responsibility.

Not simply managing delivery.

Not simply managing risk.

But acting as custodians of reality itself.

Because organisations do not invest in tasks.

They do not invest in prompts.

They do not invest in token consumption.

They invest in outcomes.

And outcomes exist at the top of the hierarchy, not the bottom.

The organisations that thrive in the age of AI will not necessarily be those that use the most AI.

They will be those that remain closest to reality while using it.