The Bluff Gets Called
For as long as I’ve worked in data, organisations have been saying they’ll fix their data foundations “soon.” The master data is a mess, but the dashboards work well enough. The customer records have duplicates, but the marketing team has learned to compensate. The documentation is outdated, but people know who to ask.
This was always a problem. But it was a deferrable problem, because previous generations of analytics and even classical machine learning were forgiving enough to tolerate messy foundations. BI dashboards with slightly inaccurate numbers were still “directionally correct.” Propensity models trained on imperfect data still outperformed random selection. The cost of poor data foundations was real but diffuse, spread across thousands of small inefficiencies rather than concentrated in any single visible failure.
Generative AI has changed the calculus entirely. It’s not forgiving. And it’s calling the bluff on every organisation that has been deferring its Tier 1 data product investment (see A Practitioner’s Data Product Taxonomy).
Why GenAI Is Different
Previous analytics paradigms degraded gracefully when data quality was poor. GenAI degrades dangerously.
The distinction matters. A dashboard built on slightly inaccurate data shows slightly inaccurate numbers. An experienced user applies a mental correction factor and moves on. A classical ML model trained on noisy data produces predictions that are less accurate but still directionally useful. The noise adds uncertainty, but the signal is still there.
GenAI, and large language models in particular, have a fundamentally different failure mode: confident fabrication. When a retrieval-augmented generation (RAG) system pulls outdated, contradictory, or incorrect information from your knowledge base, it doesn’t flag the problem. It doesn’t caveat the response. It weaves the bad information into a fluent, authoritative answer. The user, unless they happen to already know the answer, has no reliable way to tell the difference between a correct response and a hallucinated one.
This is “garbage in, garbage out” with a dangerous twist. The garbage comes out polished, articulate, and delivered with the confidence of someone who has never been wrong about anything. It’s the organisational equivalent of the colleague who speaks first and loudest in every meeting, regardless of whether they’ve actually done the reading.
I wrote previously about how GenAI and LLMs are an unedifying mirror of humanity. They reflect the best and worst of the data they’re trained on. The same principle applies inside the enterprise: point a large language model at your internal data and it will reflect the state of your information management back at you with brutal honesty. If your data house is in order, GenAI will be useful. If it’s a mess, GenAI will narrate that mess back to you with absolute conviction.
The RAG Trap
The most common enterprise GenAI pattern right now is RAG: point a large language model at your internal documents, knowledge bases, and data repositories, and let it answer questions by retrieving and synthesising relevant information.
The pitch is compelling. Imagine every employee having instant access to the collective knowledge of the organisation. No more searching through SharePoint. No more asking around to find the right policy document. No more onboarding that takes months because institutional knowledge is locked in people’s heads.
The reality is less compelling when your internal knowledge base has the following characteristics. And I have yet to encounter an enterprise where it doesn’t:
Multiple versions of the truth. The policy was updated in 2024 but the 2022 version is still in the shared drive. The process document was revised but the training materials reference the old process. The pricing framework changed last quarter but three different spreadsheets with three different pricing models are still floating around. A human navigating this knows to ask “which version is current?” An LLM just retrieves whatever scores highest on relevance and treats it as fact.
Contradictory information. Department A’s guidance says one thing. Department B’s says another. Both are technically current. Neither references the other. A human navigating this knows to ask which department owns the decision. An LLM synthesises both into a single response that confidently presents a blended answer that neither department would endorse.
Undocumented context. The document says the approval threshold is $50,000, but everyone knows that in practice it’s $25,000 because the CFO informally lowered it last year. The process document describes the official workflow, but the actual workflow has three additional informal steps that nobody ever wrote down. RAG can only retrieve what’s written. It can’t retrieve what everyone knows but nobody documented.
Stale information. Knowledge bases in large organisations are where documents go to slowly become wrong. Nobody deletes them. Nobody marks them as superseded. They just sit there, gradually diverging from reality, waiting for an LLM to retrieve them and present their outdated contents as current fact.
This is the RAG trap: the same qualities that make enterprise knowledge bases valuable (breadth, depth, institutional history) also make them dangerous inputs for generative AI. Without active governance, curation, and lifecycle management of the underlying content, RAG doesn’t give you an AI-powered knowledge assistant. It gives you an AI-powered confusion generator with a professional tone of voice.
The Connection to Data Product Foundations
This problem maps directly to the data product taxonomy I’ve written about previously. Organisations that have invested in Tier 1 foundational data products have a significant advantage when it comes to GenAI readiness. Not because Tier 1 products are always directly consumed by LLMs (although they can be), but because the discipline of building Tier 1 products creates the organisational muscle needed to manage information at the quality level GenAI demands.
Tier 1 thinking means: single authoritative source, clear ownership, defined quality standards, versioning, and lifecycle management. Apply that same thinking not just to structured data but to documents, knowledge bases, policies, and process documentation, and you have the foundation for GenAI that actually works.
The organisations that skipped Tier 1 (and there are many of them, because Tier 1 is unglamorous and expensive and nobody gets promoted for building master data) are now trying to build GenAI applications on top of information assets that have no clear owner, no quality standards, no versioning, and no lifecycle management. The predictable result is GenAI that works brilliantly in demos where you control the inputs and fails unpredictably in production where you don’t.
flowchart LR
subgraph Without Tier 1 Discipline
A1["Ungoverned\nknowledge base"] --> B1["RAG retrieves\ncontradictory docs"] --> C1["LLM produces\nconfident nonsense"]
end
subgraph With Tier 1 Discipline
A2["Governed, curated\nknowledge products"] --> B2["RAG retrieves\nauthoritative sources"] --> C2["LLM produces\nreliable responses"]
endThe Offensive Opportunity
The following quadrant maps where most organisations sit on the two dimensions that matter: how mature their data foundations are, and how aggressively they’re pursuing GenAI. The uncomfortable reality is that most enterprises are in the top-left: high GenAI ambition sitting on top of low data foundation maturity.
quadrantChart
title GenAI Ambition vs Data Foundation Reality
x-axis Low Data Foundation Maturity --> High Data Foundation Maturity
y-axis Low GenAI Ambition --> High GenAI Ambition
quadrant-1 "GenAI Ready"
quadrant-2 "The Reckoning Zone"
quadrant-3 "No Urgency"
quadrant-4 "Foundation Without Purpose"
"Most Enterprises (2025)": [0.22, 0.80]
"Target State": [0.75, 0.75]
"Data-Mature, GenAI-Cautious": [0.72, 0.22]The path to the target state runs through the bottom-right, not the top-left. You need to move the data foundation maturity axis before you can sustain GenAI at scale. But the executive energy is all in the top-left. The trick is using that energy to fund the horizontal movement.
Here’s the twist, and it connects to the defensive and offensive data strategy framework. GenAI is both the thing that exposes poor data foundations and the thing that can finally get them funded.
For years, data leaders (myself included) have been making the case for investment in data governance, quality, and foundational data products. The business case has always been sound but abstract: better data quality leads to better decisions leads to better outcomes. Executives nod along and then fund the AI pilot instead, because the pilot has a more exciting business case and a more visible output.
GenAI inverts this dynamic. For the first time, there is a highly visible, executive-sponsored initiative that visibly breaks when data foundations are poor. When the CEO’s new AI assistant confidently quotes a policy that was superseded two years ago, the data foundation problem stops being abstract. When the customer-facing chatbot gives incorrect product information because the product catalogue has duplicates and inconsistencies, the Tier 1 investment case makes itself.
This is the offensive use case driving defensive investment, which is exactly the principle I advocate in the defensive and offensive data strategy framework. Use GenAI ambition as the catalyst for the foundational work that should have happened years ago. The executive appetite for GenAI is real and it’s coming from the top of most organisations right now. That appetite is your best argument for the data governance and quality investment you’ve been trying to fund for the last decade.
What GenAI-Ready Data Foundations Look Like
Being GenAI-ready doesn’t mean achieving data perfection. Waiting for perfection is just another way of deferring the work indefinitely. It means having sufficient governance, quality, and curation across the information assets that GenAI applications will consume. In practice, this looks like five things:
1. Authoritative Sources, Not Just Accessible Ones
Every piece of information that GenAI might retrieve needs a clear owner and a defined status: current, superseded, draft, archived. If the same information exists in multiple places (and it will), there needs to be a way to identify which version is authoritative. This isn’t a technology problem. It’s an ownership and governance problem, and it requires someone to make decisions about what’s current and what isn’t.
2. Active Lifecycle Management
Documents and knowledge assets need the same lifecycle management that we apply to structured data products. Created, reviewed, updated, superseded, archived. If a document hasn’t been reviewed in two years, it should either be revalidated or removed from the GenAI-accessible corpus. The default should be exclusion, not inclusion. This is a reversal of how most knowledge management systems work, where everything is included by default and nothing is ever removed.
3. Quality Signals for Retrieval
RAG systems need to be able to assess the quality and currency of what they retrieve, not just its semantic relevance. This means enriching documents and data assets with metadata: last reviewed date, owning department, classification, confidence level. This metadata becomes the governance layer that helps the retrieval system (or the LLM itself) distinguish between a current policy document and an abandoned draft from 2019.
4. Domain-Scoped Knowledge Products
Rather than pointing GenAI at the entire organisational knowledge base and hoping for the best, build domain-scoped knowledge products: curated, governed collections of information organised around specific use cases. The HR policy assistant doesn’t need access to engineering documentation. The product information chatbot doesn’t need access to internal strategy documents. Scoping reduces the surface area for hallucination and makes governance actually tractable instead of theoretically desirable.
5. Feedback Loops
Every incorrect GenAI response is diagnostic data about your information quality. Build mechanisms for users to flag wrong answers, trace them back to the source information, and either correct the source or exclude it from the retrieval corpus. This turns GenAI from a passive consumer of your data foundations into an active, continuous quality signal about them. The GenAI application tells you where your knowledge base is broken, if you build the feedback loop to listen.
The Reckoning
Organisations have been accumulating data foundation debt for years. The interest payments have been manageable because previous analytics paradigms were tolerant enough to produce value despite the mess underneath.
GenAI is the event that makes the debt callable.
You can respond by avoiding GenAI entirely, which means ceding the capability to competitors and losing the executive sponsorship that comes with it. You can respond by deploying GenAI on top of ungoverned foundations and hoping nobody notices when it confidently gets things wrong, which is a bet you will eventually lose. Or you can use this moment to finally make the investment in data foundations that data leaders have been advocating for years, funded by the genuine executive enthusiasm for GenAI and motivated by the visible consequences of getting it wrong.
The third option is the right one. And it has a better chance of getting funded now than at any point in the last decade, because for the first time, the people who control the budget can see exactly what happens when the foundations aren’t there. Don’t waste the crisis.
This analysis reflects direct experience with the collision between GenAI ambition and data foundation reality at Cochlear (where the quality bar for information consumed by AI systems is set by regulatory requirements and patient safety) and at Westpac (where the sheer scale and complexity of institutional knowledge across retail and institutional banking made the RAG trap visible well before GenAI made it mainstream). The pattern is the same in both environments: GenAI doesn’t create the data foundation problem. It makes it impossible to pretend it doesn’t exist.