Fake Is Feeding Fake and the World Is Burning

We have spent a great deal of time worrying about whether (ai) will create false information.

That may be the wrong problem.

The bigger problem is what happens after the false information has been created.

Because it doesn't simply sit there. It gets published. Indexed. Retrieved. Summarised. Quoted. Embedded. Translated. Rewritten. Recommended. And eventually consumed by another machine.

The second machine may have no idea that the original information was uncertain. So it uses it. Generates something from it. Publishes that. And another machine consumes the result.

Fake is feeding Fake. And the world is burning.

Not because every piece of generated information is false. But because we are creating an information environment in which origin can disappear faster than knowledge can be verified.

We used to have a chain

Traditional knowledge had a relatively simple structure. Someone observed something. Someone recorded it. Someone else could inspect the record. A source could be identified. A person, institution or publication could be held accountable.

There were problems, obviously. Humans have always lied. Institutions have always made mistakes. Propaganda is ancient. But the chain existed. You could ask: Where did this come from?

Now the chain is becoming recursive. A model generates an answer. Another system retrieves it. A person publishes a modified version. A search engine indexes it. A new model encounters it during retrieval or training. It generates a more polished explanation. That explanation becomes another source. And the next system consumes it.

The output is no longer simply travelling from source to consumer. It is travelling:

source → machine → content → machine → content → machine → content

The knowledge environment becomes a feedback loop.

Recursive knowledge consumption

This is the problem I would call recursive knowledge consumption. A system consumes information that previous systems have already generated, transformed or inferred from other machine-generated information.

Imagine a photograph of a tree. Then imagine a description of that photograph. Then imagine a summary of the description. Then imagine a generated article based on the summary. Then imagine another model answering a question using that article.

At every step, something can be lost. Detail. Context. Uncertainty. Provenance. Qualification. Contradiction.

Eventually, the system may be reasoning from a statement whose connection to the original observation has become almost impossible to reconstruct. Yet it can still sound completely coherent. The machine is consuming the residue of previous machine consumption.

The dangerous part isn't the first fake

The first false statement is not necessarily the catastrophe. The catastrophe is propagation without provenance.

Suppose an (ai) system invents a plausible statistic. One person publishes it. A search engine indexes it. Another model retrieves it. A journalist encounters the generated summary. The statistic appears in an article. Another model reads the article. A consultant quotes the article. A report cites the consultant. A policymaker reads the report.

Now ask: Where did the number come from?

The answer might ultimately be: a model generated it. But the number has acquired layers of apparent authority. This is how falsehood becomes difficult to remove. It does not remain one false statement. It becomes a lineage of derivative statements. The original error disappears. Its descendants remain.

From hallucination to inheritance

We often describe (ai) hallucinations as isolated model errors. That framing is becoming inadequate.

A hallucination becomes much more dangerous when it becomes inherited knowledge. The first system says something false. The second system doesn't necessarily hallucinate the same thing — it may confidently repeat it. The third may add evidence that appears to support it. The fourth may generalise from it. The fifth may treat the conclusion as established background knowledge.

A falsehood can become more convincing every time it is transformed. Not because it becomes more true. Because it becomes more polished.

The laundering effect

Every transformation can make information look more authoritative. Raw generated text may look suspicious. A summary looks cleaner. A report looks serious. A chart looks scientific. A citation looks authoritative. A statistic looks objective.

But formatting is not provenance. Structure is not evidence. Citation is not necessarily verification. And repetition is not truth.

This creates what could be described as epistemic laundering. The original uncertainty is stripped away as information passes through systems. Each generation can remove the fingerprints of uncertainty. Eventually, what remains is a clean statement that appears to have always been known. The information has not become more trustworthy. It has simply become better dressed.

When Fake feeds Fake

The recursive system simply treats available information as input. A model does not automatically know that a webpage was generated yesterday. A retrieval system does not automatically know that an article was derived from another generated article. A downstream model does not necessarily know that a citation ultimately points back to itself.

The systems can therefore become participants in a knowledge ecosystem that increasingly contains synthetic ancestry. Synthetic information generates more synthetic information. The child inherits the uncertainty of the parent. Then produces grandchildren. And nobody keeps the family tree.

Synthetic ancestry

We have developed extraordinarily sophisticated ways to measure semantic similarity. We can determine that two pieces of text mean approximately the same thing. But semantic similarity does not tell us where either piece came from.

Two statements can be almost identical in meaning while one comes from a primary observation and the other comes from ten generations of machine-generated paraphrasing. A vector can tell us: These things are similar. It cannot, by itself, tell us: This one has accountable origin.

The future knowledge graph may contain enormous amounts of information that are semantically connected but epistemically disconnected from reality.

The recursive knowledge loop

At the beginning of the loop is the world. At the end of the loop, the machines may increasingly be consuming the outputs of the loop itself. Once the information environment becomes sufficiently recursive, the distance between the thing that happened and the thing the machine believes happened can grow without anyone noticing.

The world is burning

The consequences land in the real world. People make decisions. Businesses invest. Engineers design systems. Doctors assess information. Governments write policies. Children learn. Consumers choose. Markets respond.

If the knowledge feeding those decisions becomes recursively detached from accountable origin, errors don't stay inside the machine. They become physical. Financial. Social. Political. Environmental. Human.

The fire isn't caused by one hallucination. It is caused by the accumulation of untraceable information that looks sufficiently credible to act upon.

The death of the original source

The internet was built around documents. The emerging information environment is being built around derived representations. A page becomes a summary. A summary becomes an answer. An answer becomes a recommendation. A recommendation becomes another document.

Historically: source → interpretation. Increasingly: interpretation → interpretation → interpretation. The source becomes optional. And every interpretation is a lossy compression of what came before.

Knowledge without memory

We are creating systems with extraordinary computational memory while simultaneously creating an information environment with increasingly poor provenance memory.

We can store billions of documents. But can we preserve: Why does this claim exist? What was the original observation? Who created it? When? Under what conditions? What evidence supported it? Was it altered? What did a machine infer versus what did a human assert?

Without that information, storage is not enough. We may remember everything and still know almost nothing about why we should believe it.

This is why origin matters

A trustworthy knowledge system needs more than meaning. It needs identity. It needs origin. It needs lineage. It needs continuity. It needs authority. It needs policy. It needs evidence.

It needs to distinguish:

• observed — from primary human experience
• derived — inferred from observations
• inferred — logically concluded
• generated — produced by a machine
• verified — confirmed by accountable sources
• unknown — origin not established

Most information systems were never designed around this requirement. They were designed to move information: search, store, retrieve, rank, share, generate. The next generation must be designed to preserve epistemic identity as information moves.

The molecule of knowledge

This is where the concept of a molecule becomes important. Instead of treating knowledge as merely a piece of text, treat it as an identifiable object with a persistent origin.

The text can change. The presentation can change. The model can summarise it. A user can quote it. Another system can retrieve it. But the underlying identity remains traceable. The molecule carries its provenance with it. It can tell the next system:

• This is what I am.
• This is where I came from.
• This is what has happened to me.
• This is what has been derived from me.
• This is what is authoritative.
• This is what remains uncertain.

Now recursive systems have something fundamentally different to consume. Not merely content. Content with memory.

Break the recursive loop

The answer to recursive knowledge consumption is not to stop machines generating information. That isn't realistic. Nor would it necessarily be desirable.

The answer is to change what the machine consumes. Today, systems largely consume content. Tomorrow, trustworthy systems should consume content plus origin.

They should know the difference between: a primary observation, a verified source, a human assertion, a machine inference, a synthetic derivative, an unverified claim, and a claim that has no identifiable origin at all.

That makes recursion visible. It doesn't stop the loop. It gives the loop memory. And memory creates accountability.

From recursive generation to recursive verification

Why should recursive systems only generate? They could verify. A model encounters a claim. Instead of simply using it, it asks:

• What is the origin?
• Can the origin be resolved?
• Has the source changed?
• Is this claim supported?
• Is this derived from another generated claim?
• What is the epistemic status?
• Should this claim contribute to my answer?

The system isn't simply consuming information. It is evaluating the trust state of the information before consumption. That is a very different kind of intelligence.

The machine doesn't need to know everything

Trustworthy (ai) doesn't require perfect knowledge. It requires honest boundaries around knowledge.

A system should be able to say: This is known. This is supported. This is derived. This is plausible. This is uncertain. This is unverified. This has no reliable origin.

Those are not weaknesses. They are the foundations of a system that can participate safely in human knowledge. The real failure is not uncertainty. The real failure is uncertainty masquerading as knowledge.

A new information architecture

The internet gave us a global publishing system. Search gave us a global discovery system. Cloud computing gave us a global computation system. Generative (ai) is creating a global generation system.

What is missing is the infrastructure that allows these systems to understand the trust state of what they consume. That is the territory of Knowledge Trust Infrastructure. Not another content filter. Not another fact checker. Not another badge. Infrastructure.

A persistent mechanism for maintaining identity, origin and lineage as knowledge moves through increasingly recursive machine environments. Because the problem isn't only: Is this true? The problem is: Can we still establish why it should be trusted?

Fake is feeding Fake

The warning isn't: "AI will lie." Humans already lie. The warning is far more systemic:

We are building machines that can consume the outputs of other machines without reliably knowing where those outputs came from.

That creates recursive knowledge consumption. And recursive knowledge consumption creates the possibility of a world in which synthetic information increasingly becomes the evidence base for further synthetic information.

The system can remain coherent. The language can remain fluent. The answers can remain persuasive. And the connection to reality can quietly weaken.

Until one day we realise we have built an information environment that can explain almost anything, while becoming increasingly unable to answer the most important question:

Where did this come from?

That is why origin matters. That is why continuity matters. That is why provenance matters.

Because the future of trustworthy intelligence may depend on something remarkably simple: making sure the machine can remember the difference between what the world said — and what the machines said about what the world said.

Fake is feeding Fake.

Origin must feed Truth.