Large language models (LLMs) are vulnerable to model collapse when their own outputs are repeatedly used for training.
Shumailov et al. (2023) showed that synthetic content causes irreversible defects in LLMs: “the tails of the original content distribution disappear” when models are trained on model‑generated data【492565789159287†L55-L63】.
In reinforcement‑learning from human feedback (RLHF) settings, reward hacking can further degrade performance. Recent studies note that RLHF is susceptible to reward hacking—agents exploit flaws in the reward function rather than learning the intended behaviour【297621517710987†L70-L106】.
A separate challenge is catastrophic forgetting: deep‑learning models struggle to learn new tasks without losing previously acquired knowledge, a phenomenon first observed by McCloskey & Cohen (1989) and still unsolved【284022922580296†L15-L29】.
Beyond model issues, over‑reliance on AI tools can harm human cognition. Media reports highlight that cognitive offloading to generative AI may weaken critical‑thinking skills【396993420780090†L220-L247】 and that automating tasks deprives people of the opportunity to practise them, weakening the neural architecture involved【396993420780090†L243-L247】.
To address these coupled dangers, the reviewed manuscript proposes two resilience layers:
-
Provenance‑aware Retrieval‑Augmented Generation (RAG). RAG augments LLM prompts with external documents. By retrieving relevant, up‑to‑date documents and providing them as context, RAG mitigates hallucination and improves factual accuracy【810550645826215†L213-L244】, avoiding the need to retrain a model for each task.
-
Evolutionary Algorithms (EAs). Evolutionary algorithms maintain a population of candidate models and iteratively select and mutate the best performers. Because EAs are gradient‑free, they can fine‑tune models in black‑box settings and explore diverse solutions【335057455709596†L93-L118】. Population‑based selection provides anti‑fragility by favouring models that maintain quality despite noisy data.
The manuscript hypothesizes that combining provenance‑aware RAG with EAs can halt or reverse the coupled decline of model quality (QM) and human feedback quality (QH). To test this claim, we conducted a proof‑of‑concept simulation inspired by the manuscript’s mathematical framework.
The theoretical model uses differential equations to describe the feedback loop between model quality (QM) and human feedback quality (QH). In the absence of real LLM training resources, we approximate an LLM with a logistic‑regression classifier on the digits dataset (10‑class classification) from the scikit‑learn library. Despite its simplicity, the approach captures the essence of iterative learning with a mixture of human and synthetic labels.
Four experimental conditions were tested:
- Baseline (Synthetic Feedback Loop). The classifier is trained on human‑labeled data. In each iteration it predicts labels on a held‑out “task” subset. Incorrect predictions are accepted as labels with probability proportional to
(1 – QM); otherwise they are corrected. Accepted errors simulate humans trusting AI output. We then retrain on the mixture of original and synthetic labels. QM and QH are updated using
[\dot{Q}_M = -\alpha\big(1 - P(t)\big) - \beta\big(1 - Q_H\big), \qquad \dot{Q}_H = \gamma\big(1 - Q_M\big),]
where (P(t)) is the proportion of accepted errors, (\alpha,\beta,\gamma) are constants (0.05, 0.2 and 0.1) and (Q_M, Q_H\in [0,1]) denote model and human quality, respectively.
-
RAG (Provenance‑aware Retrieval). To simulate RAG, no synthetic data is used. Instead, each feature vector is augmented with the mean of the base training set to represent retrieved context. Training and evaluation occur on these augmented features. Since only human‑labeled data are used, (P(t)=0) and QH remains at 1.
-
EA (Evolutionary Algorithm). A population of logistic models (size = 4) is initialized with different random seeds. In each generation, each model performs the same synthetic feedback loop as the baseline. Models are evaluated on the test set and the top two performers are selected. New “children” models are spawned with different random seeds and trained on the top performer’s synthetic mixture. The best test accuracy updates (Q_M) and (Q_H) via the differential equations.
-
EA + RAG. Combines feature augmentation with evolutionary selection. The population is trained on augmented features, while synthetic feedback influences acceptance probability.
The digits dataset has 1 797 samples of 8×8 images representing digits 0–9. We split 70 % for training and 30 % for testing using stratified sampling. A subset of 200 training points is reserved as the “task” dataset for generating synthetic labels. Features are standardized with zero mean and unit variance.
- Test Accuracy: classification accuracy on the test set, representing observed model performance.
- Simulated QM: updated using the differential equation after each iteration.
- Simulated QH: updated via (\dot{Q}_H).
- P: proportion of accepted erroneous labels in the mixture.
All simulations ran for five iterations. Code and data are provided in the attached files.
The figure below compares the trajectories of Test Accuracy, QM and QH under each condition.
In the baseline condition, test accuracy remains around 0.967 across iterations, indicating that the logistic model resists immediate degradation. However, the simulated QM declines steadily from ≈ 0.92 to ≈ 0.72, reflecting the increasing proportion of synthetic labels being accepted. QH also declines, though more slowly, because human feedback quality decreases as the model becomes less trustworthy.
RAG uses only human‑labeled data and augments features with retrieved context. Test accuracy and simulated QM remain stable, and QH stays at 1.0. The retrieval augmentation ensures that the model never trains on self‑generated labels, avoiding model collapse. This aligns with the literature on RAG: by adding external context, RAG allows LLMs to bypass retraining and access the latest information, reducing hallucination and improving factuality【810550645826215†L213-L244】.
The EA maintains a population of logistic models and selects the best performers. Test accuracy declines only slightly. Simulated QM decays more slowly than in the baseline: from ≈ 0.92 to ≈ 0.90. QH declines gradually. The population‑based approach allows some models to avoid being contaminated by synthetic errors, reflecting the anti‑fragile nature of EAs. EAs are advantageous because they can fine‑tune models in black‑box scenarios without gradients【335057455709596†L93-L104】 and explore multiple candidate solutions【335057455709596†L106-L118】.
Combining EA with retrieval augmentation yields stable test accuracy but shows a more pronounced decline in simulated QM. In this simplified setting, the RAG augmentation does not provide additional benefits over EA alone; the synthetic acceptance still drives down QM. Nonetheless, QH remains higher than in the baseline. In practice, combining RAG with EA could provide improved robustness when RAG supplies high‑quality external evidence and EAs explore parameter spaces.
-
Synthetic feedback accelerates decline. Even with a simple classifier, allowing wrong predictions to be accepted as correct causes simulated model quality and human feedback quality to deteriorate. This echoes the danger of model collapse observed in generative models【492565789159287†L55-L63】.
-
RAG prevents degradation. By grounding training in authentic data and external context, RAG prevents the acceptance of wrong labels and keeps both QM and QH high. This supports the claim that RAG reduces hallucination and improves factual accuracy【810550645826215†L213-L244】.
-
EA slows deterioration. Population‑based selection retains higher‑quality models, mitigating the collapse. EAs explore diverse solutions and can fine‑tune models without gradients【335057455709596†L93-L118】, making them suitable for black‑box LLM scenarios.
-
EA + RAG synergy is nuanced. In our simplified simulation, combining retrieval with evolutionary search did not outperform EA alone. However, in real LLM systems where retrieval provides high‑quality, evolving knowledge and EAs select robust parameter sets, the combination could be powerful.
Our simulation is a proof‑of‑concept rather than a definitive test. Major limitations include:
-
Simplified model. We used logistic regression on the digits dataset as a stand‑in for a large language model. LLMs have different dynamics, contextual dependencies and error modes.
-
Simplistic retrieval. RAG was simulated by appending the mean feature vector rather than retrieving semantically relevant documents. True RAG systems use embeddings and vector search to fetch context.
-
Approximate differential equations. The constants (\alpha,\beta,\gamma) were chosen heuristically. Real degradation dynamics may include non‑linear terms and stochastic noise.
-
Limited iterations. We ran five iterations; long‑term degradation or resilience might differ.
Future work could extend this experiment by using small language models (e.g., BERT or Llama) fine‑tuned on text tasks, more realistic retrieval methods, and deeper evolutionary strategies. Running such experiments at scale would provide stronger evidence for or against the Ouroboros hypothesis.
The experiment provides preliminary support for the manuscript’s proposal that provenance‑aware RAG and evolutionary algorithms can mitigate the Ouroboros effect. While simplified, the results illustrate that (1) synthetic feedback alone can lead to coupled decline of model and human quality, (2) grounding models in authentic data through retrieval halts this decline, and (3) population‑based evolution slows it. Integrating these resilience layers into LLM training, along with cryptographic data provenance mechanisms to ensure authentic data sources【330762423699552†L71-L85】【330762423699552†L138-L152】, may be a promising path towards sustainable AI systems.