Generative AI's Potential to Fabricate Biological Discoveries Raises Concerns
Generative AI holds promise for aiding scientists in identifying new drugs, but it also poses risks of misleading researchers by fabricating biological effects that do not exist. This technology, known for creating text and images, is being explored for its potential in designing proteins, simulating cells, and generating synthetic biological data. However, the danger lies in the possibility that AI could produce plausible molecular patterns or inferences that do not accurately reflect biological realities. Such errors could lead to significant consequences in drug development and scientific understanding.
Computational biologist Thomas Burger from Grenoble Alpes University addresses these concerns in an opinion piece published in Patterns. He outlines ten potential applications of generative AI in biological research, noting that while AI can help interpret vast datasets from omics experiments, it can also introduce subtle changes that may go undetected. The level of risk associated with AI outputs varies, particularly between ideas that are tested in real experiments and synthetic data used as evidence.
Screening potential drugs using AI is considered a lower-risk application, as models can quickly assess numerous candidates and narrow them down for laboratory testing. However, if AI mistakenly discards a viable candidate or directs researchers toward an ineffective treatment, the consequences can be costly in terms of time and resources. The real danger arises when AI-generated data starts to replace actual experimental measurements. If AI fabricates features that were never present, scientists may mistakenly believe they have discovered a biological effect that does not exist, undermining the integrity of scientific evidence.
Burger emphasizes that the primary issue is not merely comparing a hallucinated finding with a genuine discovery but rather the risk of genuine data being corrupted by AI-induced hallucinations during complex computational workflows. This distortion could lead researchers to incorrect conclusions without obvious signs of fabrication.
A relevant example is AlphaFold 3, which has been reported to generate 'hallucinated structures' in disordered protein regions, although low confidence scores can indicate potential problems. AI errors may not only create false positives but can also obscure real biological effects, causing researchers to overlook genuine evidence of a treatment's efficacy.
Burger reflects on the possibility that an AI hallucination could inadvertently lead to a genuine discovery, comparing it to serendipitous findings from laboratory errors. He argues that the moral implications and validation processes should be consistent, regardless of the source of the discovery. Ultimately, the significance lies in how researchers utilize AI outputs; if treated as mere hypotheses, hallucinations may not lead to serious issues, but if regarded as valid observations, they could mislead scientific understanding.
The article underscores the need for rigorous verification of AI-generated results in biological research, emphasizing that even the most exciting findings proposed by AI require independent validation through real experiments before being accepted as discoveries.




