Top AIs invent same fake PyPl and npm package names
Churilov set out his findings in a research paper, The Range Shrinks, the Threat Remains: Re-evaluating LLM Package Hallucinations on the 2026 Frontier-Model Cohort, which is yet to be peer-reviewed. He found 127 hallucinated package names were shared across Claude Sonnet 4.6, Claude Haiku 4.5, GPT-5.4-mini, Gemini 2.5 Pro, and DeepSeek V3.2.
As of April this year, 53 of those names — 41 on the PyPI software repository and 12 on npm —are still available for registration.
According to the study, there are two reasons for this amount of conformity in the output of the models. First, models may learn the same incorrect package references from shared public training material, such as tutorials and documentation.
