Mushroom hunting with LLMs: what can go wrong?

Researchers evaluated whether large language models like ChatGPT can safely identify mushrooms from photos, testing them on a dataset of 55 Polish mushroom species that include both edible varieties and deadly poisonous ones. The study found significant safety concerns, as LLMs made unreliable identifications despite producing confident-sounding descriptions, raising questions about trusting AI for health-critical tasks like foraging. The research highlights the dangers of using unverified AI systems for real-world decisions where mistakes could be fatal.