Allergen Detection w/ GPT-4 & Zero-Shot Learning

This research explores how GPT-4 and zero-shot/few-shot machine learning can transform food allergen detection across unstructured online data sources—ranging from restaurant menus and e-commerce listings to YouTube recipe transcripts. By integrating prompt-engineered, multimodal AI into AllergenAlert’s platform, the system can detect the top 8 allergens with high precision and recall, even when allergens are implied or ambiguously stated. This work represents a major step toward automating allergy safety at scale, with real-world applications in consumer health, digital health tools, and AI ethics. 

This summer research initiative explores how cutting-edge AI tools—specifically computer vision, OCR (Optical Character Recognition), and GPT-4—can detect hidden food allergens from real-world visual inputs such as grocery packaging, restaurant menus, and recipe videos. We benchmark top OCR engines, fine-tune object detection models, and integrate GPT-4’s multimodal capabilities to build a pipeline that can identify allergens with high precision and recall. Designed to power AllergenAlert’s allergy detection platform, this project advances food safety, accessibility, and AI for public health—bringing us one step closer to a world where no one has to fear what’s on their plate.