Minseok Bae

This paper presents two complementary efforts at Vectara to measure and benchmark LLM faithfulness in Retrieval-Augmented Generation (RAG). It describes an original hallucination leaderboard and introduces FaithJudge, an LLM-as-a-judge framework, to improve automated hallucination evaluation and support the development of trustworthy generative AI systems.

This blog post introduces Vectara's Factual Consistency Score, a calibrated score designed to automatically evaluate hallucinations in Generative AI. It aims to establish a new standard for automated detection, offering enhanced performance, cost-effectiveness, and low latency for building trust in Gen AI technologies.

This article discusses the benchmarking of Anthropic's Claude 3 suite of AI models using the Hughes Hallucination Evaluation Model (HHEM) to assess factual inconsistency in summaries, comparing their performance against other foundational models.