Mitigating Hallucinations in Foundation Language Models: A Structured Approach for Hallucination- Free Query Responses in Regulatory Domains

Abstract(s): Sriram Gudimella

Foundation language models like GPT has sparked public interest in large language models (LLMs) and have paved the way to experiment, experience and democratize the benefits of AI. However, the LLM models including GPT models currently face several challenges. The most prominent among them is Hallucinations. Hallucinations are assertions or claims or information that sound plausible and authoritative but are verifiably incorrect.

In this paper, we detail our strategy and solution approach in controlling foundation models like GPT to not hallucinate while responding. We followed a novel structured way by regulating the GPT models to formulate the query response leveraging information only from the underlying knowledge base of documents that were in the formats – PDF, HTML and Word Doc. This paper details our approach implementing a production grade Query Bot application for a customer in a regulatory domain that requires formulating query responses that are Hallucination free. We have used the Retriever Augmented Generation technique coupled with a unique prompt engineering technique and post processing approach to accomplish it.

We have also defined a scoring mechanism to determine the answer score when multiple passages are sent as context to GPT model as the GPT model does not provide the score for the answer generated. The solution that we built has humongous business impact by reducing the turnaround time by 150% and enabling Query Status Tracking, and Improved User Experience.

Access The Research Paper:

Picture of Vaibhav Kumar

Vaibhav Kumar

The Chartered Data Scientist Designation

Achieve the highest distinction in the data science profession.

Elevate Your Team's AI Skills with our Proven Training Programs

Strengthen Critical AI Skills with Trusted Generative AI Training by Association of Data Scientists.

Our Accreditations

Get global recognition for AI skills

Chartered Data Scientist (CDS™)

The highest distinction in the data science profession. Not just earn a charter, but use it as a designation.

Certified Data Scientist - Associate Level

Global recognition of data science skills at the beginner level.

Certified Generative AI Engineer

An upskilling-linked certification initiative designed to recognize talent in generative AI and large language models

Join thousands of members and receive all benefits.

Become Our Member

We offer both Individual & Institutional Membership.