Transforming Decision-Making: Unveiling the Future of AI in Enterprises

Bengaluru witnessed a captivating session at the Machine Learning Developers Summit (MLDS) 2024 as Vijay Morampudi, Head of AI at Veltris, took the stage to unravel insights into the development of a Large Language Model (LLM)-based solution. Over the next 20 minutes, Morampudi delved into the intricacies of augmenting investment decision-making processes within a prominent platform. The core of his talk centered around the quest to enhance the quality of decisions made by investment committees through the integration of LLMs.

Enhancing Investment Decision Making

Morampudi commenced his talk by outlining the fundamental business challenge – improving investment decision-making within a platform. Every platform, he explained, faces the daunting task of identifying target companies, conducting due diligence, and presenting investment memos to committees. The pivotal goal is to elevate the quality of these investment decisions. To achieve this, Morampudi and his team developed an LLM-based solution, focusing particularly on refining the retrieval performance of the Q chart bot.

Unraveling the Investment Memo

The linchpin of investment decision-making is the investment memo. Morampudi elucidated its critical role in identifying growth opportunities, understanding the target company’s business model, assessing financial metrics, and gauging the competitive landscape and associated risks. However, this information is often presented in a document containing a mix of text, tables, and figures. Morampudi’s challenge was to extract and represent this data efficiently for LLMs to generate meaningful responses.

Tackling Noise and Refining Information

Morampudi revealed his strategy for data extraction, utilizing document intelligence to separate free-form text from tables. Notably, he highlighted the challenge of noise in the extracted text from tables and detailed the process of detection and removal. Tables, text, and images were processed separately, each requiring a tailored approach. For text, embeddings were created and stored in an embedding database, ensuring relevance and impact.

The Role of Metadata and Embeddings

Recognizing the non-sequential nature of information in documents, Morampudi detailed the utilization of metadata tags. The team applied this concept to tables, text, and images, ensuring a holistic approach to information retrieval. Through chunking and embedding techniques, they achieved not only efficiency but also role-based access for users.

Augmentation and Streamlining Responses

Morampudi emphasized understanding user stories, augmenting queries with acronyms, and decomposing queries for LLMs. The introduction of metadata tags facilitated effective filtering, streamlining the information retrieval process. With a focus on recall rate improvement, Morampudi explained the application of various models, including Tapas for tables and Vision for images.

Postprocessing and User Confidence

In scenarios of conflicting information, postprocessing techniques were applied to maintain accuracy. The system also introduced confidence scores, combining similarity and re-ranking scores to provide users with a measure of trust in the system. Morampudi stressed the importance of user feedback, with authorized users approving documents for entry into the vector store and contributing to continuous system improvement.

Conclusion

The comprehensive approach to user experience improvement included autocomplete features, toggles for internal and external data queries, and session context for enhanced understanding. Citations were introduced to bolster trust in the system’s responses. Morampudi concluded by presenting the system’s performance, achieving an 87% overall accuracy in a dataset of 200 queries.

As Veltris gears up to deliver the next version, incorporating information from multiple sources, Morampudi’s talk at MLDS 2024 serves as a beacon of innovation in the realm of AI-powered investment decision-making. His meticulous strategies and holistic approach showcase the potential for LLMs to revolutionize complex decision-making processes within enterprises.