Madhusudanan Kandasamy, the Head of Machine Learning at Qualcomm India, recently delivered an insightful talk on the opportunities and challenges of deploying Generative AI (GenAI) on edge devices at the Machine Learning Developers Summit (MLDS) 2024. With over two decades of experience in the industry, Kandasamy leads the Artificial Intelligence Software Development Group at Qualcomm, focusing on optimizing deep learning inference on Snapdragon chipsets for various applications.
Overview of GenAI
In his talk, Kandasamy highlighted the transformative shift brought about by GenAI compared to traditional artificial intelligence (AI). While large organizations predominantly utilized traditional AI to enhance services, GenAI is distinguished by its direct interaction with end-users, allowing common individuals to leverage AI in their daily lives. This shift is crucial as GenAI is designed to be operated by consumers on edge devices like smartphones, augmented reality headsets, smart glasses, cars, and IoT devices.
Key Technological Challenges
Kandasamy discussed the technological challenges associated with deploying GenAI on edge devices. One of the significant challenges is the size of the models, with some reaching up to 150 billion parameters. These models are both memory and compute-intensive, making them challenging to run on conventional cloud servers due to scalability issues. He emphasized the need to make GenAI accessible on edge devices to meet the demand of billions of users.
Opportunities and Use Cases
The talk delved into the opportunities and potential use cases for GenAI on the edge. Kandasamy provided examples of applications already in progress at Qualcomm, such as voice recognition, text-to-voice, large language model (LLM)-based use cases, and image processing tasks like inpainting and outpainting. He stressed the importance of bringing GenAI to edge devices to enable users to perform these tasks seamlessly without relying on cloud services.
To address the challenges, Kandasamy discussed various technological solutions, including quantization, which reduces the memory footprint of models without sacrificing accuracy. He also introduced the concept of hybrid AI, combining rule-based models with smaller deep-learning models for efficient edge computing. Additionally, he touched upon the importance of AI accelerators, such as Qualcomm’s Hexagon processor, designed specifically for AI applications, ensuring faster and more energy-efficient inference.
In conclusion, Kandasamy highlighted the benefits of on-device AI, such as cost reduction, improved performance, enhanced privacy, and increased personalization. He emphasized Qualcomm’s commitment to making GenAI accessible on a range of edge devices, ensuring that the technology remains affordable and efficient. The talk provided valuable insights into the evolving landscape of AI deployment and the exciting possibilities that GenAI brings to the edge.