GenAI’s Role in Personalizing Edge Experiences

Discover how GenAI reshapes daily life, empowering edge devices with efficiency, personalization, and transformative innovation.
Gen AI edge

Madhusudanan Kandasamy, the Head of Machine Learning at Qualcomm India, recently delivered an insightful talk on the opportunities and challenges of deploying Generative AI (GenAI) on edge devices at the Machine Learning Developers Summit (MLDS) 2024. With over two decades of experience in the industry, Kandasamy leads the Artificial Intelligence Software Development Group at Qualcomm, focusing on optimizing deep learning inference on Snapdragon chipsets for various applications.

Overview of GenAI

In his talk, Kandasamy highlighted the transformative shift brought about by GenAI compared to traditional artificial intelligence (AI). While large organizations predominantly utilized traditional AI to enhance services, GenAI is distinguished by its direct interaction with end-users, allowing common individuals to leverage AI in their daily lives. This shift is crucial as GenAI is designed to be operated by consumers on edge devices like smartphones, augmented reality headsets, smart glasses, cars, and IoT devices.

Key Technological Challenges

Kandasamy discussed the technological challenges associated with deploying GenAI on edge devices. One of the significant challenges is the size of the models, with some reaching up to 150 billion parameters. These models are both memory and compute-intensive, making them challenging to run on conventional cloud servers due to scalability issues. He emphasized the need to make GenAI accessible on edge devices to meet the demand of billions of users.

Opportunities and Use Cases

The talk delved into the opportunities and potential use cases for GenAI on the edge. Kandasamy provided examples of applications already in progress at Qualcomm, such as voice recognition, text-to-voice, large language model (LLM)-based use cases, and image processing tasks like inpainting and outpainting. He stressed the importance of bringing GenAI to edge devices to enable users to perform these tasks seamlessly without relying on cloud services.

Technological Solutions

To address the challenges, Kandasamy discussed various technological solutions, including quantization, which reduces the memory footprint of models without sacrificing accuracy. He also introduced the concept of hybrid AI, combining rule-based models with smaller deep-learning models for efficient edge computing. Additionally, he touched upon the importance of AI accelerators, such as Qualcomm’s Hexagon processor, designed specifically for AI applications, ensuring faster and more energy-efficient inference.

Conclusion

In conclusion, Kandasamy highlighted the benefits of on-device AI, such as cost reduction, improved performance, enhanced privacy, and increased personalization. He emphasized Qualcomm’s commitment to making GenAI accessible on a range of edge devices, ensuring that the technology remains affordable and efficient. The talk provided valuable insights into the evolving landscape of AI deployment and the exciting possibilities that GenAI brings to the edge.

Picture of Shreepradha Hegde

Shreepradha Hegde

Shreepradha is an accomplished Associate Lead Consultant at AIM, showcasing expertise in AI and data science, specifically Generative AI. With a wealth of experience, she has consistently demonstrated exceptional skills in leveraging advanced technologies to drive innovation and insightful solutions. Shreepradha's dedication and strategic mindset have made her a valuable asset in the ever-evolving landscape of artificial intelligence and data science.

The Chartered Data Scientist Designation

Achieve the highest distinction in the data science profession.

Elevate Your Team's AI Skills with our Proven Training Programs

Strengthen Critical AI Skills with Trusted Generative AI Training by Association of Data Scientists.

Our Accreditations

Get global recognition for AI skills

Chartered Data Scientist (CDS™)

The highest distinction in the data science profession. Not just earn a charter, but use it as a designation.

Certified Data Scientist - Associate Level

Global recognition of data science skills at the beginner level.

Certified Generative AI Engineer

An upskilling-linked certification initiative designed to recognize talent in generative AI and large language models

Join thousands of members and receive all benefits.

Become Our Member

We offer both Individual & Institutional Membership.