Deep Reinforcement Learning for Next-gen Cruise Control

Cracking the code of intelligent driving: Praveen Prasanth discusses deep reinforcement learning applications in multi-objective ACC.
Reinforcement

The Machine Learning Developers Summit (MLDS) 2024 brought together visionaries and experts from various corners of the AI landscape, and one such luminary was Praveen Prasanth KV, the Deputy Manager of AI/ML at Renault Nissan Tech. Praveen, with his extensive experience and passion for advancing traditional simulation frameworks, delivered a talk that delved into the intricate world of designing reward functions for Multi-objective Adaptive Cruise Control (ACC) systems.

Multi-objective Adaptive Cruise Control

Praveen initiated his talk by providing a comprehensive overview of Multi-objective Adaptive Cruise Control, a transformative technology in the automotive industry. ACC empowers vehicles to autonomously adjust speed, considering safety, efficiency, and comfort. Praveen underscored the challenges of developing systems that cater to these diverse objectives simultaneously and set the stage for the critical role of designing a reward function in achieving the delicate balance required for ACC systems.

Unveiling the Design Process: From Theory to Practice

The heart of Praveen’s presentation lay in decoding the essence of reward function design. With a backdrop of traditional cruise control systems, which primarily focus on maintaining constant speed, ACC systems must contend with additional layers of complexity. Praveen emphasized the iterative process involved in designing a reward function, highlighting the need for a nuanced understanding of vehicle dynamics, traffic scenarios, and user preferences. Drawing upon his expertise in internal combustion engine (ICE), hybrid, and electric powertrains, Praveen showcased the adaptability required in the design process.

Real-world Applications and Case Studies

To augment the theoretical aspects, Praveen Prasanth delved into real-world case studies, providing tangible examples of how reward function design translates into practical applications. These case studies, drawn from his work at Renault Nissan Tech, covered scenarios ranging from navigating urban traffic to adapting to diverse driving conditions and seamlessly transitioning between powertrain modes. The audience gained valuable insights into the practical implementation of AI/ML techniques in the automotive sector, specifically focused on enhancing the driving experience.

The Road to Multi-objective ACC Excellence

Praveen highlighted the challenges inherent in Multi-objective ACC, particularly the computational demands when multiple objectives are introduced. Traditional approaches like Model Predictive Control (MPC) faced limitations in handling the complexity of multi-objective scenarios, leading Praveen to advocate for a paradigm shift. Data-driven models, while collecting real driving scenarios, were deemed inefficient due to the limitations of driver expertise and model performance.

Enter Deep Reinforcement Learning: A Paradigm Shift

Praveen introduced Deep Reinforcement Learning (DRL) as a powerful candidate to tackle the challenges posed by multi-objective ACC. DRL, a form of computational learning, allows the agent to interact with the environment through episodes of trial and error, simulating real-world driving conditions. The emphasis on live interaction and learning, without relying on a predefined dataset, distinguishes DRL from traditional methods.

Cracking the Code with DDPG Algorithm

Delving into the learning algorithm, Praveen introduced the Deep Deterministic Policy Gradient (DDPG) algorithm. As a complex yet effective approach, DDPG incorporates two neural networks—actor and critic—to optimize the policy based on the rewards received. The reinforcement learning agent, in this context, continuously observes the environment, acts based on the current state, and learns through episodes of simulation.

Validation and Real-world Performance

The validation phase became a crucial component of Praveen’s talk, demonstrating the real-world performance of the developed ACC system. He showcased scenarios such as follower mode, speed-limited scenarios, and braking scenarios, illustrating how the reinforcement learning agent adapted to varying conditions while ensuring safety, tracking accuracy, and comfort. The results presented a promising outlook for the application of DRL in ACC, with impressive performance metrics even in uncertain and risky highway driving scenarios.

Conclusion

Praveen concluded his talk by outlining the future directions of their work. The integration of multi-agent systems for both longitudinal and lateral control, along with additional objectives such as energy efficiency and diverse driving scenarios, showcased the commitment to evolving autonomy skills. The ultimate goal, as Praveen articulated, is to develop agents that are not just intelligent but also liberating, allowing computers to efficiently handle driving tasks, thereby providing individuals with more time and freedom.

Praveen Prasanth’s presentation at MLDS 2024 emerged as a beacon in the pursuit of intelligent and adaptive cruise control systems, laying the groundwork for a future where AI seamlessly integrates into the driving experience, ensuring safety, efficiency, and comfort on the roads.

Picture of Shreepradha Hegde

Shreepradha Hegde

Shreepradha is an accomplished Associate Lead Consultant at AIM, showcasing expertise in AI and data science, specifically Generative AI. With a wealth of experience, she has consistently demonstrated exceptional skills in leveraging advanced technologies to drive innovation and insightful solutions. Shreepradha's dedication and strategic mindset have made her a valuable asset in the ever-evolving landscape of artificial intelligence and data science.

The Chartered Data Scientist Designation

Achieve the highest distinction in the data science profession.

Elevate Your Team's AI Skills with our Proven Training Programs

Strengthen Critical AI Skills with Trusted Generative AI Training by Association of Data Scientists.

Our Accreditations

Get global recognition for AI skills

Chartered Data Scientist (CDS™)

The highest distinction in the data science profession. Not just earn a charter, but use it as a designation.

Certified Data Scientist - Associate Level

Global recognition of data science skills at the beginner level.

Certified Generative AI Engineer

An upskilling-linked certification initiative designed to recognize talent in generative AI and large language models

Join thousands of members and receive all benefits.

Become Our Member

We offer both Individual & Institutional Membership.

Subscribe to our Newsletter