The Machine Learning Developers Summit (MLDS) 2024 brought together visionaries and experts from various corners of the AI landscape, and one such luminary was Praveen Prasanth KV, the Deputy Manager of AI/ML at Renault Nissan Tech. Praveen, with his extensive experience and passion for advancing traditional simulation frameworks, delivered a talk that delved into the intricate world of designing reward functions for Multi-objective Adaptive Cruise Control (ACC) systems.
Praveen initiated his talk by providing a comprehensive overview of Multi-objective Adaptive Cruise Control, a transformative technology in the automotive industry. ACC empowers vehicles to autonomously adjust speed, considering safety, efficiency, and comfort. Praveen underscored the challenges of developing systems that cater to these diverse objectives simultaneously and set the stage for the critical role of designing a reward function in achieving the delicate balance required for ACC systems.
The heart of Praveen’s presentation lay in decoding the essence of reward function design. With a backdrop of traditional cruise control systems, which primarily focus on maintaining constant speed, ACC systems must contend with additional layers of complexity. Praveen emphasized the iterative process involved in designing a reward function, highlighting the need for a nuanced understanding of vehicle dynamics, traffic scenarios, and user preferences. Drawing upon his expertise in internal combustion engine (ICE), hybrid, and electric powertrains, Praveen showcased the adaptability required in the design process.
To augment the theoretical aspects, Praveen Prasanth delved into real-world case studies, providing tangible examples of how reward function design translates into practical applications. These case studies, drawn from his work at Renault Nissan Tech, covered scenarios ranging from navigating urban traffic to adapting to diverse driving conditions and seamlessly transitioning between powertrain modes. The audience gained valuable insights into the practical implementation of AI/ML techniques in the automotive sector, specifically focused on enhancing the driving experience.
Praveen highlighted the challenges inherent in Multi-objective ACC, particularly the computational demands when multiple objectives are introduced. Traditional approaches like Model Predictive Control (MPC) faced limitations in handling the complexity of multi-objective scenarios, leading Praveen to advocate for a paradigm shift. Data-driven models, while collecting real driving scenarios, were deemed inefficient due to the limitations of driver expertise and model performance.
Praveen introduced Deep Reinforcement Learning (DRL) as a powerful candidate to tackle the challenges posed by multi-objective ACC. DRL, a form of computational learning, allows the agent to interact with the environment through episodes of trial and error, simulating real-world driving conditions. The emphasis on live interaction and learning, without relying on a predefined dataset, distinguishes DRL from traditional methods.
Delving into the learning algorithm, Praveen introduced the Deep Deterministic Policy Gradient (DDPG) algorithm. As a complex yet effective approach, DDPG incorporates two neural networks—actor and critic—to optimize the policy based on the rewards received. The reinforcement learning agent, in this context, continuously observes the environment, acts based on the current state, and learns through episodes of simulation.
The validation phase became a crucial component of Praveen’s talk, demonstrating the real-world performance of the developed ACC system. He showcased scenarios such as follower mode, speed-limited scenarios, and braking scenarios, illustrating how the reinforcement learning agent adapted to varying conditions while ensuring safety, tracking accuracy, and comfort. The results presented a promising outlook for the application of DRL in ACC, with impressive performance metrics even in uncertain and risky highway driving scenarios.
Praveen concluded his talk by outlining the future directions of their work. The integration of multi-agent systems for both longitudinal and lateral control, along with additional objectives such as energy efficiency and diverse driving scenarios, showcased the commitment to evolving autonomy skills. The ultimate goal, as Praveen articulated, is to develop agents that are not just intelligent but also liberating, allowing computers to efficiently handle driving tasks, thereby providing individuals with more time and freedom.
Praveen Prasanth’s presentation at MLDS 2024 emerged as a beacon in the pursuit of intelligent and adaptive cruise control systems, laying the groundwork for a future where AI seamlessly integrates into the driving experience, ensuring safety, efficiency, and comfort on the roads.
