ADaSci Banner 2024

Visualization techniques for training of Deep Reinforcement Learning (DRL) agents for real-life continuous state and action spaces

Author(s): Gaurav Adke, Ameya Divekar, Guillaume Ramelet

Abstract

Visualization of reinforcement learning (RL) environment and learning dynamics of an agent is a vital step for debugging and better understanding of the learnt policy. For virtual game environments, it is possible to visualize agent’s performance by rendering game screens. But for environments with optimisation of real world multidimensional spaces with continuous variables, such as optimisation of chemical process parameters, it is challenging and complex to observe agent’s behaviour with visualization. This field largely remains unexplored in the research community. In the current work, reinforcement learning agent is developed for optimisation of production process of rubber mix for tyre industry. This paper presents an attempt to visualize an RL agent’s training and inference for high dimensional state space problems with continuous state and action spaces. A number of techniques are presented here to assist for debugging and monitoring the convergence of an agent over complex domain. We explore plots for studying simulation environment, RL training dynamics, analysing trained policy and performance evaluation of trained policy in a given environment. Techniques described here are developed for actor-critic algorithms but can easily be extended to any RL algorithm.

Picture of Association of Data Scientists

Association of Data Scientists

The Chartered Data Scientist Designation

Achieve the highest distinction in the data science profession.

Elevate Your Team's AI Skills with our Proven Training Programs

Strengthen Critical AI Skills with Trusted Generative AI Training by Association of Data Scientists.

Our Accreditations

Get global recognition for AI skills

Chartered Data Scientist (CDS™)

The highest distinction in the data science profession. Not just earn a charter, but use it as a designation.

Certified Data Scientist - Associate Level

Global recognition of data science skills at the beginner level.

Certified Generative AI Engineer

An upskilling-linked certification initiative designed to recognize talent in generative AI and large language models

Join thousands of members and receive all benefits.

Become Our Member

We offer both Individual & Institutional Membership.