1. IDSL
  2. Research
  3. Maintenance-free Controllers

Deep Learning for Process Control

Traditional controllers used in the process industries, such as PID loops or MPC, require constant attention and upkeep for its entire lifecycle including modeling, design, tuning and maintenance. These controllers are typically designed for a narrow operating range by assuming linear process behavior, and are not resilient to changes in plant equipment or operating conditions. In complex industrial systems, there often exists a trade-off between high performance controllers and the development of complex models that are computationally intensive and difficult to interpret or maintain.

Photo of industrial plant and piping

Inspired by the recent successes of deep learning in computer vision and natural language processing, our group is exploring deep reinforcement learning (DRL) as a model-free and maintenance-free framework for process control in industrial settings. Recent work that we’ve published shows promising results for DRL in terms of setpoint tracking performance and adaptability, but there are still many fundamental questions left to explore, including sample efficiency (big data is not always good data), stability guarantees, interpretability and computational challenges.

Ultimately, we are interested in the development of smart plants and advanced controllers that can provide a high level of safety and reliability for the industry.

Reinforcement Learning based Design of Linear Fixed Structure Controllers
Reinforcement Learning based Design of Linear Fixed Structure Controllers - Lawrence et al. (2020): A standard closed-loop structure is shown inside the dashed box. Arrows passing the dashed line indicate the passing of some time-horizon [0, T]. Outside the dashed box, we store cumulative rewards based on slightly perturbed policies, which are used to update the policy with a finite-difference scheme described in section 4.2 in the manuscript.
Optimal PID and Antiwindup Control Design as a Reinforcement Learning Problem
Optimal PID and Antiwindup Control Design as a Reinforcement Learning Problem - Lawrence et al. (2020): The actor (PID controller) on the left is simply linear combination of the state and the PID & antiwindup parameters followed by a nonlinear saturation function. The critic on the right is a deep neural network approximation of the Q-function whose inputs are the state-action pair generated by the actor.

Selected Publications

See below for a selection of our papers related to deep (reinforcement) learning.


Highlights - 09 February 2022
The Banff International Research Station will host the *Climate Change Scenarios and Financial Risk* Online workshop at the UBC Okanagan campus in Kelowna, BC, from May 1 to May 6, 2022
LINK

Recruitment

Our group is recruiting year-round for postdocs, MASc and PhD students, visiting students and undergraduate students.

All admitted students will receive a stipend.

Please see our opportunities page for more information.