Safe Reinforcement Learning for Start-up and Operation of Chemical Processes
| Subject area | Chemical and Thermal Process Engineering Automation, Control Systems, Robotics, Mechatronics, Cyber Physical Systems |
| Term | since 2021 |
| Funding | Deutsche Forschungsgemeinschaft (DFG) |
This project is part of the SPP 2331: Maschinelles Lernen in der Verfahrenstechnik. Wissen trifft auf Daten: Interpretierbarkeit, Extrapolation, Verlässlichkeit, Vertrauen
Project description
Despite of advances in computational methods, autonomous operation of complex chemical processes remains a very challenging problem. It is still not possible to robustly operate the plant in all possible situations using pure model-based optimization strategies, e.g., under heavy disturbances or product switches. Complex operations are performed by following recipes and manual adaptations by operators. This leads to suboptimal plant operation, but also severely limits flexibility of many industrial processes, as start-up and shut-down procedures need to be avoided due to the lack of systematic strategies to deal with them.The increasing availability of large amounts of data opens the door for operation strategies based thereon, such as reinforcement learning (RL). However, some critical challenges of RL, as the rigorous consideration of process constraints or the large amount of data required, prevent RL from being applied on chemical processes.To mitigate these drawbacks of RL, synergies between three different disciplines are explored: expert / domain knowledge in the form of operational recipes, model-based control, and reinforcement learning.The main goal of this project is to develop a systematic approach to the real implementation and investigations under real conditions of RL in complex chemical processes in order to optimize dynamic operation by safeguarding process constraints, resulting in a methodology for the applicationof RL with nonlinear model predictive control (NMPC) for cases where many practical experiments are (1) expensive, (2) might frequently fail, and (3) plant-model mismatch is significant.This project employs a wide variety of methods from chemical engineering by using both detailed dynamic simulation models and experimental / historical data from real-life experiments performed on an already existing batch distillation column at TUB. The optimization of its operation cycle is representative of typical complex applications in chemical engineering.From the field of machine learning we introduce the idea of the parameter manifold on virtual experiments to obtain inclusive descriptions for cases where the representation of the real-life experiment is uncertain. Efficient sampling strategies for RL are provided, and highly accurate surrogate models based on large amounts of data are developed. We will also develop a new safe RL method that can be continuously improved but is safe because it uses an NMPC as policy approximator.The proposed workflow is constructed in a generic fashion, finding application in single unit operations as well as in processes with multiple units, tackling especially issues concerning plant-model mismatch. Both fields, CE and ML, can together develop the necessary methods for achieving a robust optimal control.
Researchers
Search & People Search
Location & approach
The campus of TU Dortmund University is located close to interstate junction Dortmund West, where the Sauerlandlinie A 45 (Frankfurt-Dortmund) crosses the Ruhrschnellweg B 1 / A 40. The best interstate exit to take from A 45 is “Dortmund-Eichlinghofen” (closer to South Campus), and from B 1 / A 40 “Dortmund-Dorstfeld” (closer to North Campus). Signs for the university are located at both exits. Also, there is a new exit before you pass over the B 1-bridge leading into Dortmund.
To get from North Campus to South Campus by car, there is the connection via Vogelpothsweg/Baroper Straße. We recommend you leave your car on one of the parking lots at North Campus and use the H-Bahn (suspended monorail system), which conveniently connects the two campuses.
The Laboratory of Process Automation Systems is located at Building G2 on the North Campus. Find more information here.
TU Dortmund University has its own train station (“Dortmund Universität”). From there, suburban trains (S-Bahn) leave for Dortmund main station (“Dortmund Hauptbahnhof”) and Düsseldorf main station via the “Düsseldorf Airport Train Station” (take S-Bahn number 1, which leaves every 15 or 30 minutes). The university is easily reached from Bochum, Essen, Mülheim an der Ruhr and Duisburg.
You can also take the bus or subway train from Dortmund city to the university: From Dortmund main station, you can take any train bound for the Station “Stadtgarten”, usually lines U41, U45, U 47 and U49. At “Stadtgarten” you switch trains and get on line U42 towards “Hombruch”. Look out for the Station “An der Palmweide”. From the bus stop just across the road, busses bound for TU Dortmund University leave every ten minutes (445, 447 and 462). Another option is to take the subway routes U41, U45, U47 and U49 from Dortmund main station to the stop “Dortmund Kampstraße”. From there, take U43 or U44 to the stop “Dortmund Wittener Straße”. Switch to bus line 447 and get off at “Dortmund Universität S”.
The Laboratory of Process Automation Systems is located at Building G2 on the North Campus. Find more information here.
The H-Bahn is one of the hallmarks of TU Dortmund University. There are two stations on North Campus. One (“Dortmund Universität S”) is directly located at the suburban train stop, which connects the university directly with the city of Dortmund and the rest of the Ruhr Area. Also from this station, there are connections to the “Technologiepark” and (via South Campus) Eichlinghofen. The other station is located at the dining hall at North Campus and offers a direct connection to South Campus every five minutes.
The Laboratory of Process Automation Systems is located at Building G2 on the North Campus. Find more information here. The building is within 5min walking distance of the H-Bahn Station "Dining Hall at North Campus".
The facilities of TU Dortmund University are spread over two campuses, the larger Campus North and the smaller Campus South. Additionally, some areas of the university are located in the adjacent “Technologiepark”.
Site Map of TU Dortmund University (Second Page in English).
Interactive map
The facilities of TU Dortmund University are spread over two campuses, the larger Campus North and the smaller Campus South. Additionally, some areas of the university are located in the adjacent "Technologiepark".
Zum Lageplan