| 引用本文: | 张新艳,郭鹏,余建波.应用深度强化学习的压边力优化控制[J].哈尔滨工业大学学报,2020,52(7):20.DOI:10.11918/201908012 |
| ZHANG Xinyan,GUO Peng,YU Jianbo.Optimal control of blank holder force using deep reinforcement learning[J].Journal of Harbin Institute of Technology,2020,52(7):20.DOI:10.11918/201908012 |
|
| 摘要: |
| 为改善板料拉深制造的成品质量,采用深度强化学习的方法进行拉深过程的压边力优化控制. 提出一种基于深度强化学习与有限元仿真集成的压边力控制模型,结合深度神经网络的感知能力与强化学习的决策能力,进行压边力控制策略的学习优化. 基于深度强化学习的压边力优化算法,利用深度神经网络处理巨大的状态空间,避免了系统动力学的拟合,并且使用一种新的网络结构来构建策略网络,将压边力策略划分为全局与局部两部分,提高了压边力策略的控制效果. 将压边力的理论知识用于初始化回放经验池,提高了深度强化学习算法在压边力控制任务中的学习效率. 实验结果表明,与传统深度强化学习算法相比,所提出的压边力控制模型能够更有效地进行压边力控制策略优化,成品在内部应力、成品厚度以及材料利用率3个质量评价指标的综合表现优于传统深度强化学习算法. 将深度强化学习中的策略网络划分为线性部分与非线性部分,并结合理论压边力知识来初始化回放经验,能够提高深度强化学习在压边力优化控制中的控制效果,提高算法的学习效率. |
| 关键词: 板材拉深成形 质量控制 深度强化学习 有限元仿真 优化控制 |
| DOI：10.11918/201908012 |
| 分类号:TG301 |
| 文献标识码:A |
| 基金项目:国家自然科学基金(51375290) |
|
| Optimal control of blank holder force using deep reinforcement learning |
|
ZHANG Xinyan,GUO Peng,YU Jianbo
|
|
(School of Mechanical Engineering, Tongji University, Shanghai 201804, China)
|
| Abstract: |
| To improve the quality of products in deep drawing process, the deep reinforcement learning method is used to optimize the blank holder force (BHF). A new BHF control model based on the integration of deep reinforcement learning and finite element simulation is proposed, and the BHF control strategy is optimized by combining the perception ability of deep neural network with the decision-making ability of reinforcement learning. The proposed control model uses the deep neural network to deal with huge state space and avoids the fitting of system dynamics. By utilizing a novel strategy network structure, the BHF control strategy is divided into global and local parts, and the control effect is improved. Meanwhile, the theoretical knowledge of BHF is used to initialize the replay experience, which improves the learning efficiency of deep reinforcement learning algorithm in BHF control tasks. Experiments show that the proposed BHF control model can optimize BHF control strategy more effectively than traditional deep reinforcement learning algorithm. The comprehensive performance of the proposed control model in three quality indicators (internal stress, thickness and material utilizing rate) is better than that of the traditional deep reinforcement learning algorithms. |
| Key words: deep drawing quality control deep reinforcement learning finite element analysis optimal control |
