Research on intelligent decision-making for group top-coal caving based on batch reinforcement learning

In the fully-mechanized top-coal caving mining process, there are many factors that affect the dynamic characteristics of top-coal crushing and transportation. It is difficult to accurately model the mining environment and coal caving process through the pre-detection results. As a result, the intelligent control of top coal caving mining lacks a model basis, and the contradiction between the top coal recovery rate and gangue rate could not be substantially resolved by a simple coal caving process. In this paper, a novel intelligent decision-making method called Batch Reinforcement Learning for group caving was proposed, in which the hydraulic support group was converted into a multi-agent, and the optimal control of the top coal recovery rate and gangue rate were converted into an optimal decision of the Markov process. Under the framework of multi-agent, reinforcement learning was used to achieve multiple coordinated control of windows. To this end, the top coal occurrence state was regarded as the state of the Markov process, and the correlation mechanism of “top coal occurrence state-coal outlet control” was established, and the coal outlet control strategy was generated online. In order to improve the learning ability of the agent, a batch Q value update method was proposed to optimize the “state jump” phenomenon that occurs in the state acquisition, and further improve the agent's online learning efficiency. In order to verify the validity of the algorithm, a numerical simulation was carried out in combination with the coal seam conditions of the No.8222 working face of Tashan Coal Mine and the main technical parameters of the hydraulic support. The comparative experiments were carried out on sequential coal caving, segmented and interval coal caving and group intelligent coal caving on the platform. A series of simulation experiment results show that the batch-type reinforcement learning coal caving decision method proposed in this paper can dynamically adjust the action of the coal caving port according to the occurrence state of the top coal, effectively separate coal and gangue in the group coal caving process and maximize the profits of top coal caving method.