Abstract: In view of the problem that it is difficult to construct a deep learning model with small sample data and the low recognition rate of multi-scale morphology and color coal gangue under actual working conditions, an optimization method of coal gangue depth recognition model combining migration learning ideas and structural optimization is proposed. A machine vision platform was built by imitating the actual production environment of underground mines, and the CCD industrial camera was used to obtain images of coal and gangue in real-time, the diversity of coal and gangue data sets were increased by using of image rotation, inversion and noise increase. In order to reduce the cost of model training time, a Transfer Weight-Reduce Neurons (TW-RN) model optimization method was proposed to improve the pre-training CNN model, and four kinds of improved model：Im_AlexNet, Im_VGG16, Im_VGG19 and Im_ResNet50 were constructed.Relying on the expanded coal and gangue data sets, this simulation test compared the training results of four models under different optimizer types and setting methods of learning rates, and determined the best optimizer type and learning rate setting method for each model. Based on the four evaluation parameters of test accuracy, F1 score, model memory size and training time, the performance of each model before and after the improvement was quantitatively evaluated, and an optimal coal and gangue recognition network model based on TW-RN optimized CNN was determined. The results show that the recognition accuracy of the four coal and gangue recognition models based on TW-RN has been effectively improved, and the training time and the memory size of the model have been significantly reduced. The coal gangue recognition rate has a non-positive correlation with the model complexity. Compared with the improved Im_VGG16, Im_VGG19 and Im_ResNet50 depth recognition models, the performance of the shallow Im_AlexNet model improved by the TW-RN method has been significantly improved, and its recognition accuracy has increased by 2.149 percentage points, up to 97.461%, the memory consumption is reduced by 190MB. The recognition time of a single image is saved by 0.788 milliseconds.