In view of the weaknesses of poor computing and storage capabilities of edge devices, lightweight processing was carried out on the backbone network CSPDarkNet53 for feature extraction in the traditional YOLOv5 model, and a lightweight gesture recognition algorithm MPE-YOLOv5 was proposed to realize the deployment of the model in low-power edge devices. Considering the problem that it is difficult to identify large-scale transformation targets and tiny targets due to less feature extraction in lightweight model, efficient channel attention (ECA) mechanism was added to alleviate the loss of information after high-level feature mapping due to the reduction of feature channel. A detection layer for tiny targets was added to improve the sensitivity to tiny target gestures. EIoU was selected as the loss function of the detection frame to improve the positioning accuracy. The effectiveness of the MPE-YOLOv5 algorithm was verified on the self-made dataset and NUS-Ⅱ public dataset, and the MPE-YOLOv5 algorithm was compared with lightweight M-YOLOv5 algorithm and original YOLOv5 algorithm on the self-made dataset. Experimental results show that the model parameters, model size, and computational complexity of the improved algorithm were 21.16%, 25.33%, and 27.33% of the original algorithm, and the average accuracy was 97.2%. Compared with the lightweight model M-YOLOv5, MPE-YOLOv5 improved the average accuracy by 8.72% while maintaining the original efficiency. The proposed MPE-YOLOv5 algorithm can better balance between the detection accuracy and real-time reasoning speed of the model, and can be deployed on edge terminals with limited hardware.