The work aims to propose a YOLOv5-SPM detection network, solve the challenges concerning diminished detection accuracy and complex model deployment encountered in chip packaging defect detection, to enhance detection accuracy and facilitate the implementation of lightweight models. The channel attention mechanism was placed after each feature extraction module to increase the importance of defect-related channels, reduce the interference of redundant features and improve the target detection accuracy. Then, the SimSPPF pyramid pooling structure was used in the connection of the backbone network and the neck network to integrate multi-resolution features of the self-built chip data set more effectively. After that, the feature extraction module of the backbone network was replaced with MobileNetV3 and the conventional convolution was replaced with deep convolution and point convolution to significantly reduce the model size and calculation scale. The improved new network YOLOv5s-SPM achieved a 0.6% increase in mean average precision and a 3.2% increase in accuracy compared with the original network, while reducing the model parameters by 29.5%. The experimental results validate the superiority of the proposed network in achieving higher accuracy and faster detection speed in the task of chip defect detection. Since the model parameters are reduced by 29.5%, it can also be deployed on industrial embedded devices.