The work aims to improve the detection and recognition ability of boxes with various textures and scattered stacking, which are common in industrial sorting. A rotating target detection algorithm based on YOLOv5 was proposed. This algorithm included three modules:target classification, pose angle recognition and boundary box location. In the target classification module, self-built data sets and eight target classification labels were designed for model classification learning; In the pose angle recognition module, an angle prediction branch was added to the YOLOv5-head network. The angle classification method of circular smooth label was introduced to realize accurate detection of rotation angle of sorting boxes; In the boundary box location module, the LCIoU regression box loss function was used to calculate the regression loss of the rotating box, and the boundary box that tightly wrapped the target position was obtained. The detection accuracy of the improved YOLOv5 algorithm in the self-built data set reached 95.03%. In the robot multi object sorting experiment, the accuracy rate reached 100%. The algorithm in this paper has high positioning and identification accuracy when boxes are in scattered, dense and stacked conditions.