The work aims to design a robot arm visual grasping system of disorderly stacked express package based on multimode to realize the automatic package feeding of express cross belt sorter. The multi-modal information of RGB image and depth map of stacked express packages was sent to the residual network model of the full convolution for recognition. After model post-processing, grasping strategy design, model training and deployment, the accurate identification of the grasping area of express packages was realized. Combined with the package attitude estimation, the grasping and handling of express packages were completed through the robot arm. Experiments showed that the visual recognition method proposed in this paper realized the accurate recognition of the grab area of express package, and the test accuracy was 95.73%. The robot arm visual grasping system of express parcel based on this method can complete the automatic package feeding operation of express cross belt sorter.