The work aims to improve the recognition accuracy of fruit and vegetable products, so as to automatically classify these products. The powerful feature learning and feature expression capabilities of deep convolutional neural networks were used to automatically learn the characteristics of fruit and vegetable types, the location-wise soft attention algorithm was proposed to improve the Inceptionv3 neural network, and the parameter transfer learning method was combined to establish the fruit and vegetable recognition model. In view of the wide variety of fruits and vegetables, and the lack of a complete database of fruit and vegetable images at home and abroad, the fruit and vegetable image data sets were constructed. Based on the data sets above, the proposed fruit and vegetable recognition algorithm was compared with other fruit and vegetable recognition algorithms. The experimental results showed that, when the learning rate was 0.1 and the number of iterations was 5,000, the accuracy of the proposed algorithm was as high as 97.89%. Compared with the existing fruit and vegetable recognition algorithms, the proposed fruit and vegetable recognition algorithm has the best recognition performance and the strongest robustness.