The work aims to apply machine vision technology to the surface defect detection of woven bags to solve the low efficiency and low accuracy of surface defect detection in the production of woven bags, and thus improve the production efficiency of woven bags. A surface defect detection system of woven bags was designed based on machine vision:Firstly, in order to reduce the effects of background gray changes on defect detection, a preprocessing algorithm with both noise filtering and image enhancement functions was studied. Secondly, the two-dimensional maximum entropy method was selected to segment the preprocessed woven bag map, and the improved genetic algorithm was used to optimize it to enhance the convergence speed and effect of the algorithm. Then, the recognition and classification of surface defects of woven bags were realized by feature extraction combined with morphological processing. Finally, the connected domain analysis was applied to count and locate the classified defects to obtain the size and location information of the defects. 200 defect samples of woven bag were collected, and the surface defect detection system of woven bags in this paper was used to identify defects of woven bag samples. The average recognition accuracy was 94.0%, and the processing time of an image of woven bag was about 600 ms. The system has high recognition efficiency and accuracy, and can realize rapid detection of surface defects of woven bags and meet the needs of industrial production.