The work aims to train the convolutional neural network (CNN) by extracting color histogram features of sample images, so as to achieve the purpose of detecting image color defect quickly and accurately. RGB color space of standard image was converted to HSV color space and training and testing samples were obtained by change of H, S and V component in image. The color histogram of image was non-equally quantized in HSV color space, and then all of the color histogram features of training and testing samples were obtained. The CNN was trained by color histogram features of sample image, and then the testing sample image was detected to study the detection speed and precision. Finally, this method was compared to each-pixel, super-pixel, BP and support vector machine (SVM) recognition methods. For the color image of 512×512, the mean detection time of each image by CNN was 57.66 ms. When the number of training samples was 50 000, the detection precision of CNN for 10 000 testing samples was 99.77%. Therefore, the detection method of convolutional neural network can significantly improve the detection precision while ensuring the accuracy and has good application value in on-line color defect monitoring of printing.