The work aims to solve the problem that the current capsule counting machine can only count capsules and cannot sort damaged capsules at the same time. A capsule counting machine with the Faster R-CNN deep neural network as the core was designed. On the basis of the original capsule counting machine, the CCD line-array camera was replaced by area-array camera to meet the demand of image acquisition, and the image segmentation and multi-thread technology were further used to speed up the image processing speed. Finally, the target was detected and sorted through the well trained Faster R-CNN network. After verification of the test set, the identification rate of normal capsule reached 95.47%, the identification rate of damaged capsule reached 97.94%, and the single image processing reached the real-time speed of 65 ms. The proposed method properly combines the advanced in-depth learning technology based on the traditional counting and realizes the automatic sorting of the target.