The work aims to design a new type of non-circular gear system and study its dynamic characteristics combined with the specific needs of an automatic packaging machine, so as to verify the dynamic characteristics of certain automatic packaging machine's transmission system. Firstly, the characteristics of non-circular gear were introduced and the mathematical model of non-circular gear was established to obtain the dynamic equations based on the derivation and analysis. Then, the 3D model of non-circular gear was established by UG software and the dynamics simulation was carried out by Adams. Meanwhile, the dynamic characteristics of non-circular gear and circular gear at the same center distance were compared. The results showed that the newly designed non-circular gear had a meshing force of about 500 N more than that of the circular gear, and the acceleration of the driving gear was about 10 times that of the circular gear. By comparing non-circular gears with traditional gears, the non-circular gears have the advantages of variable gear ratio and flexible spatial layout of the transmission. Because the non-circular gears cannot be made into helical gears, the transmission performance is slightly lower than that of the circular gears.