The work aims to design an image encryption algorithm based on chaotic system and multi-direction continuous diffusion for the purpose of preventing the diffusion of image pixel in a single direction and improving the security of encrypted cipher. Firstly, with the output value of Logistic model as the trigger, the mixed chaotic mechanism was designed in combination with Tent map, Sine map and improved Logistic mapping to obtain a set of chaotic sequences with strong randomness for scrambling the initial plaintext to highly confuse the pixel position and effectively reduce the scrambling periodicity. Then, the quantization mechanism was constructed to deal with chaotic sequences for obtaining the key-stream; and the multi-direction continuous diffusion mechanism was designed to change the pixel values from 4 different directions with different diffusion models, thus reducing the diffusion periodicity remarkably. The experimental results showed that the proposed algorithm had higher security and sensitivity compared with the current chaotic encryption mechanism. The proposed encryption algorithm can ensure the secure transmission of images in the network, and has better application value in packaging information anti-counterfeiting and other fields.