In order to solve the problems of the intensive computation and poor real-time property of conventional fractal image compression algorithm, this paper proposed an improved fractal gray image compression algorithm. Firstly, the algorithm used Gabor transform to extract texture features based on image blocks, then it used K-means algorithm to implement clustering process for R-block and D-block based on the texture features, and it only conducted matching operation for the R-block and D-block in the same category. In the construction process of the candidate blocks pool, this paper proposed a simplified algorithm for eight isometric transformations. The experimental results showed that this method could accelerate the speed of fractal compression, while ensure the visual effects of the reconstructed image with good robustness.