Abstract: A new image compression algorithm based on lifting directionlet transform (LDT) is proposed. This transform captures the multi-directional anisotropic image features efficiently and processes the structure of lattice-based separable filtering and sampling. The quad-tree segmentation is designed for direction optimization of local region, and a set-partitioned embedded block algorithm for the statistic distribution property of transform coefficients is adopted. The coding performance is improved by designing the new chained list sorting and context-based arithmetic coder. The experimental results show that our proposed compression algorithm outperforms the standard wavelet-based SPECK, SPIHT, JPEG\,2000 and original directionlet-based methods both in terms of peak signal to noise ratio (PSNR) and visual quality. Especially at the low-rate, our algorithm can preserve better the detailed information.