Abstract: A research on the performance evaluation of automatic target recognition(ATR) algorithms has been done. 12 kinds of image metrics related to ATR performance were choosen under the law of image quality description. And then a systematic approach based on response surface model for the scientific evaluation of ATR algorithm performance was presented. And also we provide a modular project of ATR evaluation system with many of needed capabilities for rapid testing, analysis and -evaluation and data reduction. Finally, we do a comparative study for the performance of two kinds of ATR algorithms to prove its feasibility.