Abstract: Implementation of a real time stereo vision system for obstacle detection and localization is discussed for planetary rover. The system employs binocular or trinocular stereo with fish-eye lenses to capture stereo pairs with a very large field of view (FOV～160°). Camera calibration parameters are used to perform distorted image correction, epipolar rectification and LoG filtering, and the SAD similarity is computed to estimate dense depth maps under the epipolar and smooth constraints. Using the Intel MMX instruction set and LUT,.the system can complete dense depth mapping and obstacle detection in real time. The test result shows that the system can produce dense depth maps at 10Hz with a resolution of 320X120 pixels and a disparity of 64 levels using standard PCs, are allow the robot to safely travel at lm/s.