Abstract: An on-the-fly, self-localization system is developed for mobile robot which is operative in a 3D environment with elaborative 3D landmarks. The robot estimates its pose recursively through a MAP estimator that incorporates the information collected from odometry and unidirectional camera. We build the nonlinear models for these two sensors and maintain that the uncertainty manipulation of robot motion and inaccurate sensor measurements should be embedded and tracked throughout our system. We describe the uncertainty framework in a probabilistic geometry viewpoint and use unscented transform to propagate the uncertainty, which undergoes the given nonlinear functions. Considering the processing power of our robot, image features are extracted in the vicinity of corresponding projected features. In addition, data associations are evaluated by statistical distance. Finally, a series of systematic experiments are conducted to prove the reliable and accurate performance of our system.