Abstract: One limitation of context dependent discriminative model combination is that a large number of parameters will be introduced, which is liable to overtraining with limited training data. We propose context modeling using phonetic decision trees in lattice based discriminative model combination. Question in tree node is chosen to optimize the minimum phone error criterion. First order approximation of the objective function increment is used for fast question selection. Results on speech recognition show that the method is capable of improving the robustness to overtraining and obtains error reduction with many fewer parameters. It is also shown that the model combination using tree based context modeling is superior to feature combination approach.