To enhance the robustness of IR fast small target tracking, an improved mean shift tracking algorithm based on membership degree weighted kernel histogram target representing model is proposed. Firstly, the local background interference problem in tracking fast small target with original mean shift algorithm is analyzed and the membership degree weighted kernel histogram target representing model merged into background information is presented. This model is able to enhance the representing capability of target and suppress local background interference. Then, the shift vector of this model is deduced in the framework of mean shift by regarding Bhattacharyya coefficients as the similarity measure. The target shift tracking is achieved effectively according to target gray level of large membership degree with high shift weight, and vice versa with low shift weight. Finally, the local background time-varying is conquered by employing model updating method and the robustness of target tracking is further improved. The experimental result indicates that the algorithm can improve the shift weight of target pixel gray level and suppress background interference, thus the performance of tracking the IR fast small target is robust.