For the existence of defects in material can affect thermal conductivity, we calculate the thermal conductivity of perfect crystalline silicon and exam the effects of two types of point defect of vacancy and interstitial on the thermal conductivity of bulk crystalline silicon by molecular dynamics simulation applying reverse non-equilibrium method. The simulation results demonstrate that for scatter between phonon and defect, the thermal conductivity decreases with the increasing concentration of both types of the point defect and it decreases rapidly at low concentration condition and gradually gets flat with the increasing defect concentration. The loss of temperature sensitivity to thermal conductivity is observed at relative high concentration condition. In terms of thermal resistivity, the relative additional thermal resistivity is proportional to both types of point defect concentration considered. Furthermore, considering the microcosmic aspect, the inverse relationship between the concentration of point defect and the mean free path of scattering interacted by phonon and defect is deduced from this macroscopic proportional relationship. Taking the slope of the proportional relationship as impact factor to judge the extent of point defect effect on thermal resistivity (thermal conductivity), it is found by comparing the impact factor that the effects of both types of point defect on the thermal conductivity reduce with the increasing of temperature, and the interstitial has a rather more decreasing effect on the thermal conductivity than the vacancy.