To further understand the impact factor (IF) of bridges under moving vehicular loads, key problems such as the basic concept of IF, IF for different responses and positions were discussed in detail by combining theoretical derivation and numerical simulation and considering the physical significance of IF. Results show that the IF in the current specification was only a median parameter for convenient design, which had no actual physical meanings. IFs of different responses were not the same, and IF of moment was smaller than that of deflection in most cases. Moreover, IF in various sections were different, where the IF of pier-top section was the largest, followed by that of abutment span, and then that of center span. For two sections located symmetrically, the IF of the latter section in the direction of moving vehicles was bigger. Due to different principles, a more conservative value of IF can be used for design, but it cannot be directly utilized in evaluation. In the end, recommendations of IF for design and evaluation of dynamic performance of bridges subjected to moving vehicular loads were proposed, respectively.