In order to study the strain compatibility of the nickel powder-filled cement based sensors and their surrounding concrete, the models of the sensors and concrete were set up by finite element software ANSYS. The stress and strain of the sensors and their surrounding concrete were calculated based on different embedded locations and angle deflections of the sensors, strength grades of concrete, and magnitudes and types of load. The strain compatibility of the sensors and their surrounding concrete were analyzed through the relative difference of strain. The results show that the sensors should be embedded in the internal homogeneous strain area of concrete members, rather than being embedded in the superficial area. The small angle deflection has little effect on the monitoring accuracy. As the sensors are used for health monitoring of concrete structures, the effect of strength grade of concrete and magnitudes and types of load should be systematically considered. The degree of strain compatibility of the sensors and their surrounding concrete can be better reflected by relative difference of strain.