Graphene suspension was prepared by nitric acid oxidation and ultrasonic to improve the hydrophobicity. Graphene suspension was added to cement to produce cement-graphene composite. The effect of graphene on mechanical properties and microstructure of cement paste was studied. In addition, the toughing mechanism of grapheme was also discussed, which provide theoretical and practical foundation for the study of graphene cement-based composite materials. With the increase of graphene content, the compressive and flexural strength of cement-based composite increased firstly then decreased, and the optimal additive amount is 0.02 wt%. The results from structural analysis of set cement by SEM and FT-IR indicated that graphene can promote the growth of the hydration products, change the shape and size of hydration crystal, but did not change its type through reacting with cement and graphene.