To calculate the straightness error of the axis of hole accurately, an error evaluation method based on rotating projection was proposed. According to the homogeneous coordinate transformation and rotating projection, the measuring points are rotated to the same plane around the least square line. The spatial problem was transformed into a plane problem. Three control points were pre-selected in the plane point set. Considering the distance from control points and measurement points to the control line, the control points were continuously updated by using the distance ratio coefficient until the absolute value of the distance between the three control points and the control line was the largest. The three control points met the criteria of "high and low and high" or "low and high and low" in the minimum area. This method does not need complex nonlinear optimization process, and has small calculation amount and high accuracy of evaluation result. Rotating projection preserves the distance relationship between the measuring points, which avoids the problem of missing the control points combination in direct projection. The results show that the method has higher accuracy than other evaluation methods and takes less than 0.1s.