To accurately measure the installation matrix between magnetometers and star sensors in satellite geomagnetic measurement system on the ground, a calibration system based on the single axis non-magnetic turntable was designed, which is composed of non-magnetic turntable, geomagnetic field monitoring equipment, etc. First, the error sources in the system were analyzed, and the corresponding coordinate systems were established for transferring the errors. The indicated output of the magnetometer was obtained by combining the error model of the magnetometer with the geomagnetic field. Thus, a method of calibrating installation matrix by using single axis vertical non-magnetic turntable was proposed. In this method, the error coefficients in the indicated output of the magnetometer were obtained by harmonic analysis, and the attitude of the magnetometer coordinate system in the initial geographical coordinate system was derived. The attitude of the star sensor coordinate system in the inertial space was determined by star sensor observation, and that in the initial geographical coordinate system was obtained according to the local longitude and latitude and Greenwich Sidereal Time. Finally, the installation matrix among the magnetometers and the star sensors was solved by using the geographic coordinate system as a bridge. Based on the Monte Carlo method and uncertainty synthesis formulae, simulation experiment and error analysis were conducted on the proposed method. Results show that the uncertainty of each element of the installation matrix was within 1.14×10^-5 rad when the observation error of the magnetometer was 1 nT and the measurement accuracy of the star sensor was 1″, indicating the correctness of the calibration method.