To solve the technical problem that the surface finish of single crystal germanium lens cannot meet requirements, this paper carries out numerical control high-speed polishing optimization tests on polyurethane and asphalt polishing molds based on the numerical control high-speed polishing technique. According to the Preston theory, the processing efficiency and the surface quality control ability of the two polishing molds were compared and analyzed through continuous optimization of process flow and parameters with the combination of motion trajectory simulation and power spectral density calculation. Experimental results show that both of the polishing molds obtained high surface accuracies. The polyurethane mold had higher processing efficiency, but its ability to control the microstructure of the surface was poor. The surface roughness RMS of the asphalt polishing mold improved nearly 3 nm compared with that of the polyurethane polishing mold. Finally, after optimization, the combination of polyurethane initial polishing and asphalt polishing technique was proposed and verified by experiments.