To improve the fast response and high stability control of agile flexible spacecraft during continuous on-orbit maneuvers, a combination of observer-based adaptive control law and composite manipulation law using variable speed control moment gyroscopes (VSCMGs) as the attitude control actuators is proposed. Firstly, considering the unknown flexible modes and precise inertia during maneuvers, modal observer and rotational inertia estimators are employed to identify the unmeasurable states or parameters and the identification results are used to accurately estimate the feedforward compensation torque. The Lyapunov analysis method is utilized to prove the stability of closed-loop control system. Subsequently, based on the moment allocation capability, singularity avoidance capability, wheel speed balancing capability and final gimbal positioning capability of VSCMGs, weighted pseudo-inverse steering law and three corresponding zero motions are designed accordingly. An optimal method for the final gimbal angle is proposed based on the Jacobian matrix condition number. The deployment schemes for the zero motions in different stages of maneuvers are provided. Finally, the effectiveness of the algorithm proposed in this paper is verified by numerical simulations of continuous attitude maneuvers. The results show that the VSCMGs can switch modes smoothly during continuous maneuvers and fulfill the corresponding functions in different maneuver phases. The modal observation values and inertia estimation values converge to the true values after multiple maneuvers. The controller, after parameter identification, enables the spacecraft to reach pointing accuracy faster and more stably at the end of maneuvers.