In this paper, a driving controller for vehicles to avoid collision pedestrians in CVIS environment is designed. Based on the basic principle of the velocity obstacle method, a vehicle collision condition for pedestrian crossing was proposed, and an autonomous vehicle collision avoidance algorithm for pedestrians was established under the framework of model predictive control. Considering the operation constraints of driving, the vehicle driving strategy was optimized to minimize the change of vehicle speed and meet the driving comfort requirement under the premise of achieving pedestrian collision avoidance. Two control scenarios (i.e., vehicles going straight and changing lanes to avoid collision) were set up, and simulation experiments were carried out to verify the control effect in MATLAB environment. Results show that vehicles could avoid collisions by accelerating or decelerating when pedestrians cross the street under different conditions. By comparing with seven polynomials lane change trajectories, it showed that the safety of the autonomous collision avoidance driving was higher.