Vision-based simultaneous localization and mapping (SLAM) is a research hotspot in the field of intelligent driving. However, for the scenes that contain moving targets or inconspicuous close-range features, it is easy to cause ineffective or inaccurate pose estimation between frames. To solve this problem, this paper proposes a SLAM algorithm based on road structured features and scene semantic information. First, for the problem of target moving, a semantic segmentation neural network with an improved pyramid pooling module was designed to obtain the target category corresponding to each pixel in the image. The segmentation results were taken as basis for the elimination of moving points, which avoids the problem of low accuracy of pose calculation caused by moving points participating in feature matching. Then, in view of the lack of effective feature points, the road area in the image was determined based on v-disparity algorithm, and disparity function was obtained to calculate the accurate disparity value of the pixels on the road. Furthermore, a pose calculation method based on road structured features (e.g., lane lines, road boundaries, pavement traffic markings) was proposed. Finally, scene experiments were carried out and results show that the absolute trajectory error of the improved algorithm proposed in this paper was smaller than that of the original algorithm, which proves that the proposed method has higher pose estimation accuracy in scenes with moving targets or inconspicuous close-range features. In addition, an effective dense point cloud map containing semantic information was established, which has good environmental adaptability.