A Visual Map can be seen as an image database with rich location information. All images or image features stored in this database have their corresponding location information. An accurate localization needs a large image database, but building a large database must be laborious and time-consuming. In order to solve this problem, a method based on Optical Flow Technique is proposed to establish a Visual Map. The accuracy of Optical Flow algorithms is always influenced by the difference of indoor illumination and lateral deviation of optical flow caused by turning of the cameras. A method to improve optical flow algorithm is proposed and the new algorithm is used to calculate the displacement of image sequences to acquire location information of the camera and each picture. The experimental results show that the probability of localization error less than 1 meter and 2 meters is 26% and 70%, respectively. Compared with traditional visual indoor localization system, using optical flow algorithm to build a visual map is much more convenient and time-saving, although the positioning accuracy of the proposed method is slightly worse than the traditional method. Although the performance of the proposed method is similar to the video stream method, it needs fewer sensors to build image database and can be used in more complicated environment. Overall, the proposed method will perform well in indoor visual localization systems especially in large buildings and changeable places.