Track quality is essential to track safety. A reasonable forecast of track quality is a good instructor for the department of railway maintenance when arranging track maintenance schedule. Track quality index (TQI) can evaluate track quality in a unique track interval. With research on the changing tendency of TQI, this paper proposes a forecasting method that combines unequal-interval Grey Model and Elman Neural Network. The Grey model GM (1, 1) is previously exploited to obtain an approximate forecast of original TQI series and then the residual error correction is corrected by using Elman Neural Network optimized by Genetic algorithm. The new method takes random fluctuation of changing tendency of TQI into consideration, thus the historical data can be treated more efficiently. The proposed method is demonstrated with practically measured data of Shanghai-Kunming Railway Line. The forecasting results show that, comparing to other forecasting methods, the method which uses Elman Network to correct residual error correction reaches higher forecasting accuracy at root mean square error, correlation coefficient and determination coefficient.