To meet the requirements of the daily management of water supply systems for short-term water demand prediction timeliness, a kernel-based extreme learning machine model (KELM) was established, which requires short training time. From the perspective of improving the prediction accuracy, a residual correction module based on the Fourier series (FS) was constructed, which was used to model the difference between the initial predicted value and the observed value of water demand, and the residual correction of the initial predicted value was completed. The module was superimposed on the KELM model to form the hybrid prediction model (KELM+FS). The performance of the models was tested using real water demand data. Experimental results show that the KELM model could produce similar prediction accuracy as the artificial neural network model and the support vector regression model, but the prediction time was only about 5% of the average time of the two models. Compared with the KELM model, the relative prediction accuracy of the hybrid model KELM+FS was improved by about 12% without significantly increasing the prediction time. Therefore, when applied to short-term water demand prediction, both the single model KELM and the hybrid model KELM+FS could achieve the goal of improving the prediction efficiency.