In view of the problems of target loss and low tracking accuracy due to scale transformation during long-term UAV tracking, a scale-aware spatial tracker based on moth-flame optimization (MFO) was proposed. First, the Gaussian initialization was used to replace the random initialization strategy of the original moth-flame optimization algorithm, so as to reduce the high computational complexity and waste of computing power of the optimization algorithm in the tracking problem. Second, on the basis of the characteristics of fast gradient histogram, an improved moth-flame optimization tracker was constructed. Then, considering the problem of target scale change under the long-term tracking of UAV aerial photography, a discriminative scale space tracking (DSST) algorithm combined with adaptive scale transformation was designed. A scale-aware spatial tracker was further proposed to solve the problem of tracking drift caused by the fixed aspect ratio of the scale filter. In addition, the variation of the filter response peak value under different backgrounds was analyzed, and an index that can reflect the tracking confidence under environmental changes was proposed. The moth-flame optimization tracking framework was combined with the scale-aware spatial tracker through confidence, which can solve the problems of scale change and target loss in long-term tracking. Finally, the performance of the algorithm was verified on the UAV long-term tracking dataset. Results show that the proposed algorithm can effectively prevent the occurrence of drift and improve the tracking efficiency. Compared with 12 similar algorithms in the tracking field, the proposed algorithm can effectively solve the scale change and the target loss of the long-term UAV tracking, and meet the requirement of real-time with high accuracy.