Rail corrugation detection is an important method to ensure traffic safety. In order to solve the problem of the extraction of the trend of track irregularity in complex rail lines, a novel de-trending method named EEMD-SVD is proposed, which uses permutation entropy (PE) to select relevant singular-value components to reconstruct the trend. Compared with the existing methods based on EMD, this method considers the problem of mixed signal components in the original IMFs, and initially proposes the use of SVD to extract the trend accurately hidden in multi-dimensional IMF matrix. Since the singular-value components are arranged in the order of energy reduction without considering low-complexity and high-energy of signal, the proposed method modifies the EEMD-SVD, uses PE to select relevant low-complexity singular-value components, and finally reconstructs the trend with the relevant singular-value components selected above. Numerical simulation and track irregularity data test were carried out to evaluate the performance of the method. The numerical simulation experimental results showed that the proposed method outperformed the low-pass filter algorithm, the linear programming de-trending algorithm combined with EMD, and the WD de-noising algorithm. In particular, in the simulation of multiple signal-noise ratio, the improvement accuracy was about 30% when the signal-noise ratio was low. The track irregularity data test proved that the method is effective for de-trending the trend of track irregularity.