To address the problems that models easily fall into local optima and have insufficient accuracy in the high-dimensional nonlinear spectral inversion of soil organic matter (SOM), this paper proposed an inversion method of SOM based on a Support Vector Machine (SVM) optimized by an improved goose swarm optimization (IGOOSE) algorithm. Taking soil samples from typical farm areas in Heilongjiang Province as the research object, this paper used the Mahalanobis distance method to remove abnormal samples and screened the optimal spectral preprocessing method based on partial least squares regression (PLSR) modeling results; by constructing full-band SVM, random forest (RF), and extreme learning machine (ELM) models under the optimal preprocessing condition, this paper determined the optimal basic inversion model; furthermore, this paper used the successive projection algorithm (SPA), random frog feature importance selection (RFROG), and competitive adaptive reweighted sampling (CARS) to conduct spectral feature dimensionality reduction and selected the features obtained by the optimal dimensionality reduction method as the model input. In the process of model parameter optimization, by dynamically adjusting the population size and integrating the Lévy flight strategy, random perturbation mechanism, and hybrid neighborhood search method, this paper conducted a three-stage adaptive improvement on the basic goose swarm optimization (GOOSE) algorithm to construct the IGOOSE algorithm. After verifying its effectiveness via standard test functions, this paper applied IGOOSE to SVM parameter optimization and compared it with the original GOOSE algorithm to identify the inversion model of SOM with better performance. The results show that the accuracy of the PLSR model constructed after first derivative (D1) preprocessing is significantly superior to other preprocessing methods; the optimal basic model constructed based on full-band D1 data is SVM, and its five-fold cross-validation average coefficient of determination (R²) reaches 0.762 4; CARS is the optimal spectral feature dimensionality reduction method, and the five-fold cross-validation average R² of the CARS-SVM model constructed based on its selected features is 0.753 7; IGOOSE-SVM is significantly superior to GOOSE-SVM in terms of generalization ability and prediction stability, and the R² of the training set and test set increase by 0.106 6 and 0.041 1, respectively, while the root mean square error (RMSE) decreases by 2.541 3 g/kg and 0.528 7 g/kg, respectively. Finally, the R²of the training set and test set reach 0.925 2 and 0.841 6, respectively, and the RMSE is 4.418 9 g/kg and 7.040 7 g/kg, respectively. The research indicates that the proposed improvement strategy effectively mitigates the modeling difficulties caused by high-dimensional nonlinear spectral data. The IGOOSE-SVM model provides a reliable approach for achieving high-precision inversion of SOM and provides a theoretical basis for the application and improvement of optimization algorithms in the field of spectral modeling.