To address the conflict between the complexity of aerospace tasks and the design of traditional fixed-configuration satellites, aerospace institutions are focusing on the study of modular self-reconfigurable satellites with flexible configuration change capabilities. Among these efforts, configuration planning emerges as a particularly challenging area of research. Aiming at the configuration problem of modular satellites, cube-lattice satellites are taken as the research object, and based on graph theory, a configuration matrix and an extension matrix are proposed to describe the satellite topology. Through the study of the motion characteristics of the satellite module, an algorithm for solving the accessible space of the module motion is given. Considering the satellite configuration problem as a sequential decision-making problem, and based on the theory of deep reinforcement learning, the modification process is modeled as a Markov decision process. An intelligent modification planning method based on the actor-critic model is designed, incorporating a multi-layer neural network to approximate the actor and critic functions. Through training the neural network, the performance of the satellite reconfiguration strategy is progressively improved. The simulation experimental results show that the proposed configuration method yields progressively improved satellite reconfiguration strategies for the given satellite case studies. This approach exhibits generality across different satellite configurations with varying numbers of modules. Additionally, compared with the traditional configuration method based on heuristic search, it has advantages in the number of configuration steps, calculation time and configuration success rate, which validates that the proposed intelligent planning method has potential value in future modular satellite design work.