According to the problem of sparse matrix with high complexity of LDGM codes, a low-complexity construction algorithm of LDGM codes is proposed, which can decrease the encoding complexity while keeping a good performance. Meanwhile, a serially-concatenated construction with a modified SCLDGM decoding algorithm is introduced to solve the problem of high error floor of LDGM codes. This algorithm considers the incorrect output of the inner decoder as the erasures from a BEC channel. The outer decoder uses the priori probability to initialize in the decoding process, which further reduces the residual errors. By simulations for the SCLDGM encoding and decoding system, the near-optimal combination of inner and outer coding rates and the near-optimal column weights of inner and outer codes are obtained. Simulation results show that, a good BER performance can be obtained and the error floor of LDGM codes can be decreased as well by using the proposed algorithms on the premise of keeping a low complexity.