To eliminate the adverse effect of the additional rigidity of infilled wall on its frame and make full use of the strong deformation capacity and good ductility of concrete filled steel tubular (CFST) frame, the structural measures of reserving the gaps between CFST column frame and infilled wall as well as the flexible connection between infilled wall and steel beam were adopted. A full-scale two-story two-span frame infilled wall specimen was tested under low frequency repeated loads to verify the reliability of the flexible connection and investigate the seismic performance and interaction mechanism of CFST column frame-integral assembled infilled wall. Based on the test, the damage evolution process of infilled wall, the deformation performance of CFST column frame and connection, and the seismic performance of the structure were analyzed. Results show that the hysteretic curve of the overall structure was full, and the energy dissipation capacity of the structure was strong. When the ultimate displacement angle reached 1/41, the structure still had stable bearing capacity and good ductility. The flexible connection between the assembled infilled wall panel and the frame weakened the load transferred from the frame to the infilled wall panel, and delayed and reduced the damage of the integral assembled infilled wall. The integral assembled infilled wall and fabricated steel frame were connected reliably as a whole by the flexible connection, exerting the advantages of strong deformation capacity and good ductility of the CFST frame. In the later stage, the overall structure exhibited stable bearing capacity, good seismic performance, and safety reserve under earthquake load.