Numerical simulations were performed to further investigate the effects of beam-to-column connections, reinforced ways, etc. on the hysteretic behavior of three tested steel frames with unbonded steel plate brace encased in panel (referred to as panel BRB), which is termed as steel frame with panel BRB (panel BRBF) hereafter. The values of the parameters used to reflect the hardening behavior of panel BRBs were determined based on cyclic tests of panel BRBs. In general, hysteretic curves of panel BRBFs and mechanism on yielding or buckling of steel members acquired from simulations agreed with those from tests. Yielding of panel BRBs occurred at inter-story drifts of 1/463~1/350 in tests and 1/416~1/305 in simulations, and plastic deformations of steel frames were not obvious prior to the drift of 1/50. Ductility and energy dissipation capacity of panel BRBFs were good and the design aim that yielding of panel BRBFs is mainly concentrated on panel BRBs was realized. Within the drift of 1/30 and prior to deterioration of load carrying capacity of steel frames, the panel BRBFs with moment-resisting beam-to-column connections and the panel BRBF with non-moment-resisting beam-to-column connections, in which steel frames nearly remained elastic, showed trilinear and bilinear skeleton curves, respectively. The trilinear curve of each panel BRBF can be acquired by putting the bilinear skeleton curves of both panel BRB and steel frame together. The reinforced ways of adding steel plates at the ends of steel beams near moment-resisting beam-to-column connections let yielding of beams occur far away from the beam-to-column connections and ensured both enough strength of the connections and stable energy dissipation capacity of steel frames. The beam in the panel BRBF with non-moment-resisting beam-to-column connections had large flexural yielding when one brace in a chevron panel BRB failed locally, and the overall lateral resistance of structure was not deteriorated. Besides, simplified methods by employing beam and truss elements to simulate hysteretic behavior of panel BRBFs were proposed.