A hybrid algorithm which combines the augmented Lagrangian multiplier method with the fish swarm algorithm is presented to solve the problem of constrained nonlinear optimization. The method approximately solves the optimal solution of the augmented Lagrangian function with the fish swarm algorithm, and the solution is applied to update the Lagrangian multipliers and penalty parameters. Stochastic convergence of the artificial fish swarm is analyzed. Compared with an adaptive penalty method for genetic algorithms, simulation results verify the superiority and validity of the proposed hybrid algorithm.