To solve the problems of building diversity evaluation system for different task and coupling relationship between different measurement indexes, a multi-agent evaluation frame work about effectiveness of naval ship was proposed. The method of model construction about measure of effectiveness of complex system was presented firstly, which can deal with the problem of the coupling relationship between different targets. Then Hierarchical control model was built based on that model. Task breakdown mechanism based on ship work breakdown structure (SWBS) and application of fuzzy set theory to solve the issue of multi-task coalition parallel generation was presented. Agent which had the strongest ability was chosen to compose temporary federate. Utility allocation of federate software without preferences to evaluation effectiveness of naval ship was presented. The weight of index by the method was objective. At last the paper proved the feasibility and effectiveness of the method.