The aim of this paper is to optimize femoral prosthesis structure and material based on special patient information. Firstly, femoral external and internal contour are extracted from CT data. Femur model and prosthesis model are got respectively and assembled by using 3D reconstruction software, and then the optimized prosthesis structure is developed based on biomechanical analysis results. Secondly, the prosthesis rated load, the life of prosthesis, stress shielding and deformation are determined by the finite element analysis results, and the optimized material matching model is developed based on multi-objective optimization results that are resolved by fuzzy matter-element method. Finally, this prosthesis design method is utilized to a special patient. The optimized structure for this patient is the spheroidic femoral head and individual prosthesis stem, and the optimized materials are the carbon fiber enhanced pek composite for femoral head and Zr-Nb alloy for prosthesis stem.