To effectively reduce the effects of cluttered background on traditional shape-based object detection methods, a novel object detection algorithm based on contour matching was introduced, which combines the methods of saliency detection and template matching. First, the input image was preprocessed at the super pixel level to obtain the saliency region map without background by saliency feature detection. Then, the edge detection algorithm was applied to get the edge image in the saliency region, and the shape descriptor was used for contour matching after optimizing the edge image. Finally, a depth-first search strategy was applied to identify the hypothetical location of the object and perform hypothesis verification to determine the final location of the object. The experimental result in the ETHZ shape dataset proved the feasibility of this algorithm. Compared with other shape-based methods under the 50%-IoU and 20%-IoU evaluation criteria, according to the data results, the average detection rate of different categories was 96% when the false positive per image (FPPI) was 0.3. The detection rate was 99% when the FPPI was 0.4, and the detection rate would reach 100% if higher FPPI is accepted, which were all higher than the other algorithms. The experimental and comparative analysis results show that the proposed method could improve the detection accuracy and had obvious performance advantages, providing a new solution for object detection in cluttered background.