A granule-viewed segmentation method is established based on morphological jump connected operator and spectral graph clustering to solve the problem of content-driven color image segmentation. In the framework of granular computing, basic granules in image segmentation are constructed by color jump connected operator. Based on evaluation of the size of basic granules, regularization of basic granules is conducted by orthogonal polynomial surface fitting. Then spectral graph clustering is used to fuse basic granules and segmentation of a color image is achieved. The validity of the proposed method is demonstrated by experiments. The granule-viewed technique exceeds to regular-blocking-based and pixel-based methods in both elimination of edge-blocking effect and reduction of computational complexity.