0 Introduction

Histogram equalization (HE) has been widely used in digital images to enhance contrast^[1-2]. HE typically enhances the contrast by reducing the gray-levels and stretching histogram dynamic ranges. These may cause the following problems: 1) feature loss resulting from low-frequency gray-levels being phagocytized by other gray-levels; 2) over-enhancement caused by excessive stretching of high-frequency gray levels; 3) annoying artifacts and intensity saturation effects because of mean brightness shift. To solve these problems, some approaches have been proposed and verified by some researchers.

Sub-histograms equalization has been adopted to preserve mean brightness for a long period of time. Brightness Preserving Bi-Histogram Equalization (BBHE) is the earliest approach to working on this^[3]. The input histogram is divided into two sub-histograms according to the mean brightness, and then each sub-histogram is equalized. Other bi-histogram equalization algorithms, such as Dualistic Sub-Image Histogram Equalization (DSIHE)^[4] and Minimum Mean Brightness Error Bi-Histograms Equalization (MMBEBHE)^[5], are similar to BBHE, except that the division thresholds are different.

Since BBHE separates the histogram only once, it is not suitable for applications that require higher brightness preservation. To better preserve the brightness, the Recursive Mean-Separate Histogram Equalization (RMSHE) divides the input histogram based on average brightness^[6], which is like applying BBHE on the original histogram and its sub-histograms. The subsequent algorithms are similar to RMSHE except for different thresholds^[7].

To suppress the inherent side-effects of the above algorithms, Dynamic Histogram Equalization (DHE) is presented^[8], in which the input histogram is divided into multiple bins according to the local minimum, and then a new range is allocated to each bin. The improved algorithms of DHE are similar except that the division threshold is set as the local maxima^[9], and different smoothing approaches are adopted^[10-11].

The sub-histogram equalization algorithm can effectively maintain the average brightness, but it is not easy to control the enhancement level. So clipping histogram spikes^[12-15] and non-linear transformations^[16-19] are often adopted to deal with these deficiencies. The latest researches include Quadri-HE with Limited Contrast (QHELC)^[2] and Adaptive HE using Modified Probability Density Function^[16].

The above algorithms have been proven to be effective in some specific applications. However, they cannot balance feature preservation and contrast enhancement well. To deal with imbalance between these two indicators, a new method processing the foreground pixels and background pixels independently is proposed in this paper, in which the weighted coupling of HE and the Laplace transform are applied to the foreground to preserve features while improving contrast. Only HE is adopted to maximize the background contrast because of a more uniform distribution on it.

The contributions are as follows: 1) The foreground and background pixels are processed independently according to the different uniformity of gray-level distribution; 2) The weighted coupling of HE and Laplace transform is applied to obtain the balance between contrast and features preservation.

1 Proposed Algorithm

The procedure of the proposed algorithm is depicted in Fig. 1, including gamma transformation, Otsu algorithm, sub-histograms equalization, and weighted coupled HE with Laplace transform.

Gamma transformation uses a nonlinear transformation to reduce or enhance the gray value of overexposed or too dark areas in the image, so as to enhance the image's overall details. It has been proven effective in making the output histogram uniform distribution^[17-18]. So it is adopted to restrict the enhancement rate. It is defined as

$ h_{\text {output }}=c h^\gamma $

(1)

where c and γ are constants, h and h_output are original and smoothed histograms, respectively.

The Otsu algorithm^[20] is an algorithm that determines the threshold of image binarization segmentation. It is also called the maximum between-class variance method. It divides the image into the background and foreground, according to the gray-scale characteristics of the image. The higher the inter-class variance between the background and the foreground is, the greater the difference between the two parts of the image becomes. When part of the foreground is misclassified into the background or part of the background is misclassified into the foreground, the difference between the two parts will become smaller. Therefore, the partition that maximizes the variance between classes means the least probability of misclassification.

Assuming there exists an image I(x, y), the segmentation threshold of its foreground and background is defined as T, the proportion of pixels belonging to the foreground in the entire image is ω₀ and its average gray scale is μ₀, the proportion of background pixels in the entire image is ω₁, and its average gray level is μ₁, the total average gray level of the image is regarded as μ, and the variance between classes is considered to be g. Assuming that the background of the image is dark, the size of the image is M×N, the number of pixels in the image whose gray value is less than the threshold T is recorded as N₀, the number of pixels whose gray value is greater than the threshold T is recorded as N₁. Then there are

$ \omega_0=N_0 /(M \times N) $

(2)

$ \omega_1=N_1 /(M \times N) $

(3)

$ N_0+N_1=M \times N $

(4)

$ \omega_0+\omega_1=1 $

(5)

$ \mu=\omega_0 \mu_0+\omega_1 \mu_1 $

(6)

$ g=\omega_0\left(\mu_0-\mu\right)^2+\omega_1\left(\mu_1-\mu\right)^2 $

(7)

Substituting Eq. (6) into Eq. (7), the equivalent equation is obtained:

$ g=\omega_0 \omega_1\left(\mu_0-\mu_1\right)^2 $

(8)

Eq. (8) is the inter-classes variance. The threshold T that maximizes the g in Eq. (8) can be obtained by the traversal method. Thus, the Otsu algorithm has more advantages than the traditional threshold segmentation method in the binarization application. This claim can be illustrated by Figs. 2-4, the original images shown in Fig. 2 were processed by the Otsu and the traditional threshold segmentation methods, respectively. In comparison with Fig. 4, Fig. 3 shows that the former is better than the latter.

Therefore, the Otsu algorithm was used to divide the input histogram into the background histogram and the foreground histogram by minimizing the intra-class variance and maximizing the inter-class variance.The threshold T is

$ T=\arg _{0 \leqslant T \leqslant L-1} \max \left(\operatorname{Var}_{\text {inter }} / \operatorname{Var}_{\text {intra }}\right) $

(9)

where T is the division threshold, Var_intra is the intra-class variance, Var_inter is the inter-class variance.

It should be pointed out that the Otsu threshold might make the division of the foreground and background of the image not very accurate, so this may have a certain negative impact on the proposed algorithm. Fortunately, the subsequent experiments show that the negative impact is feeble.

For example, the original histogram is divided into ranges [0, T] and [T+1, L-1] using the Otsu algorithm. To better preserve mean brightness, the foreground histogram is further divided by its median value; so the range [T+1, L-1] is split into [T+1, median] and [median, L-1], then these sub-histograms are equalized independently.For the proposed algorithm, the weighted Laplace was used to balance contrast and feature preservation. In addition, the Laplace transform has been shown to be independent of brightness^[11]. There is

$ I_0=\lambda I_{\mathrm{HE}}+(1-\lambda) I_{\text {Laplace }} $

(10)

where I_HE and I_Laplace are the HE results and the Laplace transform result, respectively, λ is the weighting factor.

2 Results and Discussion

To evaluate the performance of the proposed algorithm, experiments were conducted on 100 images acquired from CVG-UGR or USC-SIPI image database. Due to the article length restrictions, only 3 samples are exhibited and compared with state-of-the-art algorithms, such as ESIHE (Exposure Based Sub Image Histogram Equalization)^[14], MMSICHE (Median-Mean Based Sub Image Clipped Histogram Equalization)^[15], FPBHE (Feature-Preserving Bi-Histogram Equalization)^[17], BHEMHB (Bi-Histogram Equalization Using Modified Histogram Bins)^[19], and BPASHE (Brightness Preserving Adaptive Sub-Histogram Equalization)^[9]. The contrast, features quantities, noise restraint level, and brightness preservation were evaluated by the contrast, entropy, peak signal-to-noise-ratio (PSNR), and absolute mean brightness error (AMBE), respectively.

2.1 Parameters Determination

Since gamma transformation is a very popular histogram smoothing algorithm, the previous literature on the choice of parameters mainly focused on γ.The choice of parameter c is often ignored, or the default value of 1 is taken. Therefore, c is set as 1 in this work as well.

As for selecting parameter γ in Eq. (1), some literatures regards it as the key technical means to reduce the loss of detailed information. Since Eq. (1) is only used to preprocess the histogram in this paper, there is no need for adaptive selection of this parameter. To simplify the process, a constant value will be taken for γ in the paper. After many experiments, it is found that the result is the best when γ = 0.2.

The parameter λ in Eq. (10) is a weighted factor, which can balance the contrast enhancement degree of the image and the enhancement degree of the detailed information.To balance contrast and information preservation well, the parameter λ in Eq. (10) is determined by the ratio of normalized contrast and entropy. There is

$ \text { ratio }=\frac{\text { contrast }_{\text {normalized }}}{\text { entropy }_{\text {normalized }}} $

(11)

Experiments are conducted on the image databases. Results show that λ can be determined by the ratio range. There is

$ \lambda=\left\{\begin{array}{l} 1.0, \text { if ratio } \leqslant 0.5 \\ 0.9, \text { if } 0.5<\text { ratio } \leqslant 0.8 \\ 0.8, \text { if } 0.8<\text { ratio } \leqslant 1.2 \\ 0.7, \text { if } 1.2<\text { ratio } \leqslant 1.5 \\ 0.6, \text { if ratio }>1.5 \end{array}\right. $

(12)

As illustrated in Eq. (11) and Eq. (12), when the ratio is large, the image contrast is high, and the information amount of all the details in the image will be weakened. The main goal is to increase the information amount so that the λ will take a small value and vice versa.

The foreground is usually a bit more informative, so to enhance the detail and contrast simultaneously, the coefficient λ is essential. Eq. (11) indicates that the information and contrast ratio of the foreground are different. Therefore, the λ for the foreground should be assigned different values to achieve an excellent overall performance. Since the background pixels usually contain less information, it is not necessary to enhance the image details. In this case, it is reasonable to take λ=1 to ensure the maximum contrast.

In short, the λ in the foreground histogram in this paper is a variable and can be automatically selected, but the λ in the background is defined as 1; when λ=1 or λ=0, it means that only HE or Laplace transform is adopted.

2.2 Visual Performances

The visual performances of the three samples are respectively depicted in Figs. 5-7. As shown in Fig. 5, the image "F16" is globally over-enhanced by HE and BHEMHB, and the distortion has been introduced, which makes it look unnatural. The reason is that both HE and BHEMHB divide the original histogram only once. This inadequate division tends to result in the average brightness shift and distortion, although this scheme can achieve higher contrast.

As to ESIHE, it divides the image into under-exposure and over-exposure regions by an exposure threshold. It has been verified to be effective in enhancing images under low illumination conditions, but it tends to make the brighter areas too intense, such as the bright region in the lower right corner of the image in Fig. 5. In respect of MMSICHE, it divides the original histogram into four bins through the median and mean values in this paper. Apart from this, histogram peak clipping is also adopted. Although sufficient histogram segmentation and peak clipping techniques make the mean brightness well preserved, the low-frequency sub-histograms are not effectively protected, which leads to uneven changes in the gray levels of the enhanced image.

With regard to BPASHE, the enhancement effect depends on the selection of the radius around the local maxima, but this parameter selection is difficult for users. Compared with the above algorithms, the enhancement results of FPBHE are more effective, which is second only to the proposed one in this paper. Unlike MMSICHE, in addition to the histogram division and peak clipping techniques, the histogram addition technique was also conducted to protect low-frequency gray-levels to preserve detailed information. Although the bright region in the lower right corner of the image "F16" enhanced by FPBHE in Fig. 5 is slightly over-enhanced, the degree of over-enhancement is much lower than other algorithms. It can be seen from Fig. 5 that the proposed algorithm produces the most natural appearance. The reason is the Laplace transform does not change the mean brightness.

As depicted in Fig. 6, the "TANK" images processed by HE, ESIHE, and BHEMHB are over-enhanced globally so that the tanks become too dark and the grassland is too intense. In comparison to HE and BHEMHB, details of the image enhanced by ESIHE are slightly richer, but the preservation level is still not enough. Like the enhancement effect of the ESIHE on the "F16" in Fig. 5, the brighter areas on the grassland of "TANK" are also overexposed by ESIHE.

Since MMSICHE does not protect low-frequency gray-levels from being excessively merged, the details are unavoidably lost. Fig. 6 indicates that it overexposes some brighter regions on the grassland, resulting in uneven gray-levels distribution. Besides, MMSICHE causes the loss of details, especially in the turret area of the "TANK". The BPASHE algorithm, it also leads to the loss of detail in the turret area of the "TANK". However, the loss caused by BPASHE is slightly smaller than that caused by MMSICHE. Finally, Fig. 6 shows that both FPBHE and the proposed algorithm produce similar results, but the gray-levels distribution produced by the proposed algorithm tends to be more uniform than FPBHE.

Fig. 7 indicates that both HE and BHEMHB also over-enhance "Field" images globally besides "F16" and "TANK" images. As a result, they make the grassland become intense and lose some features of the grassland. In respect of ESIHE, Fig. 7 shows that it overexposes the input image as well, especially the bright regions on the grassland. However the distortion level is slightly lower than those caused by algorithms HE and BHEMHB algorithms.

Fig. 7 illustrates that MMSICHE locally over-enhances some brighter areas on the grassland. The gray levels in these regions are low-frequency ones. The advantage of MMSICHE is that the original histogram is well segmented, and the histogram peak is effectively clipped. Still, the low-frequency sub-histograms are not protected well, unavoidably leading to the loss of detail. Besides, the effects of BPASHE depend on the radius around the local maxima. The smaller the radius, the smaller the improvements in contrast with the original image. Fig. 5 illustrates the results of BPAHE introducing artifacts. Compared with the above algorithms, FPBHE and the proposed algorithm produce similar results and outperform other algorithms. The FPBHE, sufficient division, gamma transformation, and histogram addition make its enhancement results comparable to the proposed approach.

3 Evaluations

Evaluation functions and their results are exhibited in this section. To make it more readable, the best items are bolded in the following tables, and the suboptimal items are underlined.

(a) Evaluation of contrast

Contrast evaluation function is defined as

$ C=\frac{1}{W H} \sum\limits_{u=1}^W \sum\limits_{v=1}^H I_{(u, v)}^2-\left(\frac{1}{W H} \sum\limits_{u=1}^W \sum\limits_{v=1}^H I_{(u, v)}^2\right)^2 $

(13)

where W and H are the width and height of the image, respectively, and I(u, v) is the gray-level of the output image at the position (u, v). As to the coefficient C, it is expected that it can take a larger value, so that dynamic range of gray-level is fully exploited.

The contrast of the sample images are listed in Table 1. It indicates that the proposed algorithm effectively improves the contrast of the original images. The contrast of HE and BHEMHB is better than other algorithms, but both introduce severe distortions with respect to visual performance. The reason is that HE and BHEMHB divide histograms only once. In this way, the original histogram is not divided sufficiently, and the high-frequency sub-histograms cannot be efficiently suppressed. In order to obtain a comprehensive evaluation, both the contrast and the visual performance will be discussed in the following sections. In a word, a conclusion can be made that the proposed algorithm produces the most natural appearances, although the contrast is not always optimal.

(b) Evaluation of information quantities

The entropy was adopted to evaluate the information quantities contained in an image. Entropy is defined by Eq. (14). The greater the entropy is, the richer the information and content of the images will contain.

$ \text { Entropy }=\sum\limits_{i=0}^{L-1}-p(i) \log _2 p(i) $

(14)

It can be found that the entropies of these images processed by the proposed algorithm are always the highest among the algorithms in Table 2, and sometimes even greater than those in the original input images. This achievement is mainly attributed to the detail enhancement produced by the Laplace transform. Obviously, it can be inferred that the image processed by the proposed algorithm contains the most details. As far as entropy is concerned, FPBHE ranks second in Table 2, which illustrates its superiority in preserving detailed features. Histogram addition is adopted by FPBHE to prevent low-frequency gray-levels from being excessively merged, and successfully reduces the features loss. But the feature quantities are unavoidably lower than the input image. In summary, the proposed algorithm can be applied to the fields that require richer details, such as medical images.

(c) Evaluation of noise restraint level

The PSNR in Eq. (15) is designed to evaluate the noise level of the image. The higher the PSNR is, the better the noise suppression becomes.

$ \operatorname{PSNR}=10 \times \log _{10}\left(\frac{255^2}{\mathrm{MSE}}\right) $

(15)

The MSE defined as Eq. (16) reflects the mean square error between the input and the output image.

$ \mathrm{MSE}=\frac{1}{W H} \sum\limits_{x=1}^W \sum\limits_{y=1}^H[f(x, y)-g(x, y)]^2 $

(16)

where f(x, y) and g(x, y) represent the input image and the output image at the position (x, y), respectively.

Table 3 indicates that the PSNR of the image processed by the proposed algorithm always ranks top 2. Although the PSNR of the "Field" processed by the proposed algorithm is less than that of BPASHE, the average value is much higher than BPASHE. It should be pointed out that the enhancement result of BPASHE depends too much on the choice of thresholds, and the adjustment of the threshold is difficult. Therefore, the proposed algorithm is robust to noise.

(d) Evaluation of brightness preservation

AMBE was employed to measure the mean brightness preserving level. The smaller the AMBE is, the better the suppression of the brightness shift becomes. There is

$ \operatorname{AMBE}=|E(x)-E(y)| $

(17)

where E(x) and E(y) are the mean brightness of the input and the output image, respectively.

The results are depicted in Table 4. As shown, the AMBE of the proposed algorithm is always in the top 2. Although the AMBE value of "Field" is slightly worse than ESIHE, the average value of the proposed algorithm is much better than that of ESIHE. The results show that the MMSICHE has the same ability as the proposed algorithm in preserving mean brightness. MMSICHE divides the input histogram into multiple sub-histograms, and each sub-histogram is equalized separately. Further histogram division makes the mean brightness well maintained. In general, the proposed algorithm shows its excellent restraining capability of the mean brightness shift and saturation effects. The reason is that the Laplace transform does not change the mean brightness and saturation while enhancing the detail information. Therefore, it can be used in consumer electronic products.

(e) Mean value of test images

In order to verify the effectiveness of the proposed algorithm, evaluations were conducted on 100 images according to different indicators. The average results are listed in Table 5.

Table 5 illustrates that MMSICHE performs the best on the meaning brightness preservation. As discussed above, MMSICHE divides the input histogram into multiple sub-histograms, and then each sub-histogram is equalized separately. This further histogram segmentation makes the mean brightness of the image well preserved. The disadvantage of MMSICHE is the loss of details and uneven gray-level distribution because low-frequency gray levels are not effectively protected. Similarly, FPBHE also includes two parts: sub-histogram equalization and enhancement rate control. Compared with MMSICHE, FPBHE successfully reduces the feature loss because histogram addition is used to prevent low-frequency gray-levels from being excessively merged. So the overall performance of FPBHE illustrated in Table 5 is better than MSICHE. It can be seen from Table 5 that the average entropy generated by the proposed algorithm is 0.056 higher than that produced by the FPBHE algorithm. It is even higher than that in the original image. It is reasonable because Laplace transform has an enhanced effect on image edge information. In addition, the average PSNR from the proposed algorithm are 32.468 and is 4.221 higher than that of the 2nd-rank algorithm, namely, MMSICHE

In terms of the average contrast, the proposed algorithm is slightly inferior to other algorithms, but it has been improved by 0.971 over the original result by the proposed algorithm. In short, Figs. 5-7 verifies that the proposed algorithm produces the most natural appearances.

As far as the average AMBE is concerned, the result of the proposed algorithm is slightly less than that of MMSICHE. However, the entropy, PSNR, and the enhanced appearances from the former are all superior to the latter.

In combination with the visual performances section, it can be concluded that the proposed algorithm not only effectively preserves the detailed features but also improves the overall effect.

4 Conclusions

An image contrast enhancement algorithm was proposed and analyzed. Experimental results show that the proposed algorithm can effectively suppress the noise and intensity saturation generated by the traditional HE methods. In addition, indicators such as contrast, entropy, PSNR, and AMBE produced by the proposed algorithm are superior to the state-of-the-art ones. Their respective averages have also been improved by 0.971, 0.056, 4.221, and 0.523 over the images processed by suboptimal algorithms. Furthermore, the entropy in the images processed by the new approach is higher than that in the original images. In a word, both features preserving level and overall performance are greatly improved by the proposed method.

Due to the above advantages, the proposed technique can be used in the fields that require richer details or saturation effects restraint, such as consumer electronic products. Besides, the weighted coupling of HE and Laplace transform can solve the problems existing in traditional HE very well and facilitate further image processing.