python - Automatic contrast and brightness adjustment of a color photo of a sheet of paper with OpenCV

Question

When photographing a sheet of paper (e.g. with phone camera), I get the following result (left image) (jpg download here). The desired result (processed manually with an image editing software) is on the right:

I would like to process the original image with openCV to get a better brightness/contrast automatically (so that the background is more white).

Assumption: the image has an A4 portrait format (we don't need to perspective-warp it in this topic here), and the sheet of paper is white with possibly text/images in black or colors.

What I've tried so far:

1. Various adaptive thresholding methods such as Gaussian, OTSU (see OpenCV doc Image Thresholding). It usually works well with OTSU:

   ret, gray = cv2.threshold(img, 0, 255, cv2.THRESH_OTSU + cv2.THRESH_BINARY)
   

   but it only works for grayscale images and not directly for color images. Moreover, the output is binary (white or black), which I don't want: I prefer to keep a color non-binary image as output

2. Histogram equalization

   • applied on Y (after RGB => YUV transform)
   • or applied on V (after RGB => HSV transform),

   as suggested by this answer (Histogram equalization not working on color image - OpenCV) or this one (OpenCV Python equalizeHist colored image):

   img3 = cv2.imread(f)
   img_transf = cv2.cvtColor(img3, cv2.COLOR_BGR2YUV)
   img_transf[:,:,0] = cv2.equalizeHist(img_transf[:,:,0])
   img4 = cv2.cvtColor(img_transf, cv2.COLOR_YUV2BGR)
   cv2.imwrite('test.jpg', img4)
   

   or with HSV:

   img_transf = cv2.cvtColor(img3, cv2.COLOR_BGR2HSV)
   img_transf[:,:,2] = cv2.equalizeHist(img_transf[:,:,2])
   img4 = cv2.cvtColor(img_transf, cv2.COLOR_HSV2BGR)
   

   Unfortunately, the result is quite bad since it creates awful micro contrasts locally (?):

   

   I also tried YCbCr instead, and it was similar.

3. I also tried CLAHE (Contrast Limited Adaptive Histogram Equalization) with various tileGridSize from 1 to 1000:

   img3 = cv2.imread(f)
   img_transf = cv2.cvtColor(img3, cv2.COLOR_BGR2HSV)
   clahe = cv2.createCLAHE(tileGridSize=(100,100))
   img_transf[:,:,2] = clahe.apply(img_transf[:,:,2])
   img4 = cv2.cvtColor(img_transf, cv2.COLOR_HSV2BGR)
   cv2.imwrite('test.jpg', img4)
   

   but the result was equally awful too.

4. Doing this CLAHE method with LAB color space, as suggested in the question How to apply CLAHE on RGB color images:

   import cv2, numpy as np
   bgr = cv2.imread('_example.jpg')
   lab = cv2.cvtColor(bgr, cv2.COLOR_BGR2LAB)
   lab_planes = cv2.split(lab)
   clahe = cv2.createCLAHE(clipLimit=2.0,tileGridSize=(100,100))
   lab_planes[0] = clahe.apply(lab_planes[0])
   lab = cv2.merge(lab_planes)
   bgr = cv2.cvtColor(lab, cv2.COLOR_LAB2BGR)
   cv2.imwrite('_example111.jpg', bgr)
   

   gave bad result too. Output image:

   

5. Do an adaptive thresholding or histogram equalization separately on each channel (R, G, B) is not an option since it would mess with the color balance, as explained here.

6. "Contrast strechting" method from scikit-image's tutorial on Histogram Equalization:

   the image is rescaled to include all intensities that fall within the 2nd and 98th percentiles

   is a little bit better, but still far from the desired result (see image on top of this question).

---

TL;DR: how to get an automatic brightness/contrast optimization of a color photo of a sheet of paper with OpenCV/Python? What kind of thresholding/histogram equalization/other technique could be used?

Answer 1

Contrast and brightness can be adjusted using alpha (α) and beta (β), respectively. These variables are often called the gain and bias parameters. The expression can be written as

OpenCV already implements this as cv2.convertScaleAbs() so we can just use this function with user defined alpha and beta values.

import cv2

image = cv2.imread('1.jpg')

alpha = 1.95 # Contrast control (1.0-3.0)
beta = 0 # Brightness control (0-100)

manual_result = cv2.convertScaleAbs(image, alpha=alpha, beta=beta)

cv2.imshow('original', image)
cv2.imshow('manual_result', manual_result)
cv2.waitKey()

But the question was

How to get an automatic brightness/contrast optimization of a color photo?

Essentially the question is how to automatically calculate alpha and beta. To do this, we can look at the histogram of the image. Automatic brightness and contrast optimization calculates alpha and beta so that the output range is [0...255]. We calculate the cumulative distribution to determine where color frequency is less than some threshold value (say 1%) and cut the right and left sides of the histogram. This gives us our minimum and maximum ranges. Here's a visualization of the histogram before (blue) and after clipping (orange). Notice how the more "interesting" sections of the image are more pronounced after clipping.

To calculate alpha, we take the minimum and maximum grayscale range after clipping and divide it from our desired output range of 255

α = 255 / (maximum_gray - minimum_gray)

To calculate beta, we plug it into the formula where g(i, j)=0 and f(i, j)=minimum_gray

g(i,j) = α * f(i,j) + β

which after solving results in this

β = -minimum_gray * α

For your image we get this

Alpha: 3.75

Beta: -311.25

You may have to adjust the clipping threshold value to refine results. Here's some example results using a 1% threshold with other images: Before -> After

Automated brightness and contrast code

import cv2
import numpy as np
from matplotlib import pyplot as plt

# Automatic brightness and contrast optimization with optional histogram clipping
def automatic_brightness_and_contrast(image, clip_hist_percent=1):
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    
    # Calculate grayscale histogram
    hist = cv2.calcHist([gray],[0],None,[256],[0,256])
    hist_size = len(hist)
    
    # Calculate cumulative distribution from the histogram
    accumulator = []
    accumulator.append(float(hist[0]))
    for index in range(1, hist_size):
        accumulator.append(accumulator[index -1] + float(hist[index]))
    
    # Locate points to clip
    maximum = accumulator[-1]
    clip_hist_percent *= (maximum/100.0)
    clip_hist_percent /= 2.0
    
    # Locate left cut
    minimum_gray = 0
    while accumulator[minimum_gray] < clip_hist_percent:
        minimum_gray += 1
    
    # Locate right cut
    maximum_gray = hist_size -1
    while accumulator[maximum_gray] >= (maximum - clip_hist_percent):
        maximum_gray -= 1
    
    # Calculate alpha and beta values
    alpha = 255 / (maximum_gray - minimum_gray)
    beta = -minimum_gray * alpha
    
    '''
    # Calculate new histogram with desired range and show histogram 
    new_hist = cv2.calcHist([gray],[0],None,[256],[minimum_gray,maximum_gray])
    plt.plot(hist)
    plt.plot(new_hist)
    plt.xlim([0,256])
    plt.show()
    '''

    auto_result = cv2.convertScaleAbs(image, alpha=alpha, beta=beta)
    return (auto_result, alpha, beta)

image = cv2.imread('1.jpg')
auto_result, alpha, beta = automatic_brightness_and_contrast(image)
print('alpha', alpha)
print('beta', beta)
cv2.imshow('auto_result', auto_result)
cv2.waitKey()

Result image with this code:

Results with other images using a 1% threshold

An alternative version is to add gain and bias to an image using saturation arithmetic instead of using OpenCV's cv2.convertScaleAbs(). The built-in method does not take an absolute value, which would lead to nonsensical results (e.g., a pixel at 44 with alpha = 3 and beta = -210 becomes 78 with OpenCV, when in fact it should become 0).

import cv2
import numpy as np
# from matplotlib import pyplot as plt

def convertScale(img, alpha, beta):
    """Add bias and gain to an image with saturation arithmetics. Unlike
    cv2.convertScaleAbs, it does not take an absolute value, which would lead to
    nonsensical results (e.g., a pixel at 44 with alpha = 3 and beta = -210
    becomes 78 with OpenCV, when in fact it should become 0).
    """

    new_img = img * alpha + beta
    new_img[new_img < 0] = 0
    new_img[new_img > 255] = 255
    return new_img.astype(np.uint8)

# Automatic brightness and contrast optimization with optional histogram clipping
def automatic_brightness_and_contrast(image, clip_hist_percent=25):
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

    # Calculate grayscale histogram
    hist = cv2.calcHist([gray],[0],None,[256],[0,256])
    hist_size = len(hist)

    # Calculate cumulative distribution from the histogram
    accumulator = []
    accumulator.append(float(hist[0]))
    for index in range(1, hist_size):
        accumulator.append(accumulator[index -1] + float(hist[index]))

    # Locate points to clip
    maximum = accumulator[-1]
    clip_hist_percent *= (maximum/100.0)
    clip_hist_percent /= 2.0

    # Locate left cut
    minimum_gray = 0
    while accumulator[minimum_gray] < clip_hist_percent:
        minimum_gray += 1

    # Locate right cut
    maximum_gray = hist_size -1
    while accumulator[maximum_gray] >= (maximum - clip_hist_percent):
        maximum_gray -= 1

    # Calculate alpha and beta values
    alpha = 255 / (maximum_gray - minimum_gray)
    beta = -minimum_gray * alpha

    '''
    # Calculate new histogram with desired range and show histogram 
    new_hist = cv2.calcHist([gray],[0],None,[256],[minimum_gray,maximum_gray])
    plt.plot(hist)
    plt.plot(new_hist)
    plt.xlim([0,256])
    plt.show()
    '''

    auto_result = convertScale(image, alpha=alpha, beta=beta)
    return (auto_result, alpha, beta)

image = cv2.imread('1.jpg')
auto_result, alpha, beta = automatic_brightness_and_contrast(image)
print('alpha', alpha)
print('beta', beta)
cv2.imshow('auto_result', auto_result)
cv2.imwrite('auto_result.png', auto_result)
cv2.imshow('image', image)
cv2.waitKey()