CS180: Prokudin-Gorskii Colorization Project

Project Overview

This project is part of CS180: Intro to Computer Vision and Computational Photography. The goal is to colorize the Prokudin-Gorskii photo collection by aligning the red, green, and blue channels of each image.

My Approach

For the alignment of TIF images, only the Normalized Cross-Correlation (NCC) algorithm was used, with a pyramid structure to optimize computational efficiency. For the alignment of JPG images, both the Normalized Cross-Correlation (NCC) and Euclidean distance methods were employed as a comparison, without the use of a pyramid structure, but rather using a single-scale approach.

Results

-Pyramid vs NCC)

Cathedral

L2
Cathedral L2 NoPyramid

g_shift: (1, -1), r_shift: (7, -1)

NCC
Cathedral NCC NoPyramid

g_shift: (1, -1), r_shift: (7, -1)

Monastery

L2
Monastery L2 NoPyramid

g_shift: (-6, 0), r_shift: (9, 1)

NCC
Monastery NCC NoPyramid

g_shift: (-6, 0), r_shift: (9, 1)

Tobolsk

L2
Tobolsk L2 NoPyramid

g_shift: (3, 2), r_shift: (6, 3)

NCC
Tobolsk NCC NoPyramid

g_shift: (3, 2), r_shift: (6, 3)

TIF Images (Pyramid)

Church

Church L2 Pyramid

g_shift: (0, -5), r_shift: (52, -6)

Emir

Emir L2 Pyramid

g_shift: (-3, 7), r_shift: (108, -1043)

Harvesters

Harvesters L2 Pyramid

g_shift: (118, -3), r_shift: (120, 7)

Icon

Icon L2 Pyramid

g_shift: (42, 16), r_shift: (89, 22)

Lady

Lady L2 Pyramid

g_shift: (57, -6), r_shift: (123, -17)

Melons

Melons L2 Pyramid

g_shift: (83, 4), r_shift: (176, 7)

Onion Church

Onion Church L2 Pyramid

g_shift: (52, 22), r_shift: (108, 35)

Sculpture

Sculpture L2 Pyramid

g_shift: (33, -11), r_shift: (140, -26)

Self Portrait

Self Portrait L2 Pyramid

g_shift: (50, -2), r_shift: (130, -5)

Three Generations

Three Generations L2 Pyramid

g_shift: (52, 5), r_shift: (108, 7)

Train

Train L2 Pyramid

g_shift: (111, -7), r_shift: (107, 1)

Bells & Whistles

I implemented additional features such as automatic cropping and contrast adjustment to improve the final image quality.

It was observed that simple alignment based on RGB similarity performs reasonably well for most landscape images but produces poor results when applied to some images containing people. Therefore, we attempted to use gradients or edges by introducing the Sobel filter, which significantly improved the alignment results. This improvement is due to the fact that images containing people often feature complex details and color variations. Direct alignment using the original images without edge detection is prone to noise and detail interference, leading to inaccuracies. The Sobel filter calculates the gradients in the horizontal and vertical directions, emphasizing the edge information in the images, thus making some alignment process more precise.

Comparison: Original vs Sobel Filter

Emir (NCC Pyramid)

Original
Emir Original NCC Pyramid

g_shift: (-3, 7), r_shift: (108, -1043)

Sobel Filter
Emir Sobel NCC Pyramid

g_shift: (-3, 7), r_shift: (108, -1043)

Automatic cropping and automatic contrast adjustment

To enhance the visual quality of the final images, I also added automatic cropping and automatic contrast adjustment. This ensures a better overall visual effect, making the aligned images more appealing and precise.

Monastery (NCC NoPyramid)

Original
Monastery NCC NoPyramid

g_shift: (-6, 0), r_shift: (9, 1)

Sobel Filter
Monastery Sobel NCC NoPyramid

g_shift: (-3, 0), r_shift: (3, 1)