iRahul.com :: Image Stitch

XML Schema Trees (XST)

Scalable Integration of XML Schemas

FXCounter

Adaptive Learning Systems

Mobile Devices

GSM+SMS = GSMS

Java FTP

openXMPP

MPlus

Image Stitch

openGL

VRML

Software Project Management

Software Testing

Image mosaic - Stitch

Download Source (License: GNU GPL) - April 14, 2002

1. Introduction

This project aims at designing software for creating image mosaic. Image registration/mosaic is a technique for creating images which cannot be created by a single frame of the camera, for example satellite imagery.
The user gives a series of pictures with overlapping regions and the software returns one large image with all the pictures merged as accurately as possible.

2. Tools used for the project

1. Intel openCV library for C
2. MATLAB for calculating affine transformation matrix
3. GTK library for GUI
4. Linux/Unix Platform
5. GCC compiler for C

3. Assumptions

The following are some of the assumptions we made when we solved the problem.

1. The input images given by the user should have at least 10% overlap for good results.
2. The user selects the corresponding points in the two images fairly accurately for good results.
3. If the user does not get good results in the first time then user needs to try again with new points for better accuracy.
4. The user selects 2-3 points that are not close to each other for better results.
5. The brightness of the images given by the user should not differ too much.
6. The user gives the names of the images on command line in the order of the sequential order of overlapping.

4. Our method of solving the problem

Program Flowchart is as follows:

Program usage at command line is as follows

Usage: stitch <num of images> <image names...> [options]

          -b Enable Averaging on common area upon merging, note that this reduces 
          quality if more than two images: Default Disabled.

          -i Disable Interpolation in merged image: Default Enabled.

          -e Disable Edge Blurring of merged image (For 2 Images only): Default 
          Enabled.

          -z If given second image come on top of first: Deafult (Goes Behind)

          -F Specify output filename, it MUST end with .jpg: Default is output.jpg

          Example: stitch 2 picA.jpg picB.jpg -F merged.jpg -b -e -i

1. The user inputs on the command line command like the following

stitch 4 a.jpg b.jpg c.jpg d.jpg

2. Program then loads the images.
3. User is presented with the first two images with suggested good points marked.
4. User then selects the corresponding points after which MATLAB is used to calculate the affine transformation matrix.

Where x1, y1… xn, yn are points in the first image and x'1, y'1… x'n, y'n are the corresponding points in the second image.
MATLAB is used to calculate the values of a11, a12, a13, a21, a22, a23.

5. Then the affine transformation is applied to the second image and the two images are merged. During merging depending on the user's choice the features are applied (section 5).
6. If more than two images are there then the process is repeated.

5. Features

The following features are available in our program. Some features can be selected at the command line.

5.1 Color and Size

The input images given by the user can be of any size and our implementation can candle color images as well.

5.2 Good features detection

The Stitch program uses the cvGoodFeatureToTrack to show the users some points in both images simultaneously which are good features. This helps the user in getting a better idea of good corresponding points in the two images. The points are marked in green color.

5.3 Manual Point Selection

Our Implementation of the program enables manual selection of points by the user. Users are asked to select corresponding points on two images shown on our user friendly GUI. To minimize the error in points selected we take 9 close points for every selection the user makes for our affine transformation calculation.

5.4 Interpolation

After putting the second image into the final image some pixels are missing because of the affine transformation as more than one pixel of the original image may transform to one pixel in the transformed image. We use a weighted bilinear interpolation method to fill up the color values at these points. For every missing pixel we take the closest neighbors and assign a weighted average color value to the missing point based on the neighbors' color values. So the farthest the neighbor from the missing pixel the least will be its contribution in the average value.
The -i switch can disable this option

Bilinear interpolation method used is illustrated as follows:

With Interpolation

Without Interpolation

5.5 Stitching any number of Images

Stitch can be used to merge any number of overlapping images. The program loops until it creates the image mosaic of the specified number of pictures. The first two images are merged and the third is merged with the result of the these and so on.

5.6 Averaging overlap Area

The -b switch can select this option and overlapping area of the images is then blurred. However we noticed it does not give pleasing results especially if more than two images are used.

5.7 Blur Edge (First two images only)

Sometimes when two images are merged the boundary of the two images look obviously stitched. In some cases it helps to blur the image boundary for the two images to give a smoother view of the registration and to make it less obvious to the user that the images were indeed stitched. This works for most cases but may not be proper where the image stitching is so perfect that there is no need to give blurring effect. So we decided to put this feature as an option which can be disabled by the user. The -e switch can disable this option and it blurs the area of join for the first two images. In the current implementation it only works for the first two images.
A window size of 3x3 is used for taking the average the edge where the two images overlap.

6. Sample Results
The following are some of our best results.