In computer vision
, sets of data
acquired by sampling the same scene or object at different times, or from different perspectives, will be in different coordinate systems. Image registration
is the process of transforming the different sets of data into one coordinate system. Registration is necessary in order to be able to compare or integrate the data obtained from different measurements.
Medical image registration (e.g. for data of the same patient taken at different points in time) often additionally involves elastic (or nonrigid) registration to cope with deformation of the subject (due to breathing, anatomical changes, etc.). Nonrigid registration of medical images can also be used to register a patient's data to an anatomical atlas, such as the Talairach atlas for neuroimaging.
Area-based vs Feature-based
algorithms fall within two realms of classification: area based methods and feature based methods. The original image is often referred to as the reference image and the image to be mapped onto the reference image is referred to as the target image. For area based image registration methods, the algorithm looks at the structure of the image via correlation metrics, Fourier properties and other means of structural analysis. Alternatively, most feature based methods, instead of looking at the overall structure of images, fine tune their mappings to the correlation of image features: lines, curves, points,
line intersections, boundaries, etc.
algorithms can also be classified according to the transformation model used to relate the reference image space with the target image space. The first broad category of transformation models includes linear transformations
, which are a combination of translation, rotation, global scaling, shear and perspective components. Linear transformations
are global in nature, thus not being able to model local deformations. Usually, perspective components are not needed for registration, so that in this case the linear transformation is an affine
The second category includes 'elastic' or 'nonrigid' transformations. These transformations allow local warping of image features, thus providing support for local deformations. Nonrigid transformation approaches include polynomial wrapping, interpolation of smooth basis functions (thin-plate splines and wavelets), and physical continuum models (viscous fluid models and large deformation diffeomorphisms).
Search-based vs direct methods
methods can also be classified in terms of the type of search that is needed to compute the transformation between the two image domains. In search-based methods the effect of different image deformations is evaluated and compared.
Many image registration
methods operate in the spatial domain, using features, structures, and textures as matching criteria. In the spatial domain, images look 'normal' as the human eye might perceive them. Some of the feature matching algorithms are outgrowths of traditional techniques for performing manual image registration
, in which operators choose matching sets of control points
(CPs) between images. When the number of control points exceeds the minimum required to define the appropriate transformation model, iterative algorithms like RANSAC
are used to robustly estimate the best solution.
Other algorithms use the properties of the frequency-domain to directly determine shifts between two images. Applying the Phase correlation
method to a pair of overlapping images produces a third image which contains a single peak. The location of this peak corresponds to the relative translation between the two images. Unlike many spatial-domain algorithms, the phase correlation method is resilient to noise, occlusions, and other defects typical of medical or satellite images. Additionally, the phase correlation uses the Fast fourier transform
to compute the cross-correlation between the two images, generally resulting in large performance gains. The method can be extended to determine affine
rotation and scaling between two images by first converting the images to log-polar coordinates. Due to properties of the Fourier transform
, the rotation and scaling parameters can be determined in a manner invariant to translation. This single feature makes phase-correlation methods highly attractive vs. typical spatial methods, which must determine rotation, scaling, and translation simultaneously, often at the cost of reduced precision in all three.
Another useful classification is between single-modality and multi-modality registration algorithms. Single-modality registration algorithms are those intended to register images of the same modality (i.e. acquired using the same kind of imaging device), while multi-modality registration algorithms are those intended to register images acquired using different imaging devices.
There are several examples of multi-modality registration algorithms in the medical imaging field. Examples include registration of brain CT/MRI images or whole body PET/CT images for tumor localization, registration of contrast-enhanced CT images against non-contrast-enhanced CT images for segmentation of specific parts of the anatomy and registration of ultrasound and CT images for prostate localization in radiotherapy.
Further ways of classifying an algorithm consist of the amount of data it is optimized to handle, the algorithm's application, and the central theory the algorithm is based around. Image registration has applications in remote sensing (cartography updating), medical imaging
(change detection, tumor monitoring), and computer vision. Due to the vast applications to which image registration can be applied, it's impossible to develop a general algorithm optimized for all uses.
Image similarity-based methods
Image similarity-based methods are broadly used in medical imaging
. A basic image similarity-based method consists of a transformation model
, which is applied to reference image coordinates to locate their corresponding coordinates
in the target image space, an image similarity metric, which quantifies the degree of correspondence between features in both image spaces achieved by a given transformation, and an optimization algorithm
, which tries to maximize image similarity by changing the transformation parameters.
The choice of an image similarity measure depends on the nature of the images to be registered. Common examples of image similarity measures include cross-correlation, mutual information, sum of square differences and ratio image uniformity. Mutual information and its variant, normalized mutual information, are the most popular image similarity measures for registration of multimodality images. Cross-correlation, sum of square differences and ratio image uniformity are commonly used for registration of images of the same modality.
Open source software
- B. Glocker, N. Komodakis, G. Tziritas, N. Navab, N. Paragios: Dense Image Registration through MRFs and Efficient Linear Programming Medical Image Analysis, (in press), 2008.
- Barbara Zitová, Jan Flusser: Image registration methods: a survey Image Vision Comput. 21(11): 977-1000 (2003)
- Jan Modersitzki: Numerical Methods for Image Registration, Oxford University Press, 2004.
- Crum WR, Griffin LD, Hill DL, Hawkes DJ: Zen and the art of medical image registration: correspondence, homology, and quality Neuroimage, Vol. 20, No. 3. (November 2003), pp. 1425-1437.
- Ardeshir Goshtasby: 2-D and 3-D Image Registration for Medical, Remote Sensing, and Industrial Applications, Wiley Press, 2005.