Under the SMART Phase II project ANDRO developed an autonomous, expert system based toolkit for the spatial alignment of multi-sensor images including synthetic aperture radar (SAR), electro-optic (EO), infared (IR), multi-spectral (MSI), and hyperspectral (HIS) data collected from the battle space. SMART exploits geo-referenced invariant features together with other salient features that can be extracted from correlated across multiple sensor types to provide relative and absolute registration. The focus was on performing near real time automatic geo-registration of multiple images from different sensors in order to enhance the ability of military systems to identify targets of opportunity with greater accuracy and confidence and to support decision making for precision target geo-location applications.
The SMART expert system provides a pre-screening algorithm that can determine the best algorithm(s) to use for registration. The expert system takes into consideration the image types, features within the image, support data, and overall computation time.
Three basic classes of registration algorithms were included in SMART:
Feature-based methods are based on matching salient features including feature points, contours, and regions.
Fourier-based methods work in the frequency domain and utilize the translation and rotational property of Fourier transforms.
Intensity based methods use the raw intensities to perform registration.
The Mutual information intensity based method is very successful with multi-modal images. Mutual information has its roots in information theory, where it was developed to set fundamental limits on the performance of communications systems.
SMART determines the quality of the registration process using both the Pearson Product Moment (pixel value correlation) and Mutual Information similarity metrics.