Reconstruction of the gaps in malfunction Landsat7 images: Review

1. Introduction

Generally, remote sensing images are digital images that show appearance of parts of the Earth surface as seen from space. The sensor on board of the satellite reads the amount of reflected energy transmitted from different portions on Earth to the sensor. This information is stored and converted into a picture format. Remote sensing data convey spatial components; these mean the data that identify the geographic location of the features or objects on Earth. Generally, from spatial data, the locations, sizes, and shapes of objects appearing in the Earth scene (e.g., building, lake, mountain, or township) can be extracted (1, 2). In 2003, because of the permanent failing of the Scan Line Corrector (SLC) instrument of the Enhanced Thematic Mapper plus (ETM+) sensor that located on board a Landsat 7 satellite, regular gaps appeared in Landsat 7 imagery. So, as a result of this malfunction, the scientific application of ETM+ data has been limited. The reconstruction (gap filling) approach can be expressed as the strategy that addresses the SLC malfunction and proposed algorithm to reconstruct the missing data. In practice, the objective of gap filling algorithms is to predict or estimate new data to exchange the data gap locations accurately. The main goal of such a recovering approach is to provide auxiliary images (which are used as fill imagery) to balance the efficiency and performance required in these approaches (3, 26). Mostly, single-source and multisource reconstruction approaches are the two main approaches used in gap filling algorithms.

1.1. Literature review

Gap filling approaches

Single-source reconstruction approaches. In this group, the scanned areas in the Landsat 7 fault imagery are used to reconstruct the unscanned area. In general, interpolation methods are used as mathematical functions to estimate the gap location values. Therefore, in this category, the gaps on the malfunctioning Landsat 7 image itself are recreated using either straightforward linear or nonlinear filters or spatial interpolation techniques.

Simple interpolation techniques. This method replaces the missing value by averaging the values of the surrounding pixels or extracting the neighbouring pixels (4). In SLC-off imagery, unscanned spots are filled using weighted averages of practical windows like 2 × 2 and 4 × 4 in bilinear and bicubic interpolation algorithms, such as nearest neighbor, bilinear, and others. These gap filling methods are typically straightforward and simple to use, but they produce results that are imprecise since the missing values are derived from nearby pixels rather than an anticipated amount of land reflectance (5).

Spatial interpolation techniques. Spatial interpolation is a procedure or mathematical function which is used to estimate the values of points at locations that lack measured values. It depends on the autocorrelation principle to measure the degree of relations between distant and close objects (6).

Inverse distance weight (IDW)

It is considered as a spatial interpolation approach that depends on the spatial autocorrelation among the points in the study area. This approach has been approved by geoscientists in geographic information science which depends on the hypothesis that the known nearest point has a further effect or weighting to predict or estimate the value of the unknown centre point (7, 8).

Geostatistical interpolation. The geostatistical technique consists of a group of spatial statistical methods based on Toblar geographic law to evaluate the spatial data autocorrelation used to predict local values from changed properties of the sampled data (9).

Zhang et al. (10) developed an effective geostatistical algorithm to estimate data and filled the gap area in SLC-off images using ordinary kriging and standardized ordinary cokriging approaches to predict the missing pixel values. The experimental results indicate that the geostatistical techniques can get suitable results for various applications, for example, evaluation of mapping in big-scale agricultural landscape areas, but it is difficult to predict an accurate reflectance value with small-scale applications.

Multisource reconstruction approaches. In this category, multitemporal photos have been utilized as supplemental imagery to fill in the gaps in inaccurate Landsat7 photographs, where the data are derived from these images taken at various points of time. SLC-on and/or SLC-off multitemporal pictures are utilized as auxiliary images to retrieve the missing spots in the SLC-off image due to the sporadic emergence of gaps over the multispectral bands in SLC-off images (10, 11).

Landsat auxiliary images (multitemporal Landsat 7 data)

The reflectance of the same region may vary in multitemporal photos, where the images were taken at close intervals and using the same sensor. To reduce the spectral change compared with the target image, these photos used to extract auxiliary information should be chosen as close to the target image date as possible (12).

USGS/NASA methodologies

Immediately after the SLC failure, two phases of gap filling approaches have been developed by a study team from USGS/NASA Company. A histogram-based compositing technique is used in both phases by combining an SLC-off target image with one or more SLC-on and/or SLC-off auxiliary fill images to reconstruct the striping gaps in SLC-off images (13, 25).

2. Methodology

Phase 1 Product: Experts from the USGS provided the phase 1 methodology, which consists of the Global Histogram Matching (GHM) and Local Linear Histogram Matching (LLHM) recovering procedures. Both techniques have been used to fill the gap locations with one acquired SLC-on/SLC-off imagery. The principle of phase 1 techniques is to calculate a linear transformation of both the fill and target images as shown in Figure 1, where the gain (G) and the bias (B) are calculated from the known pixels in both the target and fill imagery (27).

Phase 2 Product: After the November 2004 phase 2 product, an enhancement of phase 1 algorithms was declared. In this product, one or more SLC-off (multiple scenes) were used because unscanned locations are different per images acquired for the same landscape. The phase 2 product consists of the Adaptive Window Local Histogram Matching (AWLHM) approach, which depends on a similar hypothesis of the phase 1 product; one exception is the change in the size of the moving window to gain the required common scanned pixels from the target and fill auxiliary images to calculate the linear regression coefficients. Figure 2 presents the process of moving window expanded, where the red rectangles indicate the latest determined windows, the red rectangles represent the unscanned pixels that required to be forecasted, and the rectangles with red dashed lines reflect the window searching process (3).

There are some studies which have been established based on USGA/NASA techniques (GHM, LLHM, and AWLHM). Hu et al. (5) proposed a Local Correlation Analysis (LCA) approach to reconstruct the SLC-off data using multitemporal Landsat 7 ETM+ imagery; then the results are compared with those of the LLHM algorithm. Statistical and visual indices showed that the LCA-filled gap areas well coincided compared with the original real image over LLHM.

He et al. (14) developed a comparative study between popular regressions algorithms (GHM, LLHM, adaptive window regression algorithms) which were applied to reconstructing two types of gaps: internal and edge gap locations. The results indicate that the internal gaps with a small pixel width were restored in a good manner using an adaptive window regression method, while the GHM method was suitable with edge gap locations.

Gilbert (2012) tested the LLHM algorithm to estimate values in gap locations on selected EMT+ SLC-off imagery using SLC-on images. The mean absolute difference between pixels in filled and SLC-off images was calculated to evaluate the validity of the results; however, some striping occurred in the landscape with transient changes.

Ali and Mohammed (15) compared a number of single-source gap-filling recovering methods (mean, median, midpoint, and number of interpolation methods) with the LLHM technique using SLC-on images as a donator of data. In the LLHM technique, the gain and bias are computed using whole valid pixel values in both SLC-off and donator images first and within the appropriate moving window second. A PSNR criterion was utilized to validate both self and different scene-dependent methods. The outcomes indicate the progress of LLHM above the other approaches in spite of the fact that some pixels have not filled in target image margins because the larger size of the moving window skipped these gaps.

Aghamohamadnia and Abedini (16) presented an algorithm based on the hypothesis “strip-crossing stitch lines” to recover sharp striping ETM+ SLC-off images by employing the mean pixel value of the stitch line using surrounding moving windows related with the morphological and geometrical sketch. The proposed algorithm was applied to pixels located on the margin line between the gap locations and the nearby area, and then, the similarity of close pixels was exploited to improve a stitch line. The algorithms were applied on band 4 for validation test, where the predictable values were close to the actual values. At the same time, there were smoothing influences at the stripping boundaries as a result of using the mean estimator in the recovering process.

Sulong and Sadiq (11) introduced a study to recover the gap locations in Landsat 7 imagery by mean and IDW as two single-source interpolation methods and LLHM as a multisource approach. Then the experimental results are compared using RMSE and SE metrics. The results present suitable results of LLHM over the homogeneous landscape.

3. Result

Spectral and spatial information methodology

This methodology has been developed to integrate both spectral and temporal information derived from the input sources considered as fill images utilized to forecast the unscanned locations in target Landsat7 images; the data set includes:

1. Landsat 5 TM or Landsat 7 SLC-on images.

2. Number of ETM+ SLC-off images.

3. Sentinel-2 MSI images

These input images are acquired with dates reasonably close to target SLC-off images under the same sun illumination and in a similar season to ensure that there were no significant land cover variations between the auxiliary and target scenes. A simple and effective technique was developed by Chen et al. (17), which is recognized as the Neighborhood Similar Pixel Interpolator (NSPI) approach. This algorithm was implemented using two data sources, the Landsat 5 TM or Landsat 7 SLC-on image, with a number of ETM+ SLC-off imageries. Although the NSPI method gets suitable results in recovering a heterogeneous landscape, more computing time is required as a result of calculations required to perform for selecting the identical values for each missing pixel.

Zhu et al. (18) presented the GNSPI approach (Geostatistical Neighborhood Similar Pixel Interpolator) as an improvement of the NSPI algorithm using auxiliary images from both Landsat 5 TM and Landsat 7 SLC-off images. GNSPI results indicate less striping effects as compared with NSPI and geostatistical methods.

Zeng et al. (19) used a multitemporal algorithm first; then a non-reference technique was performed as an integrated approach to recover the missing locations in SLC-off images. First, the gap locations in a complex landscape area were recovered by the Weighted Linear Regression (WLR) algorithm using information derived from multitemporal auxiliary SLC-off images; after that, the remaining unscanned locations were recovered using the regularization technique. An accurate prediction was obtained, but the difficulties appear when abrupt changes appear in the target scene.

Sadiq et al. (12) presented a new gap filling approach to reconstruct the failure in Landsat 7 imagery by adopting multitemporal SLC-off auxiliary fill imagery. The multiple linear regressions model is used to establish a correlation between the pixels in failure with two auxiliary fill imageries in parallel. Final results indicate accurate estimating values, although more temporally far fill images are used.

Also, by using multitemporal auxiliary fill images, Sadiq et al. (20) introduced an innovative gap filling to recover the large gaps. Two steps have been established; first, about half unscanned pixels are reconstructed by implementing the MLR model; thereafter, a Weighted Multiple Linear Regression (WMLR) has been planned to reconstruct the residual unscanned pixels. Results in simulated and real case studies provide accurate results.

In other studies, the fault ETM+ images were recovered using data derived from non-Landsat auxiliary photos with proper time acquisition (21). The resolution, spectral compatibility, and cost of these fill auxiliary images from non-Landsat sensors are constrained (11).

Roy et al. (22) presented that in spite of the spatial resolution of imagery delivered from (MODIS) BRDF/Albedo, the land surface is coarser than that of the ETM+ image and due to its plentiful images, a semiphysical fusion technique could be used to estimate the missing values by extracting the information from images produced by the MODIS sensor. This approach characterized by its ease of implementation; also, it was not affected by the polluted existence in neighboring ETM+ pixel or temporal changes as a result of variation of the surface.

Additionally, to estimate the missing pixels in SLC-off photos, Chen et al. (23) used the spectral similarity between the visible and near-infrared bands of Landsat 7 ETM+ and the China Brazil Earth Resources Satellite-02B (CBERS-02B), which was regarded an auxiliary image with rapid capture. Simple filling, GHM, LLHM, and AWLHM were utilized as common recovering approaches, and the RMSE and systematic error SE index were then determined to compare the outcomes of each methodology. The results show that the AWLHM technique has higher accuracy than the others, particularly in homogeneous landscapes more than heterogeneous landscapes, because in AWLHM, the numbers of effective pixels were taken into account. Furthermore, the temporal difference between the Landsat 7SLC-off and CBERS-02 has a significant effect on the obtained results.

Wang et al. (24) proposed using Sentinel-2 MSI images as auxiliary filling images because both Sentinel-2 MSI data and Landsat 7 ETM+ data have the same map projection with similar band wavelengths. First and for the downscaling step, a downscaling-then-upscaling methodology involving Area-to-Point Regression Kriging (ATPRK) has been implemented; then the spatial-spectral radial basis function (SSRBF) method was performed as gap filling interpolation, which allows using both the spectral and spatial information. The experimental results demonstrate that the Sentinel-2 MSI data can be used as a valuable source for filling CLS-off images.

4. Conclusion

This work overviewed the relevant literature studies on reconstructing the unscanned values in malfunction Lansat7 images and the major issues related to this scope. A brief background presentation and review on main reconstruction approaches associated with various types of auxiliary fill images have been displayed. The first one was related to reconstruction approaches based on malfunction images themselves, which provided inaccurate results, while the second one was related to using additional data provided from Landsat or non-Landsat auxiliary images. Particularly, the multitemporal approach that used a number of Landsat 7 as fill images in progression to fill all gap locations has been proved as the most suitable and attractive approach. However, these approaches have difficulties in estimating the gap locations in heterogeneous landscapes, with obvious temporal change and the complicated textures.

5. Acknowledgments

We acknowledge the assistance of the Department of Computer Science in the College of Science at Mustansiriyah University in Baghdad, Iraq.

References