ETM+ Automated Cloud Cover Assessment (ACCA) Algorithm

Documentation of the ACCA Algorithm

The operational Landsat-7 ETM+ Automated Cloud-Cover Assessment (ACCA) algorithm is described here. It has been divided into five processes, which parallel the five flow chart figures:

A1. Pass-1 Spectral Cloud Identification (filters 1-11)

A2. Band-6 Cloud Signature Development (filters 12-16)

A3. Pass-2 Thermal Band Cloud Separation (filters 17-20)

A4. Image-Based Cloud-Cover Assignments and Aggregation (filters 21-15)

A5. Nearest-Neighbor Cloud-Filling (filter 26)

Most of the computer-intensive part of the processing is in Pass-1 and therefore most of the ACCA algorithm documentation below addresses the processes within Pass-1. Additionally, outputs from all 26 filters are presented for a path 28, row 31 scene in Table 5.

A1. Pass-1 Spectral Cloud Identification

The algorithm handles the cloud population in each scene uniquely by examining the raw uncalibrated Level-0R image data twice. Data preparation involves converting Bands-2 through -5 to planetary reflectance and Band-6 to apparent at-satellite radiance temperature.

For each spectral band, the 8-bit observed raw uncalibrated image quantized level Q, in units of digital number (DN), is related to Top-of-the-Atmosphere (TOA) spectral radiance L_l (in Watts/(m² sr µm)) by

Q = G L_l + Q0 (1)

where

G is sensor responsivity (in DN per unit spectral radiance) for each detector in the band and

Q0 is the average zero-radiance shutter background (in DN) from the Calibration Parameter File (CPF).

Sensor responsivity and the zero-radiance bias for each detector is maintained by the Image Assessment System (IAS) and recorded in the Calibration Parameter File (CPF) at the U. S. ground processing system at EDC. Radiometric detector normalization is applied in each spectral band. Bias-corrected image values are then given by

∆Q = Q - Q0 = G L_l. (2)

Thus, TOA spectral radiances (L_l) are related to image data by

L_l= ∆Q / G . (3)

TOA planetary albedo or reflectance (r) for Bands-2 through -5 is related to TOA spectral radiance (L_l) in each band by

r = π L_l d² / (E cos q) (4)

where

E is the exoatmospheric solar irradiance in each spectral band in Watts/(m² µm) which are referenced in Table 1 and the Landsat Science Data Users Handbook (Irish, 1999),

q is the solar zenith angle, and

d is the Earth-Sun distance in Astronomical Units.

At-satellite temperature for Band-6 (T) is related to TOA spectral radiance (L_l) by

T = K2 / 1n ((K1 / L_l) + 1) (5)

where

T is the at-sensor temperature in Kelvin,

K2 is the calibration constant 1282.71 in degrees Kelvin,

K1 is the calibration constant 666.09 in Watts/(m² sr µm), and

L_l is spectral radiance from equation 3.

The first pass through the data is designed to capture pixels that are unambiguously clouds and not something on the ground. Eight different filters are used to isolate clouds and to eliminate cloudless areas including problem land surface features such as snow and sand. The pixels of clouds from pass one are used to develop a Band-6 thermal profile and thresholds for clouds for use in pass two where the remaining clouds are identified. Five categories result from pass one processing – warm clouds, cold clouds, desert, non-clouds, and an ambiguous group of image pixels that are reexamined in pass two.

The Band-6 temperature profile is formulated from the observed pass one cloud population if it exists. The profile is defined by the cloud population’s mean, variance, and skewness, and undergoes modulation if snow or desert features are present in a scene.

. A description of each filter, presented in the order implemented follows.

A1.1. Filter 1 – Brightness Threshold

Each Band-3 pixel in the scene is first compared to a brightness threshold. Pixel values that exceed the Band-3 threshold, which is set at .08, are passed to filter 3. Pixels that fall below this threshold are identified as are passed to filter 2.

A1.2 Filter 2 – Non-cloud/Ambiguous Discriminator, Band–3

Comparing each pixel entering this filter to a Band-3 threshold set at .07 identifies potential low-reflectance clouds. Pixels that exceed this threshold are labeled as ambiguous and are re-examined in pass-2. Those falling below .07 are identified as non-clouds and are flagged as such in the cloud mask.

A1.3. Filter 3 – Normalized Difference Snow Index

The normalized difference snow index (NDSI) is used to detect snow (Hall et al., 1995). The NDSI filter is expressed as:

NDSI = (r2 – r5) / (r2 + r5) (6)

This filter is designed to eliminate snow. The reflectances of clouds and snow are similar in Band-2. However, in Band-5, reflectance for clouds is high while snow is low. Hall discovered that NSDI values greater than .4 represent snow cover quite well. This value was initially tried for ACCA to eliminate snow but clouds composed of ice crystals (e.g. cirrostratus) were also eliminated. The threshold was raised to .7 to capture potential clouds of this type. Pixels that fall between an NDSI range of -.25 and .7 qualify as potential clouds and are passed to filter 5. Pixels outside this NDSI range are labeled as non-cloud and passed to Filter 4. Snow pixels that slip through are generally trapped later.

A1.4. Filter 4 – Snow Threshold

Knowledge of snow in a scene is important for pass two processing so a tally of snow pixels is retained. NDSI values above a .8 threshold qualify as snow and are recorded as non-cloud in the cloud mask.

A1.5. Filter 5 – Temperature Threshold

The Band-6 temperature (T) values are used to identify potential clouds. If a pixel value exceeds 300K, a realistic cloud temperature maximum, it is labeled as non-cloud. Pixels with a temperature a value less than 300K are passed to filter 6.

A1.6. Filter 6 – Band 5/6 Composite

The low values of the product of values is sensitive to the detection of clouds. The Band 5/6 Composite is expressed as:

Band 5/6 Composite = (1 – r5) T (7)

This filter works because clouds have cold temperatures (< 300K) and are highly reflective in Band-5 and therefore have low values of (1 – r5) T. It is particularly useful for eliminating cold land surface features that have low Band-5 reflectance such as snow and tundra. Sensitivity analysis demonstrated that a threshold setting of 225 works optimally. Pixels below this threshold are passed to filter 8 as possible clouds. Pixel values above this threshold are examined using filter 7.

A1.7. Filter 7 – Non-cloud/Ambiguous Discriminator, Band–5

Comparing each pixel entering this filter to a Band-5 threshold set at .08 identifies potential low-reflectance clouds. Pixels that exceed this threshold are labeled as ambiguous and are re-examined in pass-2. Those falling below .08 are identified as non-clouds (probably water) and are flagged as such in the cloud mask.

A1.8. Filter 8 – Band 4/3 Ratio for Growing Vegetation

This filter eliminates highly reflective vegetation and is simply Band-4 reflectance divided by Band-3 reflectance. In the near-infrared (Band-4), reflectance for green leaves is high because very little energy is absorbed. In the red region (Band-3), the chlorophyll in green leaves absorbs energy so reflectance is low. The 4/3 ratio results in higher values for vegetation than for other scene features, including clouds. A threshold setting of 2.0 is used. Pixels that exceed this threshold are labeled ambiguous and are revisited in pass-2. Pixels with ratios below this threshold are passed to filter 9.

A1.9. Filter 9 – Band 4/2 Ratio for Senescing Vegetation

This filter eliminates highly reflective senescing vegetation and is formed by dividing the Band-4 reflectance by the Band-2 reflectance. In the near-infrared (Band-4), green leaves that are dead or dying absorb even less energy and are thus highly reflective. In the green region (Band-2), the leaves absorb less energy because of chlorophyll loss and exhibit increased reflectivity. The 4/2 ratio values are higher for vegetation than other scene features including clouds. A threshold setting of 2.16248 works effectively. The at-launch setting was 2.16 but was change in May of 2001(?) when the operational decision was made to Band-4 in low gain. Pixels that exceed this number are ambiguous and are revisited in pass-2. Pixels with ratios below this threshold are passed to filter 10.

A1.10. Filter 10 – Band 4/5 Ratio for Soil

This filter eliminates highly reflective rocks and sands in desert landscapes and is formed by dividing the Band-4 reflectance by the Band-5 reflectance. Rocks and sand tend to exhibit higher reflectance in Band-5 than in Band-4, whereas the reverse is true for clouds. A threshold setting of 1.0 works effectively. Pixels that fall below this threshold are labeled ambiguous and are revisited in pass two. Knowledge of desert pixels in a scene is important for pass-2 processing. Therefore, a desert pixel tally is retained. Pixels with ratios that exceed this threshold are passed to filter 11.

A1.11. Filter 11 – Band 5/6 Composite for Warm and Cold Clouds

All pixels reaching this filtering level are classified as clouds. A further separation into two classes is achieved by again using the Band 5/6 Composite filter. For each cloud pixel, the Band 5/6 Composite is compared against a threshold setting of 210. Pixels above and below this threshold are classified as warm and cold clouds, respectively. These two cloud classes are recorded in the cloud mask and used to develop two cloud signatures, one for the cold clouds and the other conjoined cloud classes.

A2. Band-6 Cloud Signature Development

Pass-2 processing requires two new band 6 thresholds against which all ambiguous pixels are compared. These thresholds are computed using the pass-1 cloud temperature statistics. Only the cold clouds are used if snow or desert soil is present, otherwise the cold and warm clouds are combined and treated as a single population. The cloud thermal profile developed includes key statistics including the maximum cloud temperature, mean, standard deviation and histogram skewness.

A2.12. Filter 12 – Snow and Desert Indicator

An infrared/short-wave infrared ratio is used identify highly reflective rocks and sands in desert landscapes. Snow was previously accounted for. If snow or desert rocks are present, the warm cloud class is eliminated. The desert indicator employed is simply the ratio of potential cloud pixels

exiting and entering filter 10 compared against a threshold value of 0.5. If the remaining pixels are less than 50% the warm clouds are removed. The snow percentage for the scene is computed and compared against a threshold of 1%. If the scene is more than 1% snow the warm clouds are removed.

A2.13. Filter 13 – Pass-1 Cloud-free Indicator

The pass-1 cloud tally is compared against zero. If no clouds were tallied processing ends and the scene is declared cloud-free.

A2.14. Filter 14 – Pass-1 Cold Cloud, Desert and Mean Cloud Temperature Indicator

Three conditions have to exist to continue pass-2 processing. The cold cloud scene percentage must be greater than 0.4%, the pass-1 cloud temperature mean must less than 295K and desert conditions must not exist. If any of these tests are not met processing passes to filter 22. If all three test are met pass-2 processing continues at filter 15.

A2.15. Filter 15 – Temperature Histogram Negative Skewness

Prior to computing the Band-6 thresholds a skew factor is computed from the skewness of the cloud temperature histogram. If the histogram skewness is negative the cloud population is biased towards the left or colder tail of the distribution. No skew factor is necessary because the thresholds are set at appropriate levels for identifying clouds that skew colder. The shift factor is set to 0.0

A2.16. Filter 16 – Temperature Histogram Positive Skewness

If the histogram skewness is positive the cloud population is biased towards the right or warmer tail of the cloud temperature distribution. Because of the upward bias in temperature an upward threshold adjustment is necessary. The skewness becomes the skew factor if it is less than 1.0, otherwise it is set to 1.0.

A3. Pass-2 Thermal Band Cloud Separation

One of the thresholds is set low and is used to generate a conservative estimate of clouds in a scene. The other is set high and is used to compute a less restrictive estimate of cloud cover. The thresholds are determined from the pass-1 cloud temperature histogram. The histogram's 97.5 and 83.5 percentiles are the starting points for the two temperature thresholds and adjustments are made if necessary. All ambiguous pixels are tested against the two thresholds. If a pixel falls below the high threshold it is labeled as a warm cloud. It is re-labeled cold cloud if it also falls below the lower threshold.

If a pixel temperature falls below the upper threshold, the cloud mask is flagged with a unique number that identifies a class of warmer clouds. If the pixel temperature also falls below the lower threshold, the cloud-mask value is changed to a colder cloud-class identifier.

A3.17. Filter 17 – Threshold Shift Deployment

If the skew factor is positive, upward adjustments are made to compensate for the warm cloud bias. The threshold shift is the product of the skew factor and cloud temperature standard deviation. Both thresholds are adjusted by this amount. If the skew factor is 0.0 processing continues at filter 19.

A3.18. Filter 18 – Band-6 Maximum Threshold

A final check is made to see if the new upper threshold exceeds the histogram's 98.75 percentile (a threshold above or near the cloud temperature maximum is unwanted). If so, the 98.75 percentile becomes the new upper threshold and the lower threshold is adjusted by the amount of skewness compensation actually allowed.

A3.19. Filter 19 – Band-6 Warm Cloud Indicator

Each ambiguous pixel’s Band-6 temperature is tested against the higher threshold and is labeled a warm cloud if it is lower. Pass-2 then processing continues at filter 20. If it is higher it is skipped and the next ambiguous pixel is likewise examined. The process continues until all ambiguous pixels are accounted for.

A3.20. Filter 20 – Band-6 Cold Cloud Indicator

Each pass-2 warm cloud is tested against the lower threshold and is re-labeled cold cloud if it is lower. Processing continues at filter 19 until all ambiguous pixels are accounted for.

A4. Image-Based Cloud-Cover Assignments and Aggregation

After Band-6 is processed the scene percentages for the warm and cold cloud classes are computed. The integrity of the two additional cloud classes is then appraised. The presence of snow or desert features, and the magnitude of the two new cloud classes are used to accept or reject one or both classes. Cloud classes that qualify as legitimate are combined with the pass one clouds to form a single unified cloud class in the mask.

A4.21. Filter 21 – Pass-2 Cloud-free Indicator

The pass-2 cloud tally is compared against zero. If no clouds are tallied, an event unlikely to happen, the final scene cloud cover score reported is the cold cloud % from Pass-1. Processing resumes at filter 26.

A4.22. Filter 22 – Pass-1 Cloud Temperature Mean

This filter is used when the three pass-1 criteria are not met in filter 14. The cold cloud scene percentage might be less than 0.4%, the pass-1 cloud temperature might be greater than 295K and desert conditions may exist. The mean of the pass-1 cold cloud population is again tested against the limit of 295K. If it is less the clouds are accepted as real but with less certainty. If any of these tests are not met processing passes to filter 22. The final scene cloud cover score reported is the cold cloud % from Pass-1. Processing resumes at filter 26. If the mean is greater than 295K uncertainties exists. The final scene cloud cover score is set to zero and processing ends.

A4.23. Filter 23 – Pass-1 Cloud Acceptance Indicator

If the snow or desert conditions determined earlier exist the pass-1 cloud percentage is determined using the pass-1 cold clouds only. If the scene is free from snow and desert soil the pass-1 cloud percentage is determined using both the cold and warm pass-1 clouds.

A4.24. Filter 24 – Pass-2 Cold and Warm Cloud Acceptance

The temperature means and maximums are computed for the pass-2 cold and warm cloud populations. Additionally, the percentages of the scene represented by the pass-2 cold class and combined classes (cold and warm) are computed. The pass-2 cold and warm clouds are united with the pass 1 if four conditions are met. The pass-2 cold and warm cloud contribution cannot be more than 35%, snow cannot be greater than 1% of the scene, the mean temperature for combined cloud population cannot be greater than 295K and the between the combined cloud maximum temperature and the upper threshold cannot be less than two degrees. If these four conditions are met all of the clouds identified in pass-2 are united with the pass-1 clouds and processing proceeds to filter 26. If any one condition is breached processing passes to filter 25.

A4.25. Filter 25 – Pass-2 Cold Cloud Acceptance

The pass-2 cold clouds are used if their contribution to scene cloud percentage is less than 25% and their mean temperature is less than 295 degrees. If these two conditions are satisfied the cold clouds are united with the pass-1 clouds and processing advances to filter 26. Uncertainty exists if either rule is breached and the pass-1 cold clouds are used in computing the final scene score. Processing continues at filter 26.

A5. Nearest-Neighbor Cloud-Filling

A final step involves identifying and filling cloud mask holes. This operation boosts the cloud-cover content to more accurately reflect the amount of unusable image data in a scene. Afterwards, the cloud pixels in the mask are tabulated and a final cloud cover percentage score for the scene is computed.

A5.26. Filter 26 – Threshold Shift Deployment

Each non-cloud image pixel is examined and converted to cloud if at least 5 of its 8 neighbors are clouds. Filled pixels qualify as cloudy neighbors in subsequent tests.

Back to Chapter 12

Last Update: July 24, 2006