# Data Flags¶

## Introduction¶

VanderSat strives to provide the best and most reliable data. Therefore our data comes with a set of flags that supply additional information on our data. These flags indicate possible data issues as well as information on data usage and quality. This document lists the flags that come with our Soil Moisture, Vegetation Optical Depth and Land Surface Temperature products and how to interpret them.

## How to retrieve the data flags?¶

VanderSat products have an associated data flags file. The flags can be retrieved via de VanderSat API (API User Guide). The API name for the flag file associated with products can be derived by prefixing the product name with QF-. So if the product name is SM-SMAP-LN-DESC_V003_100 the associated flag file can be retrieved using the product name QF-SM-SMAP-LN-DESC_V003_100.

There is an exception to this rule: VOD uses the same flag file as the corresponding SM product. For example, VOD-SMAP-LN-DESC_V001_100 uses QF-SM-SMAP-LN-DESC_V001_100.

Example flag file product mappings

Product

Flag file

SM-SMAP-LN-DESC_V001_100

QF-SM-SMAP-LN-DESC_V001_100

SM-AMSR2-XN-DESC_V001_100

QF-SM-AMSR2-XN-DESC_V001_100

VOD-AMSRE-C1N-DESC_V001_100

QF-SM-AMSRE-C1N-DESC_V001_100

TEFF-AMSR2-DESC_V001_100

QF-TEFF-AMSR2-DESC_V001_100

For details on how to retrieve the actual data files, see the API User Guide.

## Flag types¶

The VanderSat data comes with three possible types of flags:

static flags

Static flags flag areas based on (mostly) static features such as terrain slope.

critical flags

Critical flags indicate that data is not to be used. As such we replace the data with a missing value. However, when retrieving our data you can access the original data by looking at the second band in the GeoTIFF and NetCDF files. All values in the flag file > 127 are critical.

non-critical flags

Non-critical flags mean that the data can be used with caution, taking into account the information given in the flag. The data is shown as normal in the viewer and stored similar to non-flagged data in the GeoTIFF and NetCDF files. All values in the flag file <= 127 are non-critical.

Note

Critical and non-critical flags are combined into one flag file for each product and provided each time a product is made.

## Static flags¶

The flags in this section are so-called static flags that are available as separate products in our system.

### Complex (steep) terrain¶

The complex terrain flag indicates areas in which the complex terrain (steep topography) may have influenced our algorithm. In many cases the data is still fine but care must be taken when using the data.

Affected products:

• Soil Moisture

• VOD

• Land Surface Temperature

• Products derived from any of the above, such as DRZSM

### Built-up area¶

The built-up area flag indicates paved/built-up areas. Although it does not affect our algorithm, the obtained values for soil moisture and VOD cannot be interpreted the same as for natural areas. Land Surface Temperature is not affected.

Affected products:

• Soil Moisture

• VOD

• Products derived from any of the above, such as DRZSM

## Non-critical flags¶

The flags in this section are band/product-specific non-critical flags.

### 1. Dense vegetation (high VOD)¶

This flag indicates high vegetation cover, where soil moisture retrieval is less reliable. Depending on the frequency, the flag value for vegetation corrected soil moisture and VOD are applied:

• C/X band: medium VOD threshold = 0.45 and high VOD threshold = 0.6

• L band: medium VOD threshold = 0.3 and high VOD threshold = 0.4

Affected products:

• Soil Moisture

• Products derived from any of the above, such as DRZSM

### 2. Low soil moisture¶

The retrieved soil moisture is lower than the estimated wilting point.

Affected products:

• Soil Moisture

• Products derived from any of the above, such as DRZSM

### 3. High soil moisture¶

The retrieved soil moisture is higher than the estimated porosity.

Affected products:

• Soil Moisture

• Products derived from any of the above, such as DRZSM

### 4. Possibly influenced by snow or severe rainfall¶

Part of the footprints touch an area flagged as severe rainfall or snow. See 9. Snow or severe rainfall.

Affected products:

• Soil Moisture

• VOD

• Land Surface Temperature

• Products derived from any of the above, such as DRZSM

### 5. Possibly influenced by RFI¶

Part of the footprints touch an area flagged as RFI. See 12. RFI detected.

Affected products:

• Soil Moisture

• VOD

• Products derived from any of the above, such as DRZSM

### 7. Possible frozen soil¶

This flag indicates areas where frozen soil may occur (TEFF < 273.15 K).

• Soil Moisture

• VOD

• Land Surface Temperature

• Products derived from any of the above, such as DRZSM

## Critical flags¶

The flags in this section are band-specific critical flags (VOD, SM and TEFF).

### 8. Frozen soil¶

This flag indicates areas where frozen soil is detected. Pixels with TEFF < -10C is flagged as critical, while pixels with TEFF values between 0C and -10C are flagged as non-critical. A conservative value of -10C was chosen to avoid masking valid data. For areas marked as frozen soil, no Soil Moisture or VOD is given. The data is filtered out during the processing in LPRM.

Affected products:

• Soil Moisture

• VOD

• Land Surface Temperature

• Products derived from any of the above, such as DRZSM

### 9. Snow or severe rainfall¶

This flag indicates areas where snow or severe precipitation is detected. For areas marked as snow no Soil Moisture, VOD, or Land Surface Temperature is given. They are filtered out during the processing of the raw satellite data. The flagging criterion applied during downscaling is:

$((BT36V - BT18V) < -2.5 \land (F - BT36V) >= 10 \land BT36V < 275) \lor (BT36V - BT18V) < -10.5$

where

$F = 450.2 - 0.506 * BT19V -1.874 * BT22V + 0.00637 * BT22V * BT22V$

, which represents the simulated 85 GHz vertical polarization brightness temperature under the ideal condition of no scattering effect. See Jin et.al 1.

Affected products:

• Soil Moisture

• VOD

• Land Surface Temperature

• Products derived from any of the above, such as DRZSM

### 10. High vegetation¶

This flag indicates if the MPDI of brightness temperatures at the given frequency is too low for the algorithm to converge. The formula is: $$\frac{BT_V-BT_H}{BT_V+BT_H} < 0.0001$$. The data is filtered out during LPRM processing.

Affected products:

• Soil Moisture

• VOD

• Products derived from any of the above, such as DRZSM

### 11. No overpass¶

If one or more brightness temperature frequency is missing (due to a combination of different satellite swaths), the no coverage flag is produced during downscaling.

Affected products:

• Soil Moisture

• VOD

• Products derived from any of the above, such as DRZSM

### 12. RFI detected¶

If a footprint is contaminated for more than 25% with RFI, it is removed from processing during downscaling. Footprints with less contamination than 25% are retained but the data is marked as influenced by RFI.

Affected products:

• Soil Moisture

• VOD

• Products derived from any of the above, such as DRZSM

### 13. Instrumental flaws (e.g. along the edges of the swath)¶

If the brightness temperature at 36.5 GHz V produces values either over 400K or under 0.9 * the water temperature, the data is considered as unrealistic and is removed.

Affected products:

• Soil Moisture

• VOD

• Land Surface Temperature

• Products derived from any of the above, such as DRZSM

### 14. Out of valid range¶

Soil moisture values retrieved are outside of the valid range. This affects both the soil moisture and the retrieved VOD. In essence, this means that no valid solution was found.

Affected products:

• Soil Moisture

• VOD

• Products derived from any of the above, such as DRZSM

### 15. Open water¶

This flag indicates areas that are deemed to be open water (rivers, lakes sea etc). For areas marked as open water no Soil Moisture, VOD, or Land Surface Temperature is given. They are filtered out during the processing of the raw satellite data.

Affected products:

• Soil Moisture

• VOD

• Land Surface Temperature

• Products derived from any of the above, such as DRZSM

### DRZSM quality indicator¶

The Derived Root Zone Soil Moisture (DRZSM) quality indicator is an indicator for the amount of days for which a valid soil moisture measurement was used. Like the DRZSM product, it is calculated recursively and depends on the amount of days with a valid measurement and the chosen T value. This indicator ranges from 0-100 % and describes for each pixel the level of confidence that the DRZSM has had sufficient data of acceptable quality within the characteristic time scale T. The quality indicator is calculated according to:

$q(t) = c + q(t-1) * e^{(-dt/T)}$

with

$c = (1-e^{-(1/T)})$

if a valid measurement is used and otherwise

$c = 0$

Note that for satellite products that do not have daily coverage, the quality maximum will converge to a value lower than 100 %.

Affected products:

• Derived Root Zone Soil Moisture

## Flags data file description for LST, VOD, and SM¶

During the processing of the data for each product an associated flag file including all the corresponding flags is also created. The relation between product name and flag name is described in How to retrieve the data flags?.

This file is a int16 map with each number associated with one or multiple flags. The bit values contain decimal numbers (1 and 0), the position of which indicates the flag. Below, the flags are split into two sections, the first section with the non-critical flags, the second section with the critical flags.

Position of VanderSat data flags for VOD and SM. A 1 indicates a flag position, a 0 indicates a reserved position

Section 2

Section 1

011111111

1011111

Position of VanderSat data flags for TEFF (LST). A 1 indicates a flag position, a 0 indicates a reserved position

Section 2

Section 1

010101011

1001000

Some examples of how to read the flags are given below:

64 decimal

This is 0000000001000000 in binary. Indicates that only the flag at position 7 is set, which is the non-critical flag for frozen soil.

128 decimal

This is 0000000010000000 in binary. Indicates that only the flag at position 8 is set, which is the critical flag for frozen soil.

## Decoding a flag file using python¶

The example code below shows how to decode the flags:

flagstr = [
'1. Dense Vegetation (high VOD)',
'2. Low Soil Moisture',
'3. High Soil Moisture',
'4. Possibly influenced by Snow or severe rainfall',
'5. Possibly influenced by RFI',
'6.',
'7. Possible frozen soil',
'8. Frozen Soil',
'9. Snow or severe rainfall',
'10. High vegetation',
'11. No overpass',
'12. RFI detected',
'13. Instrument flaw',
'14. Out of valid range',
'15. Open water/Water body',
'16.',
]

numbers = [12,64,128,141,32768]
for number in numbers:
pos = 15
flags= format(number,'016b')
print("=================================================")
print("Decimal number: " + str(number))
print("Binary string: " + flags)
print("--------------- Flags set -----------------------")
for fl in flags:
if fl == '1':
print(flagstr[pos])
pos = pos - 1
print("=================================================\n")


Running the script gives the following output:

=================================================
Decimal number: 12
Binary string: 0000000000001100
--------------- Flags set -----------------------
4. Possibly influenced by Snow or severe rainfall
3. High Soil Moisture
=================================================

=================================================
Decimal number: 64
Binary string: 0000000001000000
--------------- Flags set -----------------------
7. Possible frozen soil
=================================================

=================================================
Decimal number: 128
Binary string: 0000000010000000
--------------- Flags set -----------------------
8. Frozen Soil
=================================================

=================================================
Decimal number: 141
Binary string: 0000000010001101
--------------- Flags set -----------------------
8. Frozen Soil
4. Possibly influenced by Snow or severe rainfall
3. High Soil Moisture
1. Dense Vegetation (high VOD)
=================================================

=================================================
Decimal number: 32768
Binary string: 1000000000000000
--------------- Flags set -----------------------
16.
=================================================


## Flags data file description for DRZSM¶

The quality indicator is stored as a byte integer map without scaling and includes a value between 0 and 100 % indicating the estimated data quality.

## References¶

1

Jin, R., Li, X., & Che, T. (2009). A decision tree algorithm for surface soil freeze/thaw classification over China using SSM/I brightness temperature. Remote Sensing of Environment, 113(12), 2651–2660.