API User Guide

Introduction

Welcome to the VanderSat API user guide. In this document you can find general information about the functioning of our API.

This user guide will not provide a comprehensive list of endpoint parameters. For detailed information of our API endpoints we refer you to: https://maps.vandersat.com/api/v2 . All of our v2 API endpoints support Swagger/OpenAPI. This documentation allows you to try out API requests and also serves as the main documentation of the exact specifications of the supported parameters and return values.

Screenshot of interactive API documentation

Interactive API documentation used for trying out the product listing request

Authentication

We support HTTP basic authentication. There are two ways of providing your authentication. Your username and password can be provided in the URL (see example below) or they can be applied in the HTTP headers of the request.

https://<USERNAME>:<PASSWORD>@maps.vandersat.com/

To test the VanderSat authentication, the available product list can be requested using:

GET https://maps.vandersat.com/api/v2/products

This will return an JSON object with all products available to this user account if authentication is successful. e.g.:

{
    "products": [
        {
          "name": "Surface Soil Moisture",
          "max_zoom": 15,
          "min_val": 0,
          "abbreviation": "SM-LN_V001_100",
          "max_val": 1,
          "api_name": "SM-LN_V001_100",
          "default_legend_id": "1",
          "area_allowed": null,
          "time_series_type": "default",
          "unit": "m3/m3"
        }
    ]
}

or in a Python script (with requests installed) like:

import requests
import json

r = requests.get('https://maps.vandersat.com/api/v2/products',
                 auth=('username', 'password')) # change username and password
r.raise_for_status()
product_dict = json.loads(r.content)

Python client to make API requests

The Python client (vds-api-client) is a command line interface to directly download data from the API. This package simplifies and speeds up submitting asynchronous data requests, synchronous requests for ROI-time series and downloading the data. The project is available on GitHub and can be installed through PyPI.

The package can be installed using pip in an existing environment using:

$ pip install vds-api-client

More information on getting started can be found in the documentation which can be found in one of the previous links.

Data availability

Account management

Username and password are supplied to younvia separate channels. These credentials can most easily be changed in the VanderSat Viewer under User settings in the top right corner.

Checking access to VanderSat products

Access to the VanderSat API gives access to a number of different products, depending on your contract with us.

To discover which products your account has access to perform the following request:

GET https://maps.vandersat.com/api/v2/products/

This will return a JSON list of all products you can access. The most important field for each product is the api_name since this is the string used for accessing the data.

An extensive description of all the products can be found in the Data Products chapter.

Checking data availability for a specific VanderSat product

To check the availability of a product get the api_name as described in the Checking access to VanderSat products section.

Then a request like:

GET https://maps.vandersat.com/api/v2/products/<api_name>/availability

will return the product availability for this product. The returned value is a date and a time but at the moment the time component can be ignored. The local overpass time for each product is specified in the Data Products chapter.

Detailed specifiations about this endpoint can be found in the `Interactive API documentation`_.

Data requests

Requesting images

Getting spatial data from the API can be accomplished in two ways:

  1. Synchronous Requesting images as a xyztile

  2. Asynchronous Requesting images as gridded-data

Requesting images as a xyztile

Synchronous requests can be used to have the VanderSat API act like a XYZ tiled web map service. For these kind of requests PNG tiles of size 256x256 pixels are returned. The tiles can be used very easily in a frontend application in an openstreetmap or google way.

Further information can be found at https://wiki.openstreetmap.org/wiki/Slippy_map_tilenames and https://www.maptiler.com/google-maps-coordinates-tile-bounds-projection/. These sites describe how to go from lat/lon to the x, y and z values which can be used in the requests.

Examples: XYZ web tiles
GET https://maps.vandersat.com/api/v2/products/<api_name>/xyztiles?x=36&y=18&z=6&date=2019-10-15&legend_start=0&legend_end=1

The parameters are documented in the `Interactive API documentation`_. Because this API endpoint is synchronous the PNG tiles are returned right away.

Since this API endpoint follows the specifications of ‘slippy map tilenames’ as defined by OpenStreetMap it can be added very easily as a layer in a Leaflet map using the following JavaScript example:

var VDS_map = L.tileLayer('https://{s}.maps.vandersat.com/api/v2/products/<api_name>/xyztiles?x={x}&y={y}&z={z}&date=<date>', {
        maxZoom: 15,
        attribution: '&copy; VanderSat'
});

The parameters are documented in the `Interactive API documentation`_. For serving these tiles we have three URLs available for allowing more concurrent requests, this means that {s} can be either a, b or c.

Requesting images as gridded-data

Asynchronous requests can be used to extract mapped data for a bounding box. For these kind of requests GeoTIFF or NetCDF files are returned.

GET https://maps.vandersat.com/api/v2/products/<api_name>/gridded-data

The parameters are documented in the `Interactive API documentation`_. Because this API endpoint is asynchronous the return value is a process uuid under which the data will be available as soon as it is processed. Getting the data once the request is processed is explained at api-get-data-current. Note that requested data is stored up to 7 days after submitting an asynchronous request.

Examples: gridded-data

Get L band soil moisture for May 2018 in GeoTIFFs:

GET https://maps.vandersat.com/api/v2/products/SM-LN_V001_100/gridded-data?lon_min=3&lon_max=7&lat_min=51&lat_max=53&start_date=2018-05-01&end_date=2018-05-31&format=gtiff&zipped=false

Get Land-surface temperature for 2017 zipped in NetCDF:

GET https://maps.vandersat.com/api/v2/products/TEFF_V001_100/gridded-data?lat_min=51&lat_max=53&lon_min=3&lon_max=7&start_date=2017-01-01&end_date=2017-12-31&format=NetCDF4&zipped=true

Requesting time-series

Requesting time-series as point-data

Getting time-series for single point time-series from the API can be accomplished by calling one of the following URLs:

   GET https://maps.vandersat.com/api/v2/products/<api-name>/point-time-series


For time-series, one can request only one file

per request containing the time-series of a single product.

The parameters are documented in the `Interactive API documentation`_. Since this API endpoint is asynchronous the return value is a process uuid under which the data will be available as soon as it is processed. Getting the data once the request is processed is explained at api-get-data-current. Note that requested data is stored up to 7 days after submitting an asynchronous request.

Note

The climatology option in the API determines if a climatology should be supplied together with the requested time series data. It is important to realize that the climatology is determined for the period for which the time series is requested. Available earlier or later data is not taken into account.

Requesting time-series for regions of (ROIs)

Regions of Interest (ROI) are regions that are of particular interest. It is possible to define these regions of interest by drawing or uploading a shapefile in the Viewer (Managing regions of interest). It is also possible to define an ROI using the API.

The management of ROIs via the API can be performed via the https://maps.vandersat.com/api/v2/rois endpoint. Please see the `Interactive API documentation`_ for more information.

Note

The climatology option in the API determines if a climatology should be supplied together with the requested time series data. It is important to realize that the climatology is determined for the period for which the time series is requested. Available earlier or later data is not taken into account.

Getting time-series for ROIs from the API can be accomplished by calling one of the following URLs:

GET https://maps.vandersat.com/api/v2/products/<api-name>/roi-time-series

Examples of requesting time-series

Get L band soil moisture time-series between april 2015 until april 2019 at (lat, lon) = (51.386, 4.647):

GET https://maps.vandersat.com/api/v2/products/TEFF_V001_100/point-time-series?start_time=2015-04-01&end_time=2019-03-31&lat=51.386&lon=4.647&format=csv&avg_window_days=0&include_masked_data=false&climatology=false

Same, but also include a 15 day moving average, masked data (marked as invalid), the climatology based on the average column, and the derived root zone calculation for a given T=10 days and output as JSON:

GET https://maps.vandersat.com/api/v2/products/TEFF_V001_100/point-time-series?start_time=2015-04-01&end_time=2019-03-31&lat=51.386&lon=4.647&format=json&avg_window_days=15&include_masked_data=true&climatology=true&exp_filter_t=10

Finally, downloading an area-averaged time-series for an uploaded region:

GET https://maps.vandersat.com/api/v2/products/TEFF_V001_100/roi-time-series?start_time=2015-04-01&end_time=2019-03-31&roi_id=4325

Data downloading

Retrieving the data

A return value of a gridded, point-time-series, or roi-time-series request could be, e.g.:

{
  "url": "api/v2/api_requests/83833aae-1376-4288-b7c7-3e99af57e29f/status",
  "message": "Download request received",
  "uuid": "83833aae-1376-4288-b7c7-3e99af57e29f"
}

The status of the API request can now be queried using:

GET https://maps.vandersat.com/api/v2/api_requests/83833aae-1376-4288-b7c7-3e99af57e29f/status

Which would return either the progress for Scheduled or Processing requests such as:

{
  "scheduled_dt": "2019-06-04T11:59:56.409228",
  "percentage": 0,
  "processing_status": "Scheduled"
}

or something like the following for an API request that is Ready:

{
  "scheduled_dt": "2019-01-24T17:18:45.810523",
  "started_dt": "2019-01-24T17:19:07.409485",
  "processing_status": "Ready",
  "finished_dt": "2019-01-24T17:19:12.698098",
  "data": [
    "/api/v2/api-requests/83833aae-1376-4288-b7c7-3e99af57e29f/data/SM-LN_V001_100_2018-12-02T000000_6.481934_52.669720_8.602295_51.710012_83833.tif",
    "/api/v2/api-requests/83833aae-1376-4288-b7c7-3e99af57e29f/data/SM-LN_V001_100_2018-12-03T000000_6.481934_52.669720_8.602295_51.710012_83833.tif",
    "/api/v2/api-requests/83833aae-1376-4288-b7c7-3e99af57e29f/data/SM-LN_V001_100_2018-12-04T000000_6.481934_52.669720_8.602295_51.710012_83833.tif",
    "/api/v2/api-requests/83833aae-1376-4288-b7c7-3e99af57e29f/data/SM-LN_V001_100_2018-12-05T000000_6.481934_52.669720_8.602295_51.710012_83833.tif",
    "/api/v2/api-requests/83833aae-1376-4288-b7c7-3e99af57e29f/data/SM-LN_V001_100_2018-12-06T000000_6.481934_52.669720_8.602295_51.710012_83833.tif",
    "/api/v2/api-requests/83833aae-1376-4288-b7c7-3e99af57e29f/data/SM-LN_V001_100_2018-12-07T000000_6.481934_52.669720_8.602295_51.710012_83833.tif"
  ],
  "percentage": 100
}

The data can then be downloaded by calling /download on each of the produced files, e.g.:

GET https://maps.vandersat.com/api/v2/api-requests/83833aae-1376-4288-b7c7-3e99af57e29f/data/SM-LN_V001_100_2018-12-02T000000_6.481934_52.669720_8.602295_51.710012_83833.tif/download

Note

requested data is stored up to 7 days after submitting an asynchronous request.

File naming

gridded data

The file naming convention of the returned gridded-data file(s) is the following:

<product-api-name>_<date:YYYY-MM-DDTHHMMSS>_<lon_min:.6f>_<lat_max:.6f>_<lon_max:.6f>_<lat_min:.6f>_<uuid4_short>.<ext>

Or for zipped files:

<product-api-name>_<start_date:YYYY-MM-DDTHHMMSS>_<end_date:YYYY-MM-DDTHHMMSS>_<lon_min:.6f>_<lat_max:.6f>_<lon_max:.6f>_<lat_min:.6f>_<uuid4_short>.<ext>

with:

  • <product-api-name> as described in Product naming convention.

  • <*:YYYY-MM-DDTHHMMSS> sensing date of data file in datetime-format e.g. 2018-12-07T000000

  • <lon_min:.6f> Minimum longitude rounded to 6 digits after the comma.

  • <lat_max:.6f> Maximum latitude rounded to 6 digits after the comma.

  • <lon_max:.6f> Maximum longitude rounded to 6 digits after the comma.

  • <lat_min:.6f> Minimum latitude rounded to 6 digits after the comma.

  • <uuid4_short> The first 5 characters of the API request UUID.

  • <ext> File extension depending on requested data format (tif, nc or zip)

Note

Although datetime is specified, currently only the sensing date is reflected in the sensing-date. By default, the time is always T000000.

time-series

The file naming convention for point-time-series is the following:

ts_<product-api-name>_<start_time:YYYY-MM-DDTHHMMSS>_<end_time:YYYY-MM-DDTHHMMSS>_<lon:.6f>_<lat:.6f>_<uuid4_short>.<ext>

and for roi-time-series:

roi-ts_<product-api-name>_<start_time:YYYY-MM-DDTHHMMSS>_<end_time:YYYY-MM-DDTHHMMSS>_<roi-id:d>_<uuid4_short>.<ext>

with:

  • <product-api-name> as described in Product naming convention.

  • <*:YYYY-MM-DDTHHMMSS> start- and end-datetime e.g. 2018-12-07T000000

  • <lon:.6f> Minimum longitude rounded to 6 digits after the comma.

  • <lat:.6f> Maximum latitude rounded to 6 digits after the comma.

  • <roi-id:d> region-id as an integer value.

  • <uuid4_short> The first 5 characters of the API request UUID.

  • <ext> File extension depending on requested data format (csv, json)

Data sample

Sample data files available for download

API endpoint

File format

Download

gridded-data

GeoTIFF

gridded_data.tif

gridded-data

NetCDF

gridded_data.nc

point-time-series

CSV

point_time_series.csv

point-time-series

JSON

point_time_series.json

roi-time-series-sync

CSV

roi_time_series_sync.csv

roi-time-series-sync

JSON

roi_time_series_sync.json

Python scripting example

This code example shows a minimal example to download the files contained in one request. For it to work, one should install the requests package through pip. Change the authentication, the request URI, and the output folder to make the script work:

$ pip install requests

or through conda

$ conda install -c conda-forge requests

import os
import time

import json
import requests


def _get_content(uri, auth):
    r = requests.get(uri, auth=auth)
    r.raise_for_status()
    response = json.loads(r.content)
    return response


def submit_job(api_call, auth):
    response = _get_content(api_call, auth)
    uuid = response['uuid']
    return uuid


def _get_status(uuid, auth):
    status_uri = ('https://maps.vandersat.com/api/v2/'
                 'api-requests/{uuid}/status'.format(uuid=uuid))
    response = _get_content(status_uri, auth)
    return response


def get_data(uuid, auth):
    status = _get_status(uuid, auth=auth)
    while status['percentage'] < 100:
        print('Percentage = {}'.format(status['percentage']))
        time.sleep(5)  # wait 5 seconds before requesting new status
        status = _get_status(uuid, auth=auth)
    print('Finished')
    data = status['data']
    return data


def download_files(files, auth, out_folder=''):
    for fn_link in files:
        file_path = os.path.join(out_folder, os.path.split(fn_link)[1])
        file_uri = ('https://maps.vandersat.com/{fn_link}/download'.format(fn_link=fn_link))
        r = requests.get(file_uri, verify=True, stream=True, auth=auth)
        r.raise_for_status()
        with open(file_path, 'wb') as f:
            for chunk in r.iter_content(1024):
                f.write(chunk)


if __name__ == '__main__':

    auth = ('username', 'password')  # Change username and password
    api_call = ('https://maps.vandersat.com/api/v2/'
                'products/SM-C1N_V001_100/gridded-data?'
                'lat_min=51.5&lat_max=52.5&lon_min=4&lon_max=5&'
                'start_date=2012-10-01&end_date=2012-10-03&'
                'format=gtiff&zipped=false')  # Change desired request
    out_folder = ''  # set output folder (has to exist)

    uuid = submit_job(api_call, auth=auth)
    files = get_data(uuid, auth=auth) # waits for processing to finish
    download_files(files, auth=auth, out_folder=out_folder)

Contact

Address

VanderSat Data Service Wilhelminastraat 43a, 2011 VK, Haarlem, The Netherlands

E-mail

servicedesk@vandersat.com

Standard office hours

9:00 - 17:00 (CET)