Accessing Data¶

WebCOOS provides two main data acccess methods for information and data within the WebCOOS data system: an HTTP API for programmatic access to WebCOOS information and data and a website for discovery, visualization, and documentation of the WebCOOS program and its data holdings.

Website¶

Coming soon…

API¶

Registering¶

First, you must register with the WebCOOS data system to be able to use the API. You can choose to use an external account provider (i.e. Google) to register and login to WebCOOS or create a username and password specifically for WebCOOS. If using an external account provider keep in mind that WebCOOS will never have access to your account password or other private information, your accounts will remain safe.

Authenticating¶

After registering with the WebCOOS system you must obtain your API token from the website by visiting your profile page. Please keep this API token confidential, it identifies your with the data API.

To authenticate with any of the WebCOOS API endpoints you must provide your API token in the HTTP Authorization Header.

Example using httpie:

$ http GET "https://app.webcoos.org/u/api/me/" \
            Authorization:"Token [your_api_token]" \
            Accept:"application/json"
HTTP/1.1 200 OK
Allow: GET, HEAD, OPTIONS
Connection: keep-alive
Content-Length: 754
Content-Type: application/json
...
{
    "username": "kyle",
    "email": "kyle@axiomalaska.com",
    "last_login": "2022-07-13T13:22:03.379876Z",
    ...
}

Example using curl:

$ curl -H "Authorization: Token [your_api_token]" \
       -H "Accept: application/json" \
           https://app.webcoos.org/u/api/me/

{"username":"kyle","email":"kyle@axiomalaska.com","last_login": "2022-07-13T13:22:03.379876Z", ...}%

Example using python and requests

$ python -q
>>> import requests
>>> r = requests.get(
    'https://app.webcoos.org/u/api/me/',
    headers={
        'Authorization': 'Token [your_api_token]',
        'Accept': 'application/json'
    }
)
>>> r.json()
{"username":"kyle","email":"kyle@axiomalaska.com","last_login": "2022-07-13T13:22:03.379876Z", ...}

Usage¶

The WebCOOS API conforms to the OpenAPI 3.0 specification and its definition can be found here. There is also a swagger interface available (be sure to login to the WebCOOS Data System before visiting).

To use the API a few concepts need to be understood. They form the basis for organizing the data contained in the WebCOOS Data System:

Term	Definition
Asset	A device (camera). This concept exists to capture the higher level idea that one named thing captures and produces all sorts of different Products over its lifetime. Each deployment of a camera is an individual “Asset”. An asset has Products, and each or those Products has Services which expose Elements.
Feed	A specific Feed of data coming from an Asset. Typically there will be feeds that represent the raw or level 0 data, such as the raw streaming video data, as well as other feeds representing alternative feeds from the device location, such as air temperature.
Element	A chunk of data. These are typically file-based and the persistent “at rest” representation of data. This could be a video file, image file, CSV file, JSON file, ESRI Shapefile, etc. Elements sometimes don’t make a ton of sense by themselves and require the context of the entire Product or Service they are made available through.
Product	A collection of one or more Elements. One file is an Element. A collection of files is an Product. Products are collection of Elements that are analyzed or otherwise used together and don’t make a ton of sense by themselves. Products would typically be made available or shared and transferred around together. The raw/original data from a camera is an Product and the post-processed data would be another Product. A Product containing a single Element is entirely possible.
Service	A Service exposes Products. A file system, API, of HTTP WAF directory are all examples of Services. Different types of Services exist for describing different types of access patterns.

Pagination¶

Some WebCOOS API requests can return lots and lots of JSON data. Return payloads contain a top-level pagination field to manage iterating over all pages of data and retrieving a full set of data you requested. You can control the size of each page with the page_size parameter and which page is returned with the page parameter. There are also helper URL returned in the pagination section to easily work through all pages of a response.

Example pagination data

{
  "pagination": {
    "next": "https://app.webcoos.org/webcoos/api/v1/assets/?format=json&page=2&page_size=2",
    "previous": null,
    "count": 16,
    "page": 1,
    "total_pages": 8,
    "previous_page": null,
    "next_page": 2,
    "start_index": 1,
    "end_index": 2
  }

Camera List¶

To retrieve a list of Assets (cameras) in the WebCOOS system use the Assets List API endpoint

$ curl -H "Authorization: Token [your_api_token]" \
       -H "Accept: application/json" \
           https://app.webcoos.org/webcoos/api/v1/assets/

This response is intentionally verbose and can use used to populate applications with the location for all cameras and what data feeds, products, and services they provide. High level statistics for each Service are provided including the number of Elements, the total size in bytes of all Elements, and the temporate range of all Elements.

Service Data Inventory¶

Once you have identified a Camera, Feed, Product and Service you would like to explore, you can retrieve an statistical inventory of when data is available through the Service using the Inventory API endpoint. The URL format is as follows: https://app.webcoos.org/webcoos/api/v1/services/[service_slug]/inventory/ where [service_slug] is obtained from the Camera list.

The example below returns the inventory for the Waikiki Sheraton’s video-archive service:

$ curl -H "Authorization: Token [your_api_token]" \
       -H "Accept: application/json" \
           https://app.webcoos.org/webcoos/api/v1/services/waikiki_sheraton-video-archive-s3/inventory/

Elements API¶

Once you have identified a Camera, Feed, Product or Service you would like to explore, you can retrieve a list of Elements contained in that object using the Elements API endpoint.

The Elements response contains metadata including download URLs, binary LQIP data, thumbnail URLs, size (in bytes), the duration (if available), and the temporal range of the Element. You can also filter to only return Elements that are valid between a range of dates.

The example below returns all elements that are part of the Waikiki Sheraton’s video-archive for June 2, 2022.

HTTP Parameters:

starting_after=2022-06-02T00:00:00Z (inclusive)
starting_before=2022-06-03T00:00:00Z (exclusive)
service=waikiki_sheraton-video-archive-s3

$ curl -H "Authorization: Token [your_api_token]" \
       -H "Accept: application/json" \
           "https://app.webcoos.org/webcoos/api/v1/elements/?starting_after=2022-06-02T00%3A00%3A00Z&starting_before=2022-06-03T00%3A00%3A00Z&service=waikiki_sheraton-video-archive-s3"