Delivery Stats API Guide

Overview

This document describes using the Instart Logic statistics API to make specific queries against the aggregated delivery statistics collected by our log processing system.

Statistics queries

The query structure is appended to the API URL as a standard query string, as follows:

https://api.instartlogic.com/<your unique customer name>/v1/stats/data?
dimensions=<dimension1,dimension2,...>
&granularity=[day|hour|5minute]
&metrics=<metrics1,metrics2,...>
&startime=<UTC date/time of the desired beginning time>
&endtime=<UTC date/time of the desired end time>
[&filters=<filter expression 1,...>]
[&sortby=<dimension1,metric2,...>]
[&limit=<number of results desired>]
[&start=<integer index into the result>]

(Line breaks are added above for readability.)

To be valid, each query requires

The additional query parameters filters, sortby, limit and start may be provided to further refine your query, but are optional.

All query parameters must be URL-encoded.

Metrics

The metrics parameter is a list of one or more of the available statistics, such as number of requests, time to first byte, etc. A list of valid metrics (as of this writing) can be found here.

Dimensions

The dimensions parameter contains a list of one or more elements which correspond to features that characterize the aggregated statistics. For example, country, device type and browser_name are dimensions that specify from which country, device type and browser a user made the request from. A list of valid dimensions can be found here.

The list of dimensions you specify must correspond to a valid grouping set. The list of valid grouping sets can be found here.

Granularity

The granularity parameter corresponds to the time granularity of the aggregated access log data you want to retrieve.

The possible values are

Statistics data is aggregated as follows:

When making a query you need to specify an appropriate granularity depending on the span of time you want the data for. For example, if you specify a span from last night at midnight to noon today, it doesn't make sense to specify a granularity of day. Similarly, if you specify a span that ended longer than three months ago, data is only available at the granularity of a day, so it doesn't make sense to specify it as hour or 5min. If you specify a span that ended longer than one week ago but less than three months ago, data is available at both day and hour granularities.

Filters

The filters parameter contains a comma-separated list of filters to be applied to the requested data. A filter is of the form:

<dimension><operator><value>

For example:

http_status!=400

where http_status is the dimension, != is the operator (not equal to) and 400 is the value. This filter, if added to a query, would get back only the results that have an HTTP status code other than 400.

Valid operators are == and != (remember that all query parameters must be URL-encoded.)

Multiple individual filters can be combined using the OR(,) or AND(;) operators – but not both – to form a logical filter expression.

NOTE If the special characters "," or ";" need to be treated as literal characters in a filter expression (for example, the comma character occurs in some country strings you might want to filter on), you need to escape them by prepending them with "\"

For example:

filters=http_status!=400;browser_name==Chrome

contains two filters: the first one to allow only results with an HTTP status code other than 400, and a second one to allow only results that came from requests from the Chrome browser. They are separated by a semicolon, which means AND. So this pair of filters would return only results with an HTTP status code other than 400 AND were requests from Chrome.

NOTE For the initial version of the Delivery Stats API, filters have the following limitations:
  • We do not support specifying filters on metrics.
  • The filters string can only consist of either the OR(,) or the AND(;) operator, not both. There is currently no support for complex logical expressions based on either operator precedence or using explicit parentheses.
  • We do not support specifying filters containing regular expressions.

starttime

The starttime parameter is the start time of the desired time range of the desired statistics, which corresponds to when the user requests were received as logged in the access logs. It is required and cannot be left empty. It must be specified in standard ISO8601 date/time format, but always in the UTC time zone. Don't specify a time zone designator or the request will be rejected. Start times must be earlier than the end time to be valid.

endtime

The endtime parameter is the end time of the desired time range of the statistics, which corresponds to when the user requests were received as logged in the access logs. It is required and cannot be left empty. It must be specified in standard ISO8601 date/time format, but always in the UTC time zone (specifying another time zone designator will cause the request to be rejected). End times must be later than the start time to be valid.

sortby

The sortby parameter contains a list of one or more dimensions and/or metrics over which the resultant statistics data should be sorted. For example:

sortby=time_to_first_byte_ms

Sorting order is specified by the left to right order of the metrics and dimensions listed. Sorting direction defaults to ascending and can be changed to descending by using a minus sign (-) prefix on the requested field.

limit

The limit parameter is an integer that will restrict the total number of results that are returned by the query. For example, using

limit=25

will tell the Delivery Stats API server to return only the first 25 results.

If unspecified, the limit parameter defaults to returning 2500 results.

The limit parameter has a maximum specifiable value of 2500 per request. If set to anything greater than 2500, only 2500 results will be returned.

start

The Delivery Stats API service has a maximum number of 2500 rows it will send back in its response to the query. To get more data, you can use the start parameter to specify an integer index into the data returned by the query. For example, if you set start to 5000, the next 2500 rows of data is returned to you, along with the URIs to fetch the next 2500 and the previous 2500 when they are present.

For example, if the full data set contains 3000 results, setting start to 50 means that the response will have the rows 51-1450, plus a URI that will request the next rows 1451-2850 from the Stats service and the URI for the rows 1-50. The next URI can be used to request the next 1400 rows (1451-2850), and the previous URI for the previous 50 (1-50). However, if there are no more/less rows in the returned data set, the next/previous URI would be blank.

Also, note that if you have also chosen to limit the results returned, a start index greater than the limit value makes no sense.

Some query examples

The following are some examples of using the Delivery Stats API to query for statistics. Line breaks are added for readability.

Example 1

The following query requests the number of hits and request times, aggregated by day, grouped by country, device OS, and browser name, that occurred between 10 September 2017 at midnight (UTC) and 15 September 2017 at noon (UTC).

Using cURL:

Execute the command

$ curl -v -u 'username:password' 'https://api.instartlogic.com/acme/v1/stats/data?
dimensions=country,device_os,browser_name
&metrics=hits,request_time_ms
&granularity=day
&starttime=2017-09-10T00:00:00
&endtime=2017-09-15T12:00:00'

where you would of course use your own authorization string and company name in place of the placeholders, and remove the line breaks that were added for readability.

Using Advanced REST client plugin for Chrome:

  1. In the URL field at the top enter

    https://api.instartlogic.com/acme/v1/stats/data

    where you would of course use your own company name in place of the placeholder.

  2. Click the grey triangle to the left of the URL field to display the Query Parameters form. Click the Add button to the right to add a new query string parameter field, enter the name and associated value, and repeat for each of the following each of the query string parameter names and values:
    • dimensions=country,device_os,browser_name,customer
    • metrics=hits,request_time_ms
    • granularity=day
    • starttime=2017-09-10T00:00:00
    • endtime=2017-09-15T12:00:00
  3. Click the GET radio button.
  4. In the Headers field, add a header named Authorization with a value

    Basic <base64-encoded version of your authorization string>

  5. Click Send.

If successful the API server will respond with a JSON object similar to the following (for brevity only a few rows are shown of the full number actually returned):

{
    "columns": [
        {
            "name": "starttime",
            "type": "string"
        },
        {
            "name": "endtime",
            "type": "string"
        },
        {
            "name": "country",
            "type": "string"
        },
        {
            "name": "device_os",
            "type": "string"
        },
        {
            "name": "browser_name",
            "type": "string"
        },
        {
            "name": "hits",
            "type": "long"
        },
        {
            "name": "request_time_ms",
            "type": "long"
        }
    ],
    "query": {
        "dimensions": ["country","device_os","browser_name"],
        "endtime": "2017-09-15T12:00:00",
        "filters": null,
        "granularity": "day",
        "limit": 2500,
        "metrics": ["hits","request_time_ms"],
        "sortby": null,
        "start": 0,
        "starttime": "2017-09-10T00:00:00"
    },
    "rows": [
        [
            "2017-09-10T00:00:00",
            "2017-09-02T00:00:00",
            "India",
            "Windows",
            "Firefox",
            140,
            30732
        ],
        ...
        [
            "2017-09-15T00:00:00",
            "2017-09-06T00:00:00",
            "United States",
            "OS X",
            "Chrome",
            4,
            1170
        ]
    ],
    "total_results": 2142,
    "uri": "/instartlogic/stats/data"
}

Example 2

The following query requests the number of hits, last mile bandwidth, and middle mile bandwidth, aggregated by day, and grouped by domain and country, that occurred between 11 September 2017 at midnight (UTC) and 29 September 2017 at midnight (UTC),filtered by country to show only results where thc country was "Phillipines."

Using cURL:

Execute the command

  curl -v -u 'username:password'
  'https://api.instartlogic.com/instartlogic/v1/stats/data?
  dimensions=domain,country
  &metrics=hits,last_mile_bandwidth_bytes,middle_mile_bandwidth_bytes
  &granularity=day
  &filters=country==Philippines
  &starttime=2017-09-11T00:00:00
  &endtime=2017-09-29T00:00:00'

where you would of course use your own authorization string and company name in place of the placeholders, and remove the line breaks that were added for readability. Notice also the &20 in the filters value for the blank space character – a plus sign (+) also works to replace a space.

Using Advanced REST client plugin for Chrome:

  1. In the URL field at the top enter

    https://api.instartlogic.com/acme/v1/stats/data

    where you would of course use your own company name in place of the placeholder.

  2. Click the grey triangle to the left of the URL field to display the Query Parameters form. Click the Add button to the right to add a new query string parameter field, enter the name and associated value, and repeat for each of the following each of the query string parameter names and values:
    • dimensions=domain,country
    • metrics=hits,last_mile_bandwidth_bytes,middle_mile_bandwidth_bytes
    • granularity=day
    • starttime=2017-09-11T00:00:00
    • endtime=2017-09-29T00:00:00
  3. Click the GET radio button.
  4. In the Headers field, add a header named Authorization with a value

    Basic <base64-encoded version of your authorization string>

  5. Click Send.

If successful the API server will respond with a JSON object similar to the following (for brevity only a few rows are shown of the full number actually returned):

{
  "uri": "/acme/stats/data",
  "query": {
    "metrics": [
      "hits",
      "last_mile_bandwidth_bytes",
      "middle_mile_bandwidth_bytes"
    ],
    "dimensions": [
      "domain",
      "country"
    ],
    "granularity": "day",
    "sortby": null,
    "filters": [
      "country==Philippines"
    ],
    "starttime": "2017-09-11T00:00:00",
    "endtime": "2017-09-29T00:00:00",
    "start": 0,
    "limit": 2500
  },
  "columns": [
    {
      "name": "starttime",
      "type": "string"
    },
    {
      "name": "endtime",
      "type": "string"
    },
    {
      "name": "domain",
      "type": "string"
    },
    {
      "name": "country",
      "type": "string"
    },
    {
      "name": "hits",
      "type": "long"
    },
    {
      "name": "last_mile_bandwidth_bytes",
      "type": "long"
    },
    {
      "name": "middle_mile_bandwidth_bytes",
      "type": "long"
    }
  ],
  "rows": [
    [
      "2017-09-11T00:00:00",
      "2017-09-12T00:00:00",
      "support.acme.com",
      "Philippines",
      12,
      85667,
      0
    ],
    [
      "2017-09-11T00:00:00",
      "2017-09-12T00:00:00",
      "www.acme.com",
      "Philippines",
      1050,
      19019980,
      13589723
    ],
    [
      "2017-09-11T00:00:00",
      "2017-09-12T00:00:00",
      "app.acme.com",
      "Philippines",
      118,
      9025470,
      3380639
    ],
    [
      "2017-09-11T00:00:00",
      "2017-09-12T00:00:00",
      "login.acme.com",
      "Philippines",
      68,
      2491572,
      1439469
    ],
    [
      "2017-09-11T00:00:00",
      "2017-09-12T00:00:00",
      "acme.com",
      "Philippines",
      97,
      44698,
      0
    ],
    [
      "2017-09-12T00:00:00",
      "2017-09-13T00:00:00",
      "login.acme.com",
      "Philippines",
      178,
      6365658,
      2867015
    ],
    [
      "2017-09-12T00:00:00",
      "2017-09-13T00:00:00",
      "acme.com",
      "Philippines",
      80,
      36866,
      0
    ],
    ...

    [
      "2017-09-28T00:00:00",
      "2017-09-29T00:00:00",
      "www.acme.com",
      "Philippines",
      421,
      5228304,
      3523181
    ],
    [
      "2017-09-28T00:00:00",
      "2017-09-29T00:00:00",
      "acme.com",
      "Philippines",
      80,
      36872,
      0
    ]
  ],
  "total_results": 84
}

Try this query with and without the filters field; when it's not present you will notice the addtional rows of results from other countries.

Example 3

The following query requests the time to first byte aggregated by 5 minute intervals, sorted in descending order, grouped by country codes, content type, and property, which occurred between 4 September 2017 at midnight (UTC) and 11 September 2017 at 6am (UTC), starting with the 10th result, and returning only 10 results.

Using cURL:

Execute the command

$ curl -v -u 'username:password' 'https://api.instartlogic.com/acme/v1/stats/data?
dimensions=country,content_type,property
&metrics=time_to_first_byte_ms
&granularity=5minute
&sortby=-time_to_first_byte_ms
&starttime=2017-09-04T00:00:00
&endtime=2017-09-11T06:00:00
&start=10
&limit=10'

where you would of course use your own authorization string and company name in place of the placeholders, and remove the line breaks that were added for readability.

Using Advanced REST client plugin for Chrome:

  1. In the URL field at the top enter

    https://api.instartlogic.com/acme/v1/stats/data

    where you would of course use your own company name in place of the placeholder.

  2. Click the grey triangle to the left of the URL field to display the Query Parameters form. Click the Add button to the right to add a new query string parameter field, enter the name and associated value, and repeat for each of the following each of the query string parameter names and values:
    • dimensions=country,content_type,property
    • metrics=time_to_first_byte_ms
    • granularity=5min
    • sortby=-time_to_first_byte_ms
    • starttime=2017-09-04T00:00:00
    • endtime=2017-09-11T06:00:00
    • start=10
    • limit=10
  3. Click the GET radio button.
  4. In the Headers field, add a header named Authorization with a value

    Basic <base64-encoded version of your authorization string>

  5. Click Send.

Try this with and without the sortby field; when it's not present the result rows are not sorted.

Also notice the additional prev_uri field after the columns block:

{
   {
    "columns": [
        {
            "name": "starttime",
            "type": "string"
        },
        {
            "name": "endtime",
            "type": "string"
        },
        {
            "name": "country",
            "type": "string"
        },
        {
            "name": "content_type",
            "type": "string"
        },
        {
            "name": "property",
            "type": "string"
        },
        {
            "name": "time_to_first_byte_ms",
            "type": "long"
        }
    ],
    "next_uri": "/stats/data?dimensions=country,content_type,property&metrics=time_to_first_byte_ms&granularity=5minute&starttime=2017-09-04T00:00:00&endtime=2017-09-11T06:00:00&sortby=-time_to_first_byte_ms&start=20&limit=10",
    "prev_uri": "/stats/data?dimensions=country,content_type,property&metrics=time_to_first_byte_ms&granularity=5minute&starttime=2017-09-04T00:00:00&endtime=2017-09-11T06:00:00&sortby=-time_to_first_byte_ms&start=0&limit=10",
    "query": {
        "dimensions": ["country","content_type","property"],
        "endtime": "2017-09-11T06:00:00",
        "filters": null,
        "granularity": "5minute",
        "limit": 10,
        "metrics": ["time_to_first_byte_ms"],
        "sortby": ["-time_to_first_byte_ms"],
        "start": 10,
        "starttime": "2017-09-04T00:00:00"
    },
    "rows": [
        [
            "2017-09-06T15:25:00",
            "2017-09-06T15:30:00",
            "United States",
            "text/html",
            "public",
            35384
        ],
        [
            "2017-09-15T19:25:00",
            "2017-09-15T19:30:00",
            "United States",
            "text/html",
            "public",
            35260
        ],
        [
            "2017-09-06T20:35:00",
            "2017-09-06T20:40:00",
            "United States",
            "text/html",
            "public",
            34720
        ],
        [
            "2017-09-10T08:10:00",
            "2017-09-10T08:15:00",
            "Netherlands",
            "text/html",
            "public",
            32280
        ],
        [
            "2017-09-07T05:50:00",
            "2017-09-07T05:55:00",
            "India",
            "image/jpeg",
            "public",
            30185
        ],
        [
            "2017-09-06T06:50:00",
            "2017-09-06T06:55:00",
            "India",
            "image/jpeg",
            "public",
            26299
        ],
        [
            "2017-09-06T12:05:00",
            "2017-09-06T12:10:00",
            "Germany",
            "text/html",
            "newportal",
            26196
        ],
        [
            "2017-09-08T03:50:00",
            "2017-09-08T03:55:00",
            "India",
            "application/json",
            "community",
            25787
        ],
        [
            "2017-09-09T15:00:00",
            "2017-09-09T15:05:00",
            "Spain",
            "application/json",
            "community",
            25403
        ],
        [
            "2017-09-15T19:45:00",
            "2017-09-15T19:50:00",
            "United States",
            "text/html",
            "public",
            24816
        ]
    ],
    "total_results": 89125,
    "uri": "/instartlogic/stats/data"
}

This is present because we used the start field in the query string to return the 11th through 20th results from the data set of 89125 rows, and the API accordingly provides the URI to return the 1st through 10th as well as the URI to return the 21st through 30th.

Valid dimensions

You can use the dimensions method

$ curl -v -u 'username:password' 'https://api.instartlogic.com/acme/v1/stats/dimensions

to return a list of the valid dimensions of the aggregated statistics.

The following is the list of dimensions supported as of this writing (September 2017):

Valid dimension groupings

You can use the groupings method

$ curl -v -u 'username:password' 'https://api.instartlogic.com/acme/v1/stats/groupings

to return the list of the valid groupings of the aggregated statistics.

The following is the list of groupings supported as of this writing (September 2017). Since there are 118 valid groupings, the list is collapsed; click the + to expand.

Valid metrics

You can use the metrics method

$ curl -v -u 'username:password' 'https://api.instartlogic.com/acme/v1/stats/metrics

to return the list of the valid metrics of the aggregated statistics.

The following is the list of metrics supported as of this writing (September 2017).

Values of cache_status/gna_cache_status/scheme

If you query for the metrics cache_status, gna_cache_status, or scheme, note that the values returned for these will be integers. This is because these fields are defined as enumerated values so the API returns the index. The following is the mapping from these integers to the meaningful value names.

Values for cache_status and gna_cache_status

Value returned Meaning
1A cache hit
2A cache miss
3The object has expired
4Cache was bypassed
5A failure occured
NOTE If you see a cache_status of 0, this is because the request data does not have cache_status and it either also does not have a gna_cache_status, or the gna_cache_status is anything but 1 (hit).

Values for scheme

Value returned Meaning
1HTTP
2HTTPS