Timeseries

Use the Timeseries service to query metrics directly from the Flowmill Timeseries database. A Timeseries query specify:

  • A timerange

  • A metric, such as TCP Packets Dropped or TCP Bytes Transferred

Timeseries may optionally specify filters and groups.

Specifying a Time Duration and Number of Samples

The Flowmill service is built around a timeseries database capable of doing flexible aggregations over time. Queries can be configured to cover both a desirable range of time as well as requesting a given number of data points.

V1 - Start, End, Step Method

We have deprecated version 1 of our Timeseries APIs. Please use version 2.

V2 - End, Duration, Number of Samples Method

In version 2 of the Timeseries APIs, time ranges are specified using an exclusive end timestamp, the duration of the time range requested and the number of values to return.

Specifying the Metric

Requests to the Timeseries API must specify both the metric and timerange to query.

Metrics are currently gathered for 4 procotols:

  • TCP

  • UDP

  • HTTP

  • DNS

To get the full list of metrics to query, please contact Flowmill.

A Sample Request

The following example shows how to query tcp bytes transferred for the last hour, aggregated to a single datapoint:

import datetime
import flowmill
end = datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S.%f')[:-3]+'Z'
duration = '1h'
num_steps = '1'
request = flowmill.GrpcTimeSeriesRequestV2(end=end,
duration=duration,
num_steps=num_steps,
metric='bytes',
)

Limiting the Total Number of Timeseries

Use the top_k parameter to limit the number of timeseries returned.

The following example shows how to query for the 100 highest-traffic connections between services over the the past hour:

import datetime
import flowmill
end = datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S.%f')[:-3]+'Z'
duration = '1h' num_steps = '1'
request = flowmill.GrpcTimeSeriesRequestV2(end=end,
duration=duration,
num_steps=num_steps,
top_k=100,
metric='bytes',
)

Filters

Use filters to limit which timeseries are used to generate the result. Requests may define distinct filters for source and destination or use the same filter if desired. ​ The following example shows how to filter for bytes transferred by a single service in a single availability zone:

import datetime
import flowmill
end = datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S.%f')[:-3]+'Z'
duration = '1h'
num_steps = '1'
source_filters = [flowmill.GrpcFilter('role', 'IN', [role]),
flowmill.GrpcFilter('az', 'IN', [zone])]
request = flowmill.GrpcTimeSeriesRequestV2(end=end,
duration=duration,
num_steps=num_steps,
metric='bytes',
source_filters=source_filters,
)

Groupings

Use groupings to rollup metrics from individual entities into an aggregation of these entities, such as role, availability zone or environment.

The following example shows how to aggregate traffic from a single source service, but breakdown results for the destination by service, zone and ip address:

import datetime
import flowmill
‌end = datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S.%f')[:-3]+'Z'
source_grouping = ['role'] destination_grouping = ['role', 'az', 'ip']
request = flowmill.GrpcTimeSeriesRequestV2(end=end,
duration=duration,
num_steps=num_steps,
metric='bytes',
source_grouping=source_grouping,
destination_grouping=destination_grouping,
)

Matching Source and Destination

Use label_equality to query for things like "cross zone traffic" or "same zone traffic" by performing comparisions on a subset of the labels. ​ The following example shows how to return timeseries where source and destination are in different availability zones:

import datetime
import flowmill
end = datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S.%f')[:-3]+'Z'
duration = '1h'
num_steps = 1
label_equality = [flowmill.GrpcLabelEqualityPair('az', False)]
request = flowmill.GrpcTimeSeriesRequestV2(end=end,
duration=duration,
num_steps=num_steps,
metric='bytes',
label_equality=label_equality
)