Prerequisites

Overview

Flowmill supports several agents designed to automatically collect data from the environment.

Name

Deployment Type

Description

Kernel Agent

Daemonset (1 per VM / host)

eBPF-based collector that tracks sockets / processes / containers

AWS Collector

1 cluster AWS account

Collects AWS metadata on network interfaces and regions in the AWS account

Kubernetes Collector

1 per K8s cluster

Collects metadata from the Kubernetes masters on pods and deployments

Flowmill Kernel Agent

Flowmill agents use eBPF (extended Berkeley Packet Filter) to collect data from the operating system on every connection.

It is supported on the following Linux systems including:

  • RHEL / Centos 7.6, 7.7, 8.x

  • Ubuntu 16.04 or later

  • Debian Stretch or later

  • Amazon Linux / Amazon Linux 2

  • GCP COS

  • Other Linux systems with 4.4+ kernels

The Flowmill agent requires kernel header packages to be installed. The agents will do this automatically on Debian, Red Hat/ Centos, Amazon Linux, and GCP COS.

Note: Red Hat Openshift requires special configuration. Please contact us if you need help installing in this environment.

Amazon Web Services Collector

Flowmill includes an Amazon Web Services (AWS) collector. This agent requires that the instance on which it runs has read-only AIM permissions for “EC2:DescribeNetworkInterfaces” and “EC2:DescribeRegions”. This can be most easily be accomplished by attaching the “AmazonEC2ReadOnlyAccess” policy to the IAM role used for your instances.

For background on creating IAM roles and attaching policies, please see: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html

Kubernetes Collector

Flowmill supports Kubernetes environments with no additional configuration. It includes a k8s-collector that collects metadata from the Kubernetes APIs and relays it to the Flowmill service. These agents require no configuration when deployed via the helm charts.

Agent Keys

Agent keys, accessible via the Agent tab, are used to authenticate and secure agent traffic sent to the Flowmill service. You can create as many agent keys as you choose although it is recommended you rotate them on a regular basis.

The agent key consists of two pieces of information:

  • Key ID: A 'public' unique identifier for the key.

  • Secret: A private secret that should be used alongside it. As the name implies, the secret should be kept in a secure location.

Both pieces of information must be passed to the agent when it started to allow it to properly connect to the Flowmill service. Agent keys will no longer be valid once they are deleted.

Agent keys can be created in the Flowmill UI agent tab.

Configuring Environments

Flowmill uses "environment" to identify separate collections of hosts and services. Flowmill will not aggregate data across environments so it could be thought of as a high level grouping mechanism. Environments are user-defined although this capability can be used to:

  • Separating dev/staging/prod

  • Identifying independent Kubernetes / Nomad clusters

  • Labeling groups of workloads in a datacenter

In general, environments are a flexible mechanism to separate out services in the Flowmill application.

SELinux

SELinux must be configured to enable the Flowmill kernel collector to run. Please contact us if you need help installing in this environment.