Flowmill agents can easily be installed in Kubernetes environments using Helm. Note that you should already have a
flowmill.yaml file customized with an agent key after completing the Prerequisites.
By default, the Helm chart will automatically launch the kernel agents on each Kubernetes node and the Kubernetes collector and AWS collector.
Before starting, make sure you have a valid agent key and
flowmill.yaml. This file can be automatically generated by the Flowmill UI in the agent tab.
flowmill.yaml in a text editor. If you did not autogenerate the keyId and secret, enter values for a valid agent key and note the tenantName will be different. Additionally, edit the 'clusterName' to be a value unique in your tenant. This will be used as the "Environment" label in the Flowmill application.
# TODO: Update these values based on your deploymentflowmill:clusterName: YOUR_CLUSTER_NAMEtenantName: YOUR_TENANT_NAME# TODO: Log in to https://app.flowmill.com/agents?activeTab=agent-keys# to generate an agent key, and fill in the values hereagent:keyId: "TODO_AGENT_KEY"secret: "TODO_AGENT_SECRET"# You can also use a kubernetes secret to provide your key# If this is not null, the chart will ignore the key id and secret settings above# and use this insteadagentExistingSecret: null
Now its time to install the agents. First, add the chart repo to Helm:
helm repo add flowmill https://charts.flowmill.com \&& helm repo update
Then to install using Helm 3 do the following using the
flowmill.yaml file that you generated from the Agents tab:
kubectl create namespace flowmill \&& helm install -n flowmill --values flowmill.yaml flowmill flowmill/flowmill-k8s
For Helm 2 do the following:
helm install flowmill/flowmill-k8s --name flowmill-k8s -n flowmill --values flowmill.yaml
To update an existing installation:
helm repo update \&& helm upgrade flowmill-k8s flowmill/flowmill-k8s -f flowmill.yaml -n flowmill
Finally, confirm everything is working as expected. There should be one k8s-agent per node.
$ kubectl get pods -n flowmillNAME READY STATUS RESTARTS AGEflowmill-k8s-agent-4ndcj 1/1 Running 0 15hflowmill-k8s-agent-kp6v6 1/1 Running 0 15hflowmill-k8s-agent-mj5qk 1/1 Running 0 15hflowmill-k8s-collector-577dd97f74-v2qhq 3/3 Running 0 15h
Flowmill also supports running agents in ECS environments. In this environment, the Flowmill kernel agent collects metadata directly from the local Docker daemon on each instance and does not create load on the ECS API. These instructions will create an ECS task for the kernel collector and the AWS collector, along with services for each.
Again it is recommended to download task definitions with autogenerated agent keys. However, it is also possible to edit the definitions with existing agent keys.
Next open the task definition for the AWS and kernel collectors.
Update the FLOWMILL_AUTH_KEY_ID and FLOWMILL_AUTH_SECRET if needed
Note that the FLOWMILL_INTAKE_NAME may be different based on your tenant.
Set FLOWMILL_AGENT_LABELS_ENVIRONMENT to a value unique to this ECS cluster. This will be used as the environment label in the Flowmill application.
Next, register the task definitions
aws ecs register-task-definition --cli-input-json file://flowmill-kernel-collector-task-definition.jsonaws ecs register-task-definition --cli-input-json file://flowmill-aws-collector-task-definition.json
Create the associated services
aws ecs create-service --cli-input-json file://flowmill-kernel-collector-service.jsonaws ecs create-service --cli-input-json file://flowmill-aws-collector-service.json
The Flowmill kernel collector and AWS collector can also be run directly through Docker.
To run the Flowmill kernel collector, customize the following commands below. Note that the flowmill-agent (ie. kernel collector) must be run on each node while one aws collector should be run per AWS account.
Update the FLOWMILL_AUTH_KEY_ID and FLOWMILL_AUTH_SECRET with values from your agent key
Update FLOWMILL_INTAKE_NAME based on your tenant name
sudo docker run \--env FLOWMILL_AUTH_KEY_ID="TODO_KEY_ID_GOES_HERE" \--env FLOWMILL_AUTH_SECRET="TODO_SECRET_GOES_HERE" \--env FLOWMILL_INTAKE_NAME="flowmill.intake.flowmill.com" \--env FLOWMILL_INTAKE_HOST="intake.flowmill.com" \--env FLOWMILL_AUTHZ_SERVER="app.flowmill.com" \--env FLOWMILL_INTAKE_PORT=443 \--privileged \--pid host \--network host \--volume /sys/fs/cgroup:/hostfs/sys/fs/cgroup \--volume /usr/src:/var/run/flowmill/host/usr/src \--volume /lib/modules:/var/run/flowmill/host/lib/modules \--volume /etc:/var/run/flowmill/host/etc \--volume /var/cache:/var/run/flowmill/host/cache \--volume /var/run/docker.sock:/var/run/docker.sock \--name flowmill-agent \flowmill/agent \--log-console
sudo docker run \--env FLOWMILL_AUTH_KEY_ID="TODO_KEY_ID_GOES_HERE" \--env FLOWMILL_AUTH_SECRET="TODO_SECRET_GOES_HERE" \--env FLOWMILL_INTAKE_NAME="flowmill.intake.flowmill.com" \--env FLOWMILL_INTAKE_HOST="intake.flowmill.com" \--env FLOWMILL_AUTHZ_SERVER="app.flowmill.com" \--env FLOWMILL_INTAKE_PORT=443 \--name flowmill-aws-collector \flowmill/aws-collector \--log-console
The Flowmill agent tab provides real time information on agents that have connected.