Installing Agents

Note: Customized deployment instructions with your agent keys already filled in can be found at the Deployment page in the Flowmill UI.
Kubernetes / Helm
Flowmill agents can easily be installed in Kubernetes environments using Helm.
By default, the Helm chart will automatically launch the kernel agents on each Kubernetes node and the Kubernetes collector.
Before starting, make sure you have a valid agent key. Agent keys can be created in the Flowmill UI.
First, add the chart repo to Helm:
helm repo add flowmill https://charts.flowmill.com \
&& helm repo update
Then to install using Helm 3 do the following using the agent key/secret that you generated from the Agents tab:
kubectl create namespace flowmill \
  && helm install flowmill flowmill/flowmill \
  --namespace flowmill \
  --set "auth.keyId=<AGENT_KEY_ID_GOES_HERE>" \
  --set "auth.keySecret=<AGENT_KEY_SECRET_GOES_HERE>" \
  --set "flowmill.clusterName=default"
For Helm 2 do the following:
kubectl create namespace flowmill \
  && helm install flowmill/flowmill \
  --namespace flowmill \
  --name flowmill \
  --set "auth.keyId=<AGENT_KEY_ID_GOES_HERE>" \
  --set "auth.keySecret=<AGENT_KEY_SECRET_GOES_HERE>" \
  --set "flowmill.clusterName=default"
To update an existing installation:
helm repo update \
  && helm upgrade flowmill flowmill/flowmill \
  --namespace flowmill \
  --set "auth.keyId=<AGENT_KEY_ID_GOES_HERE>" \
  --set "auth.keySecret=<AGENT_KEY_SECRET_GOES_HERE>" \
  --set "flowmill.clusterName=default"
Finally, confirm everything is working as expected. There should be one k8s-agent per node.
$ kubectl get pods -n flowmill
NAME                                            READY   STATUS    RESTARTS    AGE
flowmill-k8s-agent-tmxct                        1/1     Running   0          146m
flowmill-k8s-agent-tpt2p                        1/1     Running   0          147m
flowmill-k8s-agent-wlfrc                        1/1     Running   0          146m
flowmill-k8s-aws-collector-6958dbf488-44qgw     1/1     Running   0          148m
flowmill-k8s-k8s-collector-58f755b69f-xd6b4     2/2     Running   0          148m
For more options to configure your Flowmill installation using Helm, please see the Helm Chart's README for a list of configurable values.
Amazon ECS (Elastic Container Service)
Flowmill also supports running agents in ECS environments. In this environment, the Flowmill kernel agent collects metadata directly from the local Docker daemon on each instance and does not create load on the ECS API. These instructions will create an ECS task for the kernel collector and the AWS collector, along with services for each.
Again it is recommended to download task definitions with autogenerated agent keys. However, it is also possible to edit the definitions with existing agent keys.
Next open the task definition for the AWS and kernel collectors.
Update the FLOWMILL_AUTH_KEY_ID and FLOWMILL_AUTH_SECRET if needed 
Set FLOWMILL_AGENT_LABELS_ENVIRONMENT to a value unique to this ECS cluster.  This will be used as the environment label in the Flowmill application.
Next, register the task definitions
aws ecs register-task-definition --cli-input-json file://flowmill-kernel-collector-task-definition.json
aws ecs register-task-definition --cli-input-json file://flowmill-aws-collector-task-definition.json
Create the associated services
aws ecs create-service --cli-input-json file://flowmill-kernel-collector-service.json
aws ecs create-service --cli-input-json file://flowmill-aws-collector-service.json
Docker
The Flowmill kernel collector and AWS collector can also be run directly through Docker.
To run the Flowmill kernel collector, customize the following commands below. Note that the flowmill-agent (ie. kernel collector) must be run on each node while one aws collector should be run per AWS account.
Update the FLOWMILL_AUTH_KEY_ID and FLOWMILL_AUTH_SECRET with values from your agent key
sudo docker run  \
     --env FLOWMILL_AUTH_KEY_ID="TODO_KEY_ID_GOES_HERE" \
     --env FLOWMILL_AUTH_SECRET="TODO_SECRET_GOES_HERE" \
     --env FLOWMILL_INTAKE_HOST="intake.flowmill.com" \
     --env FLOWMILL_AUTHZ_SERVER="app.flowmill.com" \
     --env FLOWMILL_INTAKE_PORT=443 \
     --privileged \
     --pid host \
     --network host \
     --volume /sys/fs/cgroup:/hostfs/sys/fs/cgroup \
     --volume /usr/src:/var/run/flowmill/host/usr/src \
     --volume /lib/modules:/var/run/flowmill/host/lib/modules \
     --volume /etc:/var/run/flowmill/host/etc \
     --volume /var/cache:/var/run/flowmill/host/cache \
     --volume /var/run/docker.sock:/var/run/docker.sock \
     --name flowmill-agent \
     flowmill/agent \
       --log-console
sudo docker run  \
     --env FLOWMILL_AUTH_KEY_ID="TODO_KEY_ID_GOES_HERE" \
     --env FLOWMILL_AUTH_SECRET="TODO_SECRET_GOES_HERE" \
     --env FLOWMILL_INTAKE_HOST="intake.flowmill.com" \
     --env FLOWMILL_AUTHZ_SERVER="app.flowmill.com" \
     --env FLOWMILL_INTAKE_PORT=443 \
     --name flowmill-aws-collector \
     flowmill/aws-collector \
       --log-console
Nomad
Flowmill agents can be run in the Nomad environment as well. The jobspec below can be used to get started. You will need to fill in the "TODO" items below.
# nomad jobspec for flowmill-kernel-collector
​
job "flowmill-kernel-collector" {
  datacenters = [
    "<TODO: DATA CENTERS WHERE THIS SHOULD BE DEPLOYED>",
  ]
  group "kernel-collector" {
    task "kernel-collector" {
      driver = "docker"
      env {
        FLOWMILL_CLUSTER_NAME="<TODO: YOUR CLUSTER NAME GOES HERE>"
        FLOWMILL_AGENT_CLUSTER="<TODO: YOUR CLUSTER NAME GOES HERE>"
        FLOWMILL_AUTH_KEY_ID="<TODO: AGENT KEY ID>"
        FLOWMILL_AUTH_SECRET="<TODO: AGENT SECRET>"
        FLOWMILL_INTAKE_HOST="intake.flowmill.com"
        FLOWMILL_INTAKE_PORT="443"
        FLOWMILL_AUTHZ_SERVER="app.flowmill.com"
        FLOWMILL_CRASH_COLLECTOR_HOST="app.flowmill.com"
        FLOWMILL_KERNEL_HEADERS_AUTO_FETCH="true"
        FLOWMILL_DISABLE_HTTP_METRICS="false"
      }
      config {
        image = "flowmill/agent:latest-v0.8"
        args = [
          "--log-console",
          "--force-docker-metadata",
        ]
        volumes = [
          "/etc:/var/run/flowmill/host/etc",
          "/lib/modules:/var/run/flowmill/host/lib/modules",
          "/sys/fs/cgroup:/hostfs/sys/fs/cgroup",
          "/usr/src:/var/run/flowmill/host/usr/src",
          "/var/cache:/var/run/flowmill/host/cache",
          "/var/run/docker.sock:/var/run/docker.sock",
        ]
        privileged = true
        network_mode = "host"
        pid_mode = "host"
      }
      resources {
        cpu    = 1500 # 1500 MHz
        memory = 500  # 500 MB
      }
    }
    count = 1
    restart {
      attempts = 2
      interval = "30m"
      delay = "15s"
      mode = "delay"
    }
    ephemeral_disk {
      size = 300
    }
  }
  type = "system"
  update {
    max_parallel = 1
    min_healthy_time = "10s"
    healthy_deadline = "3m"
    progress_deadline = "10m"
    auto_revert = true
    canary = 0
  }
}
Verifying Agents
The Flowmill agent tab provides real time information on agents that have connected.
If you do not see your agents running, please check out the Troubleshooting page.
Getting Started - Previous
Prerequisites
Next - Getting Started
Managing Users
Last updated 4 months ago