(Elastic monitoring - 4/5) Collect logs with Elastic Filebeat for monitoring Kubernetes
In the next section of this series, we are now going to install Filebeat, it is a lightweight agent to collect and forward log data to ElasticSearch within the k8s environment (node and pod logs). Moreover, specific modules can be configured to parse and visualise logs format coming from common applications or system (databases, message bus).
Similarly to Metricbeat, Filebeat requires a settings file to configure the connections to ElasticSearch (endpoint, username, password), the connection to Kibana (to import pre-existing dashboards) and the way to collect and parse logs from each container of the k8s environment.
The following ConfigMap represents all the settings needed to capture logs (find more here to customise this config).
configmap/filebeat-config configured configmap/filebeat-indice-lifecycle configured daemonset.extensions/filebeat created clusterrolebinding.rbac.authorization.k8s.io/filebeat created clusterrole.rbac.authorization.k8s.io/filebeat created serviceaccount/filebeat created
Wait until the filebeat pod is Running and you should be able to observe logs in Kibana.
1 2 3 4 5 6
$ kubectl get all -n monitoring -l app=filebeat NAME READY STATUS RESTARTS AGE pod/filebeat-l88qj 1/1 Running 0 4m52s
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/filebeat 1 1 1 1 1 <none> 4m52s
Now Filebeat is up and running, you can observe logs in different ways. From the left menu, click on “Logs” and you can see an aggregated view of all the logs printed from every nodes and containers. You can filter the logs by any attributes attached to the log (for example a kubernetes label) and navigate over the time:
In the Infrastructure view, the logs are now integrated and can be accessed easily for each pod by clicking on “View logs” on a pod or container.
Filebeat comes also with pre-built dashboards imported to Kibana, go to “Dashboards” and you should have a lot of Filebeat dashboards available. We enabled the mongodb module so the dashboard “Overview [Filebeat MongoDB] ECS” should be populated. It give an overview of the state of MongoDB based on the logs (error rate).