Overview

Fluentd is a log aggregator and router. The fluentd will simply poke an arbitrary tcp input as a liveness check. Fluentbit has an actual http health check, so it can detect more nuanced issues.

Technical Details

Implementation

Global base check	no
Puppet profiles using this check	profile_fluentd

Check plugins

Name	check_tcp
Packages	standard icinga2 check
CheckCommand name	check_tcp
Upstream source link	https://icinga.com/docs/icinga-2/latest/doc/10-icinga-template-library/#tcp
Documentation link	https://icinga.com/docs/icinga-2/latest/doc/10-icinga-template-library/#tcp

List of variables

Icinga2 variable	Configured in	Default Value	Description
tcp_port	profile_fluentd::fluentd::monitoring_port	1337	An arbitrary tcp port that will be injected into the input config of fluentd
tcp_port	profile_fluentd::fluentbit::monitoring_port	1337	An arbitrary tcp port that will be injected into the input config of fluentbit

Troubleshooting/Known issues

TCP port no reachable

The fluentd or fluentbit service is not running.

Solution

Check the service td-agent (fluentd) or td-agent-bit (fluentbit) on the host.

Fluentbit health check critical

Fluentbit is running, but there is a problem with the log forwarding or ingestion. Check the logs with

journalctl -u td-agent-bit

to find the cause of the issue.

Tuning

Fluentd health check critical

Fluentd is not running, check logs with:

journalctl -u td-agent
less /var/log/td-agent/td-agent.log