Member-only story
Streamlining Kubernetes Logging with Fluentd and Fluent Bit
Integrating Fluentd and Fluent Bit with a Go-Based Task Manager

Introduction and Architecture
In this blog, we’ll delve into the practical aspects of setting up and managing Kubernetes logging using Fluentd and Fluent Bit, demonstrated through a simple Go task management application running in a Minikube cluster. The below diagram illustrates the architecture of our setup, where the application logs are collected from the
golang-logging-demo
namespace and forwarded to the Fluentd and Fluent Bit services deployed in a separate namespace.

Here’s a breakdown of the setup:
- Golang Logging Demo: This namespace hosts a deployment of our Go-based task management app, consisting of two pods (
APP-POD-1
andAPP-POD-2
). These pods generate logs that are stored in the/var/log/containers/*.log
directory. - Fluentd & Fluent Bit: In a separate namespace, we have deployments for Fluentd and Fluent Bit, two powerful log processing tools. Fluentd is responsible for aggregating and routing logs, while Fluent Bit, running as a DaemonSet, is lightweight and efficient, designed for…