Member-only story

Streamlining Kubernetes Logging with Fluentd and Fluent Bit

Kamalesh D
Stackademic
Published in
5 min readAug 31, 2024

Integrating Fluentd and Fluent Bit with a Go-Based Task Manager

Fluentd and Fluent-BIT

Introduction and Architecture

In this blog, we’ll delve into the practical aspects of setting up and managing Kubernetes logging using Fluentd and Fluent Bit, demonstrated through a simple Go task management application running in a Minikube cluster. The below diagram illustrates the architecture of our setup, where the application logs are collected from the golang-logging-demonamespace and forwarded to the Fluentd and Fluent Bit services deployed in a separate namespace.

High Level Architecture of the Infrastructure..

Here’s a breakdown of the setup:

  1. Golang Logging Demo: This namespace hosts a deployment of our Go-based task management app, consisting of two pods (APP-POD-1 and APP-POD-2). These pods generate logs that are stored in the /var/log/containers/*.log directory.
  2. Fluentd & Fluent Bit: In a separate namespace, we have deployments for Fluentd and Fluent Bit, two powerful log processing tools. Fluentd is responsible for aggregating and routing logs, while Fluent Bit, running as a DaemonSet, is lightweight and efficient, designed for…

--

--

Published in Stackademic

Stackademic is a learning hub for programmers, devs, coders, and engineers. Our goal is to democratize free coding education for the world.

Written by Kamalesh D

DevOps and Cloud Enthusiast | Cyber Security | DevSecOps Engineer@FordMotors | https://dev.to/kamaleshseervi

No responses yet

What are your thoughts?