Distributed Systems Monitoring
Collected and streamed system metrics in real time through MQTT-based publish/subscribe services running in Docker Compose.
Overview
A distributed monitoring project for collecting CPU, memory and uptime metrics from multiple services in real time.
Problem
Centralize continuous performance data from separate systems without tightly coupling producers and consumers.
Constraints
- Asynchronous communication with low delivery latency.
- Independent maintenance of metric producers and consumers.
- A reproducible containerized multi-service environment.
Approach
Built an MQTT publish/subscribe pipeline with Python services running in a containerized multi-service environment managed through Docker Compose.
Key Decisions
Use MQTT with Eclipse Mosquitto for publish/subscribe communication.
The lightweight protocol fits continuous metric delivery and keeps data producers decoupled from consumers.
Run the services with Docker Compose.
A shared multi-service configuration made the environment reproducible and simplified integration testing.
Tech Stack
- Python
- MQTT (Eclipse Mosquitto)
- Docker
- Docker Compose
- Linux
Result & Impact
Demonstrated end-to-end metric collection and delivery across decoupled services in a reproducible containerized environment.
Learnings
- Asynchronous communication and real-time data streaming.
- Clear service boundaries in a distributed workflow.
- Multi-service integration with Docker Compose.