DFMS — Distributed File Management System
A highly scalable file storage platform built as production-grade microservices, with low-latency gRPC communication and multi-tiered Redis rate limiting.
- ▸ Production microservices: Architected a scalable storage platform using gRPC for low-latency inter-service communication and Redis for multi-tiered rate limiting.
- ▸ Optimized storage engine: Engineered an advanced storage algorithm leveraging Content-Defined Chunking (CDC) and SHA-256 deduplication to drastically minimize redundant data.
- ▸ Fault-tolerant replication: Designed a resilient, event-driven replication manager using Kafka, a consistent hashing ring, and active health probing for automated node failover.
- Go
- gRPC
- PostgreSQL
- Redis
- Kafka
- MinIO
- Docker
- OpenTelemetry