7da8cc91a6
* Initial plan * feat: implement centralized log aggregation with Grafana Loki stack - Add Loki for log aggregation and storage with 30-day retention - Configure Promtail for log collection from all services - Set up Grafana with pre-configured logging dashboard - Add structured logging with JSON format and label extraction - Support Docker container log collection via Docker socket - Implement log filtering by service, level, and custom fields - Add comprehensive documentation for logging system - Update Docker Compose for dev and prod environments Co-authored-by: onnwee <211922112+onnwee@users.noreply.github.com> * docs: add validation script and quickstart guide for centralized logging - Add comprehensive validation script to check logging setup - Create quick start guide for users - Validate all configuration files (YAML, JSON, Docker Compose) - All 22 validation checks passing successfully Co-authored-by: onnwee <211922112+onnwee@users.noreply.github.com> * docs: add comprehensive implementation summary - Document all components deployed - Detail configuration highlights and design decisions - Compare with alternative solutions (ELK, CloudWatch) - Provide troubleshooting guide - List future enhancement opportunities - Confirm all success criteria met Co-authored-by: onnwee <211922112+onnwee@users.noreply.github.com> * fix: address code review feedback - Support both docker-compose and docker compose commands - Fix log path duplication in Promtail (separate backend and security logs) - Remove invalid Tempo datasource reference from Grafana config - Add authentication note for production Loki deployments - Update security documentation with Loki auth best practices Co-authored-by: onnwee <211922112+onnwee@users.noreply.github.com> * Update LOGGING.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update docs/LOGGING_IMPLEMENTATION_SUMMARY.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update promtail/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: onnwee <211922112+onnwee@users.noreply.github.com> Co-authored-by: ⓪ηηωεε忧世 <onnweexd@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Loki Configuration
This directory contains the configuration for Grafana Loki, the log aggregation system.
Files
loki-config.yml- Main Loki configuration file
Key Configuration
Retention Policy
- Period: 30 days (720 hours)
- Delete Delay: 2 hours after retention period
- Compaction: Every 10 minutes
Storage
- Type: Filesystem (TSDB)
- Location:
/loki(inside container) - Chunks:
/loki/chunks - Rules:
/loki/rules
Limits
- Ingestion Rate: 15 MB/s
- Burst Size: 20 MB
- Max Entries per Query: 5000
- Max Streams per User: 10000
Customization
To adjust retention period, edit loki-config.yml:
limits_config:
retention_period: 720h # Change this (e.g., 1440h for 60 days)
table_manager:
retention_period: 720h # Keep same as above
Ports
3100- HTTP API9096- gRPC