7da8cc91a6
* Initial plan * feat: implement centralized log aggregation with Grafana Loki stack - Add Loki for log aggregation and storage with 30-day retention - Configure Promtail for log collection from all services - Set up Grafana with pre-configured logging dashboard - Add structured logging with JSON format and label extraction - Support Docker container log collection via Docker socket - Implement log filtering by service, level, and custom fields - Add comprehensive documentation for logging system - Update Docker Compose for dev and prod environments Co-authored-by: onnwee <211922112+onnwee@users.noreply.github.com> * docs: add validation script and quickstart guide for centralized logging - Add comprehensive validation script to check logging setup - Create quick start guide for users - Validate all configuration files (YAML, JSON, Docker Compose) - All 22 validation checks passing successfully Co-authored-by: onnwee <211922112+onnwee@users.noreply.github.com> * docs: add comprehensive implementation summary - Document all components deployed - Detail configuration highlights and design decisions - Compare with alternative solutions (ELK, CloudWatch) - Provide troubleshooting guide - List future enhancement opportunities - Confirm all success criteria met Co-authored-by: onnwee <211922112+onnwee@users.noreply.github.com> * fix: address code review feedback - Support both docker-compose and docker compose commands - Fix log path duplication in Promtail (separate backend and security logs) - Remove invalid Tempo datasource reference from Grafana config - Add authentication note for production Loki deployments - Update security documentation with Loki auth best practices Co-authored-by: onnwee <211922112+onnwee@users.noreply.github.com> * Update LOGGING.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update docs/LOGGING_IMPLEMENTATION_SUMMARY.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update promtail/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: onnwee <211922112+onnwee@users.noreply.github.com> Co-authored-by: ⓪ηηωεε忧世 <onnweexd@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Grafana Configuration
This directory contains provisioning configuration for Grafana.
Structure
grafana/
├── provisioning/
│ ├── datasources/
│ │ └── loki.yml # Loki and Prometheus datasources
│ ├── dashboards/
│ │ ├── dashboard.yml # Dashboard provider config
│ │ └── json/
│ │ └── spywatcher-logs.json # Main logging dashboard
└── README.md
Datasources
Loki (Default)
- URL:
http://loki:3100 - Type: loki
- Use: Log aggregation and querying
Prometheus
- URL:
http://backend:3001/metrics - Type: prometheus
- Use: Metrics collection
Dashboards
Spywatcher - Log Aggregation
Pre-configured dashboard with:
- Log volume charts
- Log level statistics
- Application logs viewer
- Security logs viewer
- Error logs viewer
Template Variables:
$job- Filter by service$level- Filter by log level$search- Free-text search
Access
URL: http://localhost:3000
Default Credentials:
- Username:
admin - Password:
admin
Important: Change the default password on first login!
Customization
Adding Custom Dashboards
- Create a JSON dashboard file in
provisioning/dashboards/json/ - Dashboard will be automatically loaded on Grafana startup
Modifying Datasources
Edit provisioning/datasources/loki.yml:
datasources:
- name: MyCustomDataSource
type: prometheus
url: http://my-service:9090
Environment Variables
GF_SECURITY_ADMIN_USER- Admin username (default: admin)GF_SECURITY_ADMIN_PASSWORD- Admin password (default: admin)GF_USERS_ALLOW_SIGN_UP- Allow user signup (default: false)GF_SERVER_ROOT_URL- Public URL for Grafana
Ports
3000- Grafana web UI