* Initial plan * feat: implement PgBouncer connection pooling and monitoring - Add PgBouncer service with transaction mode pooling - Configure optimal pool sizes (25 default, 100 max clients) - Update docker-compose for dev and prod environments - Enhance Prisma client with connection metrics - Improve Redis connection lifecycle management - Add comprehensive connection pool monitoring utilities - Create monitoring endpoints for health checks and metrics - Add graceful shutdown handlers for all services - Update environment configuration with PgBouncer settings Co-authored-by: onnwee <211922112+onnwee@users.noreply.github.com> * feat: add comprehensive documentation and tests for connection pooling - Create comprehensive CONNECTION_POOLING.md documentation - Add PgBouncer setup guide in docs/ - Create unit tests for connection pool monitoring - Add integration tests for monitoring endpoints - Update .gitignore for pgbouncer credentials Co-authored-by: onnwee <211922112+onnwee@users.noreply.github.com> * docs: finalize connection pooling documentation and README updates - Add comprehensive pgbouncer/README.md with usage guide - Update main README with connection pooling section - Add references to all relevant documentation - Complete documentation checklist Co-authored-by: onnwee <211922112+onnwee@users.noreply.github.com> * fix: address code review feedback on connection pooling - Fix PgBouncer Dockerfile permissions by running entrypoint as root with su-exec - Add PGPASSWORD to health check for proper authentication - Fix isHealthy check to not treat 0% utilization as unhealthy - Use proper URL parsing for isPgBouncer detection instead of string matching - Remove duplicate SIGTERM/SIGINT handlers in redis.ts, coordinate through db.ts - Add su-exec package for secure user switching in container Co-authored-by: PatrickFanella <61631520+PatrickFanella@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: onnwee <211922112+onnwee@users.noreply.github.com> Co-authored-by: PatrickFanella <61631520+PatrickFanella@users.noreply.github.com>
PgBouncer Configuration
This directory contains the PgBouncer connection pooler configuration for the Discord Spywatcher application.
Files
- pgbouncer.ini - Main PgBouncer configuration file
- userlist.txt.template - Template for user authentication (actual file generated at runtime)
- entrypoint.sh - Startup script that generates credentials and starts PgBouncer
- Dockerfile - Container image definition
- .gitignore - Prevents committing generated credentials
Configuration
Pool Settings
The default configuration uses transaction-mode pooling with the following limits:
- Pool Mode:
transaction(optimal for Prisma) - Default Pool Size: 25 connections per database
- Min Pool Size: 5 connections minimum
- Reserve Pool: 5 additional connections for spikes
- Max Client Connections: 100 simultaneous clients
Connection Lifecycle
- Server Lifetime: 3600 seconds (1 hour)
- Idle Timeout: 600 seconds (10 minutes)
- Query Wait Timeout: 120 seconds
Security
- Authentication: MD5 hashed passwords
- Admin Access: Separate admin user for monitoring
- Network: Internal Docker network only (no external exposure in production)
Usage
Environment Variables
Required:
DB_USER- Database username (e.g., "spywatcher")DB_PASSWORD- Database password
Optional:
PGBOUNCER_ADMIN_USER- Admin username (default: "pgbouncer_admin")PGBOUNCER_ADMIN_PASSWORD- Admin password (recommended for production)
Starting PgBouncer
The entrypoint script automatically:
- Generates MD5 hashed passwords
- Creates
userlist.txtfrom environment variables - Sets appropriate file permissions
- Starts PgBouncer
Connecting
Application Connection (through PgBouncer)
postgresql://user:password@pgbouncer:6432/spywatcher?pgbouncer=true
Admin Console
psql -h pgbouncer -p 6432 -U pgbouncer_admin pgbouncer
Monitoring
PgBouncer Admin Commands
SHOW POOLS; -- Pool statistics
SHOW DATABASES; -- Database connections
SHOW CLIENTS; -- Client connections
SHOW SERVERS; -- Server connections
SHOW CONFIG; -- Current configuration
SHOW STATS; -- Performance statistics
RELOAD; -- Reload configuration
Health Check
The Docker container includes a health check that runs:
psql -h 127.0.0.1 -p 6432 -U $DB_USER -d pgbouncer -c "SHOW POOLS;"
Customization
To modify PgBouncer settings:
- Edit
pgbouncer.ini - Rebuild the Docker image or restart the container
- Verify changes with
SHOW CONFIG;in admin console
Common Adjustments
Increase pool size for high load:
default_pool_size = 50
max_client_conn = 200
Adjust timeouts:
server_idle_timeout = 300 # Reduce to 5 minutes
query_wait_timeout = 60 # Reduce to 1 minute
Enable verbose logging:
log_connections = 1
log_disconnections = 1
log_pooler_errors = 1
verbose = 1
Troubleshooting
Issue: "No such user"
Check that credentials are set correctly:
docker exec container_name cat /etc/pgbouncer/userlist.txt
Issue: Connection refused
Ensure PostgreSQL is running and healthy:
docker-compose ps postgres
docker logs container_name
Issue: Pool saturation
Check pool utilization:
SHOW POOLS;
-- If cl_waiting > 0, increase pool size
Security Notes
- Never commit
userlist.txt- it contains credentials (already in.gitignore) - Use strong passwords - especially for production
- Rotate credentials - regularly change passwords
- Limit network access - PgBouncer should only be accessible within Docker network