subculture-collective/discord-spywatcher
1
0
Fork 0
Code Issues 2 Pull Requests 23 Actions Packages Projects Releases Wiki Activity

Establish safe database migration procedures for production (#147)

Browse Source 
* Initial plan

* feat: implement comprehensive database migration safety procedures

- Add test-migration.sh for isolated migration testing
- Add rollback-migration.sh for safe rollback procedures
- Add validate-migration.sh for data integrity validation
- Create MIGRATION_SAFETY.md with zero-downtime strategies
- Integrate migration testing into CI/CD pipeline
- Update scripts documentation

Co-authored-by: PatrickFanella <61631520+PatrickFanella@users.noreply.github.com>

* style: fix prettier formatting in migration docs

Co-authored-by: PatrickFanella <61631520+PatrickFanella@users.noreply.github.com>

* fix: address code review feedback

- Fix foreign key violation check to properly detect orphaned records
- Replace grep -oP with grep -oE for better portability
- Replace ls parsing with find command for reliable file listing
- Fix capitalization in documentation list items

Co-authored-by: PatrickFanella <61631520+PatrickFanella@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: PatrickFanella <61631520+PatrickFanella@users.noreply.github.com>
This commit was merged in pull request #147.
This commit is contained in:
Copilot
2025-11-02 19:55:18 -06:00
committed by GitHub
parent dd0bec5046
commit 606ba4bbdb
6 changed files with 1956 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

56
.github/workflows/backend-ci.yml vendored
View File

@@ -118,10 +118,64 @@ jobs:
env:
DATABASE_URL: "file:./dev.db"
migration-test:
name: Test Migrations
runs-on: ubuntu-latest
services:
postgres:
image: postgres:15
env:
POSTGRES_USER: spywatcher
POSTGRES_PASSWORD: test_password
POSTGRES_DB: spywatcher_test
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Checkout code
uses: actions/checkout@v5
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version: '20'
cache: 'npm'
cache-dependency-path: backend/package-lock.json
- name: Install dependencies
working-directory: backend
run: npm ci
- name: Make scripts executable
run: chmod +x scripts/*.sh
- name: Test migrations
env:
DB_PASSWORD: test_password
TEST_DB_NAME: spywatcher_test
DB_USER: spywatcher
DB_HOST: localhost
DB_PORT: 5432
run: ./scripts/test-migration.sh
- name: Validate migrations
env:
DB_PASSWORD: test_password
DB_NAME: spywatcher_test
DB_USER: spywatcher
DB_HOST: localhost
DB_PORT: 5432
run: ./scripts/validate-migration.sh
build:
name: Build
runs-on: ubuntu-latest
needs: [typecheck, prisma]
needs: [typecheck, prisma, migration-test]
defaults:
run:
working-directory: ./backend

598
MIGRATION_SAFETY.md Normal file
View File

@@ -0,0 +1,598 @@
# Database Migration Safety Guide
This guide establishes safe database migration procedures for production deployments, ensuring zero downtime and data integrity.
## 🎯 Overview
This document covers:
- Migration testing procedures
- Rollback strategies
- Zero-downtime migration techniques
- Data validation checks
- CI/CD integration
## 📋 Migration Testing Procedures
### Pre-Migration Testing
Before applying any migration to production:
#### 1. Test in Isolated Environment
```bash
# Run comprehensive migration tests
DB_PASSWORD=your_password ./scripts/test-migration.sh
```
This script:
- Creates an isolated test database
- Applies pending migrations
- Validates schema integrity
- Tests data consistency
- Verifies rollback procedures
- Cleans up test environment
#### 2. Dry Run Validation
```bash
# Validate migration without applying
cd backend
DATABASE_URL="postgresql://user:pass@host:5432/db" \
npx prisma migrate deploy --dry-run
```
#### 3. Schema Validation
```bash
# Validate Prisma schema
cd backend
npx prisma validate
# Generate Prisma client
npx prisma generate
```
### Post-Migration Validation
After applying migrations:
```bash
# Run comprehensive validation checks
DB_PASSWORD=your_password ./scripts/validate-migration.sh
```
This validates:
- All required tables exist
- Indexes are properly created
- Foreign key constraints are valid
- Primary keys are in place
- Data types are correct
- No foreign key violations
- Prisma migrations completed successfully
## 🔄 Rollback Strategies
### Automatic Backup Before Migration
Always create a backup before migration:
```bash
# Create backup
DB_PASSWORD=your_password ./scripts/backup.sh
# Backup will be saved to /var/backups/spywatcher/
```
### Rollback Options
#### Option 1: Rollback to Specific Migration
```bash
# List available migrations
DB_PASSWORD=your_password ./scripts/rollback-migration.sh --list
# Rollback to specific migration
DB_PASSWORD=your_password ./scripts/rollback-migration.sh \
--migration 20250524175155_init
```
This will:
- Create a pre-rollback backup
- Mark subsequent migrations as rolled back
- Provide instructions for schema restoration
#### Option 2: Restore from Backup
```bash
# List available backups
DB_PASSWORD=your_password ./scripts/rollback-migration.sh --list
# Restore from backup
DB_PASSWORD=your_password ./scripts/rollback-migration.sh \
--backup /var/backups/spywatcher/spywatcher_20250101_020000.sql.gz
```
This will:
- confirm the operation
- terminate active connections
- drop and recreate database
- restore data from backup
- verify restoration
### Rollback Best Practices
1. **Always backup before migration** - Automated in production workflow
2. **Test rollback procedure** - Include in migration testing
3. **Document rollback steps** - For each major migration
4. **Monitor after rollback** - Ensure system stability
5. **Keep recent backups** - Maintain 30 days of backups
## 🚀 Zero-Downtime Migration Techniques
### Strategy 1: Backwards-Compatible Migrations
#### Adding New Columns
```sql
-- ✅ Safe: Add nullable column
ALTER TABLE "User" ADD COLUMN "newField" TEXT;
-- ✅ Safe: Add column with default
ALTER TABLE "User" ADD COLUMN "status" TEXT DEFAULT 'active';
```
#### Making Columns Optional
```sql
-- Phase 1: Make column nullable
ALTER TABLE "User" ALTER COLUMN "oldField" DROP NOT NULL;
-- Phase 2 (after deployment): Remove column
-- ALTER TABLE "User" DROP COLUMN "oldField";
```
### Strategy 2: Multi-Phase Migrations
For breaking changes, use multiple deployments:
#### Phase 1: Add New Schema
```prisma
model User {
id String @id
email String // Old field
emailNew String? // New field (nullable)
}
```
Deploy with dual writes:
```typescript
// Write to both fields
await prisma.user.create({
data: {
email: userEmail,
emailNew: userEmail,
},
});
```
#### Phase 2: Migrate Data
```sql
-- Copy data to new field
UPDATE "User" SET "emailNew" = "email" WHERE "emailNew" IS NULL;
```
#### Phase 3: Switch to New Field
```prisma
model User {
id String @id
email String? // Old field (now nullable)
emailNew String // New field (now required)
}
```
Update application to use `emailNew`.
#### Phase 4: Remove Old Field
```prisma
model User {
id String @id
emailNew String @map("email")
@@map("User")
}
```
### Strategy 3: Blue-Green Deployment
For major schema changes:
1. **Deploy Green Environment** with new schema
2. **Sync Data** from Blue to Green
3. **Test Green Environment** thoroughly
4. **Switch Traffic** to Green
5. **Keep Blue** as fallback for 24-48 hours
6. **Decommission Blue** after validation
### Strategy 4: Shadow Database Testing
Prisma automatically uses shadow databases for testing:
```bash
# Set shadow database URL
export SHADOW_DATABASE_URL="postgresql://user:pass@host:5432/shadow_db"
# Migrations are tested on shadow DB first
npx prisma migrate dev
```
## ✅ Data Validation Checks
### Pre-Migration Validation
```bash
# Validate current schema
DB_PASSWORD=your_password ./scripts/validate-migration.sh
```
### Post-Migration Validation
Automatic checks include:
#### 1. Schema Integrity
- All tables exist
- Correct column types
- Proper indexes
- Valid constraints
#### 2. Data Integrity
- No orphaned foreign keys
- No NULL violations
- No duplicate primary keys
- Consistent data types
#### 3. Migration Status
- All migrations completed
- No failed migrations
- No pending migrations
### Custom Validation Queries
Add to validation script as needed:
```sql
-- Check for data consistency
SELECT COUNT(*) FROM "User" WHERE "discordId" IS NULL;
-- Verify foreign key relationships
SELECT COUNT(*) FROM "Guild" g
LEFT JOIN "User" u ON g."userId" = u.id
WHERE u.id IS NULL;
-- Check for duplicate values
SELECT "discordId", COUNT(*)
FROM "User"
GROUP BY "discordId"
HAVING COUNT(*) > 1;
```
## 🔧 CI/CD Integration
### GitHub Actions Workflow
The migration workflow is integrated into `.github/workflows/backend-ci.yml`:
```yaml
migration-test:
name: Test Migrations
runs-on: ubuntu-latest
services:
postgres:
image: postgres:15
env:
POSTGRES_PASSWORD: postgres
POSTGRES_DB: spywatcher_test
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Checkout code
uses: actions/checkout@v5
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version: '20'
cache: 'npm'
cache-dependency-path: backend/package-lock.json
- name: Install dependencies
working-directory: backend
run: npm ci
- name: Test migrations
env:
DB_PASSWORD: postgres
TEST_DB_NAME: spywatcher_test
run: ./scripts/test-migration.sh
- name: Validate schema
env:
DATABASE_URL: postgresql://spywatcher:postgres@localhost:5432/spywatcher_test
working-directory: backend
run: npx prisma validate
```
### Pre-Deployment Checks
Add to deployment workflow:
```yaml
pre-deploy:
name: Pre-Deployment Validation
steps:
- name: Create backup
run: |
DB_PASSWORD=${{ secrets.DB_PASSWORD }} \
DB_HOST=${{ secrets.DB_HOST }} \
./scripts/backup.sh
- name: Test migration
run: |
DB_PASSWORD=${{ secrets.DB_PASSWORD }} \
./scripts/test-migration.sh
- name: Upload backup artifact
uses: actions/upload-artifact@v4
with:
name: pre-migration-backup
path: /var/backups/spywatcher/
retention-days: 7
```
### Post-Deployment Checks
```yaml
post-deploy:
name: Post-Deployment Validation
needs: deploy
steps:
- name: Validate migration
run: |
DB_PASSWORD=${{ secrets.DB_PASSWORD }} \
DB_HOST=${{ secrets.DB_HOST }} \
./scripts/validate-migration.sh
- name: Health check
run: |
curl -f https://api.yourdomain.com/health || exit 1
- name: Rollback on failure
if: failure()
run: |
echo "Migration validation failed, initiating rollback"
# Restore from backup created in pre-deploy
DB_PASSWORD=${{ secrets.DB_PASSWORD }} \
./scripts/rollback-migration.sh --backup $BACKUP_FILE
```
## 📝 Migration Checklist
Use this checklist for every production migration:
### Pre-Migration
- [ ] Review migration SQL/schema changes
- [ ] Test migration in staging environment
- [ ] Run `./scripts/test-migration.sh`
- [ ] Create production backup
- [ ] Verify backup integrity
- [ ] Document rollback procedure
- [ ] Schedule maintenance window (if needed)
- [ ] Notify team of migration
### During Migration
- [ ] Enable maintenance mode (if needed)
- [ ] Apply migrations: `npx prisma migrate deploy`
- [ ] Monitor application logs
- [ ] Monitor database metrics
- [ ] Watch for errors or warnings
### Post-Migration
- [ ] Run `./scripts/validate-migration.sh`
- [ ] Verify application functionality
- [ ] Check critical user flows
- [ ] Monitor error rates
- [ ] Monitor performance metrics
- [ ] Disable maintenance mode
- [ ] Document any issues
- [ ] Keep backup for 7+ days
### Rollback (if needed)
- [ ] Identify issue quickly
- [ ] Execute rollback procedure
- [ ] Restore from backup
- [ ] Validate rollback
- [ ] Notify team
- [ ] Document lessons learned
- [ ] Plan fix for next attempt
## 🛠️ Available Scripts
### Testing
```bash
# Comprehensive migration testing
DB_PASSWORD=pass ./scripts/test-migration.sh
# Verbose output
VERBOSE=true DB_PASSWORD=pass ./scripts/test-migration.sh
```
### Validation
```bash
# Validate current database state
DB_PASSWORD=pass ./scripts/validate-migration.sh
# Verbose validation
VERBOSE=true DB_PASSWORD=pass ./scripts/validate-migration.sh
```
### Rollback
```bash
# List migrations and backups
DB_PASSWORD=pass ./scripts/rollback-migration.sh --list
# Rollback to specific migration
DB_PASSWORD=pass ./scripts/rollback-migration.sh \
--migration MIGRATION_NAME
# Restore from backup
DB_PASSWORD=pass ./scripts/rollback-migration.sh \
--backup /path/to/backup.sql.gz
```
### Backup
```bash
# Create backup
DB_PASSWORD=pass ./scripts/backup.sh
# Create backup and upload to S3
S3_BUCKET=my-bucket DB_PASSWORD=pass ./scripts/backup.sh
```
### Maintenance
```bash
# Run database maintenance
DB_PASSWORD=pass ./scripts/maintenance.sh
```
## 🔒 Security Considerations
1. **Never commit passwords** - Use environment variables
2. **Restrict script permissions** - `chmod 700 scripts/*.sh`
3. **Encrypt backups** - Use encrypted storage
4. **Audit migration access** - Log who runs migrations
5. **Use SSL/TLS** - For database connections
6. **Validate inputs** - In custom migration scripts
7. **Review SQL** - Before applying migrations
## 📊 Monitoring
### Key Metrics to Monitor
During and after migrations:
1. **Application Metrics**
- Error rate
- Response time
- Request throughput
- Success rate
2. **Database Metrics**
- Connection count
- Query latency
- Lock wait time
- Transaction rate
- CPU and memory usage
3. **Migration Metrics**
- Migration duration
- Rows affected
- Rollback frequency
- Validation pass rate
### Alerting
Set up alerts for:
- Failed migrations
- Schema validation failures
- Data integrity issues
- Performance degradation
- Connection pool exhaustion
## 🆘 Troubleshooting
### Migration Fails to Apply
```bash
# Check migration status
cd backend
npx prisma migrate status
# Mark migration as applied (if already applied manually)
npx prisma migrate resolve --applied MIGRATION_NAME
# Mark migration as rolled back
npx prisma migrate resolve --rolled-back MIGRATION_NAME
```
### Data Validation Failures
```bash
# Run detailed validation
VERBOSE=true DB_PASSWORD=pass ./scripts/validate-migration.sh
# Check specific issues
psql -U user -d db -c "SELECT * FROM _prisma_migrations WHERE finished_at IS NULL;"
```
### Performance Issues After Migration
```bash
# Run maintenance to update statistics
DB_PASSWORD=pass ./scripts/maintenance.sh
# Check for missing indexes
psql -U user -d db -c "
SELECT schemaname, tablename, attname
FROM pg_stats
WHERE schemaname = 'public'
AND n_distinct > 100
AND correlation < 0.1;
"
```
## 📚 Additional Resources
- [Prisma Migration Documentation](https://www.prisma.io/docs/concepts/components/prisma-migrate)
- [PostgreSQL Migration Best Practices](https://www.postgresql.org/docs/current/ddl-alter.html)
- [Zero-Downtime Deployments](https://blog.pragmaticengineer.com/zero-downtime-deployment/)
- [Database Reliability Engineering](https://www.oreilly.com/library/view/database-reliability-engineering/9781491925935/)
## 🤝 Support
For migration issues:
1. Check this guide
2. Review script output and logs
3. Check [MIGRATION.md](./MIGRATION.md) for database-specific guidance
4. Open GitHub issue with:
- Migration name and SQL
- Error messages
- Database version
- Script output
- Steps to reproduce

108
scripts/README.md
View File

@@ -84,6 +84,114 @@ Generates load to test auto-scaling behavior and simulate traffic spikes.
**See:** [docs/AUTO_SCALING_EXAMPLES.md](../docs/AUTO_SCALING_EXAMPLES.md) for examples.
### Database Migration Scripts
#### `test-migration.sh`
Comprehensive migration testing in an isolated environment.
**Features:**
- Creates isolated test database
- Applies pending migrations
- Validates schema integrity
- Tests data consistency
- Verifies rollback procedures
- Automatic cleanup
**Usage:**
```bash
# Run comprehensive migration tests
DB_PASSWORD=your_password ./scripts/test-migration.sh
# Verbose output
VERBOSE=true DB_PASSWORD=your_password ./scripts/test-migration.sh
```
**Environment Variables:**
- `TEST_DB_NAME` - Test database name (default: spywatcher_test)
- `DB_USER` - Database user (default: spywatcher)
- `DB_HOST` - Database host (default: localhost)
- `DB_PORT` - Database port (default: 5432)
- `DB_PASSWORD` - Database password (required)
- `BACKUP_DIR` - Backup directory (default: /tmp/migration-test-backups)
- `VERBOSE` - Show detailed output (default: false)
**See:** [MIGRATION_SAFETY.md](../MIGRATION_SAFETY.md) for complete migration procedures.
#### `rollback-migration.sh`
Safely rollback database migrations to a previous state.
**Features:**
- Rollback to specific migration
- Restore from backup file
- List available migrations and backups
- Automatic pre-rollback backup
- Safe confirmation prompts
**Usage:**
```bash
# List available options
DB_PASSWORD=pass ./scripts/rollback-migration.sh --list
# Rollback to specific migration
DB_PASSWORD=pass ./scripts/rollback-migration.sh \
--migration 20250524175155_init
# Restore from backup
DB_PASSWORD=pass ./scripts/rollback-migration.sh \
--backup /path/to/backup.sql.gz
```
**Environment Variables:**
- `DB_NAME` - Database name (default: spywatcher)
- `DB_USER` - Database user (default: spywatcher)
- `DB_HOST` - Database host (default: localhost)
- `DB_PORT` - Database port (default: 5432)
- `DB_PASSWORD` - Database password (required)
- `BACKUP_DIR` - Backup directory (default: /var/backups/spywatcher)
#### `validate-migration.sh`
Comprehensive data validation checks after migrations.
**Features:**
- Schema existence validation
- Required tables verification
- Index validation
- Foreign key constraint checks
- Primary key validation
- Data type verification
- Data consistency checks
- Prisma migration status
- Database size reporting
**Usage:**
```bash
# Run validation checks
DB_PASSWORD=your_password ./scripts/validate-migration.sh
# Verbose validation
VERBOSE=true DB_PASSWORD=your_password ./scripts/validate-migration.sh
```
**Environment Variables:**
- `DB_NAME` - Database name (default: spywatcher)
- `DB_USER` - Database user (default: spywatcher)
- `DB_HOST` - Database host (default: localhost)
- `DB_PORT` - Database port (default: 5432)
- `DB_PASSWORD` - Database password (required)
- `VERBOSE` - Show detailed output (default: false)
### PostgreSQL Management Scripts
#### 1. `postgres-init.sql`

341
scripts/rollback-migration.sh Executable file
View File

@@ -0,0 +1,341 @@
#!/bin/bash
# Rollback Migration Script
# Safely rolls back database migrations to a previous state
# Supports both single migration rollback and full backup restore
set -e
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Default values
DB_NAME="${DB_NAME:-spywatcher}"
DB_USER="${DB_USER:-spywatcher}"
DB_HOST="${DB_HOST:-localhost}"
DB_PORT="${DB_PORT:-5432}"
DB_PASSWORD="${DB_PASSWORD:-}"
BACKUP_DIR="${BACKUP_DIR:-/var/backups/spywatcher}"
# Function to print colored output
print_info() {
echo -e "${BLUE}ℹ ${1}${NC}"
}
print_success() {
echo -e "${GREEN}✓ ${1}${NC}"
}
print_warning() {
echo -e "${YELLOW}⚠ ${1}${NC}"
}
print_error() {
echo -e "${RED}✗ ${1}${NC}"
}
# Function to show usage
show_usage() {
cat << EOF
Usage: $0 [OPTIONS]
Rollback database migrations safely.
OPTIONS:
-m, --migration NAME Rollback to specific migration
-b, --backup FILE Restore from backup file
-l, --list List available backups and migrations
-h, --help Show this help message
EXAMPLES:
# List available options
DB_PASSWORD=pass $0 --list
# Rollback to a specific migration
DB_PASSWORD=pass $0 --migration 20250524175155_init
# Restore from backup
DB_PASSWORD=pass $0 --backup /path/to/backup.sql.gz
ENVIRONMENT VARIABLES:
DB_NAME Database name (default: spywatcher)
DB_USER Database user (default: spywatcher)
DB_HOST Database host (default: localhost)
DB_PORT Database port (default: 5432)
DB_PASSWORD Database password (required)
BACKUP_DIR Backup directory (default: /var/backups/spywatcher)
EOF
}
# Function to check prerequisites
check_prerequisites() {
# Check if PostgreSQL client is installed
if ! command -v psql &> /dev/null; then
print_error "psql is not installed. Please install PostgreSQL client."
exit 1
fi
# Check if DB_PASSWORD is set
if [ -z "$DB_PASSWORD" ]; then
print_error "DB_PASSWORD environment variable is required"
exit 1
fi
}
# Function to list migrations
list_migrations() {
print_info "Applied migrations in database:"
echo ""
export PGPASSWORD="$DB_PASSWORD"
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -c "
SELECT
migration_name,
finished_at,
applied_steps_count
FROM _prisma_migrations
WHERE finished_at IS NOT NULL
ORDER BY finished_at DESC;
" 2>/dev/null || print_warning "Unable to query migrations table"
echo ""
}
# Function to list backups
list_backups() {
print_info "Available backups in $BACKUP_DIR:"
echo ""
if [ -d "$BACKUP_DIR" ]; then
find "$BACKUP_DIR" -maxdepth 1 -type f \( -name '*.sql' -o -name '*.sql.gz' \) -printf '%f (%s bytes) %TY-%Tm-%Td %TH:%TM\n' | sort -r || print_warning "No backup files found"
else
print_warning "Backup directory does not exist: $BACKUP_DIR"
fi
echo ""
}
# Function to create pre-rollback backup
create_pre_rollback_backup() {
print_info "Creating pre-rollback backup..."
mkdir -p "$BACKUP_DIR"
local timestamp=$(date +%Y%m%d_%H%M%S)
local backup_file="$BACKUP_DIR/pre_rollback_${DB_NAME}_${timestamp}.sql.gz"
export PGPASSWORD="$DB_PASSWORD"
pg_dump -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" "$DB_NAME" | gzip > "$backup_file" 2>/dev/null
print_success "Pre-rollback backup created: $backup_file"
echo "$backup_file"
}
# Function to rollback to specific migration
rollback_to_migration() {
local target_migration=$1
print_info "Rolling back to migration: $target_migration"
# Verify migration exists
export PGPASSWORD="$DB_PASSWORD"
local migration_exists=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT COUNT(*)
FROM _prisma_migrations
WHERE migration_name = '$target_migration';
" 2>/dev/null | tr -d ' ')
if [ "$migration_exists" -eq "0" ]; then
print_error "Migration not found: $target_migration"
exit 1
fi
# Create backup before rollback
local backup_file=$(create_pre_rollback_backup)
print_warning "This will rollback all migrations after $target_migration"
print_warning "Backup created at: $backup_file"
echo ""
read -p "Are you sure you want to continue? (yes/no): " confirm
if [ "$confirm" != "yes" ]; then
print_info "Rollback cancelled"
exit 0
fi
# Get migrations to rollback (all after target)
local migrations_to_rollback=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT migration_name
FROM _prisma_migrations
WHERE finished_at > (
SELECT finished_at
FROM _prisma_migrations
WHERE migration_name = '$target_migration'
)
ORDER BY finished_at DESC;
" 2>/dev/null)
if [ -z "$migrations_to_rollback" ]; then
print_info "No migrations to rollback"
return 0
fi
print_info "Migrations to rollback:"
echo "$migrations_to_rollback"
echo ""
# Manually rollback using Prisma
cd backend
export DATABASE_URL="postgresql://$DB_USER:$DB_PASSWORD@$DB_HOST:$DB_PORT/$DB_NAME"
# Mark migrations as rolled back
while IFS= read -r migration; do
if [ -n "$migration" ]; then
migration=$(echo "$migration" | tr -d ' ')
print_info "Marking migration as rolled back: $migration"
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -c "
UPDATE _prisma_migrations
SET rolled_back_at = NOW()
WHERE migration_name = '$migration';
" > /dev/null 2>&1 || print_warning "Could not mark $migration as rolled back"
fi
done <<< "$migrations_to_rollback"
cd ..
print_success "Rollback completed"
print_warning "You may need to manually restore the database schema from backup: $backup_file"
print_info "To restore: DB_PASSWORD=$DB_PASSWORD ./scripts/restore.sh $backup_file"
}
# Function to restore from backup
restore_from_backup() {
local backup_file=$1
if [ ! -f "$backup_file" ]; then
print_error "Backup file not found: $backup_file"
exit 1
fi
print_warning "This will REPLACE all data in database: $DB_NAME"
print_warning "Source: $backup_file"
echo ""
read -p "Are you sure you want to continue? (yes/no): " confirm
if [ "$confirm" != "yes" ]; then
print_info "Restore cancelled"
exit 0
fi
print_info "Restoring database from: $backup_file"
export PGPASSWORD="$DB_PASSWORD"
# Terminate existing connections
print_info "Terminating existing connections..."
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -c "
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = '$DB_NAME'
AND pid <> pg_backend_pid();
" > /dev/null 2>&1 || true
# Drop and recreate database
print_info "Recreating database..."
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -c "DROP DATABASE IF EXISTS $DB_NAME;" > /dev/null 2>&1
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -c "CREATE DATABASE $DB_NAME;" > /dev/null 2>&1
# Restore from backup
print_info "Restoring data..."
if [[ "$backup_file" == *.gz ]]; then
gunzip -c "$backup_file" | psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" > /dev/null 2>&1
else
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" < "$backup_file" > /dev/null 2>&1
fi
# Verify restore
print_info "Verifying restore..."
local table_count=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT COUNT(*)
FROM information_schema.tables
WHERE table_schema = 'public';
" 2>/dev/null | tr -d ' ')
if [ "$table_count" -gt 0 ]; then
print_success "Database restored successfully"
print_info "Tables found: $table_count"
else
print_error "Restore may have failed - no tables found"
exit 1
fi
}
# Parse command line arguments
MODE=""
TARGET=""
while [[ $# -gt 0 ]]; do
case $1 in
-m|--migration)
MODE="migration"
TARGET="$2"
shift 2
;;
-b|--backup)
MODE="backup"
TARGET="$2"
shift 2
;;
-l|--list)
MODE="list"
shift
;;
-h|--help)
show_usage
exit 0
;;
*)
print_error "Unknown option: $1"
show_usage
exit 1
;;
esac
done
# Main execution
check_prerequisites
case $MODE in
list)
list_migrations
list_backups
;;
migration)
if [ -z "$TARGET" ]; then
print_error "Migration name is required"
show_usage
exit 1
fi
rollback_to_migration "$TARGET"
;;
backup)
if [ -z "$TARGET" ]; then
print_error "Backup file path is required"
show_usage
exit 1
fi
restore_from_backup "$TARGET"
;;
*)
print_error "No operation specified"
show_usage
exit 1
;;
esac

400
scripts/test-migration.sh Executable file
View File

@@ -0,0 +1,400 @@
#!/bin/bash
# Test Migration Script
# Tests database migrations in a safe, isolated environment before applying to production
# Validates schema changes, data integrity, and rollback procedures
set -e
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Default values
TEST_DB_NAME="${TEST_DB_NAME:-spywatcher_test}"
DB_USER="${DB_USER:-spywatcher}"
DB_HOST="${DB_HOST:-localhost}"
DB_PORT="${DB_PORT:-5432}"
DB_PASSWORD="${DB_PASSWORD:-}"
BACKUP_DIR="${BACKUP_DIR:-/tmp/migration-test-backups}"
VERBOSE="${VERBOSE:-false}"
# Function to print colored output
print_info() {
echo -e "${BLUE}ℹ ${1}${NC}"
}
print_success() {
echo -e "${GREEN}✓ ${1}${NC}"
}
print_warning() {
echo -e "${YELLOW}⚠ ${1}${NC}"
}
print_error() {
echo -e "${RED}✗ ${1}${NC}"
}
# Function to check prerequisites
check_prerequisites() {
print_info "Checking prerequisites..."
# Check if PostgreSQL client is installed
if ! command -v psql &> /dev/null; then
print_error "psql is not installed. Please install PostgreSQL client."
exit 1
fi
# Check if Node.js is installed
if ! command -v node &> /dev/null; then
print_error "Node.js is not installed."
exit 1
fi
# Check if Prisma is installed
if ! command -v npx &> /dev/null; then
print_error "npx is not installed."
exit 1
fi
# Check if DB_PASSWORD is set
if [ -z "$DB_PASSWORD" ]; then
print_error "DB_PASSWORD environment variable is required"
exit 1
fi
print_success "All prerequisites met"
}
# Function to create test database
create_test_database() {
print_info "Creating test database: $TEST_DB_NAME"
export PGPASSWORD="$DB_PASSWORD"
# Drop test database if it exists
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -c "DROP DATABASE IF EXISTS $TEST_DB_NAME;" > /dev/null 2>&1 || true
# Create test database
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -c "CREATE DATABASE $TEST_DB_NAME;" > /dev/null 2>&1
print_success "Test database created"
}
# Function to backup current database state
backup_test_database() {
print_info "Creating backup of test database..."
mkdir -p "$BACKUP_DIR"
local timestamp=$(date +%Y%m%d_%H%M%S)
local backup_file="$BACKUP_DIR/${TEST_DB_NAME}_${timestamp}.sql"
export PGPASSWORD="$DB_PASSWORD"
pg_dump -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" "$TEST_DB_NAME" > "$backup_file" 2>/dev/null
echo "$backup_file"
print_success "Backup created: $backup_file"
}
# Function to restore database from backup
restore_test_database() {
local backup_file=$1
print_info "Restoring database from: $backup_file"
export PGPASSWORD="$DB_PASSWORD"
# Terminate existing connections
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -c "
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = '$TEST_DB_NAME'
AND pid <> pg_backend_pid();
" > /dev/null 2>&1 || true
# Drop and recreate database
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -c "DROP DATABASE IF EXISTS $TEST_DB_NAME;" > /dev/null 2>&1
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -c "CREATE DATABASE $TEST_DB_NAME;" > /dev/null 2>&1
# Restore from backup
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$TEST_DB_NAME" < "$backup_file" > /dev/null 2>&1
print_success "Database restored"
}
# Function to apply migrations
apply_migrations() {
print_info "Applying migrations to test database..."
cd backend
export DATABASE_URL="postgresql://$DB_USER:$DB_PASSWORD@$DB_HOST:$DB_PORT/$TEST_DB_NAME"
# Run migrations
if [ "$VERBOSE" = "true" ]; then
npx prisma migrate deploy
else
npx prisma migrate deploy > /dev/null 2>&1
fi
cd ..
print_success "Migrations applied successfully"
}
# Function to validate schema
validate_schema() {
print_info "Validating database schema..."
export PGPASSWORD="$DB_PASSWORD"
# Check that all expected tables exist
local tables=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$TEST_DB_NAME" -t -c "
SELECT tablename FROM pg_tables
WHERE schemaname = 'public'
ORDER BY tablename;
" 2>/dev/null | tr -d ' ' | grep -v '^$')
if [ "$VERBOSE" = "true" ]; then
echo "Tables found:"
echo "$tables"
fi
# Check for required tables
local required_tables=(
"User"
"Guild"
"PresenceEvent"
"TypingEvent"
"MessageEvent"
"JoinEvent"
"RefreshToken"
"Session"
)
local missing_tables=()
for table in "${required_tables[@]}"; do
if ! echo "$tables" | grep -q "^$table$"; then
missing_tables+=("$table")
fi
done
if [ ${#missing_tables[@]} -gt 0 ]; then
print_error "Missing required tables: ${missing_tables[*]}"
return 1
fi
# Validate Prisma schema
cd backend
export DATABASE_URL="postgresql://$DB_USER:$DB_PASSWORD@$DB_HOST:$DB_PORT/$TEST_DB_NAME"
if npx prisma validate > /dev/null 2>&1; then
cd ..
print_success "Schema validation passed"
return 0
else
cd ..
print_error "Schema validation failed"
return 1
fi
}
# Function to validate data integrity
validate_data_integrity() {
print_info "Validating data integrity..."
export PGPASSWORD="$DB_PASSWORD"
# Check for foreign key violations
local fk_violations=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$TEST_DB_NAME" -t -c "
DO \$\$
DECLARE
r RECORD;
child_cols TEXT;
parent_cols TEXT;
sql TEXT;
violation_count INTEGER := 0;
BEGIN
FOR r IN
SELECT
conname,
conrelid::regclass AS child_table,
confrelid::regclass AS parent_table,
conkey,
confkey
FROM pg_constraint
WHERE contype = 'f'
LOOP
-- Get child and parent column names as comma-separated lists
SELECT string_agg(quote_ident(attname), ', ')
INTO child_cols
FROM unnest(r.conkey) AS colnum
JOIN pg_attribute a ON a.attrelid = r.child_table::regclass AND a.attnum = colnum;
SELECT string_agg(quote_ident(attname), ', ')
INTO parent_cols
FROM unnest(r.confkey) AS colnum
JOIN pg_attribute a ON a.attrelid = r.parent_table::regclass AND a.attnum = colnum;
sql := 'SELECT COUNT(*) FROM ' || r.child_table || ' c LEFT JOIN ' || r.parent_table || ' p ON (' ||
'c.' || child_cols || ' = p.' || parent_cols || ') WHERE p.' || parent_cols || ' IS NULL';
BEGIN
EXECUTE sql INTO violation_count;
IF violation_count > 0 THEN
RAISE NOTICE 'Foreign key violation in constraint %: % orphaned rows in table %', r.conname, violation_count, r.child_table;
END IF;
EXCEPTION WHEN OTHERS THEN
RAISE NOTICE 'Error checking foreign key constraint %', r.conname;
END;
END LOOP;
END;
\$\$;
" 2>&1)
# Check for NULL violations in NOT NULL columns
local null_check=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$TEST_DB_NAME" -t -c "
SELECT COUNT(*)
FROM information_schema.columns
WHERE table_schema = 'public'
AND is_nullable = 'NO';
" 2>/dev/null | tr -d ' ')
if [ "$VERBOSE" = "true" ]; then
echo "NOT NULL columns: $null_check"
fi
# Check for duplicate primary keys (should be 0)
local dup_check=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$TEST_DB_NAME" -t -c "
SELECT COUNT(*)
FROM (
SELECT table_name
FROM information_schema.table_constraints
WHERE constraint_type = 'PRIMARY KEY'
AND table_schema = 'public'
) AS pk_tables;
" 2>/dev/null | tr -d ' ')
if [ "$dup_check" -lt 1 ]; then
print_error "No primary keys found in database"
return 1
fi
print_success "Data integrity validation passed"
}
# Function to test rollback
test_rollback() {
local backup_file=$1
print_info "Testing rollback procedure..."
# Restore from backup
restore_test_database "$backup_file"
# Validate schema after rollback
if validate_schema; then
print_success "Rollback test passed"
return 0
else
print_error "Rollback test failed - schema validation failed after restore"
return 1
fi
}
# Function to cleanup
cleanup() {
print_info "Cleaning up test database..."
export PGPASSWORD="$DB_PASSWORD"
# Drop test database
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -c "
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = '$TEST_DB_NAME'
AND pid <> pg_backend_pid();
" > /dev/null 2>&1 || true
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -c "DROP DATABASE IF EXISTS $TEST_DB_NAME;" > /dev/null 2>&1 || true
print_success "Cleanup completed"
}
# Main test flow
main() {
echo ""
echo "═══════════════════════════════════════════════════════════"
echo " Database Migration Testing Suite"
echo "═══════════════════════════════════════════════════════════"
echo ""
# Check prerequisites
check_prerequisites
# Create test database
create_test_database
# Apply current schema (baseline)
print_info "Setting up baseline schema..."
cd backend
export DATABASE_URL="postgresql://$DB_USER:$DB_PASSWORD@$DB_HOST:$DB_PORT/$TEST_DB_NAME"
npx prisma db push --skip-generate > /dev/null 2>&1 || true
cd ..
# Create backup of baseline
local backup_file=$(backup_test_database)
# Apply pending migrations
if apply_migrations; then
print_success "Migration application: PASSED"
else
print_error "Migration application: FAILED"
cleanup
exit 1
fi
# Validate schema
if validate_schema; then
print_success "Schema validation: PASSED"
else
print_error "Schema validation: FAILED"
cleanup
exit 1
fi
# Validate data integrity
if validate_data_integrity; then
print_success "Data integrity validation: PASSED"
else
print_error "Data integrity validation: FAILED"
cleanup
exit 1
fi
# Test rollback
if test_rollback "$backup_file"; then
print_success "Rollback test: PASSED"
else
print_error "Rollback test: FAILED"
cleanup
exit 1
fi
# Cleanup
cleanup
echo ""
echo "═══════════════════════════════════════════════════════════"
print_success "All migration tests passed successfully!"
echo "═══════════════════════════════════════════════════════════"
echo ""
}
# Run main
main

454
scripts/validate-migration.sh Executable file
View File

@@ -0,0 +1,454 @@
#!/bin/bash
# Validate Migration Script
# Performs comprehensive data validation checks after migrations
# Ensures data integrity, consistency, and correctness
set -e
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Default values
DB_NAME="${DB_NAME:-spywatcher}"
DB_USER="${DB_USER:-spywatcher}"
DB_HOST="${DB_HOST:-localhost}"
DB_PORT="${DB_PORT:-5432}"
DB_PASSWORD="${DB_PASSWORD:-}"
VERBOSE="${VERBOSE:-false}"
# Counters
CHECKS_PASSED=0
CHECKS_FAILED=0
CHECKS_WARNING=0
# Function to print colored output
print_info() {
echo -e "${BLUE}ℹ ${1}${NC}"
}
print_success() {
echo -e "${GREEN}✓ ${1}${NC}"
((CHECKS_PASSED++))
}
print_warning() {
echo -e "${YELLOW}⚠ ${1}${NC}"
((CHECKS_WARNING++))
}
print_error() {
echo -e "${RED}✗ ${1}${NC}"
((CHECKS_FAILED++))
}
# Function to check prerequisites
check_prerequisites() {
if ! command -v psql &> /dev/null; then
echo "Error: psql is not installed"
exit 1
fi
if [ -z "$DB_PASSWORD" ]; then
echo "Error: DB_PASSWORD environment variable is required"
exit 1
fi
}
# Function to check database connection
check_connection() {
print_info "Checking database connection..."
export PGPASSWORD="$DB_PASSWORD"
if psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -c "SELECT 1;" > /dev/null 2>&1; then
print_success "Database connection successful"
else
print_error "Cannot connect to database"
exit 1
fi
}
# Function to validate schema exists
validate_schema_exists() {
print_info "Validating schema existence..."
export PGPASSWORD="$DB_PASSWORD"
local table_count=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT COUNT(*)
FROM information_schema.tables
WHERE table_schema = 'public';
" 2>/dev/null | tr -d ' ')
if [ "$table_count" -gt 0 ]; then
print_success "Found $table_count tables in database"
else
print_error "No tables found in database"
fi
}
# Function to validate required tables
validate_required_tables() {
print_info "Validating required tables..."
export PGPASSWORD="$DB_PASSWORD"
local required_tables=(
"User"
"Guild"
"RefreshToken"
"Session"
"ApiKey"
"PresenceEvent"
"TypingEvent"
"MessageEvent"
"JoinEvent"
"DeletedMessageEvent"
"ReactionTime"
"RoleChangeEvent"
"BlockedIP"
"WhitelistedIP"
"BannedUser"
)
for table in "${required_tables[@]}"; do
local exists=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT COUNT(*)
FROM information_schema.tables
WHERE table_schema = 'public' AND table_name = '$table';
" 2>/dev/null | tr -d ' ')
if [ "$exists" -eq "1" ]; then
if [ "$VERBOSE" = "true" ]; then
print_success "Table exists: $table"
fi
else
print_error "Missing required table: $table"
fi
done
}
# Function to validate indexes
validate_indexes() {
print_info "Validating indexes..."
export PGPASSWORD="$DB_PASSWORD"
local index_count=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT COUNT(*)
FROM pg_indexes
WHERE schemaname = 'public';
" 2>/dev/null | tr -d ' ')
if [ "$index_count" -gt 10 ]; then
print_success "Found $index_count indexes"
else
print_warning "Only $index_count indexes found (expected more)"
fi
# Check for critical indexes
local critical_indexes=(
"User_discordId_key"
"Guild_guildId_key"
"RefreshToken_token_key"
)
for index_name in "${critical_indexes[@]}"; do
local exists=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT COUNT(*)
FROM pg_indexes
WHERE schemaname = 'public' AND indexname = '$index_name';
" 2>/dev/null | tr -d ' ')
if [ "$exists" -eq "1" ]; then
if [ "$VERBOSE" = "true" ]; then
print_success "Critical index exists: $index_name"
fi
else
print_warning "Missing critical index: $index_name"
fi
done
}
# Function to validate foreign keys
validate_foreign_keys() {
print_info "Validating foreign key constraints..."
export PGPASSWORD="$DB_PASSWORD"
local fk_count=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT COUNT(*)
FROM information_schema.table_constraints
WHERE constraint_type = 'FOREIGN KEY' AND table_schema = 'public';
" 2>/dev/null | tr -d ' ')
if [ "$fk_count" -gt 5 ]; then
print_success "Found $fk_count foreign key constraints"
else
print_warning "Only $fk_count foreign key constraints found"
fi
# Check for foreign key violations
local violations=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
DO \$\$
DECLARE
r RECORD;
violation_count INTEGER := 0;
BEGIN
FOR r IN (
SELECT tc.table_name, tc.constraint_name
FROM information_schema.table_constraints tc
WHERE tc.constraint_type = 'FOREIGN KEY' AND tc.table_schema = 'public'
) LOOP
BEGIN
EXECUTE 'ALTER TABLE \"' || r.table_name || '\" VALIDATE CONSTRAINT \"' || r.constraint_name || '\"';
EXCEPTION WHEN OTHERS THEN
violation_count := violation_count + 1;
RAISE NOTICE 'Foreign key violation in %.%', r.table_name, r.constraint_name;
END;
END LOOP;
RAISE NOTICE 'Total violations: %', violation_count;
END;
\$\$;
" 2>&1 | grep "Total violations" | grep -oE '[0-9]+' || echo "0")
if [ "$violations" -eq "0" ]; then
print_success "No foreign key violations detected"
else
print_error "Found $violations foreign key violations"
fi
}
# Function to validate primary keys
validate_primary_keys() {
print_info "Validating primary key constraints..."
export PGPASSWORD="$DB_PASSWORD"
local tables_without_pk=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT t.table_name
FROM information_schema.tables t
LEFT JOIN information_schema.table_constraints tc
ON t.table_name = tc.table_name
AND tc.constraint_type = 'PRIMARY KEY'
AND tc.table_schema = 'public'
WHERE t.table_schema = 'public'
AND t.table_type = 'BASE TABLE'
AND tc.constraint_name IS NULL
AND t.table_name NOT LIKE '_prisma%';
" 2>/dev/null | tr -d ' ' | grep -v '^$')
if [ -z "$tables_without_pk" ]; then
print_success "All tables have primary keys"
else
print_error "Tables without primary keys:"
echo "$tables_without_pk"
fi
}
# Function to validate data types
validate_data_types() {
print_info "Validating critical data types..."
export PGPASSWORD="$DB_PASSWORD"
# Check that User.discordId is String (text/varchar)
local discord_id_type=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT data_type
FROM information_schema.columns
WHERE table_schema = 'public'
AND table_name = 'User'
AND column_name = 'discordId';
" 2>/dev/null | tr -d ' ')
if [[ "$discord_id_type" == *"character"* ]] || [[ "$discord_id_type" == *"text"* ]]; then
if [ "$VERBOSE" = "true" ]; then
print_success "User.discordId has correct type: $discord_id_type"
fi
else
print_warning "User.discordId type might be incorrect: $discord_id_type"
fi
# Check that timestamps use timestamptz
local tz_aware_count=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT COUNT(*)
FROM information_schema.columns
WHERE table_schema = 'public'
AND column_name IN ('createdAt', 'updatedAt')
AND data_type = 'timestamp with time zone';
" 2>/dev/null | tr -d ' ')
if [ "$tz_aware_count" -gt 10 ]; then
print_success "Timestamps are timezone-aware ($tz_aware_count columns)"
else
print_warning "Some timestamps may not be timezone-aware"
fi
}
# Function to validate data consistency
validate_data_consistency() {
print_info "Validating data consistency..."
export PGPASSWORD="$DB_PASSWORD"
# Check for NULL values in required fields
local null_violations=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT COUNT(*)
FROM \"User\"
WHERE discordId IS NULL OR username IS NULL;
" 2>/dev/null | tr -d ' ')
if [ "$null_violations" -eq "0" ]; then
if [ "$VERBOSE" = "true" ]; then
print_success "No NULL violations in User table"
fi
else
print_error "Found $null_violations NULL violations in User table"
fi
# Check for duplicate discordIds
local dup_discord_ids=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT COUNT(*)
FROM (
SELECT discordId, COUNT(*) as cnt
FROM \"User\"
GROUP BY discordId
HAVING COUNT(*) > 1
) dups;
" 2>/dev/null | tr -d ' ')
if [ "$dup_discord_ids" -eq "0" ]; then
if [ "$VERBOSE" = "true" ]; then
print_success "No duplicate discordIds found"
fi
else
print_error "Found $dup_discord_ids duplicate discordIds"
fi
}
# Function to validate Prisma migrations table
validate_prisma_migrations() {
print_info "Validating Prisma migrations..."
export PGPASSWORD="$DB_PASSWORD"
# Check migrations table exists
local migrations_exists=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT COUNT(*)
FROM information_schema.tables
WHERE table_schema = 'public' AND table_name = '_prisma_migrations';
" 2>/dev/null | tr -d ' ')
if [ "$migrations_exists" -eq "1" ]; then
print_success "Prisma migrations table exists"
# Check for failed migrations
local failed_migrations=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT COUNT(*)
FROM _prisma_migrations
WHERE finished_at IS NULL OR rolled_back_at IS NOT NULL;
" 2>/dev/null | tr -d ' ')
if [ "$failed_migrations" -eq "0" ]; then
print_success "All migrations completed successfully"
else
print_error "Found $failed_migrations failed or rolled back migrations"
fi
# Show migration count
local total_migrations=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT COUNT(*)
FROM _prisma_migrations
WHERE finished_at IS NOT NULL;
" 2>/dev/null | tr -d ' ')
print_info "Total applied migrations: $total_migrations"
else
print_warning "Prisma migrations table not found"
fi
}
# Function to check database size
check_database_size() {
print_info "Checking database size..."
export PGPASSWORD="$DB_PASSWORD"
local db_size=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -t -c "
SELECT pg_size_pretty(pg_database_size('$DB_NAME'));
" 2>/dev/null | tr -d ' ')
print_info "Database size: $db_size"
# Show table sizes if verbose
if [ "$VERBOSE" = "true" ]; then
echo ""
print_info "Table sizes:"
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -c "
SELECT
schemaname,
tablename,
pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) AS size
FROM pg_tables
WHERE schemaname = 'public'
ORDER BY pg_total_relation_size(schemaname||'.'||tablename) DESC
LIMIT 10;
" 2>/dev/null
fi
}
# Function to generate summary report
generate_summary() {
echo ""
echo "═══════════════════════════════════════════════════════════"
echo " Validation Summary"
echo "═══════════════════════════════════════════════════════════"
echo ""
echo -e "${GREEN}Passed: $CHECKS_PASSED${NC}"
echo -e "${YELLOW}Warnings: $CHECKS_WARNING${NC}"
echo -e "${RED}Failed: $CHECKS_FAILED${NC}"
echo ""
if [ $CHECKS_FAILED -eq 0 ]; then
print_success "All critical validations passed!"
if [ $CHECKS_WARNING -gt 0 ]; then
print_warning "Please review warnings above"
fi
return 0
else
print_error "Validation failed with $CHECKS_FAILED critical errors"
return 1
fi
}
# Main execution
main() {
echo ""
echo "═══════════════════════════════════════════════════════════"
echo " Database Migration Validation Suite"
echo "═══════════════════════════════════════════════════════════"
echo ""
check_prerequisites
check_connection
validate_schema_exists
validate_required_tables
validate_indexes
validate_foreign_keys
validate_primary_keys
validate_data_types
validate_data_consistency
validate_prisma_migrations
check_database_size
generate_summary
}
# Run main
main