Automation Scripts: Streamlining Development Tasks

Automation Scripts: Streamlining Development Tasks

Automation scripts are executable programs that automate repetitive tasks in software development. Instead of typing the same sequence of commands dozens of times per day, you write a script once and run it whenever needed. Automation saves time, reduces human error, ensures consistency across team members, and frees developers to focus on solving interesting problems rather than performing manual drudgery.

Every development team should have a collection of automation scripts for common tasks: setting up new developer environments, running tests, building artifacts, deploying to staging, backing up databases, and cleaning up old files. To understand automation scripts properly, it is helpful to be familiar with CI/CD pipelines, Git workflow, and cloud deployment.

Manual Task                    → Automated Script
─────────────────────────────────────────────────────────
Run tests before commit        → pre-commit hook
Deploy to staging              → deploy-staging.sh
Set up new developer machine   → setup-dev-environment.sh
Backup production database     → backup-database.sh
Clean up old log files         → cleanup-logs.sh
Generate API documentation     → generate-docs.sh
Run database migrations        → migrate-database.sh

Why Automation Scripts Matter

Automation scripts transform error-prone, time-consuming manual processes into reliable, repeatable operations. They are the foundation of DevOps practices and modern software development.

• Save time: A task that takes 10 minutes manually takes 10 seconds with a script. Over a team of 10 developers doing the task daily, that saves hours every week.
• Reduce errors: Humans make mistakes when repeating the same steps. Scripts execute the same sequence every time without variation.
• Ensure consistency: Every team member performs tasks the same way. No more "it works on my machine" problems.
• Document processes: A script is executable documentation. Reading the script shows exactly what steps are performed.
• Enable CI/CD: Automation scripts are the building blocks of continuous integration and continuous deployment pipelines.
• Scale operations: Tasks that work for one server can run on hundreds of servers with the same script.
• Reduce context switching: Run a script and continue working while it executes in the background.

Common Automation Script Use Cases

Scripting Languages for Automation

Choosing the right language for automation scripts depends on your environment, team expertise, and task complexity.

Essential Automation Script Examples

1. Development Environment Setup Script

This script automates setting up a new developer machine, ensuring every team member has the same tools and configuration.

#!/bin/bash
# setup-dev.sh - Setup development environment

set -e  # Exit on error

echo "Setting up development environment..."

# Install Node.js
if ! command -v node &> /dev/null; then
    echo "Installing Node.js..."
    curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash -
    sudo apt-get install -y nodejs
fi

# Install dependencies
echo "Installing npm dependencies..."
npm ci

# Copy environment configuration
if [ ! -f .env ]; then
    cp .env.example .env
    echo "Created .env file from example"
fi

# Setup pre-commit hooks
echo "Setting up Git hooks..."
cp scripts/pre-commit .git/hooks/
chmod +x .git/hooks/pre-commit

# Run database migrations
echo "Running database migrations..."
npm run migrate

echo "Setup complete! Run 'npm start' to start the application"

2. Deployment Script

This script deploys the application to a staging or production server.

#!/bin/bash
# deploy.sh - Deploy application to server

set -e

ENVIRONMENT=${1:-staging}
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_DIR="/backups/${ENVIRONMENT}"

echo "Deploying to ${ENVIRONMENT} environment..."

# Run tests
echo "Running tests..."
npm test

if [ $? -ne 0 ]; then
    echo "Tests failed. Aborting deployment."
    exit 1
fi

# Build application
echo "Building application..."
npm run build

# Create backup before deployment
echo "Creating backup..."
mkdir -p ${BACKUP_DIR}
tar -czf ${BACKUP_DIR}/backup_${TIMESTAMP}.tar.gz dist/

# Deploy to server
echo "Deploying to server..."
rsync -avz --delete dist/ user@server:/var/www/app/

# Run database migrations
echo "Running database migrations..."
ssh user@server "cd /var/www/app && npm run migrate"

# Restart application
echo "Restarting application..."
ssh user@server "sudo systemctl restart myapp"

echo "Deployment to ${ENVIRONMENT} completed successfully!"

3. Database Backup Script

This script backs up a database and uploads it to cloud storage.

#!/bin/bash
# backup-db.sh - Backup database to cloud storage

set -e

DB_NAME="myapp_production"
DB_USER="backup_user"
BACKUP_DIR="/var/backups/postgres"
RETENTION_DAYS=30
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="${BACKUP_DIR}/${DB_NAME}_${TIMESTAMP}.sql.gz"

echo "Starting database backup..."

# Create backup directory if it doesn't exist
mkdir -p ${BACKUP_DIR}

# Create backup
echo "Creating backup..."
pg_dump -U ${DB_USER} ${DB_NAME} | gzip > ${BACKUP_FILE}

# Verify backup
if [ -f ${BACKUP_FILE} ]; then
    SIZE=$(du -h ${BACKUP_FILE} | cut -f1)
    echo "Backup created: ${BACKUP_FILE} (${SIZE})"
else
    echo "ERROR: Backup failed"
    exit 1
fi

# Upload to cloud storage (AWS S3 example)
echo "Uploading to S3..."
aws s3 cp ${BACKUP_FILE} s3://myapp-backups/${ENVIRONMENT}/${TIMESTAMP}/ 

# Delete old backups
echo "Deleting backups older than ${RETENTION_DAYS} days..."
find ${BACKUP_DIR} -name "*.sql.gz" -type f -mtime +${RETENTION_DAYS} -delete

echo "Backup completed successfully"

4. Health Check Script

This script verifies that all services are running correctly and sends alerts on failures.

#!/bin/bash
# health-check.sh - Verify service health

set -e

URL="https://myapp.com/health"
EXPECTED_STATUS="healthy"
ALERT_EMAIL="oncall@example.com"

# Check HTTP endpoint
echo "Checking health endpoint..."
RESPONSE=$(curl -s ${URL})
STATUS_CODE=$(curl -s -o /dev/null -w "%{http_code}" ${URL})

if [ "${STATUS_CODE}" != "200" ]; then
    echo "ERROR: Health check failed with status ${STATUS_CODE}"
    echo "Application is DOWN" | mail -s "ALERT: Application Down" ${ALERT_EMAIL}
    exit 1
fi

if [[ "${RESPONSE}" != *"${EXPECTED_STATUS}"* ]]; then
    echo "ERROR: Unexpected health response: ${RESPONSE}"
    exit 1
fi

echo "Health check passed - application is healthy"
exit 0

Script Organization Best Practices

• Store scripts in version control: Keep automation scripts in your repository under a scripts/ directory.
• Use consistent naming: Name scripts descriptively (e.g., deploy-staging.sh, backup-database.sh).
• Add error handling: Use set -e (Bash) or try/catch (Python) to stop on errors.
• Make scripts idempotent: Running a script multiple times should be safe and produce the same result.
• Log actions: Print what the script is doing to help debugging.
• Use command-line arguments: Make scripts flexible with parameters (e.g., ./deploy.sh production).
• Keep secrets out: Never hardcode passwords or API keys. Use environment variables or secret management.
• Document usage: Include a comment header explaining what the script does and how to use it.

#!/bin/bash
# Script: deploy.sh
# Description: Deploys application to specified environment
# Usage: ./deploy.sh 
#   environment: staging, production
# Requirements: AWS CLI, Docker, kubectl
# Author: DevOps Team
# Last Modified: 2024-01-15

Common Automation Script Mistakes to Avoid

• Hardcoding paths: Use relative paths or environment variables instead of absolute paths.
• No error handling: Scripts that continue after errors cause bigger problems.
• Not testing scripts: Test automation scripts in a safe environment before running in production.
• Overcomplicating: A simple Bash script is often better than a complex Python program for file operations.
• No logging: Without logging, you cannot debug why a script failed.
• Ignoring exit codes: Check exit codes of commands to detect failures.
• Not using version control: Scripts that live only on one server will be lost when that server fails.

Running Scripts on Schedule (Cron Jobs)

Many automation tasks need to run on a schedule. Cron is the standard scheduler on Unix/Linux systems.

# Edit crontab
crontab -e

# Run backup every day at 2 AM
0 2 * * * /home/user/scripts/backup-db.sh

# Run health check every 5 minutes
*/5 * * * * /home/user/scripts/health-check.sh

# Clean logs every Sunday at 3 AM
0 3 * * 0 /home/user/scripts/cleanup-logs.sh

# Deploy on merge to main (triggered by CI, not cron)

Integrating Scripts with Git Hooks

Git hooks are automation scripts that run automatically on Git events. They are perfect for enforcing code quality before commits or pushes.

#!/bin/bash
# .git/hooks/pre-commit - Run before commit

# Run linter
echo "Running linter..."
npm run lint

if [ $? -ne 0 ]; then
    echo "Linter failed. Commit aborted."
    exit 1
fi

# Run tests
echo "Running tests..."
npm test

if [ $? -ne 0 ]; then
    echo "Tests failed. Commit aborted."
    exit 1
fi

echo "All checks passed. Proceeding with commit."
exit 0

Frequently Asked Questions

1. What is the best language for automation scripts?
   Bash is best for Unix/Linux system tasks. Python is best for cross-platform and complex logic. Use what your team knows.
2. Should I use shell scripts or a configuration management tool?
   Shell scripts are fine for simple tasks. For managing many servers, use tools like Ansible, Puppet, or Chef.
3. How do I test automation scripts safely?
   Test in a development environment first. Use --dry-run flags where available. Run scripts with test data before production.
4. What is the difference between a script and a program?
   Scripts are typically interpreted and used for automation. Programs are compiled and used for application logic. The line is blurry.
5. How do I handle errors in Bash scripts?
   Use set -e to exit on error. Check exit codes with if [ $? -ne 0 ]; then. Use trap for cleanup on error.
6. What should I learn next after automation scripts?
   After mastering automation scripts, explore CI/CD pipelines, Infrastructure as Code, containerization with Docker, and configuration management tools.