-
Notifications
You must be signed in to change notification settings - Fork 57
feat: improved backup system with encryption and scheduling #448
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
NathanJ60
wants to merge
8
commits into
Arcadia-Solutions:main
Choose a base branch
from
NathanJ60:feat/416-better-backup-solution
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
d6c38c4
feat: improved backup system with encryption and scheduling
NathanJ60 d609c13
feat: add daemon mode for backup container
NathanJ60 ce5bf14
fix: remove :ro from scripts to allow chmod in container
NathanJ60 604f506
docs: add daemon mode documentation for backup
NathanJ60 4e9d0c6
refactor: merge backup services into single container
NathanJ60 3e03cbc
fix: support BACKUP_DESTINATION env var and remove chmod in container
NathanJ60 6ea5f39
docs: update backup documentation for single container
NathanJ60 fe3caf0
fix: simplify backup container volumes and use /tmp for temp files
NathanJ60 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,135 @@ | ||
| #!/bin/bash | ||
|
|
||
| # Arcadia Backup Cron Wrapper | ||
| # This script is designed to be run by cron for scheduled backups | ||
| # It loads configuration from backup.conf and handles logging/locking | ||
| # | ||
| # Installation: | ||
| # 1. Copy backup.conf.example to backup.conf and customize | ||
| # 2. chmod +x backup-cron.sh | ||
| # 3. Add to crontab: 0 3 * * * /path/to/backup-cron.sh | ||
| # | ||
| # Logs are written to /var/log/arcadia-backup.log by default | ||
|
|
||
| set -e | ||
|
|
||
| # Get script directory | ||
| SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||
|
|
||
| # Default configuration | ||
| LOG_FILE="${BACKUP_LOG_FILE:-/var/log/arcadia-backup.log}" | ||
| LOCK_FILE="/tmp/arcadia-backup.lock" | ||
|
|
||
| # ============================================================================ | ||
| # LOGGING | ||
| # ============================================================================ | ||
|
|
||
| log() { | ||
| echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE" | ||
| } | ||
|
|
||
| log_separator() { | ||
| echo "========================================" >> "$LOG_FILE" | ||
| } | ||
|
|
||
| # ============================================================================ | ||
| # LOCK HANDLING | ||
| # ============================================================================ | ||
|
|
||
| # Prevent concurrent runs | ||
| if [ -f "$LOCK_FILE" ]; then | ||
| # Check if the process is still running | ||
| if [ -f "$LOCK_FILE" ]; then | ||
| OLD_PID=$(cat "$LOCK_FILE" 2>/dev/null) | ||
| if [ -n "$OLD_PID" ] && kill -0 "$OLD_PID" 2>/dev/null; then | ||
| log "ERROR: Backup already running (PID: $OLD_PID), skipping" | ||
| exit 0 | ||
| else | ||
| log "WARNING: Stale lock file found, removing" | ||
| rm -f "$LOCK_FILE" | ||
| fi | ||
| fi | ||
| fi | ||
|
|
||
| # Create lock file with our PID | ||
| echo $$ > "$LOCK_FILE" | ||
| trap "rm -f $LOCK_FILE" EXIT | ||
|
|
||
| # ============================================================================ | ||
| # LOAD CONFIGURATION | ||
| # ============================================================================ | ||
|
|
||
| if [ -f "$SCRIPT_DIR/backup.conf" ]; then | ||
| source "$SCRIPT_DIR/backup.conf" | ||
| log "Configuration loaded from backup.conf" | ||
| else | ||
| log "WARNING: No backup.conf found, using defaults" | ||
| fi | ||
|
|
||
| # ============================================================================ | ||
| # RUN BACKUP | ||
| # ============================================================================ | ||
|
|
||
| log_separator | ||
| log "Starting scheduled backup" | ||
|
|
||
| cd "$SCRIPT_DIR" | ||
|
|
||
| # Build command arguments | ||
| BACKUP_ARGS="--db-docker" | ||
|
|
||
| [ "$BACKUP_ENCRYPT" = "true" ] && BACKUP_ARGS="$BACKUP_ARGS --encrypt" | ||
| [ -n "$BACKUP_PASSWORD" ] && BACKUP_ARGS="$BACKUP_ARGS --password \"$BACKUP_PASSWORD\"" | ||
| [ -n "$BACKUP_PASSWORD_FILE" ] && BACKUP_ARGS="$BACKUP_ARGS --password-file \"$BACKUP_PASSWORD_FILE\"" | ||
| [ -n "$BACKUP_DESTINATION" ] && BACKUP_ARGS="$BACKUP_ARGS --destination \"$BACKUP_DESTINATION\"" | ||
| [ -n "$BACKUP_REMOTE_TYPE" ] && BACKUP_ARGS="$BACKUP_ARGS --remote-type \"$BACKUP_REMOTE_TYPE\"" | ||
| [ -n "$BACKUP_REMOTE_HOST" ] && BACKUP_ARGS="$BACKUP_ARGS --remote-host \"$BACKUP_REMOTE_HOST\"" | ||
| [ -n "$BACKUP_REMOTE_USER" ] && BACKUP_ARGS="$BACKUP_ARGS --remote-user \"$BACKUP_REMOTE_USER\"" | ||
| [ -n "$BACKUP_REMOTE_PATH" ] && BACKUP_ARGS="$BACKUP_ARGS --remote-path \"$BACKUP_REMOTE_PATH\"" | ||
| [ -n "$BACKUP_REMOTE_KEY" ] && BACKUP_ARGS="$BACKUP_ARGS --remote-key \"$BACKUP_REMOTE_KEY\"" | ||
| [ -n "$BACKUP_REMOTE_PORT" ] && BACKUP_ARGS="$BACKUP_ARGS --remote-port \"$BACKUP_REMOTE_PORT\"" | ||
| [ "$BACKUP_DELETE_AFTER_UPLOAD" = "true" ] && BACKUP_ARGS="$BACKUP_ARGS --delete-after-upload" | ||
|
|
||
| # Run backup | ||
| log "Executing: ./backup.sh $BACKUP_ARGS" | ||
| eval "./backup.sh $BACKUP_ARGS" >> "$LOG_FILE" 2>&1 | ||
| STATUS=$? | ||
|
|
||
| if [ $STATUS -eq 0 ]; then | ||
| log "Backup completed successfully" | ||
| else | ||
| log "ERROR: Backup failed with exit code $STATUS" | ||
| fi | ||
|
|
||
| # ============================================================================ | ||
| # RETENTION POLICY | ||
| # ============================================================================ | ||
|
|
||
| if [ -n "$BACKUP_RETENTION_DAYS" ] && [ -n "$BACKUP_DESTINATION" ]; then | ||
| log "Applying retention policy: deleting backups older than $BACKUP_RETENTION_DAYS days" | ||
|
|
||
| # Count files before deletion | ||
| OLD_COUNT=$(find "$BACKUP_DESTINATION" -name "arcadia_backup_*" -mtime +$BACKUP_RETENTION_DAYS 2>/dev/null | wc -l) | ||
|
|
||
| if [ "$OLD_COUNT" -gt 0 ]; then | ||
| find "$BACKUP_DESTINATION" -name "arcadia_backup_*" -mtime +$BACKUP_RETENTION_DAYS -delete | ||
| log "Deleted $OLD_COUNT old backup(s)" | ||
| else | ||
| log "No old backups to delete" | ||
| fi | ||
| fi | ||
|
|
||
| # ============================================================================ | ||
| # SUMMARY | ||
| # ============================================================================ | ||
|
|
||
| if [ -n "$BACKUP_DESTINATION" ] && [ -d "$BACKUP_DESTINATION" ]; then | ||
| BACKUP_COUNT=$(find "$BACKUP_DESTINATION" -name "arcadia_backup_*" 2>/dev/null | wc -l) | ||
| BACKUP_SIZE=$(du -sh "$BACKUP_DESTINATION" 2>/dev/null | cut -f1) | ||
| log "Current backups: $BACKUP_COUNT files, total size: $BACKUP_SIZE" | ||
| fi | ||
|
|
||
| log "Scheduled backup job finished" | ||
| log_separator | ||
|
|
||
| exit $STATUS | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| # Arcadia Backup Configuration | ||
| # ============================= | ||
| # Copy this file to backup.conf and customize for your environment | ||
| # | ||
| # This configuration is used by backup-cron.sh for scheduled backups | ||
| # You can also set these as environment variables | ||
|
|
||
| # ============================================================================ | ||
| # ENCRYPTION | ||
| # ============================================================================ | ||
|
|
||
| # Enable encryption (AES256 via GPG) | ||
| BACKUP_ENCRYPT=true | ||
|
|
||
| # Encryption password (REQUIRED if BACKUP_ENCRYPT=true) | ||
| # Choose ONE of the following methods: | ||
| BACKUP_PASSWORD=your_secure_passphrase_here | ||
|
|
||
| # OR use a password file (more secure for automated systems) | ||
| # BACKUP_PASSWORD_FILE=/path/to/secure/password/file | ||
|
|
||
| # ============================================================================ | ||
| # LOCAL STORAGE | ||
| # ============================================================================ | ||
|
|
||
| # Directory where backups will be stored | ||
| BACKUP_DESTINATION=/home/ubuntu/arcadia-backups | ||
|
|
||
| # ============================================================================ | ||
| # REMOTE STORAGE (Optional) | ||
| # ============================================================================ | ||
|
|
||
| # Upload method: rsync or scp | ||
| # BACKUP_REMOTE_TYPE=rsync | ||
|
|
||
| # Remote server hostname | ||
| # BACKUP_REMOTE_HOST=backup.example.com | ||
|
|
||
| # Remote server username | ||
| # BACKUP_REMOTE_USER=backup | ||
|
|
||
| # Remote destination path | ||
| # BACKUP_REMOTE_PATH=/backups/arcadia | ||
|
|
||
| # SSH private key file (optional, uses default SSH key if not set) | ||
| # BACKUP_REMOTE_KEY=/home/ubuntu/.ssh/backup_key | ||
|
|
||
| # SSH port (default: 22) | ||
| # BACKUP_REMOTE_PORT=22 | ||
|
|
||
| # Delete local backup after successful remote upload | ||
| # BACKUP_DELETE_AFTER_UPLOAD=false | ||
|
|
||
| # ============================================================================ | ||
| # RETENTION POLICY | ||
| # ============================================================================ | ||
|
|
||
| # Number of days to keep backups (older backups are automatically deleted) | ||
| BACKUP_RETENTION_DAYS=7 | ||
|
|
||
| # ============================================================================ | ||
| # LOGGING | ||
| # ============================================================================ | ||
|
|
||
| # Log file path | ||
| BACKUP_LOG_FILE=/var/log/arcadia-backup.log | ||
|
|
||
| # ============================================================================ | ||
| # CRON SCHEDULE EXAMPLES | ||
| # ============================================================================ | ||
| # Add one of these to your crontab (crontab -e): | ||
| # | ||
| # Daily at 3:00 AM: | ||
| # 0 3 * * * /home/ubuntu/arcadia-services/backup-cron.sh | ||
| # | ||
| # Every 6 hours: | ||
| # 0 */6 * * * /home/ubuntu/arcadia-services/backup-cron.sh | ||
| # | ||
| # Weekly on Sunday at 2:00 AM: | ||
| # 0 2 * * 0 /home/ubuntu/arcadia-services/backup-cron.sh | ||
| # | ||
| # Monthly on the 1st at 4:00 AM: | ||
| # 0 4 1 * * /home/ubuntu/arcadia-services/backup-cron.sh |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🛠️ Refactor suggestion | 🟠 Major
Fix trap expansion to prevent variable substitution at invocation time.
Use single quotes in trap to defer variable expansion until the trap is triggered:
As per shellcheck (SC2064), this ensures
$LOCK_FILEis evaluated when the trap runs, not when it's defined.📝 Committable suggestion
🧰 Tools
🪛 Shellcheck (0.11.0)
[warning] 56-56: Use single quotes, otherwise this expands now rather than when signalled.
(SC2064)
🤖 Prompt for AI Agents