Raspberry Pi Backup
I have been experimenting with creating backups for the raspberry pi. My backup philosophy is to backup everything on these little machines as I have a tendency to install packages and files everywhere. I have not been satisfied with traditional backup methods, such as rsync, rsnapshot, etc. I wanted a full image copy of the memory card in the raspberry pi.
So, I wrote a script to do this for you. I make some assumptions, but you can change the variables in the script. More on that in a bit. Basically, my script uses the “dd” command to make a bit by bit copy of your LIVE memory card and store it into a compressed image file on a USB flash drive. This is a tricky process, mainly because the Pi is powered on and utilizing the very memory card you are copying. If you have files that are constantly changing (databases, etc), this may not turn out so well. However, it works for my purposes and has saved my bacon a few times, especially when my Pi with Pi-Hole and my Ubiquity Cloud Key controller crashed in a fiery dumpster fire, you know, similar to 2020.
To recover, all I had to do was:
- Gracefully shutdown the pi: sudo shutdown -now
- Remove the flash drive & microSD from the Pi
- Insert Flash drive into my Windows 10 machine
- Unzip the last known good backup on the flash drive using 7-Zip all while saying a prayer
- TIP: Copy the image to your local hard drive as the image file is highly compressed (in my case a 2GB file expanded to 32GB)
- Use Rufus to rewrite the image file to microSD card
- Insert both USB and microSD back into Pi
- Power on and pray again.
- Test – TADA! I was back up and running within a short time.
This backup process is CPU and media intensive. It took about 2 hours and some change on a Raspberry Pi 3 to backup a 64GB memory card. Below is the script. I have it heavily commented and it creates logs of EVERYTHING. As I say: Logs or it didn’t happen. Save the below file to /home/pi/backup.sh
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
#!/bin/sh #Full Memory Card Backup Script for Raspberry Pi #v1.1 2020-10-19 by George J. Silowash #Setup Some Variables #MEMCARD is the device name of the the memory card. #Default is mmcblk0 MEMCARD=mmcblk0 #BKLOCATE is the complete path to the backup location #Default is /mnt/usb1 BKLOCATE=/mnt/usb1 #TIMESTAMP is the value that will be prepended to the backup filename #The below format is YYYYMMDD_HHMM format TIMESTAMP=`date +"%Y%m%d_%H%M%S"` #LOGTIME Is the date stamp used for the log filename LOGTIME=`date +"%Y%m%d"` #Log File Creates a log of each backup in a new file log="$BKLOCATE/$LOGTIME"_rpibkup.log #BKAPPEND is appended to TIMESTAMP when creating the filename BKAPPEND="_piholeunifi.img" #BKFNAME is the complete name of the backup file BKFNAME=$BKLOCATE/$TIMESTAMP$BKAPPEND echo `date +"%Y-%m-%d %H:%M:%S"` "-------------------------------------" | tee -a $log echo `date +"%Y-%m-%d %H:%M:%S"` "Using log file: "$log | tee -a $log echo `date +"%Y-%m-%d %H:%M:%S"` "Raspberry Pi Backup is starting" | tee -a $log echo `date +"%Y-%m-%d %H:%M:%S"` "Backing Up /dev/"$MEMCARD "to " $BKFNAME| tee -a $log echo `date +"%Y-%m-%d %H:%M:%S"` "Script Variables Below:" | tee -a $log echo `date +"%Y-%m-%d %H:%M:%S"` "MEMCARD="$MEMCARD | tee -a $log echo `date +"%Y-%m-%d %H:%M:%S"` "BKLOCATE="$BKLOCATE | tee -a $log echo `date +"%Y-%m-%d %H:%M:%S"` "TIMESTAMP="$TIMESTAMP | tee -a $log echo `date +"%Y-%m-%d %H:%M:%S"` "LOGTIME="$LOGTIME | tee -a $log echo `date +"%Y-%m-%d %H:%M:%S"` "LOG="$log | tee -a $log echo `date +"%Y-%m-%d %H:%M:%S"` "BKAPPEND="$BKAPPEND | tee -a $log echo `date +"%Y-%m-%d %H:%M:%S"` "BKFNAME="$BKFNAME | tee -a $log echo `date +"%Y-%m-%d %H:%M:%S"` "Let's start the backup..." | tee -a $log echo `date +"%Y-%m-%d %H:%M:%S"` "Now would be a good time to make coffee, it will be a while...if it goes right" echo `date +"%Y-%m-%d %H:%M:%S"` "Executing Command: dd if=/dev/"${MEMCARD}" | gzip -1 - | dd of="${BKFNAME}".gz | tee -a $log" | tee -a $log dd if=/dev/${MEMCARD} | gzip -1 - | dd of=${BKFNAME}.gz | tee -a $log echo `date +"%Y-%m-%d %H:%M:%S"` "Process Complete." | tee -a $log |
Assumptions
- Assumes your memory card is called mmcblk0. If this is not the case, change it on line 8. You will need to dig through the /dev/ directory to locate it. On my Pi, with Raspian, it was mmcblk0.
- Your USB flash drive will need to be mounted. Mine was called /dev/sda1 and I mounted it to /mnt/usb1. You can change the mount point to whatever you want by editing line 13.
- Change line 24 to append something meaningful to you about the image file. Since my pi is running Pi-Hole and UniFi Cloud key services, I used _piholeunifi.img. Make it whatever you want.
Test It
You should test the script out before scheduling a cronjob. Execute the following command:
|
1 |
sudo /home/pi/backup.sh |
Watch for any errors. Screen output is also captured to a log file on the flashdrive, assuming it is mounted correctly and you updated the variables above. If the script produces errors, check the variables for the microSD card (MEMCARD) and the USB flashdrive mount point (BKLOCATE). Once you have resolved any errors, move on.
Automate It
Before you can execute it, you will need to give it execution permissions:
|
1 |
sudo +x backup.sh |
Copy the script, I called it backup.sh, to:
|
1 |
sudo cp /home/pi/backup.sh /usr/bin/backup.sh |
This script needs to execute as root, otherwise, it will fail. You will need to schedule a cron job as root:
|
1 |
sudo crontab -e |
At the bottom of your crontab file, enter:
|
1 2 |
# execute the backup.sh script every Sunday at 00:05 5 0 * * SUN sudo /usr/bin/backup.sh |
Conclusion
This is a simple way to have a complete backup of your memory card as an image file that can be restored using a memory card writing program. This is not a fast process. If you require file level backup rather than an image file, you will want something else.
I should note that this script is not limited to Raspberry Pis. In theory, it will work on any Linux distribution.
BONUS: Digital Forensics
Because I am a digital forensics investigator, I try to preserve everything I can using the most complete method available. I have not tested this as a forensics method for collecting information off of a Raspberry Pi, but I assume it is pretty close. Of course, you know what they say about assuming. I will need to do some additional testing. I believe that since this is a live forensic capture, you will not be able to have consistent hash values of the microSD card and the resulting image. I will test that when I am able.
Future Work
I have a few ideas for some future work: Error handling (hey, I wrote this in about 30 minutes and have little bash scripting experience), Hashing, stop/start of services during the image creation process to minimize microSD card changes, encryption of backups, cloud upload.
2020-10-19 UPDATE:
After posting the original script, I realized I had an error in my code that only affected time stamping in the logs. I forgot that setting the time stamp variable LOGENTRY at one particular moment in time does not cause it to update, therefore, the time stamps in the log will be wrong. In the corrected version above, I remove the LOGENTRY variable in favor of calling the date command as log entries are created. The version has been incremented to 1.1.
