This howto will illustrate a simple way to backup files to an external USB drive. This is not intended to be a complete backup solution but is a framework for both quick and dirty backups and some backups and also a framework for any app that may use the RSYNC process with CronTab to perform scheduled backups to opportunistic USB devices. This method should only be used in conjunction with regular inspection and backup testing and validation.
You should have some level of comfort with vi and other command line tools before using this method or at the least, try it out in a test environment. Even veteran command-line gurus should also familiarize themselves with the Centralized User Data model for ClearOS since we will create our mount points under this model and this model accounts for backup stores.
Decide whether you want to have your media for your backup readable by windows or not. The advantage to using FAT is that you can read the disk drive with a Windows Machine. The advantage to using EXT3 is that you can backup permissions. But you don't get both advantages.
1) Partition USB or other removable drive in ClearOS or other linux distribution.
a) in Linux you would identify the drive using 'fdisk -l' b) then using fdisk, create the partition. Use type **83** for the partition
2) Validate the drive partition in linux (fdisk -l)
fdisk -l
You should see something like this.
Disk /dev/sdc: 8011 MB, 8011120640 bytes 41 heads, 41 sectors/track, 9307 cylinders Units = cylinders of 1681 * 512 = 860672 bytes Device Boot Start End Blocks Id System /dev/sdc1 5 9308 7819328 83 Linux
3) Format the disk
mkfs.ext3 /dev/sdc1
4) Give the drive a label
mlabel -i /dev/sd*1 -s :: #### Replace the * with the drive letter in linux given for the drive (Step 2)
e2label /dev/sdc1 VOLUMENAME
1) Partition USB or other removable drive (ok to do in Windows if works)
a) in Linux you would identify the drive using 'fdisk -l' b) then using fdisk, create the partition. Give it type **c** for the partition
2) Validate the drive partition in linux (fdisk -l)
fdisk -l
You should see something like this.
Disk /dev/sdc: 8011 MB, 8011120640 bytes 41 heads, 41 sectors/track, 9307 cylinders Units = cylinders of 1681 * 512 = 860672 bytes Device Boot Start End Blocks Id System /dev/sdc1 5 9308 7819328 c W95 FAT32 (LBA)
3) Give the drive a label, can be done in Window (maybe)
mlabel -i /dev/sd*1 -s ::
#### Replace the * with the drive letter in linux given for the drive (Step 2)
If you receive a warning 'Total number of sectors (###) not a multiple of sectors per track (##)! Add mtools_skip_check=1 to your .mtoolsrc file to skip this test, type the following:
echo "mtools_skip_check=1" > /root/.mtoolsrc
And try running the 'mlabel' command again.
4) If the label name is wrong give the volume a label in Linux and validate:
mlabel -i /dev/sd*1 ::VOLUMENAME #### Replace the * with the drive letter in linux given for the drive (Step 2) #### Replace the VOLUMENAME with the name desired
mlabel -i /dev/sd*1 -s :: #### Replace the * with the drive letter in linux given for the drive (Step 2)
5) Make the landing directory for the VOLUMENAME
mkdir /store/VOLUMENAME && touch /store/VOLUMENAME/volume.not.mounted
6) Update the /etc/fstab to reflect the new VOLUMENAME
nano /etc/fstab
#### If nano is not installed, install it (yum install nano or use vi if you are more comfortable with that)
a) The line added will look like this:
LABEL=VOLUMENAME /store/VOLUMENAME vfat defaults,noauto 0 0
#### Use the Control+O (^O) to write the file. Use Control+X (^X) to exit. (in vi, use 'i' to enter insert mode for editing and <esc> to exit editing, use :wq to write and quit)
7) Mount the USB drive for testing
mount /store/VOLUMENAME
8) Validate the USB is mounted by making sure it is listed when running the 'mount' command:
root@gateway ~]# mount /dev/md1 on / type ext3 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) /dev/md0 on /boot type ext3 (rw) tmpfs on /dev/shm type tmpfs (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) /dev/sdc1 on /store/VOLUMENAME type vfat (rw)
/dev/sdc1 on /store/VOLUMENAME type vfat (rw) **** Look for this one
/dev/sdc1 on /store/VOLUMENAME type ext3 (rw) **** or this one
9) Make the two folders and the validation file on the USB drive that is mounted
mkdir /store/VOLUMENAME/backup /store/backup1/backup/gateway touch /store/VOLUMENAME/VOLUMENAME.do.not.delete.me
10) Validate that the file and directories are made using:
ls -laR /store/VOLUMENAME/*
11) Unmount the drive
umount /store/VOLUMENAME
12) All done, refer to the section about setting up the scripts and schedules.
1) Copy the scripts from a system which is working well to the '/root' folder on the target server or make the scripts using your favorite editor
The files required are:
A copy of these files are included in the Appendix. Be sure to modify some of the static constants in the files.
2) Make sure that the script files are executable by running the following:
chmod +x /root/backupdaily.sh /root/backupweekly.sh
3) As needed, modify the 'backup.includes' to suit your needs.
4) Install mutt.
yum --enablerepo=clearos-core install mutt
5) Install the SMTP server module in Webconfig under ClearCenter ยป Modules.
6) Refresh you browser and navigate to Server > Mail > SMTP Server. Start the server and set it to automatically start.
1) The scheduler uses CRON to run jobs. You can use the template below to populate your initial crontab (a tablature of the scheduled jobs)
2) To edit the crontab run the following:
crontab -e
3) This will place you in the 'vi' (VIsual text editor)
a) 'vi' has to basic modes; command mode and insert mode. The starting mode is command mode b) press 'i' to enter insert mode. You can tell that you are in insert mode by the '-- INSERT --' comment at the bottom of the screen c) press <Escape> at any time to go into command mode
4) While in insert mode, type or paste the data from the template.
5) Once the data in listed on the screen, exit insert mode by pressing <Escape>. To save the data
type the following (it will appear at the bottom of the screen): :wq
6) Press <Enter>. The screen will breifly flash the message 'Installing crontab'
0 2 * * 2 /root/backupdaily.sh backup1 Monday 0 2 * * 3 /root/backupdaily.sh backup1 Tuesday 0 2 * * 4 /root/backupdaily.sh backup1 Wednesday 0 2 * * 5 /root/backupdaily.sh backup1 Thursday 0 2 * * 6 /root/backupweekly.sh backup1 0 2 * * 2 /root/backupdaily.sh backup2 Monday 0 2 * * 3 /root/backupdaily.sh backup2 Tuesday 0 2 * * 4 /root/backupdaily.sh backup2 Wednesday 0 2 * * 5 /root/backupdaily.sh backup2 Thursday 0 2 * * 6 /root/backupweekly.sh backup2
Info: Column meanings:
For additional technical information please visit
Usage: In our usage we want to execute backups at 2 AM each morning. To do this we specify '0' for the minute and '2' for the hour. We place an asterisk '*' on the day of the month column and also on the month of the year column because we want to have the script run regardless of the day of the month or the month of the year. What is important is that these backups occur on specific days of the week and we specify those in the fifth column accordingly.
The paramaters we pass are those required by the backup scripts. The daily backup script requires that we specify which volume is the target (e.g. backup1 or backup2) and which day of the week we are going to use (i.e Monday through Friday). For the weekly backup we only need to specify the volumen name.
Please refer to the USB labelling document to match the volume name with the mount point in /etc/fstab.
#!/bin/bash device=$1 #### replace this value 'gateway' with the pingable name of your server. servername=gateway #### replace this value 'helpdesk@example.com' with a valid email address. notifyemail=helpdesk@example.com if [ "$1" = "" ]; then echo Syntax: /root/backupdaily [DEVICENAME] [BACKUPDAY] echo EXAMPLE: /root/backupdaily backup1 Sunday exit 0 fi #### Mounting Backup Device alreadymounted=`/bin/mount|grep $device` if [ -n "$alreadymounted" ]; then echo Backup device already mounted else echo Attempting mount of Backup device /bin/mount /store/$device fi if [ -f /store/$device/$device.do.not.delete.me ]; then echo Backup device validated else echo Backup device invalid or not inserted correctly. Please contact the administrator exit 0 fi alreadymounted=`/bin/mount|grep $device` #### Setup day of the Week #### if [ -z $2 ]; then echo "Backup using default day of Monday" day=monday else day=$2 if [[ "$day" =~ "((Mon)|(Tues)|(Wednes)|(Thurs)|(Fri)|(Satur)|(Sun))day" ]]; then datevalid=1 else datevalid=0 fi fi #### Determine FAT or EXT4 vfatfs=`echo $alreadymounted|grep vfat` extfs=`echo $alreadymounted|grep ext` #### Backup folder=/store/$device/backup/$servername backupdir=$folder/$day logfile="`date '+%d%B%Y'`".log errorfile="`date '+%d%B%Y'`".error.log backuplog=/tmp/$device.$logfile.log errorlog=/tmp/$device.$errorfile.log if [ ! -d $backupdir ]; then mkdir $backupdir fi if [ ! -d $backupdir ]; then echo Backup failed, check layout of backup media else if [ "$datevalid" != "1" ]; then echo Date invalid, check run script. else if [ -n "$extfs" ]; then echo "EXT" fi if [ -n "$vfatfs" ]; then rm -rf $backupdir/* ls -laR / > $backupdir/directory.permissions.log echo Backup started. `date` /usr/bin/rsync -rtv --include-from=/root/backup.includes / $backupdir/ 2> $errorlog echo Backup complete. `date` fi fi fi #### Notify #### touch $backuplog date > $backuplog echo " " >> $backuplog echo "`df -h|grep Filesystem`" >> $backuplog echo "`df -h |grep /store/$device`" >> $backuplog echo " " >> $backuplog if [ -f $errorlog ]; then echo "Errors:" >> $backuplog echo "`cat $errorlog`" >> $backuplog echo " " >> $backuplog fi if [ -d $backupdir/var ]; then status="Backup SUCCESSFUL!" echo $status >> $backuplog echo " " >> $backuplog else status="Backup FAILED!" echo $status >> $backuplog echo " " >> $backuplog fi /bin/cp $backuplog $backupdir/$logfile host=`hostname` if [ -f $backupdir/$logfile ]; then echo "$status $host - $device" | /usr/bin/mutt $notifyemail -a /$backupdir/$logfile -s "$status" else echo "Backup script failed. $host - $device" | /usr/bin/mutt $notifyemail -a $backuplog -s "Backup script failed, contact support" fi echo "Files backed up listed below: " >> $backupdir/$logfile echo " " >> $backupdir/$logfile ls -laR $backupdir >> $backupdir/$logfile sleep 5 rm $backuplog rm $errorlog sync sleep 2 #### Unmounting USB device #### /bin/umount /store/$device sleep 4 alreadymounted=`/bin/mount|grep $device` if [ -n "$alreadymounted" ]; then echo Backup device failed to dismount $device, attempting dismount again /bin/umount /store/$device else echo Device $device: unmounted successfully. exit 0 fi sleep 4 alreadymounted=`/bin/mount|grep $device` if [ -n "$alreadymounted" ]; then echo Unable to dismount $device. Contact your administrator echo `lsof /store/$device` else echo Device '$device': unmounted successfully. exit 0 fi
device=$1 #### replace this value 'gateway' with the pingable name of your server. servername=gateway #### replace this value 'helpdesk@example.com' with a valid email address. notifyemail=helpdesk@example.com if [ "$1" = "" ]; then echo Syntax: /root/backupweekly [DEVICENAME] echo EXAMPLE: /root/backupweekly backup1 exit 0 fi #### Mounting Backup Device alreadymounted=`/bin/mount|grep $device` if [ -n "$alreadymounted" ]; then echo Backup device already mounted else echo Attempting mount of Backup device /bin/mount /store/$device fi if [ -f /store/$device/$device.do.not.delete.me ]; then echo Backup device validated else echo Backup device invalid or not inserted correctly. Please contact the administrator exit 0 fi alreadymounted=`/bin/mount|grep $device` day=`date '+%d%B%Y'` echo $day datevalid=1 #### Determine FAT or EXT4 vfatfs=`echo $alreadymounted|grep vfat` extfs=`echo $alreadymounted|grep ext` #### Backup folder=/store/$device/backup/$servername backupdir=$folder/$day logfile="`date '+%d%B%Y'`".log errorfile="`date '+%d%B%Y'`".error.log backuplog=/tmp/$device.$logfile.log errorlog=/tmp/$device.$errorfile.log if [ ! -d $backupdir ]; then mkdir $backupdir fi if [ ! -d $backupdir ]; then echo Backup failed, check layout of backup media else if [ "$datevalid" != "1" ]; then echo Date invalid, check run script. else if [ -n "$extfs" ]; then echo "EXT" fi if [ -n "$vfatfs" ]; then rm -rf $backupdir/* ls -laR / > $backupdir/directory.permissions.log echo Backup started. `date` /usr/bin/rsync -rtv --include-from=/root/backup.includes / $backupdir/ 2> $errorlog echo Backup complete. `date` fi fi fi sync sleep 2 #### Notify #### touch $backuplog date > $backuplog echo " " >> $backuplog echo "`df -h|grep Filesystem`" >> $backuplog echo "`df -h |grep /store/$device`" >> $backuplog echo " " >> $backuplog if [ -f $errorlog ]; then echo "Errors:" >> $backuplog echo "`cat $errorlog`" >> $backuplog echo " " >> $backuplog fi if [ -d $backupdir/var ]; then status="Backup SUCCESSFUL!" echo $status >> $backuplog echo " " >> $backuplog else status="Backup FAILED!" echo $status >> $backuplog echo " " >> $backuplog fi /bin/cp $backuplog $backupdir/$logfile host=`hostname` if [ -f $backupdir/$logfile ]; then echo "$status $host - $device" | /usr/bin/mutt $notifyemail -a /$backupdir/$logfile -s "$status" else echo "Backup script failed. $host - $device" | /usr/bin/mutt $notifyemail -a $backuplog -s "Backup script failed, contact support" fi echo "Files backed up listed below: " >> $backupdir/$logfile echo " " >> $backupdir/$logfile ls -laR $backupdir >> $backupdir/$logfile sleep 5 rm $backuplog rm $errorlog #### Unmounting USB device #### /bin/umount /store/$device sleep 4 alreadymounted=`/bin/mount|grep $device` if [ -n "$alreadymounted" ]; then echo Backup device failed to dismount $device, attempting dismount again /bin/umount /store/$device else echo Device $device: unmounted successfully. exit 0 fi sleep 4 alreadymounted=`/bin/mount|grep $device` if [ -n "$alreadymounted" ]; then echo Unable to dismount $device. Contact your administrator echo `lsof /store/$device` else echo Device '$device': unmounted successfully. exit 0 fi
+ /var/flexshare/shares/** + /var/flexshare/shares/ + /var/flexshare/ + /var/ + /home/** + /home/