Backup

Useful Links and Howtos

A comparison of utilities for filesystem backup

incremental

glastree:

doesn’t work remotely, easy, limited options.

incremental and compressed

storebackup

incremental backups to a local disk. lots of options. space efficient (gzipped diffs).

remote and incremental

by hand: (‘rsync’ + ‘cp -al’)
www.mikerubel.org/computers/rsync_snaps...
good overview of how and why. this has now been implented and is available in a debian package called rsnapshot

rsnapshot

based on mike rubel’s scripts. clean & easy configuration. push style backups.

snapback2

based on mike rubel’s scripts. clean & easy configuration. pull style backups.

mirrordir

MIRRORDIR mirrors a directory tree in every detail, including devices, ownerships, permissions, symlinks, hardlinks and access times, suitable for timed backups of disk drives. The minimal set of changes needed to make the original directory tree identical to the backup directory tree is executed. Can optionally create backup files before deleting, and store multiple revisions through any number of levels. A scripting language allows for custom excluding of special files. Works over ftp and mcfs. Also implements its own secure sockets for transparent strong encrypted file transfer, using a custom daemon.

dirvish

debian howto for dirvish: www.pegasys.ws/dirvish/debian.howto.html robust configuration files. confusing terminology,

rlbackup

rlbackup provides a simple secure mechanism for generating linked backups over the network.

remote, incremental, and compressed

rdiff-backup

good for big files, because only the changes are transmitted and stores gzipped files and then gzipped diffs. includes meta data files, so that even if the backup user can’t change file ownership permissions, this information is retained and will restore correctly. from the website: rdiff-backup backs up one directory to another, possibly over a network. The target directory ends up a copy of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves subdirectories, hard links, dev files, permissions, uid/gid ownership, and modification times. Also, rdiff-backup can operate in a bandwidth efficient manner over a pipe, like rsync. Thus you can use rdiff-backup and ssh to securely back a hard drive up to a remote location, and only the differences will be transmitted.

backupninja

Backupninja allows you to coordinate system backup by dropping a few simple configuration files into /etc/backup.d/. Most programs you might use for making backups don’t have their own configuration file format. Backupninja provides a centralized way to configure and coordinate many different backup utilities (including rdiff-backup).

remote, incremental, and encrypted

Duplicity

Duplicity backs directories by producing encrypted tar-format volumes and uploading them to a remote or local file server. Because duplicity uses librsync, the incremental archives are space efficient and only record the parts of files that have changed since the last backup. Because duplicity uses GnuPG to encrypt and/or sign these archives, they will be safe from spying and/or modification by the server. Here is a useful duplicity tutorial.

Box Backup

An open source, completely automatic on-line backup system for UNIX. All backed up data is stored on the server in files on a filesystem — no tape or archive devices are used. The server is trusted only to make files available when they are required — all data is encrypted. A backup daemon runs on systems to be backed up, and copies encrypted data to the server when it notices changes. Only changes within files are sent to the server, just like rsync. Old versions of files on the server are stored as changes from the current version. Behaves like tape — old versions and deleted files are available.

remote and encrypted

DIBS

Distributed Internet Backup System (DIBS). peer to peer distributed backup using encrypted chunks so that no one knows what they are backup up from other peers.

Recommendations

I tried them all, and I didn’t like any of them. So I wrote backupninja. Backupninja is not actually a backup program. Instead, it manages your entire backup scheme using easy ini style configurations files in /etc/backup.d/. Most programs you might use for making backups don’t have their own configuration file format. Backupninja provides a centralized way to configure and
coordinate many different backup utilities.

Features:

  • easy to read ini style configuration files.
  • you can drop in scripts to handle new types of backups.
  • backup actions can be scheduled
  • you can choose when status report emails are mailed to you
    (always, on warning, on error, never).
  • console-based wizard (ninjahelper) makes it easy to create
    backup action configuration files.
  • passwords are never sent via the command line to helper programs.
  • works with Linux-Vservers.

Backup types:

  • secure, remote, incremental filesytem backup (via rdiff-backup).
    incremental data is compressed. permissions are retained even
    with an unpriviledged backup user.
  • backup of mysql databases (via mysqlhotcopy and/or mysqldump).
  • backup of postgresql databases
  • backup of ldap databases (via slapcat and/or ldapsearch).
  • basic system and hardware info.
  • encrypted remote backups (via duplicity).
  • backup of subversion repositories.
  • burn CD/DVDs or create ISOs.

It slices, it dices!