Block Level Incremental Backup-Macintosh

Overview

Retrospect now has the ability to back up only the parts of a file that have changed. Many applications like Microsoft Outlook for Windows and FileMaker have large files that are constantly changing by small increments. After you enable block level incremental backup for a backup script, the next backup will be a full backup of modified files. For large files, subsequent backups using that backup script will be incremental, storing only blocks that changed since the prior backup. When restoring a file backed up using this feature, Retrospect first restores the full backup and then the subsequent increments. To restore the 5th backup of a file, for example, Retrospect will restore that file’s first full backup, and then each of the next four increments of that file.

Block level incremental backup works with existing Retrospect file level features. If a backup uses a selector/rule, only selected files are backed up fully or incrementally. If matching is enabled, any file that has a matching version in the backup set is skipped entirely. If software compression is enabled, block level increments are compressed and then stored in the backup set. When transferring snapshots or backup sets containing block level incremental backups of a file, the complete chain of prior increments leading to and including the full version of that file is automatically transferred. During grooming, if an increment of a file is preserved based on the grooming policy, the complete chain of prior increments leading to and including the full version of that file is automatically preserved.

Block level incremental backup works with various backup set types, such as disk, file and tape.

Storage Savings

With block level incremental backup enabled, significant storage savings is possible when backing up certain large files, above 90% for daily use in some cases.

Depending on how a specific application stores and modifies its data, storage savings from block level incremental backup can vary. Retrospect automatically excludes a number of known file types that do not benefit from block level incremental backup, and you can add others easily. Find out more in Options.

Usage

Block level incremental backup is a script and wizard option, available for use with all backup/media set types. The feature is off by default. You can enable it or disable it anytime. Once you enable block level incremental backup, the first backup will be a full backup of each new or modified file. During subsequent backups, only changed blocks are backed up for applicable files. Find out more in Technical Details.

The option is available under Options in Backup, Archive, and Proactive scripts.

Applicable files

With block level incremental backup enabled, by default files 100 MB or larger will be backed up incrementally. Smaller files will automatically be backed up in full because restore overhead outweighs the benefits of incremental backup. Find out more in

For digital media files, some media authoring apps change the files substantially even when only small edits are made. In these cases, benefits from block level incremental backup will be limited.

Logging

While a backup is in progress, Retrospect shows the full size of the files being backed up. Once completed, Retrospect will show the size of the increments that are actually backed up.

The following example shows the Operations Log of a completed backup of two modified files. One of the files has a full size of 100 MB, of which 5 MB has changed since previous backup. The other file is 1 MB in size, not meeting the default criteria of block level incremental backup and therefore backed up in full. The resulting backup’s actual size is about 6 MB (6,147 KB). Since this backup has software compression turned off, the 94% compression figure in the log indicates block level incremental backup has reduced the backup’s size by 94%.

When block level incremental backup is enabled, the amount of data shown in the progress panel can differ from the amount of data listed in the Operations Log. Retrospect calculates the data to be backed up based on the size of the files; however, the final number in the log is based on how much data was backed up. This final size varies according to how much was saved both via block level incremental backup as well as software compression.

Options

With block level incremental backup enabled, files 100 MB or larger will be backed up incrementally by default. Smaller files will automatically be backed up in full because restore overhead outweighs the benefits of incremental backup. This is customizable via a rule/selector named Block Level Filter. It controls how a file is backed up, i.e. whether it is backed up in full or incrementally. To select what files to back up, choose one of the other rules/selectors instead for your backup scripts, such as All Files Except Cache Files or User Files and Settings.

The Block Level Filter is in Preferences > Rules:

Other Thresholds

Block level incremental backup has two other thresholds: the number of backups and the number of days since prior backup. To reduce the risk of storage media loss breaking a chain of incremental backups (making some of the backups not restorable), Retrospect automatically performs a full backup if a file’s 30 recent backups are all incremental, or if its most recent backup is more than 31 days old. These settings are customizable in retro.ini:

Backward Compatibility

Block level incremental backups cannot be restored by earlier versions of Retrospect. When enabling a backup script for block level incremental backup, you will be prompted to upgrade the associated backup sets if they were created by a prior version of Retrospect. To keep the backup set compatible with prior Retrospect versions, cancel the upgrade prompt, and block level incremental backup will remain disabled.

Technical Details

Speed and size of a block level incremental backup depend on how a specific application stores and modifies its data. Applications like Apple Mail save each item—an email or a document—as a separate file. When these small files change, Retrospect can backup the whole file up quickly. Other applications store many items, database or disk image in a large file, so block level incremental backup should improve backup performance for these the most. For items like movies, photos, and music, the files themselves don't change, unless you edit them; Retrospect's standard backup with the matching feature will suffice.

After you enable block level incremental backup for a backup script, the next backup will be a full backup of new or modified files. During subsequent backups, each 2MB-block of applicable files is compared to its checksum from prior backup, and only changed blocks are backed up. For files not applicable to block level incremental backup, they will be backed up in full.

Retrospect now pre-scans NTFS and HFS+ volumes connected to the backup server and Retrospect clients, speeding overall backup and restore operations by removing the lengthy volume scan from backup process. This feature employs the USN change journal (for NTFS volumes) and FSEvents (for HFS+ volumes) to predetermine which files have changed since the last backup to a particular Media Set.

Instant Scan runs on the backup source computer. When Instant Scan data is used, the activity's execution log will show "Using Instant Scan." Starting from Retrospect 8.1.0 (266) for Windows and Retrospect 10.1.0 (221) for Mac, Instant Scan is only used for scheduled script activity, and not any activity manually started by clicking a Run button.