Technical Deep Dive on Forever-Incremental File-Based Backup

White Papers

Overview

Retrospect Backup’s goal is to protect your entire environment as quickly as possible in as little storage as possible. The core engine uses forever-incremental file-based backup technology to protect individual files and folders as well as full systems. Forever-incremental backup allows you to only back up file and system changes since the previous backup, minimizing backup time and storage costs.

Understanding How Retrospect Works

Retrospect is file-based backup software, as opposed to drive imaging software that copies blocks or sectors on a hard disk drive. While an image- or block-based tool can be very fast at recovering an entire disk partition or providing a point-in-time view into a partition’s contents, a file-based backup provides several distinct advantages. For example, it’s possible to quickly search multiple backup archives and recover multiple versions of a document, while still being able to provide a complete restore of an entire disk. It’s also easy to restore files backed up from a Windows PC’s NTFS volume to a Mac, because the volume format doesn’t matter for file-based recovery.

Retrospect employs several technologies to ensure that it only and always backs up the minimum number of files necessary to restore whatever volume it is protecting.

Forever-Incremental Backup Technology

Retrospect doesn’t use the traditional concepts of full, incremental, and differential backups.

• Full Backup: This is a complete point-in-time snapshot of a volume. They are time consuming and takes up the most space.
• Standard Incremental Backup: This is a backup that depends on a full backup and any needed incremental backups. Incremental backups increase backup speed and decrease the storage space that it takes to do a full backup. Incremental backups only back up the data that has changed since the previous backup. The primary disadvantage is that restores can be inaccurate and time-consuming. Example: to restore a backup on Wednesday after a Mon and Tues backup, you would have to first restore Monday’s full backup. After that, you would have to restore Tuesday’s backup, followed by Wednesday’s …and that is with just two incremental backups.
• Differential Backup: The advantage that differential backup offers over incremental backups is a shorter restore time. Restoring a differential backup never requires more than two backup sets. The tradeoff is that, as time progresses, differential backup media grows to contain much more data than an incremental backup.

These are outdated modes of backup that have significant drawbacks regarding performance and restore precision. Traditional full backups are incredibly time consuming but offer precise restores. Incremental backups save time during the backup process, yet they restore unwanted files that were renamed, moved, or deleted since the last full backup.

Retrospect’s Forever-Incremental Backup Technology works by matching the files on a source volume with the files that are already stored on the destination Backup Set. By doing so, Retrospect only needs to back up those files that have changed or been newly created since files were last written to the destination.

Forever-incremental backups offer the best of all option. The basic idea is that, like an incremental backup, a forever-incremental backup begins by taking a full backup of the data set. After that point, only incremental backups are performed.

What makes a forever-incremental backup different from a normal incremental backup is the availability of data. The backup server stores the data of all the backup sets on the preferred media set (disk/cloud/tape). It automates the restoration process so that you don’t have to figure out which backup sets need to be restored. In essence, the process of restoring the incremental data becomes completely transparent and mimics the process of restoring a full backup.

In addition to copying just the files not already present on the destination, the Forever-Incremental process saves a complete listing of all the files and folders that were present on the source at the time of the backup. This provides a point-in-time snapshot of the exact state of a volume, which Retrospect can later use as a guide to select the proper files for a restore.

Retrospect uses the term Snapshot to describe a list of all files, folders, and settings on a computer at a point in time when a backup occurred. Each time a backup is performed, another Snapshot is added to the Backup Set. Each Snapshot represents a restore point, which is a point in time for which a restore can be performed for each source. This is the definition of the term Snapshot that is used when performing transfers of backup data from one Backup Set to another.

Transfer Snapshots from Disk to Offsite Media

Backup data from the Disk Backup Sets is copied to offsite media Backup Sets, for disaster recovery and archive purposes. The data is moved using Snapshot Transfers, which copy selected data from the Disk Backup Set to a new or existing offsite media Backup Set.

After the first Snapshot Transfer to an offsite media Backup Set, the offsite media contains exactly one Snapshot (restore point) for each source computer that was backed up. Because the data was placed onto initial media with a Snapshot Transfer operation, it was collected from the Media Set without having to repeat the backup of the original source computers or clog the network infrastructure.

Each subsequent Snapshot Transfers to an existing offsite media Backup Set adds an additional restore point onto the offsite media for each source. Each restore point is identical to a restore point made from a full backup of each source computer. However, the Snapshot Transfer operations need to copy only the files and folders that are new or changed since the last Snapshot Transfer operation. The result is a collection of restore points that make extremely efficient use of your offsite media

Retrospect can utilize the incremental backups to produce a “synthetic full” backup. It does this by combining the existing full back up with the data from the incremental backups. The result is a synthetic full backup that is indistinguishable from a full backup that has been created the traditional way. As one can imagine, the primary advantage of a synthetic full backup is a greatly reduced restore time. Restoring a synthetic full backup doesn’t require the backup operator to restore multiple tape or disk sets like an incremental backup does. Synthetic full backups provide all the advantages of a true full backup but offer the decreased backup times and bandwidth usage of an incremental backup.

Retrospect Backup leverages a number of components to optimize its forever-incremental file-based backup workflow:

Data Deduplication

Retrospect’s Forever-Incremental technology provides another benefit to users: data deduplication that saves time and storage space needed for backups.

Because Retrospect only backs up files that aren’t already contained in the target Backup Set, it doesn’t store multiple copies of files that are duplicated around the network. For example, if Retrospect encounters a PowerPoint presentation on Computer B that’s exactly the same (in terms of name, size, creation and modification dates, etc.) as one it just backed up from Computer A, it doesn’t need to copy that file again. Likewise, if Computer A and Computer B share most of the same 60 GB iTunes music library, Retrospect only needs to copy the matching files one time. The more data that is duplicated around a network, the more time and storage space Retrospect saves.

Using selectors to further refine backups and restores

Retrospect allows the user to create selectors that can be used to filter unimportant files, or to specifically select files that meet certain criteria. Several rules come pre-defined, such as the “All Files Except Cache Files” rule, which tells Retrospect to ignore temporary cache files like those created by Web browsers. Not backing up such files can save significant storage space, since it’s typically unnecessary to restore them. Businesses need to be able to include or exclude files based on how critical they are to the business. Music and movies can be excluded while preserving the ability to perform a bare-metal recovery.

Selectors are flexible and powerful. They can be used to prevent operating system files from being backed up, or to restore all Microsoft Excel files larger than 2 MB, which were modified in the past 60 days, and contain the word “Forecast” in their names.

Customer-Centric Approach to Restoring

Retrospect’s modular design allows the Forever-Incremental technology to be used in reverse during a restore operation for a restore process focused on your data, part of Retrospect’s customer-centric approach to data protection.

By using the snapshot saved with the backup as a guide, Retrospect ensures that only those files not already on the volume being restored need to be written, while (depending on the restore options selected) files that don’t belong are deleted. This gives Retrospect the restore precision of a traditional full backup, only Retrospect doesn’t have to unnecessarily re-copy matching files that are already present on the destination. Retrospect offers three types of restores:

• Restore an entire volume to a previous point in time
• Browse and restore selected files and folders from a specific point in time
• Search for and restore one or more versions of one or more files from any backup

These options cover a multitude of cases and provide incredible restore flexibility.

Other Components

• Block-Level Incremental Backup (BLIB): Retrospect uses block-level incremental backup (BLIB) to only back up the blocks of a large files (pst, vdmk files etc.) that changed, further reducing the storage footprint.
• Automatic Backup Health: Customers can enable automatic backup healing by using a "Verify" script to auto-heal their backups. If any issues are found, Retrospect will back up the affected files in the next backup. It’s a flexible approach that you can apply to each backup set on your own schedule. It detects integrity issues.
• Scheduled Backup: Organizations have their own unique business flow, and Retrospect allows them to schedule their data protection strategies based on their business needs.
• Policy-Based Backup: Retrospect’s unique ProactiveAI policy-based scheduling enables businesses to intelligently protect their infrastructure based on policy window. Endpoints that go on and off the network will be protected as soon as they join, and with remote backup, remote workers are protected even outside of the office.
• Open File Protection with Microsoft VSS: To capture an accurate backup of a system, Retrospect’s Open File agent utilizes Windows Volume Shadow Copy Service (VSS) to snapshot the system. Retrospect will initiate a VSS Snapshot during every backup, creating an unchanging point-in-time version of the system that it can back up, while the actual system continues to be used. Once the backup is complete, Retrospect lets Windows forget that snapshot and resume writing to the file system normally.

Retrospect Backup’s Powerful Forever-Incremental backups provide Synthetic Full Backups and Restores, optimizes backup time, pipeline requirements and minimizes storage costs.

Last Update: January 4, 2022