Using SyncBackPro > Easy Mode

SyncBack can make backing up even faster if you choose the Fast Backup option. Read this help page carefully before you continue as there are some important considerations to make when choosing this option. You'll also find some Frequently Asked Questions about Fast Backup below along with some examples.

The Fast Backup option is displayed when you create or modify a profile and select the option from the tree:

fast-backup

warning

Some of the descriptions below refer to "Fast Backup data". This is data that is kept by SyncBack to keep track of what has changed between each profile run. It is used internally by SyncBack.

•Fast Backup:

oDo not perform a fast backup: Enable this option to perform a standard backup, i.e. the source and destination will be scanned and compared to decide which files must be copied.

oPerform a fast backup: Enable this option to greatly improve the performance of a backup profile by not scanning the destination. Note that this option is only available when a profile is configured in a certain way (i.e. the source is not being changed by the settings). For more information, including side effects of fast backups, please see the section below.

oPerform a fast backup using the archive attribute: Enable this option to greatly improve the performance of a backup profile by not scanning the destination. This option is different from the one above because it uses the traditional backup method of using the archive attribute of a file to decide if a file should be copied or not. Note that this option is only available when a profile is configured in a certain way. For more information, including side effects of fast backups, please see the section below.

oPerform a fast backup of emails: Enable this option to greatly improve the performance of the backup of emails. Note that this option is only available when a profile is configured to backup emails and the email server is IMAP4 or Microsoft Exchange. It is not available with POP3. For more information, including side effects of fast backups, please see the section below.

•Keep fast backup data based on the actual destination directory (each destination has a full backup): If this option is unticked then a fast backup works much the same as an incremental backup. This means that only new or modified files are copied from the source to the destination regardless of where the destination is. If you are not using variables, e.g. %DAYOFWEEK%, in the destination then you can leave this option unticked as it will make no difference. This option is not available when using a Fast Backup with the archive attribute or backup of emails.

If you are using variables in the destination then you should consider ticking this option. If this option is ticked then the fast backup works in a different way. It keeps track of which files and folders are in each destination. This means each destination directory will have a complete backup and not just contain the new/changed files.

•Differential backup (do not update the fast backup data): If this option is ticked then a fast backup works the same as a differential backup. This means that only new or modified files since the last full backup are copied from the source to the destination. If you are not using variables, e.g. %DAYOFWEEK%, in the destination then you can leave this option unticked as it will make no difference. For example, if your destination is X:\%DAYOFWEEK%\, and you force a rescan on Mondays, then your Monday backup will be a full backup. Your Tuesday backup will contain new and changed files since Monday, Wednesday’s backup will contain new and changed files since Monday, etc. If the Fast Backup is using the archive attribute then the archive bit is not cleared on the original file when it is copied unless it is a full/rescan backup. This option is not available when using a Fast Backup of emails.

•Delete all the files and folders in the destination before the backup (only if it is not a rescan): If this option is ticked then all the files and folders (just the Zip file if compressing to one single Zip file) in the destination are deleted before the backup is made. The files and folders are not deleted if a re-scan has been forced or is required.

This option is best used when Keep fast backup data based on the actual destination directory is unticked. For example, if you keep 7 days worth of backups (using the %DAYOFWEEK% variable in the destination), and force a rescan each Monday, then by enabling this option you'll ensure that the Monday backup contains a complete backup and that the backups for all the other days just contain new/changed files since the previous day.

This option is not available when using a Fast Backup of emails.

warning

Note that with this option enabled it is not advisable to run incremental/differential backups more than once when SyncBack is backing up to the same folder. Considering the above example: if you run incremental backup on Tuesday morning then all the new/changed files since Monday will be copied to Tuesday folder. Again, if you run incremental backup on Tuesday afternoon then, all the existing files/folders (those files/folders copied in the last run) in the Tuesday folder will be deleted (as 'Delete all files and folders..' option is enabled) and only the new/changed files from the last run until now will be copied to the Tuesday folder. Hence, the Tuesday folder will not be a complete incremental backup from Monday as some of the files/folders were deleted from the incremental backup folder during the second run.

•Use a different folder for full (rescan) backups: If this option is ticked then you can define which folder should be used for full backups, i.e. where to backup to if there is a re-scan. This can be very useful when you always want full backups to go into one folder, and incremental/differential backups to go into the usual destination folder. If you tick this option then SyncBack will automatically set the full folder to your destination folder. However, it is unlikely this is the folder that you wish to use so you should modify it as appropriate.

For example, if your destination is X:\%DAYOFWEEK%\, and you have it set to rescan on Monday, then you probably want your full backup folder set to X:\1\. This means the full backups will always go into the Monday (1) folder even if you force a rescan on a Friday, for example.

This option is not available when using a Fast Backup of emails.

•Force Re-scan: Click this button to force SyncBack to scan the destination/email server next time the profile is run. Please read the notes below about the consequences of forcing a rescan when using FTP. If the button is disabled it is either because the profile is not a Fast Backup profile or SyncBack will already be performing a rescan on the next profile run, e.g. the button has already been pressed. If you want to force several Fast Backup profiles to re-scan then select them in the main window and select Rescan from the pop-up menu.

•Do Not Re-scan: If you are using Fast Backup with the archive attribute, then click this button to force SyncBack to not scan the destination next time the profile is run. If the button is disabled it is either because the profile is not an Archival Fast Backup profile or SyncBack will already not be performing a rescan on the next profile run, e.g. the button has already been pressed. To force a rescan, click the Force Re-scan button. This button is useful when you create a new profile and do not want it to scan the destination when it is first run.

•Force a re-scan when: This lets you define when SyncBack should perform a complete re-scan of the destination/email server. For example, to force a complete re-scan every Monday you would select %DAYOFWEEK% from the list, select equals from the drop-down, and type in 1 (1=Monday, 7=Sunday) in the edit box. See the section below for details on why this may be required and what it does. Please read the notes below about the consequences of forcing a rescan when using FTP. Note that you cannot enter a list of values, for example you cannot use %DAYOFWEEK% and enter 1,3 to rescan on Mondays and Wednesdays. Only one value may be entered. With the Pro version it is possible to use scripting to decide when a rescan should occur, meaning far more complex evaluations can be made (see the IncVar example script). If you use greater than, greater than or equal, less than or less than equal, then see below on what happens when you evaluate strings and not numbers. If you hover the mouse over the list of variables then a hint will appear with the current value of the variable. If you select a variable by double-clicking it then the variable will be selected and the variables current value will be used.

Important Information About Fast Backups

Explaining Fast Backups

When you backup files to the destination it is assumed that no other application, or person, will be changing the files in the destination. For example, if you backup your files to another drive you are not going to be editing or changing those backup files (except using SyncBack to replace them as appropriate). Because of this SyncBack should be able to remember what files, and directories, are on the destination without needing to scan it to find out.

How 'Fast Backup' works

First, you need to enable a Fast Backup option on the Fast Backup tab. The Fast Backup option is not available if the profile is configured such that it cannot use the Fast Backup option, e.g. it's an Intelligent Synchronization profile.

How Fast Backup works depends on the method chosen:

•Backup of emails: Email servers (IMAP4 and Microsoft Exchange) give each email a unique index value (a UIDL). When SyncBackPro asks the email server for a list of emails it needs to get the email headers of each email to see what the local filenames will be and so compare it to the locally stored emails. Retrieving that email header can be slow. When Fast Backup is enabled SyncBackPro will get a list of email UIDL's (not the headers) and then check its local Fast Backup database to see which of those emails have been downloaded previously. For those emails that have been downloaded before, it will skip them, and for those it hasn't downloaded it will get the email header and proceed as per normal. This can greatly reduce the backup time.

•Not using the archive attribute: When the profile is next run, SyncBack will remember which files and directories it copied to (or deleted from) the destination directory. This means the first run of a profile, after Fast Backup is enabled, will take the same amount of time as without Fast Backup enabled. However, for the second and subsequent runs of the profile it will not need to scan the destination directory because it remembers what it did the last time the profile was run.

•Using the archive attribute: Each file has what is called an archive attribute (just like files have read-only, hidden, etc. attributes). Whenever a file is changed the archive bit is automatically set (by Windows), and when SyncBack copies a file it clears the archive bit. So when SyncBack needs to know which files are to be copied to the destination it just needs to see if the archive attribute is set. There is no need to scan the destination. The main advantage to using the archive attribute is that there is no need to keep information on the state of the files (so less disk space is used). It may also be very slightly faster (as it doesn't need to read and save the information on the files). The disadvantage is that it has no record of if a file is deleted in the source as no database is kept. You cannot use the archival bit fast backup method when using the cloud. This is because SyncBackPro needs information on the files in the cloud.

Using Fast Backup means the scan time is substantially lower (at least twice as fast, often far more) especially if the destination is on a slow device, e.g. networked drive, cloud, FTP server, etc.

Rescan with archival backups

If a rescan is done, either by clicking the Force Re-scan button or by other means, then when the profile is next run it will scan both the source and destination, compare the files, then copy new and changed files. This has a side effect with archival backup in that it will not copy a file (even if it has its archive attribute set) if the source and destination files are the same. If the profile is run again (so it is not a rescan) then it will scan the source, see the file has the archive attribute set, and copy it to the destination regardless.

Can I use Fast Backups on all profile types?

No. The Fast Backup option is only possible when no changes are being made to the destination by other programs, profiles, or users or it is a backup of emails. This means it cannot be used with backups from FTP or Zip files, synchronization profiles, or profiles that use prompting. It is for backup profiles only. Fast Backup can be used with all cloud storage services except Backblaze B2. Backblaze B2 must be used via the S3 compatibility interface.

If you are using archival backups you must keep in mind SyncBack is relying on the archive attribute being set once a file is created or changed and is not reset by anything else other than that profile in SyncBack. Once the archive attribute is set then SyncBack knows the file needs to be backed up. However, some other programs, e.g. other backup software, may also use and reset file archive attributes. You must also be careful not to have more than one profile that copies the same files and resets the archive attribute. You cannot use the archival bit fast backup method when using the cloud. This is because SyncBackPro needs information on the files in the cloud.

Sometimes SyncBack scans the destination directory or email server even though I've enabled Fast Backups. Why?

There are a number of reasons why SyncBack may scan the destination directory:

•The Force Re-scan button has been pressed for that profile.

•The settings on the Fast Backup tab specify a re-scan should be performed under certain conditions.

•The –full command line parameter was used.

•The fast backup data has been deleted.

•The filters or file & folder selections have been modified.

What options does using Fast Backup disable?

When using Fast Backups you cannot enable the following options in your profile:

•Reset the archive file attribute on files once they have been copied.

•Files cannot be moved (to or from the source), and files cannot be copied to the source or deleted from the source (this option can be used with archive attribute fast backups)

•The destination cannot be watched for file changes.

What side effects are there with using Fast Backups?

If the profile is set to delete destination only files, SyncBack may not know a new file has been created in the destination (see this section for more details).

Because a Fast Backup will not scan the destination (except on the first run after it is enabled for that profile) that means only the changes will be applied to the destination without regard to what is actually on the destination. For example, you could change the destination directory, run the profile, and then only the new/modified files would be copied to the destination (and not all the files as would normally be the case).

This has important consequences when your destination directory is dynamic, i.e. it uses environment variables that can change in value. For example:

•Create a normal backup profile and set the source directory to C:\My Documents\ and the destination directory to D:\Backup\%DAYOFWEEK%\

•Enable Fast Backup for the profile.

•On the first run of the profile (let's assume it's Monday and the destination directory is empty) all the files will be copied to D:\Backup\1\

•When the profile is run on Tuesday then only the new or modified files will be copied to D:\Backup\2\

•On Wednesday new and changed files will be copied to D:\Backup\3\ and so on until Monday.

•When it is run again on the following Monday then only the new and changed files will be copied to D:\Backup\1\

•Enable the option "Keep fast backup data based on the actual destination directory". This will create full backups for each day and not just incremental backups for Tuesday to Sunday.

•Enable the option "Delete all the files and folders in the destination before the backup" and force a rescan on Mondays. There are three ways to do this (using this example):

1. The best and easiest option is to configure the "Force a re-scan when:" settings to force a re-scan every Monday (select %DAYOFWEEK% from the list, select equals from the drop-down, and type in 1 (1=Monday, 7=Sunday) in the edit box).

2. You can do this manually by clicking the Force Re-scan button on the Mondays.

3. Use the -full command line option (for Monday only when scheduling).

This will mean that the Monday backup is a complete backup, and the backups on Tuesday to Sunday contain just the new/changed files since the previous days backup.

What about cloud storage and Fast Backups?

Cloud storage services like Amazon S3 and Microsoft Azure have worked with Fast Backup since the feature was introduced. This is because "business/professional" cloud storage services like these can record meta-data with the files, e.g. they can store the last modification date & time, uncompressed size, etc. However, starting with SyncBackPro V10, Fast Backup can also be used with consumer cloud storage services like Dropbox, Google Drive, Box, etc.

When using Fast Backup with some cloud storage services, e.g. Box, SugarSync, WebDAV, ShareFile, pCloud, if you perform a rescan then SyncBackPro may request a full upload. The reason for this is some cloud systems cannot store meta-data so all the meta-data is stored in a database managed by SyncBackPro. When a rescan is performed, the local meta-data is deleted and the cloud storage service is re-scanned to get that information. As it does not have it, SyncBackPro may not be able to determine if the file has changed. If you know which files have not changed, then on the Differences window you could choose to copy the meta-data from the local file to the cloud (database) for those unchanged files. For the other files, they can be copied as normal and the new meta-data will be stored in the database. If you are using Fast Backup with multi-zip compression and Fast Backup, then you will also get the same issue with rescan on SharePoint and OneDrive as although we can store some meta-data, e.g. last modification date & time, we cannot store custom meta-data, e.g. the uncompressed size of a file. Google Drive and Dropbox have no issues because they can store meta-data.

What about cloud cold storage and Fast Backups?

Cloud storage services like Amazon S3 and Microsoft Azure have cold storage options. Basically, this means that objects (files) stored in cold storage are stored offline (e.g. on tape drives) for long-term storage. To change or retrieve objects stored in cold storage, a request must be made to the cloud storage service to copy the object from offline storage (e.g. tape) to online storage (e.g. SSD). Changes can then be made to the object and it can be moved back to cold storage once complete. Often the bucket/container is configured to automatically move objects to cold storage once they've been stored for a certain amount of time, e.g. after 7 days moved an object to cold storage. This can be a problem with Fast Backup because Fast Backup requires that the destination is not changed. Automatically moving an object to cold storage (after upload at a later time) is a change to the destination. For example: if you want to change the date & time of a file in cold storage then it will fail because to change the date & time of a file it must be retrieved from cold storage first, changed and then moved back to cold storage. Keep this in mind if you are using Fast Backup and cold storage.

What about email and Fast Backups?

The main benefit of using Fast Backup and email is that it can greatly reduce the backup time. One side effect of using Fast Backup and email is that if SyncBackPro has already downloaded an email it does not check to see if the email backup file actually exists. For example, you may have a profile that backs up your emails and then runs a 3rd party program that processes or moves those email files. This will not effect the next run of SyncBackPro as it doesn't care if the email file exists or not, only if it has been downloaded before or not. If you force a rescan then of course it will then compare the local email files with the actual emails on the server and act as appropriate.

What about FTP and Fast Backups?

One of the benefits of using Fast Backup and FTP is that it can really improve the backup time. Apart from not having to scan the FTP server to find changes, SyncBack also does not need to set the date & time of the file on the FTP server to match that of its equivalent file on your PC. This can further reduce the backup time.

The Archival Fast Backup option is not available when doing multi-zip backup to an FTP server. When multi-zip files are stored on an FTP server SyncBack must name the Zip files in a special way (to store information like their uncompressed size, for example). Because of this the destination must be scanned to know what those filenames are. The non-archival Fast Backup method can be used (in most cases) because it knows what the destination filename is without scanning (because it is in the fast backup database). However, if the destination folder is dynamic, e.g. it is using a variable, then it can cause problems as it will not always be able to know what the destination filename is.

If you don't care if the date & time of the files on your FTP server match those on your PC then you can untick the "If the FTP server cannot set a files date & time then change the local files date & time to match that on the server" option on the FTP tab.

There are important consequences to doing this: the date & time the file should be set to (on the FTP server) is kept in the Fast Backup data. Therefore, if you force a re-scan (so erasing that data) then the last date & time information is permanently lost. What does this mean?

•When you do a restore all the files will be retrieved from the FTP server along with their last modification date & time, which is not the original value. As the date & times won't match, all the files will be restored unless you've configured your profile to ignore file date & times.

•When you next run a backup all the date & times will be mismatched so forcing a complete backup (unless you've configured your profile to ignore file date & times).

What about backup to a single Zip file on an FTP or cloud server?

Making a backup to a single Zip file on a remote FTP server creates some interesting challenges:

1.To update a remote Zip file it would need to be downloaded, updated, and then uploaded. Depending on the size of the Zip file, this could be extremely slow.

2.To know what files and folders are in a remote Zip file it would need to be entirely downloaded.

Because of this SyncBack will always replace the existing Zip file and therefore assume there are no files in the remote Zip file. The solution is to use a Fast Backup profile, i.e. an incremental or differential backup. For example, to keep 7 days worth of backups, and have a full-backup on Mondays and incremental backups on all the other days of the week:

•Create a backup profile and configure your source as appropriate, and set your destination as appropriate (i.e. a single Zip file on an FTP server)

•Set the Destination to something (the %DAYOFWEEK% is required) like \My Backups\%DAYOFWEEK%.zip

•Go to the Fast Backup tab and enable the options: Perform a fast backup and set the full-backup folder to \My Backups\1.zip

•In the "Force a re-scan when:" box select the %DAYOFWEEK% item, select Equals from the drop-down list to the right of the box, and enter 1 into the box to the right of the drop-down list (we use 1 because Monday is day 1, Tuesday is day 2, etc).

•You should schedule the profile to run just once every day

Note about "Delete all the files and folders in the destination before the backup"

This option should be used with care because it will delete all the files and folders in the destination before running the profile. However, if you are compressing to a single Zip file it will just delete that Zip file and no other folders or files.

An important detail to remember about this setting is that the destination files are not deleted if the profile run is doing a rescan. For example, if you've configured your profile to rescan on Mondays (%DAYOFWEEK% equals 1) then when the profile is run on a Monday it will not delete the destination files. However, it will delete them if there is no rescan. This has consequences if you run your profile more than once and also if you do not ever do a rescan. For example, if you configured your profile to rescan on Mondays then when the profile is run on a Tuesday it will delete the destination files and copy over the new or changed files since the last profile run. However, if you run it again immediately (and assuming it is still Tuesday) it will delete those files and then copy over any new or changed files since the last run, which may be no files at all.

Versioning

As an alternative to doing incremental or differential backups, you may want to consider using Versioning instead. With versioning you can keep a defined number of versions of a file. This means you can keep old versions of files that have been changed or deleted. Note that you can use versioning with Fast Backup, but it can become complex and will slow down a Fast Backup (as SyncBack must scan the destination to know what versions are available).

Equals, Not Equals, Less Than, etc.

If you are using strings (not integer numbers) for the re-scan comparison, then the following rules apply:

•Equals (=), Not equals (!=, <>): a case insensitive string comparison is made, e.g. ABC is the same as abc

•Less than or equal (<=), Greater than or equal (>=): is the equivalent of Equals. For example, if it was %DAYOFWEEKNAME% >= Fri, and it was Friday, then it would be a rescan. So it is equivalent to %DAYOFWEEKNAME% = Fri

•Less than (<), Greater than (>): is the equivalent of Not equals. For example, if it was %DAYOFWEEKNAME% > Fri, and it was Friday, then it would not be a rescan. So it is equivalent to %DAYOFWEEKNAME% <> Fri

Example Fast Backup Configurations

The following section provides example backup configurations. You may also want to look at the special automatically incrementing variable as it can be used to define exactly how many backups you want to keep.

To keep 7 days worth of backups, and have a full-backup on Mondays and incremental backups on all the other days of the week:

•Create a backup profile and configure your source as appropriate

•Set the Destination to something (the %DAYOFWEEK% is required) like D:\My Backups\%DAYOFWEEK%\

•Go to the Fast Backup tab and enable the options: Perform a fast backup and Delete all the files and folders in the destination before the backup, and set the full-backup folder to D:\My Backups\1\

•You should schedule the profile to run just once every day

To keep 7 days worth of backups, and have a full-backup on Mondays and differential backups on all the other days of the week:

•Create a backup profile and configure your source as appropriate

•Set the Destination to something (the %DAYOFWEEK% is required) like D:\My Backups\%DAYOFWEEK%\

•Go to the Fast Backup page and enable the options: Perform a fast backup, Differential backup (do not update the fast backup data), Delete all the files and folders in the destination before the backup, and set the full-backup folder to D:\My Backups\1\

•You should schedule the profile to run just once every day

To keep 7 days worth of backups and have full-backups for each day:

•Create a backup profile and configure your source as appropriate

•Set the Destination to something (the %DAYOFWEEK% is required) like D:\My Backups\%DAYOFWEEK%\

•Go to the Fast Backup tab and enable the options: Perform a fast backup and Keep fast backup data based on the actual destination directory

•Go to the Decisions - Files page and configure the profile to delete files from the destination that do not exist on the source. This ensures that your backups don't get cluttered with redundant files.

•You should schedule the profile to run just once every day

To keep 4 backups and have full-backups for each day:

•Create a backup profile and configure your source as appropriate

•We are going to use the special %AUTOINC% variable, which is an automatically incrementing variable. Go to the Variables - Incremental settings page, enable the auto-incrementing variable, set the current and minimum value to 1 and set the maximum value to 4.

•Set the Destination to something like D:\My Backups\%AUTOINC%\

•Go to the Fast Backup tab and enable the options: Perform a fast backup and Keep fast backup data based on the actual destination directory

•You can schedule the profile to run however often you wish as for each run the destination directory changes (based on the auto-increment variable).

To keep full-backups on a set of disks:

•Create a backup profile and configure your Source as appropriate

•Set the Destination, e.g. X:\%DISKSERIAL%, where X: is a drive that accepts removable media, e.g. SD card, USB drive, etc.

•Go to the Fast Backup tab and enable the options: Perform a fast backup and Keep fast backup data based on the actual destination directory

•Each time you run the profile use a different disk. You could keep 10 days worth of backups by rotating a set of 10 disks. Each disk will have a complete backup.