SyncBack has the ability to compress files using the industry standard Zip format as well as the newer LZMA and LZMA2 (7-zip) format. Compression reduces the size of the file and has the potential to save a lot of disk space, especially when files such as text and office documents are being copied. Two methods of compression are supported: all the files can be placed into one single compressed file (single Zip), or each file can be placed into it's own individual compressed file (multi-Zip). The first option uses the least amount of disk space, but has the disadvantage that "all the eggs are placed in one basket", so to speak. Also, versioning cannot be used if all files are placed into a single compressed file, and if you store the single Zip in a remote location (e.g. FTP, cloud, etc.) then the Zip file cannot be updated and must be recreated each time.
To increase compression performance SyncBackPro can be configured not to try and compress already compressed files, e.g. MP3's, JPG images, etc. Instead of compressing files of these types it will instead store them (without compression) in the Zip file. Note that they are still stored in a Zip file but are not compressed within the Zip file.
For maximum security, you can also encrypt and compress the filenames and file details within the Zip file.
|
SyncBackPro is Unicode enabled, so it can store files and folders with names in any language, e.g. Chinese, in a Zip file. However, some compression programs are not Unicode enabled. Because of this if you open a Zip file produced by SyncBackPro using a non-Unicode enabled compression program then it will show the filenames incorrectly (probably with question marks). The problem is with the compression program, not the Zip file produced by SyncBackPro. The solution is to use a compression utility that is Unicode enabled, such as 7Zip, for example. Another possibility is that the compression utility uses UTF-8 Unicode encoding, e.g. WinZip 12. If so, you should configure the profile to use UTF-8 encoding in the Zip file.
Also note that if you open a split Zip file created by SyncBackPro in a compression program, e.g. WinZip, then it may give an error saying the file is corrupted. The problem is that it is expecting the split filenames to be named differently. See this section for details. |
How Compression Is Used
When you are using the cloud, FTP, SyncBack Touch and Media Transfer Protocol (MTP) then the compression applies to that location. This means that any files copied to that location, e.g. FTP, will be compressed to a temporary file and that compressed file will be uploaded to the location. When a file is copied from such a location then the compressed file is downloaded and uncompressed.
In some cases, e.g. with FTP, the compressed file stored in the location will have a special filename. This special filename contains details about the file stored in the compressed file. By doing this SyncBackPro can scan the location and get the details of the file within the compressed file without having to download it and extract it, which would be extremely slow. In most cases, e.g. when using Amazon S3, it doesn't need to do this as those kinds of locations allow SyncBackPro to store meta-data along with the file and this meta-data will contain details of the file within the compressed file.
In the special case that you have compressed files on FTP etc. and want to download them uncompressed then you must run the profile as a restore. So you must create a backup profile where you are backing up files to the location using compression and then run that profile as a restore. Keep in mind that if the compressed files are being created by something other than SyncBackPro then they will not have the special filename that contains the details. This means SyncBackPro will not know the size nor the last modification date & time of the file inside the compressed file.
If you want to download files from FTP etc. and have them stored locally compressed, then this is not possible with a single profile. Instead you would need to create two profiles: the first profile would download the files to a temporary location and the second profile would then compress those files. Those two profiles would then be put into a group and you would run the group.
Which Compression Method to use
If you are looking for maximum compatibility with 3rd party compression utilities, choose Deflate or Deflate64.
If you require maximum performance use Zstandard (zstd) with a lower compression level. Zstandard can only be used with multi-zip compression. If you are compressing all to a single Zip then Deflate64 has the best performance.
For maximum compression (but the slowest performance) choose XZ, LZMA or LZMA2. All three compress to similar levels. If you choose a higher compression level then avoid LZMA as it uses a lot of memory at higher compression levels. XZ can only be used with multi-zip compression. If you are compressing all to a single Zip then LZMA2 compresses the best.
Performance
When compressing files there is a trade-off. Less storage (by compressing files) means slower backups (and restores). Compression is a CPU (and memory) intensive process. You cannot compress files and expect similar performance to simply copying files as-is without compression.
To get the best compression performance:
•Use multi-zip compression so each file is stored in its own Zip file, i.e. untick the option "Compress the files on destination/right into a Zip file".
•Use multi-threaded (parallel) compression (see the setting "Number of files to compress/decompress in parallel"). Do not use too high a number otherwise, i.e. do not use a number higher than the number of threads your CPU supports.
•Use the Zstandard compression method.
•Do not use a compression level above 5 - Normal compression. The lower the compression level, the faster it will be.
•Enable the option "Auto-detect files that may not compress".
Compression Settings
•Compress the files on destination/right into a Zip file: Enable this option to compress files copied to the destination/right into a single Zip file. If you are backing up files to an FTP server or the cloud, please read the Fast Backup section for tips on getting the best results.
•Put all the files into a single compressed file (by default each file will be placed in its own compressed file): If this option is enabled, then the files will be put into a single compressed Zip file. If this option is not enabled (the default), then each file will be placed into it's own Zip file.
|
When each file goes into its own Zip file, and those Zip files are being stored on an FTP server, we have the problem of knowing what is in a Zip file on a remote FTP server. To know this SyncBackPro would need to download the Zip file and open it to see what file is inside it, what it's uncompressed size is, and what it's last modification date & time is. To avoid this SyncBackPro changes the filename of Zip files stored on an FTP server by embedding this information in the filename itself.
However, if the filename does not contain this information, e.g. it was created on the FTP server by some other utility, then SyncBackPro will not know the files uncompressed size or its last modification date & time. This means (depending on your profile configuration) it will always assume the file has been changed since the last profile run. |
•Type of compression: There are several types of compression: Deflated (which is the default), Deflated64, Burrows Wheeler, BZip2, LZMA, LZMA2, XZ and Zstandard.
Deflated provides the normal type of compression used by the older Zip format (traditional PKZIP 2.04g compression method). This is the most compatible compression method, i.e. nearly all 3rd party compression utilities understand this compression method.
Deflated64 (also know as Enhanced Deflate) provides a greater level of compression than Deflate, but note that it will increase the compression time and is not compatible with as many 3rd party compression utilities as Deflate. Deflate64™ is a trademark of PKWARE Inc.
Burrows Wheeler (popularized by the UNIX and Linux BZip2 program) offers significantly better compression than Deflate but takes longer to compress and decompress data. Tests have shown BWT (Burrows Wheeler Transform) to often achieve between 20% to 30% better compression than Deflate on many popular file types such as databases, pictures, text and executable files. BWT is considered to be one of the most efficient compression algorithms for compressing XML data. In comparison to BWT, Deflated64 is slightly faster but does not compress as well. The compressed file format is Zip.
|
Note that the Burrows Wheeler compression method is not supported by any other compression program. Only SyncBackSE and SyncBackPro can be used to restore Burrows Wheeler compressed files. |
BZip2 is similar to the Burrows Wheeler compression method except that it is compatible with some compression programs, e.g. WinZip 11. Note that in some cases, e.g. with highly random data, the compression speed can be very slow as compared to the other compression methods (see the Compressed settings page to choose which files not to compress). However, it generally provides the best compression level. The compressed file format is Zip.
LZMA (Lempel-Ziv-Markov chain-Algorithm) uses an improved and optimized version of the Lempel-Ziv (LZ77) compression algorithm, backed by a Markov chain range encoder. It uses a variable dictionary size. It is compatible with some compression programs, e.g. WinZip 12. LZMA typically provides much better compression than the Deflate and Deflate64 algorithms at the expense of speed and memory usage when compressing. It also typically provides compression ratios a little better than BZip2/BWT while being a little faster. The compressed file format is Zip.
|
LZMA maximum level compression uses a huge amount of memory. If you run two or more profiles in parallel, that use LZMA maximum level compression, then you will probably get the following error: Compression error: There is not enough free memory to process the file. It is strongly recommended that you use the 64-bit version of SyncBackSE or SyncBackPro, and have enough memory, if you need to use maximum level LZMA compression. |
LZMA2: This is the improved version of LZMA and is compatible with 7-Zip V9 and newer. The compressed file format is 7z.
XZ: XZ is an alternative compression engine that uses LZMA2 compression but stores the compressed file in an XZ container within a Zip file, i.e. it does not use the XZ file format but the Zip file format. It is compatible with 7-Zip V9.20, WinRAR 6.24, and WinZip 20.5 and newer.
Zstandard: Also called zstd, it is the fastest compression method. Although it does not compress as well as XZ, LZMA and LZMA2, it is quicker to compress and decompress. Compatible with WinRAR 6.10 and newer. The compressed file format is Zip.
•Level of compression: There are ten levels of compression ranging from level 0 (no compression, files are stored in a compressed file but are not actually compressed) to level 9 (highest compression). The more a file is compressed, the slower it takes to compress the file and it will also use more memory. This option allows you to make a trade-off between speed and file size. The Type of compression setting also influences the compression speed and file size. Note that level 9 LZMA compression (not LZMA2) uses a huge amount of memory and will very likely cause memory failures if two or more profiles are run at the same time using level 9 LZMA compression.
|
The dates and times stored in a Zip file are stored literally without timezone information. This can cause issues when a Zip file is used in different timezones. |
•Encryption method: If you wish to password protect the contents of the files in the Zip file then choose the encryption method to use. AES encryption is more secure but not as portable. For example, "Old style" encrypted Zip files can be decrypted and extracted using practically any 3rd party Zip program. However, AES encrypted Zip files can only be decrypted and extracted using newer 3rd party Zip programs, e.g. WinZip 9.
•Password: If you want to password protect the files in the compressed archive then enter a password here. It's important to note that if you change the password then the existing files in the destination will still use the old password. Each file in a compressed archive has its own password. The maximum password length is 79 characters. You can use a secret for the password.
|
Note that you are entirely responsible for remembering the password used. It is not possible under any circumstances for 2BrightSparks to recover lost passwords. |
•Prompt for the password when run (profile will fail if run unattended): If this option is enabled then every time the profile is run SyncBack will prompt you for the compression password. If the profile is being run unattended, then no prompt will be displayed and the profile run will fail.
•Encrypt and compress the filenames and details: SyncBack can optionally encrypt and compress the filenames and file details in the Zip file. By doing this, anyone who opens the Zip file (to see the file list) will be unable to see anything without knowing the password. This encryption method is compatible with PKWare SecureZip (when Deflate or BZip2 compression is used) or 7-Zip (when LZMA2 compression is used). The PKWare SecureZip encryption is not compatible with the same feature in WinZip and 7-Zip. However, when using LZMA2, it is compatible with 7-Zip. To use this option some other options must be set first:
oAll files must be put into a single Zip file
oThe Deflated, BZip2 or LZMA2 compression method must be used
oThe encryption method must be AES
oSplit Zip files cannot be used
oSelf-extracting Zip files cannot be used
oThe compression level must be greater than zero (i.e. it must be compressed)
All Content: 2BrightSparks Pte Ltd © 2003-2024