Back-up technologies don’t seem to progress much from generation to generation. In the ’80s, we transitioned from Digital Audio Tape (DAT) to Linear Tape-Open (LTO). And in the noughties, we moved to disk-based appliances using deduplication. But since then, we have simply piggybacked on the technological advances around us.
Back-up technologies need to integrate with the latest versions of operating systems, storage platforms and applications. Still, fundamentally, the game is the same: we copy point-in-time data from production systems to a secondary location.
What is deduplication?
Before unpacking the “amplified” part of this blog, we first need to understand what deduplication is and how back-up technologies employ it.
In its simplest form, deduplication is the process of eliminating duplicate or redundant data. Deduplication allows a back-up system to store one copy of unique data blocks when you back up systems that share a similar operating system with common data. Since we perform multiple point-in-time backups, we gain an additional deduplication benefit as we keep up to 30 copies of a single system which often have only small variances. We store one master version and only the small file or block changes are kept in addition to this.
There are also two types of deduplication which are typically employed. Software- and hardware-based deduplication systems do the same thing, but the magic happens at different points in the process.
In the past, the only way to deduplicate backups was through purpose-built back-up appliances such as EMC Data Domain or Quantum DXi. These days, most of the back-up software vendors have made the same technology available in software. So instead of buying expensive back-up appliances, you can use commodity storage with software magic to optimise your back-up storage.
We generally see clients with deduplication rates of 10:1 and higher. Deduplication software allows you to store 100TB as only 10TB. This is a massive saving, and when you consider the benefits of data that’s available for restore at a moment’s notice, it makes a strong case for backup to disk.
What is amplified deduplication?
This year ushers in next-gen deduplication strategies. At STS, we have the benefit of engaging with leading global vendors in the back-up and storage space and are not locked into a single technology stack. We can take the best of multiple vendors to provide a unique value proposition for our customers.
Amplified deduplication relies on software-based deduplication from your back-up software to store data on an all flash storage system which then applies a second stage of data reduction. We are essentially compounding the data reduction using two separate deduplication and data reduction engines.
With this solution, we can amplify the deduplication that’s achieved by the back-up software. If we use our example of 10:1 (100TB stored as 10TB) and apply a further 2:1 data reduction on the storage level, we can achieve an incredible overall deduplication level of 20:1 (100TB stored as 5TB).
You can’t do that!?
For those in the know, you will be aware that companies have long since told us we must choose between software and hardware deduplication. This is because the process of comparing data and storing it in a deduplicated format requires processing power and storage input and output.
Deduplication is slower than merely streaming all the data to disk because the system has to apply an algorithm to figure out which blocks to store and which to create pointers for.
The whole platform will grind to a halt if you add a second process or algorithm. Backups and restores would not be possible using this method on traditional disk-based back-up appliances.
Where the magic happens
Backup to flash is the magical ingredient that enables us to amplify standard deduplication. STS – in partnership with a leading storage provider that has released NVMe (non-volatile memory express) storage solutions at much higher capacities and lower costs – now offers a solution with enough performance to amplify deduplication.
While enabling an effective costing model, we also gain the inherent benefits of NVMe-based storage solutions. More throughput and lower latencies mean that you cannot only back up to these systems faster, but restores will also happen much quicker. New technologies like instant recovery function at an incredible pace and enable you to use your back-up solution as a disaster recovery platform.
Have your cake and eat it too
Get in touch with an STS deduplication expert today to reduce your back-up solution’s total cost of ownership while providing your business with faster and more reliable data recovery.