Nutanix has recently introduced in the latest tech preview releases, something called Erasure coding (EC-X).
Erasure coding (EC) is a method of data protection in which data is broken into fragments, expanded and encoded with redundant data pieces, and stored across a set of different locations, such as disks, storage nodes or geographic locations.Nutanix
Apparently it’s not a new idea, having been thought about nearly 50 years. But Nutanix have taken the idea, created their own algorithm, that is currently patent pending, and been able to increase available space in their NDSF (Nutanix Distributed Storage Fabric) previously known as NDFS (Nutanix Distributed File System).
The entire process of coding this up has taken them 2 years and once it goes GA, a lot of people using Nutanix will gain some space savings.
So what does Erasure coding provide?
It improves their storage usage when running RF2 (redundancy factor 2) and erasure coding only works on data held in the cold tier (HDD) and works best with data that is only being read. By only running the erasure code on the cold data, this ensures that no impact occurs on the system as it is done as a post process. EC-X is a bit like RAID10.
RF2 is how Nutanix protects data within their cluster, rather than using RAID, they ensure that every block of data is stored on another node within the cluster, and can tolerate a node failure without downtime or loss of data, (RF3 allows two multiple node failures within a cluster) this does however mean that storage is impacted and on the simplest calculation, if you have 10TB of storage, after protection means you only 5TB (RF2) and 3.33TB (RF3).
The implementation can work with just 3 nodes, but Nutanix recommends a minimum of 4 nodes in a cluster.
With the correct nodes in place, you can gain as much as 25% of additional storage using this feature on any RF2 enabled systems.
What are the limitations?
It only works on cold data and ideally data should be stuff that is written once and read many times, great for archive data. If, at any time data is written to any EC-X blocks then the process has to start again.
When will it be GA?
If you are lucky enough to be part of their tech preview program, you can see use it right now. If not, it’s planned to be out in version 4.5, scheduled for the end of the year.