(1 votes, average: 1.00 out of 5)
Loading...
I am always exploring the latest Storage landscape which keeps on changing. Below is a brief history of the storage market and what to expect in the future.
What has changed in the storage market?
The Enterprise Storage market used to be solely controlled by a few big name vendors, primarily using SAN attached storage. The array structure was mostly build with expensive proprietary components, causing customer TCO to be very high.
As a results of using a FC SAN, an expensive FC switch or directors are needed, pushing the end customer TCO to even higher prices.
To get ultra high performance many disk spindles (or extra array cache) were needed i.e. spindles were added just for performance not for disk space. for example: for high IOp’s many spindles are used, as well as some vendors used the outer disk platters to lower latency.
The first major change was the release of Enterprise grade SSD. A brief history of SSD technology When SSD were initially introduced there was SLC, super expensive, but cell write cycles ware much less of an issue. Along came MLC, much cheaper, but less reliable, with many times less write cycles per cell.
To note: initial SSD’s had major write cliff issues, some users got bitten by mystery slowness or so called GC issues. With the updated versions, release of eMLC and for some use-cases TLC, most issues got ironed out by the manufactures price coming down to match (or almost match) regular HDD’s.
As discussed above the existing storage market comprised mostly from big monolithic iron, very cumbersome to manage (you needed a degree to manage the system), and yes (mostly) extremely stable. With that in mind the new comers like Violin Memory (now gone sadly), pure, kaminario, tegile, nimble (now part of HPE), tintri, nimbos, xtremio, solidfire (now owned by netapp), TMS (now owned by IBM) etc.. (mostly VCs) took a very simplistic management approach.
Most of these startups provided much faster r/w access and IOp’s were in the 100k/s some even hitting a million. all of these startups used a variety of the SSD types mentioned above (SLC/MLC/eMLC,TLC).
Depending in the vendor – some vendors didn’t even have thin-provisioning, some just missing cloning and or replication. talking about services, its worth to mention, for example a service like QoS (noisy neighbor) most vendors claim it to be a non issue with such high IOp’s.
Now, with all these missing services and price still being expensive, adoption was slow. but once adoption started, companies using an SSD array very quickly noticed the benefits, and as comfort levels rise, more and more data got moved to these SSD arrays.
Some of the factors to move to SSD’s is lower latency. for example, using an array with 1000 HDD spindles to get the performance you need, was replaced by a small 1-3U SSD array.
One of the things to note. at first when SSD arrays were new, they were quite expensive, so many startups came out with the idea of using compression and dedup, bringing the price more in-line (for example pure’s, xtremio’s and other business models).
To note NVMe is not using SAS nor SATA its using using your PCIe bus, which is a very simple, powerful protocol with additional benefits like the queue length, the queue length of an NVMe is 65k – significantly larger than traditional uses.
The NVMeoF was mostly completed in 2016, enhancements will eventually make it to the standard. but even where it stands today, it is a very promising technology and eventually will change enterprise storage.
There are a number of vendors working on NVMeoF solution, just to name a few, e8storage, Apeiron, EMC DSSD, and Zsto (using Mangstor technology), some implementations are proprietary. NVMeoF can run on 10Gb/s Ethernet, but is preferred to run on 40Gb/s+ Ethernet or IB 100Gb/s. To summarize, the next wave of storage is definitely ahead of us, start using NVMe and NVMeoF and enjoy the technology, NVMeoF its still cutting edge technology, but will quickly come around, at least be prepared.
Quobyte https://www.quobyte.com/ Quobyte is software defined storage that turns commodity servers into a reliable and highly automated data center file system, You can use, run Quobyte on any thing including containers, i.e. Docker, Rkt, Kubernetes, etc.. E8 storage https://e8storage.com Provides very fast disk access based on the back-end build with SSD or NVMe, the array can be accessed using 40Gb or 100Gb Ethernet or even IB (in the future maybe NVMeoF. Apeirondata http://apeirondata.com/ Provides a full NVMeoF uniq solution Excelero http://www.excelero.com/ Provides a full NVMeF using Ethernet or IB RDMA Mangstor and Zstor https://mangstor.com/ and http://www.zstor.de/en/ Anther NVMeoF provider, to note zstor is using mangstor parts in there solution. paviliondata http://www.paviliondata.com/ Provides an SSD NVMe array soultion Liqid https://www.liqid.com/solution.html Provides a 3rU Flash Storage Array. up to 64 GB/s and 40 M IOPS, massive capacity up to 256 TB, and innovative PCIe . Elastifile http://www.elastifile.com/ Provides an SDS solution, Elastifile Cloud File System scale-out software-defined solution, with a unifying data within the single global namespace of a global file/object system. Hedvig Inc http://www.hedviginc.com/ Hedvig Software-Defined Storage (SDS) for Private and Hybrid Clouds, Hedvig software deploys on commodity server hardware. Drivescale https://drivescale.com/ Provides a scale out architecture used for Big Data clusters., Overcome Issues with Traditional Scale-Out Deployments Datera http://datera.io High Performance and Low Latency using NVMe or SAS/SATA, native integration with container schedulers, unique QoS controls.
First lets start with brief history of the storage market.
What has changed in the storage market?
The Enterprise Storage market used to be solely controlled by a few big name vendors, primarily using SAN attached storage. The array structure was mostly build with expensive proprietary components, causing customer TCO to be very high.
As a results of using a FC SAN, an expensive FC switch or directors are needed, pushing the end customer TCO to even higher prices.
To get ultra high performance many disk spindles (or extra array cache) were needed i.e. spindles were added just for performance not for disk space. for example: for high IOp’s many spindles are used, as well as some vendors used the outer disk platters to lower latency.
Now lets see what changed? and what is changing?
In the recent years with the use of SSD, things have drastically changed, some of the changes significantly lowered TCO.The first major change was the release of Enterprise grade SSD. A brief history of SSD technology When SSD were initially introduced there was SLC, super expensive, but cell write cycles ware much less of an issue. Along came MLC, much cheaper, but less reliable, with many times less write cycles per cell.
To note: initial SSD’s had major write cliff issues, some users got bitten by mystery slowness or so called GC issues. With the updated versions, release of eMLC and for some use-cases TLC, most issues got ironed out by the manufactures price coming down to match (or almost match) regular HDD’s.
A few other industry changes to note.
- Intel CPU’s speed superseding proprietary FPGAs
- SAS and SAS suspenders going to 6Gb/s+ now at 12Gb/s
- Availability of PCI gen3 in commodity servers
- Availability of DDR4 / faster and larger memory banks
Enterprise Storage shake-up by new SSD arrays
At the same time the above changes happened, a few Storage start-ups (VCs) came along, causing the first storage market shake-up.As discussed above the existing storage market comprised mostly from big monolithic iron, very cumbersome to manage (you needed a degree to manage the system), and yes (mostly) extremely stable. With that in mind the new comers like Violin Memory (now gone sadly), pure, kaminario, tegile, nimble (now part of HPE), tintri, nimbos, xtremio, solidfire (now owned by netapp), TMS (now owned by IBM) etc.. (mostly VCs) took a very simplistic management approach.
Startups – SSD architecture benefits
Architecturally they had in common, using commodity servers (with the exception of Violin Memory – part of the issues Violin had).Most of these startups provided much faster r/w access and IOp’s were in the 100k/s some even hitting a million. all of these startups used a variety of the SSD types mentioned above (SLC/MLC/eMLC,TLC).
Other benefits of using a SSD array
As mentioned above, one of the main benefits (from most of the vendors) were simplicity, for example, bringing it up in production in less than an hour. but from the other side they were missing many SAN services.Depending in the vendor – some vendors didn’t even have thin-provisioning, some just missing cloning and or replication. talking about services, its worth to mention, for example a service like QoS (noisy neighbor) most vendors claim it to be a non issue with such high IOp’s.
Now, with all these missing services and price still being expensive, adoption was slow. but once adoption started, companies using an SSD array very quickly noticed the benefits, and as comfort levels rise, more and more data got moved to these SSD arrays.
Some of the factors to move to SSD’s is lower latency. for example, using an array with 1000 HDD spindles to get the performance you need, was replaced by a small 1-3U SSD array.
One of the things to note. at first when SSD arrays were new, they were quite expensive, so many startups came out with the idea of using compression and dedup, bringing the price more in-line (for example pure’s, xtremio’s and other business models).
SSD adoption now going extremely well
It took a nice few years to realize the effect of all these new SSD vendors, as storage is one of these things needing a track record. but once picked-up, adoption started happening extremely fast. Once adoption started being strong, all proprietary vendors started panicking, causing them to either modify their own arrays with SSD’s or buying one of the VC startups.The next Storage wave – NVMe and NVMeoF
Now that SSD prices has come down and are standardized, we are up to the next Storage wave using NVMe or NVMeoF technology.First what is NVMe?
NVM Express, is a specification that allows a solid-state drive (SSD) to make effective use of a high-speed Peripheral Component Interconnect Express (PCIe) bus in a computer.To note NVMe is not using SAS nor SATA its using using your PCIe bus, which is a very simple, powerful protocol with additional benefits like the queue length, the queue length of an NVMe is 65k – significantly larger than traditional uses.
Finally NVMeoF to the rescue
Now we are up to the next wave (even its very new), but will pickup very quickly. After NVMe was released, the NVMe committee realized how great it is, and decided to enhance it with NVMeoF standard. One major benefit NVMeoF brings to the table is, the benefit of sharing and using any server as an initiator and or as a target (drivers now available on Linux – Mellanox reference below), which will arguably be called a true SDS solution. Mellanox NVMeoF – HowTo Configure NVMe over FabricsThe NVMeoF was mostly completed in 2016, enhancements will eventually make it to the standard. but even where it stands today, it is a very promising technology and eventually will change enterprise storage.
There are a number of vendors working on NVMeoF solution, just to name a few, e8storage, Apeiron, EMC DSSD, and Zsto (using Mangstor technology), some implementations are proprietary. NVMeoF can run on 10Gb/s Ethernet, but is preferred to run on 40Gb/s+ Ethernet or IB 100Gb/s. To summarize, the next wave of storage is definitely ahead of us, start using NVMe and NVMeoF and enjoy the technology, NVMeoF its still cutting edge technology, but will quickly come around, at least be prepared.
SSD/NVMe and NVMeoF Vendor list
Below is a list of storage and object store (cloud storage) vendor newcomers.Quobyte https://www.quobyte.com/ Quobyte is software defined storage that turns commodity servers into a reliable and highly automated data center file system, You can use, run Quobyte on any thing including containers, i.e. Docker, Rkt, Kubernetes, etc.. E8 storage https://e8storage.com Provides very fast disk access based on the back-end build with SSD or NVMe, the array can be accessed using 40Gb or 100Gb Ethernet or even IB (in the future maybe NVMeoF. Apeirondata http://apeirondata.com/ Provides a full NVMeoF uniq solution Excelero http://www.excelero.com/ Provides a full NVMeF using Ethernet or IB RDMA Mangstor and Zstor https://mangstor.com/ and http://www.zstor.de/en/ Anther NVMeoF provider, to note zstor is using mangstor parts in there solution. paviliondata http://www.paviliondata.com/ Provides an SSD NVMe array soultion Liqid https://www.liqid.com/solution.html Provides a 3rU Flash Storage Array. up to 64 GB/s and 40 M IOPS, massive capacity up to 256 TB, and innovative PCIe . Elastifile http://www.elastifile.com/ Provides an SDS solution, Elastifile Cloud File System scale-out software-defined solution, with a unifying data within the single global namespace of a global file/object system. Hedvig Inc http://www.hedviginc.com/ Hedvig Software-Defined Storage (SDS) for Private and Hybrid Clouds, Hedvig software deploys on commodity server hardware. Drivescale https://drivescale.com/ Provides a scale out architecture used for Big Data clusters., Overcome Issues with Traditional Scale-Out Deployments Datera http://datera.io High Performance and Low Latency using NVMe or SAS/SATA, native integration with container schedulers, unique QoS controls.