Having the right data in the right place continues to be a critical mandate for IT professionals responsible for instrumenting and maintaining application ecosystems. While it’s become easier to have distributed frontends spanning on premises, public and hybrid ecosystems, finding efficient and cost-effective ways to ensure scalability and data locality has only become more challenging. This combined with the fact that generally practiced application architecture is undergoing transformation means that object storage solutions being evaluated by organizations must also support modern protocols and methods for access. These factors have been steadily driving the adoption of object storage as a part of modern enterprise data strategies.
Depending on who you ask, we’re headed towards somewhere between ~30-40 billion connected mobile devices by end of 2023 leveraged by 3.8 billion users resulting in 1 petabyte of data being managed by each IT professional. By the end of this year alone, as a world population we’re likely to be on a 44 zettabyte per year run rate – 40x more bytes than the stars in the observable universal! In order to manage, scale and extract business value from this phenomenal data explosion, the use of metadata has become increasingly important.
By extracting analytics and business insights, organizations are able to monetize their data, out-maneuver the competition and find ways to become more efficient. Unlike other storage mediums, object stores enable very large amounts of metadata to be stored with any individual object allowing for more detailed, granular index and search as well as higher context analytics.
Despite all of the benefits that object storage provides, it’s also often significantly more cost effective than other options. Being the same technology that enables public cloud and service providers, it’s gotten the benefit of having to learn to be efficient for hyperscale ecosystems without performance degradation. Various techniques such as erasure coding, versioning, customization, embedded analytics along with the delegation of low-level block management to the local file systems, enabling the object store to only have to worry about high-level details. This further enables file lookup tables to be kept to a minimal size and provide scalability to many hundreds of petabytes while maintaining high performance levels. These efficiencies and cost factors make object storage an ideal candidate for a variety of secondary and long-term storage use cases
The current trends we’re seeing in IT of flattening organizations and the adoption of modern principles that bring development and operations practices closer together are all centered on increasing agility, speed and efficiency. A value proposition that object storage provides for this operating environment is the fact that it can be provisioned and scaled quickly, often by the application owner once the ecosystem is put in place. Block storage volumes can only be accessed when they’re attached to an OS while object stores include data and metadata that can be access directly through APIs and over common web protocols.
This means that application developers have direct access to data need to help them speed delivery of services while reducing overhead on the rest of the infrastructure and storage team when key changes are needed. What’s more, since object storage were greatly popularized in public cloud applications like Amazon S3, Rackspace Cloud Files, Azure and Google Cloud Storage, the capabilities baked into today’s solutions also enable customers and service providers to get closer to treating their own on-premises ecosystem like a cloud environment.
Dell has over 16 years of experience in helping customers transition to cloud operating models. This heritage has been leveraged and applied to the ECS solution which provides hyper-scalable, next gen object storage for large data set use cases. One primary advantage of the EMC approach is that they’ve put a significant level of engineering into the automation of best practices, which ensures optimal deployment configuration and keeps administrators out of trouble. By providing a multi-use platform, ECS also helps drive cost savings and efficiency buy consolidating backup, secondary, storage, DRaaS backbone and a migration target for legacy NFS stores. Capabilities for interfacing with cloud native apps over Swift serving as an analytics repository for Splunk, Hawq, Spark and Hadoop also enable interoperability with existing environments when ECS is introduced into brownfield deployments.
As with any part of an application ecosystem, the components are only useful if their available, resilient and can scale as needs of the organization change and object storge is no different.
As a strategic Dell partner, Kemp developed the ECS Connection Manager to provide object storage high availability, resilience and traffic flow optimization while continuously maintaining enterprise-wide security posture and departmental segmentation of data. The Connection Manager serves as a termination end point for all client application traffic requests and storge responses. This enables a tight coupling to the storage engines, along with performance visibility that drives intelligent decisions about local and cross-cluster traffic distribution. Multi-site single namespace is also enabled which significantly simplifies administration. Additional features such as IPv4/6 translation, SSL optimization, QoS and traffic steering driven by the unique characteristics of individual requests, makes ECS Connection an optimal component to be included in every object storage deployment.
As a solution, instrumented specifically with ECS in mind, the ECS Connection Manager provides a number of capabilities that optimize and simplify any object storage deployment:
Object storage should certainly be considered as a part of any modern data storage strategy. Dell’s solution in ECS is one of the premier options with capabilities that can be used to address a variety of use cases. Combined with Kemp’s ECS Connection Manager, the efficiency, scale and resilience of any object storage deployment can further be greatly improved.
You can learn more about the solutions discussed in this post at the following links: