S3 Storage Optimization for ECS
Advanced S3 storage services with ECS Connection Manager
S3 Optimized Scheduling
One of the many features of Dell EMC ECS is its ability to deliver storage efficiency using XOR. To enable this feature, XOR requires 3 or more Virtual Data Centers (VDC) or sites (8 maximum) replicating data chunks between them. Within each VDC, multiple chunks are combined into a single chunk therefore reducing the storage footprint while providing the necessary site redundancy. The load balancing component of an ECS solution leveraging XOR is essential to the successful delivery.
When data is written to a site (VDC), that VDC becomes the owner of that object. When that object is read, it should be read from the VDC that owns it. If it is read from another VDC, that VDC must request the latest version from the owning VDC which will produce unnecessary overhead and latency on the ECS storage solution. Kemp ECS Connection Manager delivers S3 Optimized Scheduling method to ensure the accurate routing of object write and read requests. This method utilizes a URL Hash algorithm to distribute writes evenly across multiple sites and sends all reads to the site owning the object. This purpose-built scheduling method delivers greater performance and optimization of the S3 traffic.
S3 Addressing Auto-Detection
ECS currently supports two addressing methods, Path Style and Virtual Hosted Addressing. In most ECS environments consisting of multiple sites that require the “XOR” storage efficiency and/or geographic distribution, the need to support both methods becomes essential. ECS Connection Manager delivers S3 Addressing Auto-Detection to simplify the configuration while providing the optimized distribution of objects throughout the ECS solution using both addressing methods seamlessly.
Dynamic Global Host Resolution
Most ECS deployments include multiple geographic locations providing the required site resilience for the object storage solution. ECS Connection Manager’s Global Server Load Balancing (GSLB) feature distributes traffic across these multiple locations with the use of intelligent DNS. The implementation of GSLB will be different contingent on whether the applications accessing the storage leverage Path Style or Virtual Hosted addressing methods. This is due to behavior of Virtual Hosted which now includes the S3 bucket names within the HTTP Host Header requiring DNS to support this dynamic addressing method. ECS Connection Manager takes this requirement and extends it to support Dynamic Global Host Resolution within GSLB to provide the flexible multi-site distribution and availability for ECS deployments leveraging both Path Style and Virtual Hosted addressing.
