S3 Storage Optimization for ObjectScale

S3 Optimized Scheduling

One of the many features of Dell ObjectScale is its ability to deliver storage efficiency using XOR. To enable this feature, XOR requires 3 or more Virtual Data Centers (VDC) or sites (8 maximum) replicating data chunks between them. Within each VDC, multiple chunks are combined into a single chunk therefore reducing the storage footprint while providing the necessary site redundancy. The load balancing component of an ObjectScale solution leveraging XOR is essential to the successful delivery.

When data is written to a site (VDC), that VDC becomes the owner of that object. When that object is read, it should be read from the VDC that owns it. If it is read from another VDC, that VDC must request the latest version from the owning VDC which will produce unnecessary overhead and latency on the ObjectScale storage solution. Kemp Connection Manager for ObjectScale delivers S3 Optimized Scheduling method to ensure the accurate routing of object write and read requests. This method utilizes a URL Hash algorithm to distribute writes evenly across multiple sites and sends all reads to the site owning the object. This purpose-built scheduling method delivers greater performance and optimization of the S3 traffic.

S3 Addressing Auto-Detection

ObjectScale currently supports two addressing methods, Path Style and Virtual Hosted Addressing. In most ObjectScale environments consisting of multiple sites that require the “XOR” storage efficiency and/or geographic distribution, the need to support both methods becomes essential. Connection Manager for ObjectScale delivers S3 Addressing Auto-Detection to simplify the configuration while providing the optimized distribution of objects throughout the ObjectScale solution using both addressing methods seamlessly.

Dynamic Global Host Resolution

Most ObjectScale deployments include multiple geographic locations providing the required site resilience for the object storage solution. Connection Manager for ObjectScale ’s Global Server Load Balancing (GSLB) feature distributes traffic across these multiple locations with the use of intelligent DNS. The implementation of GSLB will be different contingent on whether the applications accessing the storage leverage Path Style or Virtual Hosted addressing methods. This is due to behavior of Virtual Hosted which now includes the S3 bucket names within the HTTP Host Header requiring DNS to support this dynamic addressing method. Connection Manager for ObjectScale takes this requirement and extends it to support Dynamic Global Host Resolution within GSLB to provide the flexible multi-site distribution and availability for ObjectScale deployments leveraging both Path Style and Virtual Hosted addressing.