We all know the most complicated is storage performance sizing especially with leveraging automated storage tiering technology existing in almost all modern storage systems. I was able to prepare some estimations based on standard storage calculations and my experience however we left final responsibility on hardware vendors and their technical pre-sales teams. Our requirement was pretty easy - 60TB of capacity and 25,000 IOPSes generated from servers in R/W ratio 70/30.
Validation and acceptance test of storage was clearly defined. The storage systems must be able to handle a 25,000 IOPS workload synthetically generated leveraging free tool IOmeter. Test environment was composed from 250 linux VMs with single Worker (IOmeter dynamo). All these workers were connected to single IOmeter GUI reporting total workload nicely in single place. Each of 250 workloads were defined as described below:
- Outstanding IO: 1
- IO size: 64KB
- Workload pattern Random/Sequence ratio: 70:30
- Read/Write Ratio: 70:30
Selected hardware vendor delivered storage with following disk tiers:
- Tier 1: 8x 400GB 6G SAS 2.5” MLC SSD R5 (3+1)
- Tier 2: 128x 300GB 6G SAS 15K 2.5” HDD R5 (7+1)
- Tier 3: 40x 900GB 6G SAS 10K 2.5” HDD R5 (7+1)
The result where we are right now is that we are able to achieve 15,600 front-end IOPSes which can be simply recalculated into back-end IOPSes based on read/write ratio and write penalty which is 4 for RAID 5. On figure below is screenshot from IOmeter just for illustration but final achievement was really 15,600 IOPS average from the beginning of the test.
Backend IOPSes = 10920 reads + ( 4 x 4680 writes) = 29,640 which can be recalculated into IOPSes per disk = 29640/128 = 231 IOPS. 231 IOPSes per 15k/rpm disk is pretty high and other Storage tiers are not leveraged so we are calling hardware vendor and asking how we can achieve our numbers.
BTW: this is acceptance hardware test and vendor has to prove this numbers otherwise he has to upgrade his storage (at his expense) or take the hardware out and return money.
To be continued ... stay tuned ...
UPDATE: Long story short ... at the end of the day storage vendor had to add additional disk enclosures with more spindles. Storage vendor had to pay it and it is worth to mention that it was significant additional cost covered 100% from his margin!!! No additional single cent paid by my customer. It is just another reason to engage subject matter expert for Infrastructure Design because when logical infrastructure design along with test plan is prepared before RFI and RFP your RFP strict requirements can be properly written and clearly articulated to all tender participants.