- Tier 1 cluster (mission-critical applications) - 1:1 vCPU / pCPU ratio
- Tier 2 cluster (business-critical applications) - 3:1 vCPU / pCPU ratio
- Tier 3 cluster (supporting applications) - 5:1 vCPU / pCPU ratio
- Tier 4 cluster (virtual desktops) - 10:1 vCPU / pCPU ratio
- vCPU - virtual CPU configured for Virtual Machine
- pCPU - physical CPU core (hyper-threading is not considered)
vSphere 6.5 DRS has introduced additional option to set maximum CPU over-commitment. It limits the number of vCPUs per pCPU in particular DRS cluster. However, it is good to know that there are two different advanced DRS options (configuration parameters) how to specify vCPU:pCPU ratio and each setting behaves differently. See table below ...
|DRS Advanced Option||Scope||Min-Max Value|
|MaxVcpusPerClusterPct||cluster||0% - 500%|
|MaxVCPUsPerCore||host||0 - 32|
It is worth to mention a little bit tricky setting of these additional options via GUI. It is good to know how GUI setting of "CPU Over-Commitment" is mapped to DRS cluster advanced options.
In my lab, I have VCSA 6.5 U1c (build 7119157) so I did some tests.
If I set "CPU Over-Commitment" in vSphere Web Client (Flash/Flex) it sets MaxVcpusPerClusterPct so it is the setting per the whole vSphere Cluster.
However, in vSphere Client (HTML5) it sets MaxVCPUsPerCore so it is per ESXi host.
Therefore, it is good to know what you would like to achieve and double check DRS Advanced Options.
See different behavior in screenshots below
|6.5 U1c (build 7119157) vSphere Web Client (Flash/Flex) sets MaxVcpusPerClusterPct|
|6.5 U1c (build 7119157) vSphere Client (HTML5) sets MaxVCPUsPerCore|
TEST 1: MaxVcpusPerClusterPct = 0
Let's set MaxVcpusPerClusterPct to 0 so we are saying to allow 0 : 1 vCPU / pCPU ratio. In other words, no VM can be running in the cluster. And it works as expected. When I try to run VM I get the error "The total number of virtual CPUs present or requested in virtual machines' configuration has exceeded the limit on the host: 0". Well, it is a little bit misleading because it should be cluster-wide rule but it works as expected.
|Error when MaxVcpusPerClusterPct is set to 0|
Let's set MaxVcpusPerClusterPct to 100% so we are saying to allow 1 : 1 vCPU / pCPU ratio. I have 4 node DRS cluster where each ESXi host has two cores (pCPUs), therefore I have 8 pCPUs available in the cluster. And I can really start only four VMs because each has 2 vCPUs so I can run up to 8 vCPUs in DRS cluster.
|Error when MaxVcpusPerClusterPct is set to 100|
TEST 3: MaxVcpusPerCore = 0
MaxVcpusPerCore should solve the problem observed in previous two test because it should set vCPU / pCPU ratio per host which is much better from the predictability point of view.
Let's set MaxVcpusPerCore to 0. My expectation was that I will not be able to start any VM but that was NOT the case. I was able to start a lot of VMs and exceed the expected vCPU / pCPU ratio. This is unexpected behavior.
TEST 4: MaxVcpusPerCore = 1
Let's set MaxVcpusPerCore to 1. My expectation was that I will not be able to start more than 2 vCPUs per ESXi host so only one VM with 2 vCPUs. Unfortunately, I was able to start much more vCPUs per ESXi host. This is again unexpected behavior.
TEST 5: MaxVcpusPerCore = 4
I have been informed by DRS Engineer that the minimum value allowed by host over-commitment ratio option (MaxVCPUsPerCore) is 4:1.
So, let's set MaxVcpusPerCore to 4. I have prepared nine VMs with 4 vCPUs each and my expectation is that I will not be able to start more than 8 vCPUs per ESXi host so only two VMs with 4 vCPUs per ESXi host. And because I have 4 ESXi hosts per cluster I should be able to start the maximum of 8 VMs.
|Expected error message when MaxVcpusPerCore is set to 4 and vCPU:pCPU is over 4:1 per ESXi host.|
Please note, that MaxVcpusPerCore does not support ratio 0:1 in contrast with MaxVcpusPerClusterPct where 0:1 is possible and it will effectively disable to run any VM on the cluster.
Advanced DRS setting MaxVcpusPerClusterPct supports values between 0 and 500 and represents percentage between vCPUs and pCPUs across the whole DRS cluster. If the value is higher then 500, enforcing does not work. So, vCPU / pCPU percentage ratio can be enforced between 0:1 to 5:1. vCPU:pCPU ratio 0:1 is a special setting where no VM can be PowerOn on vSphere Cluster. This is little bit risky setting but it can be used to put the whole cluster in kind of "maintenance mode" and forbid anyone to run VMs there.
To be honest, I think for vSphere architects/designers MaxVcpusPerCore makes more sense than MaxVcpusPerClusterPct because vCPU/pCPU overbooking ratio defines CPU quality per ESXi host.
VMware is internally considering to align MaxVcpusPerClusterPct and MaxVcpusPerCore behavior and allow lower vCPU / pCPU ratios when MaxVcpusPerCore is used. I will track how this topic will evolve in the future. Stay tuned.