A hidden cost of virtualization when scaling multicore applications

As the number of cores in a multicore node increases in accordance with Moore’s law, the question arises as to whether there are any “hidden” costs of a cloud’s virtualized environment when scaling applications to take advantage of larger core counts. This paper identifies one such cost, resulting in up to a 583% slowdown as the multicore application is scaled. Surprisingly, these slowdowns arise even when the application’s VM has dedicated use of the underlying physical hardware and does not use emulated resources. Our preliminary findings indicate that the source of the slowdowns is the intervention from the VMM during synchronization-induced idling in the application, guest OS, or supporting libraries. We survey several possible mitigations, and report preliminary findings on the use of “idleness consolidation” and “IPI-free wakeup” as a partial mitigation.