VMWare published a new version of Workstation Pro 14.1.2 on may the 21st of 2018. As an intensive user of VMWare workstation, I always accept new version, and update as soon as possible. But this version was a great deception.
With this new version, my VMs, that usually run smoothly on my PC, consumes all the CPUs. As soon as I fired up a VM, my CPU was loaded at more than 80%. I tried to investigate a bit to understand the problem…
Hardware configuration
I worked on a 7 years old laptop but still in the running. I launched a few commands to get my workstation specifications.
CPU specifications
$ sudo lscpu Architecture : x86_64 Mode(s) opératoire(s) des processeurs : 32-bit, 64-bit Boutisme : Little Endian Processeur(s) : 8 Liste de processeur(s) en ligne : 0-7 Thread(s) par cœur : 2 Cœur(s) par socket : 4 Socket(s) : 1 Nœud(s) NUMA : 1 Identifiant constructeur : GenuineIntel Famille de processeur : 6 Modèle : 42 Nom de modèle : Intel(R) Core(TM) i7-2720QM CPU @ 2.20GHz Révision : 7 Vitesse du processeur en MHz : 843.906 Vitesse maximale du processeur en MHz : 3300,0000 Vitesse minimale du processeur en MHz : 800,0000 BogoMIPS : 4390.25 Virtualisation : VT-x Cache L1d : 32K Cache L1i : 32K Cache L2 : 256K
Memory specifications My PC is well equipped with RAM :
$ cat /proc/meminfo MemTotal: 32972168 kB MemFree: 246404 kB MemAvailable: 13267076 kB Buffers: 183064 kB Cached: 28702732 kB SwapCached: 0 kB
Disks specification
My PC is equipped with an SSD. I run LVM on top of it :
$ sudo hdparm -i /dev/sda1
/dev/sda1:
Model=Samsung SSD 850 PRO 1TB, FwRev=EXM02B6Q
Config={ Fixed }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=0
BuffType=unknown, BuffSize=unknown, MaxMultSect=1, MultSect=1
CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=2000409264
IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
The hardware is well dimensionned if you consider that I run standard VMs such as small linux or Windows Workstations.
OS configuration
Debian 8.6 is installed :
hostnamectl
Chassis: laptop
Operating System: Debian GNU/Linux 8 (jessie)
Kernel: Linux 3.16.0-6-amd64
Architecture: x86-64
VMWare workstation configuration
I subscribed to a Workstation Pro license. I’ve been running the previous version (14.1.1 build-7528167) since january 2018 without any performance issue. In may 2018, VMWare published a new version (14.2.1).
The issue with this new version
I updated to this new version at the end of may. The upgrade was installed smoothly. But, with this new version, my PC freezes two or three minutes after any VM (linux, windows, very small or larger configuration) starts. The load of the RAM was not high but the CPU was around 80% or above.
I didn’t catch the correlation between VMWare upgrade and my performance problems in the first time. I thought that, as my PC was old and summer was approaching, the performance was probably due to an overheating. I check the temperature of my system. It was quite hot but not alarming :
$ sensors acpitz-virtual-0 Adapter: Virtual device temp1: +25.0°C (crit = +107.0°C) coretemp-isa-0000 Adapter: ISA adapter Physical id 0: +59.0°C (high = +86.0°C, crit = +100.0°C) Core 0: +57.0°C (high = +86.0°C, crit = +100.0°C) Core 1: +57.0°C (high = +86.0°C, crit = +100.0°C) Core 2: +57.0°C (high = +86.0°C, crit = +100.0°C) Core 3: +59.0°C (high = +86.0°C, crit = +100.0°C)
I then thought about a problem with my browsers, that are very CPU-demanding. I tried various configurations.
I thought I should upgrade to the last version of Debian (9.x). I didn’t make the leap to 9.x because I was too busy to do so (but I definitely should).
I ended up by identifying that the problem occurs as soon as I started up a VM. I decided to rollback to VMWare workstation previous version (14.1.1). The performance issues have gone away with the downgrade.
Potential root cause
The root of the problem is probably in VMWare Meltdown/Spectre patching strategy. As mentionned in the release notes of workstation 14.2.1 on VMWare site:
This update provides guest access to the SSBD feature in IA32_SPEC_CTRL. This feature will be provided to all HWv9 or later virtual machines when running on an Intel based platform with the appropriate microcode updates. Guest operating systems may choose to use this new CPU feature to mitigate CVE-2018-3639. Please see VMware knowledge base article 54951 for more details.
The KB 54951 article on VMWare site mentions that SSB mitigation requires both Hypervisor-Assisted Guest Mitigations and Operating System-Specific Mitigations. I suspect that I should check that the OS that is installed on my laptop is compatible with SSB mitigation and how the whole thing works with VMWare workstation.
I’ll let you know more about this when I’ll upgrade to Debian 9!
