At 20:50 CEST we received our first alert regarding a hypervisor being unreachable. Upon investigation we found that the nf_conntrack table was full.
We're still investigating what the reason for this is since we have a fairly high value for nf_conntrack_max, and nf_conntrack_count is nowhere near the value of nf_conntrack_max.
We have increased (3x) nf_conntrack_max and rebooted the server.
At 21:31 CEST the server was rebooted and all VM's were being started.
