Skip to main content

Posts

Showing posts from April, 2013

LXC and Host Crashes

We had set up a bunch of lxc containers on two servers each with 16 core CPUs and 64 GB RAM(for reliability and loadbalancing). Both the servers are on same vlan. The servers need to have atleast one of their network interface in promiscuous mode so that it forwards all packets on vlan to the bridge(http://blogs.eskratch.com/2012/10/create-your-own-vms-i.html) which takes care of the routing to containers. If the packets are not addressed to the containers, the bridge drops the packet.

Having this setup, we moved all our platform maintenance services to these containers. They are fault tolerant as we used two host machines where each host machine has a replica of the containers on the other. The probability to crash for both the servers at the same time due to some hardware/software failure is less. But to my surprise both the servers are crashing exactly the same time with a mean life time 20 days. We had to wake up late nights(early mornings) to fix stuffs that gone down

The detec…