Running OpenVZ with IPv4 and IPv6 on Digital Ocean

Containers, Linux
Running OpenVZ with IPv4 and IPv6 on Digital Ocean

With containers all the rage at the moment (LXC/LXD, Docker, Rocket etc), I thought it would be interesting to see if it was possible to get a mature container implementation (OpenVZ) running on the cloud provider Digital Ocean.

I have been running OpenVZ with CentOS 5 & 6 in production for over 5 years now and I have found it to be rock solid and has a simple set of management tools.

First Steps

Firstly you need to sign up for a CentOS 6 Droplet in one of the many data centers that Digital Ocean provide.

I chose the following settings:

  • Hostname: testopenvz
  • Size: $5/mo
  • Region: London 1
  • Image: CentOS 6.5 x64
  • Settings: IPv6

Once the Droplet is up and running, SSH into it and enable a SWAP file.

Then ensure your Droplet is all up to date:

yum upgrade -y

Install some useful tools:

yum install wget nano -y

OpenVZ Installation

Now the time has come to install OpenVZ kernel and management utilities.

wget -P /etc/yum.repos.d/ http://ftp.openvz.org/openvz.repo
rpm --import http://ftp.openvz.org/RPM-GPG-Key-OpenVZ
yum install vzkernel vzctl vzquota ploop -y

Enable OpenVZ IPv4 and IPv6 network settings in /etc/sysctl.conf by adding the following lines:

# On Hardware Node we generally need
# packet forwarding enabled and proxy arp disabled
net.ipv4.ip_forward = 1
net.ipv6.conf.default.forwarding = 1
net.ipv6.conf.all.forwarding = 1
net.ipv4.conf.default.proxy_arp = 0
net.ipv6.conf.all.proxy_ndp = 1

# Enables source route verification
net.ipv4.conf.all.rp_filter = 1

# Enables the magic-sysrq key
kernel.sysrq = 1

# We do not want all our interfaces to send redirects
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0

Modify /etc/sysconfig/network and add these lines:

NETWORKING_IPV6=yes
IPV6FORWARDING=yes

Because we will be running NAT on the Droplet firewall we need to ensure that OpenVZ doesn't disable connection tracking by modifying /etc/modprobe.d/openvz.conf as follows: 

options nf_conntrack ip_conntrack_disable_ve0=0

Setting up KEXEC

Digital Ocean annoyingly do not allow you to boot your Droplet with a custom kernel. As OpenVZ requires this, the only way to boot the custom kernel is to use the kexec utility, which allows you to boot a standard kernel and then replace it with a custom kernel.

Install kexec:

yum install -y kexec-tools

Then create the following init file /etc/init.d/kexecvz which will start the OpenVZ kernel on boot:

#!/bin/sh
#
# kexecvz
#
# chkconfig: 2345 90 60
### BEGIN INIT INFO
# Provides:          localkexec
# Required-Start:
# Required-Stop:
# Should-Start:
# Default-Start:     S
# Default-Stop:
# X-Interactive:     true
# Short-Description: kexec
### END INIT INFO

case "$1" in
  start|"")
        if grep -q kexeced /proc/cmdline; then
                exit 0
        fi
        /sbin/kexec --load `ls -1t /boot/vmlinuz-*stab* | head -n 1` --initrd=`ls -1t /boot/initramfs-*stab* | head -n 1` --command-line="`cat /proc/cmdline` kexeced"
        /sbin/kexec -e
        ;;
  restart|reload|force-reload)
        echo "Error: argument '$1' not supported" >&2
        exit 3
        ;;
  stop)
        # No-op
        ;;
  *)
        echo "Usage: $0 [start|stop]" >&2
        exit 3
        ;;
esac

Run the following commands to start it on boot:

chmod +x /etc/init.d/kexecvz
chkconfig kexecvz on

Configure OpenVZ Default Container Settings

Modify the /etc/vz/vz.conf file and add the following lines:

VE_LAYOUT=simfs
DEF_OSTEMPLATE="centos-6-x86_64"

Checking OpenVZ is running

Now reboot your Droplet and when it comes back check it is running the OpenVZ kernel.

uname -a

The output should have the word "stab" in the version which indicates the OpenVZ kernel is running:

Linux testopenvz 2.6.32-042stab108.5

Enable NAT on the Droplet firewall

As Digital Ocean only allows a single IPv4 address on their Dropets we need to use NAT in order to allow the containers to access the Internet.

/sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
service iptables save

Create First Container

Now we can create our first container with Container ID 1000 and hostname testcontainer1.localdomain:

vzctl create 1000 --hostname testcontainer1.localdomain

We will add a private IPv4 address to each of our containers so they can communicate with each other and use NAT to access the IPv4 Internet. For this example I am using the 172.16.1.0/24 subnet.

vzctl set 1000 --ipadd 172.16.1.2 --save

Now we add an IPv6 address to your container from the range of 16 IPv6 addresses that Digital Ocean provide with each Droplet. To find your range go into your Digital Ocean control panel and go to Settings > Networking.

vzctl set 1000 --ipadd 2a03:xxxx:1:d0::424:6002 --save

Copy your SSH public key from your Droplet into the container so you can SSH into your container.

mkdir /vz/private/1000/root/.ssh/
chmod og= /vz/private/1000/root/.ssh/
cp /root/.ssh/authorized_keys /vz/private/1000/root/.ssh/

Now you can start the container and then login to it:

vzctl start 1000
vzctl enter 1000

Test Container Connectivity

Now you are logged into your container, you should test your IPv4 and IPv6 network connectivity.

ping google.com
ping6 google.com

If both succeed, then you're good to go, so exit our of your container back to your Droplet.

exit

Add NAT SSH Port Forward

In order to reach your container from the Internet you can either use the IPv6 address that you assigned to your container, or if you don't have IPv6 connectivity then you need to setup a NAT port forward for SSH from your Droplet's public IP to your container.

iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 8882 -j DNAT --to-destination 172.16.1.2:22
service iptables save

You should now be able to SSH to your Droplet's public IP using port 8882.

Final Thoughts

You should now be all setup to add additional containers and IP addresses as needed. Remember to setup a firewall to protect your Droplet and your containers though!.