I use VirtualBox to host a number of different servers as virtual machines. One of these is my Squid Proxy Server, which I use to allow servers on my non-internet facing internal network access to the internet. My Squid server itself runs on Ubuntu, and has two network interfaces configured in the /etc/network/interfaces file:
# The loopback network interface auto lo iface lo inet loopback # The primary network interface - access to host machine and web auto eth0 iface eth0 inet static address 192.168.XXX.YYY netmask 255.255.255.0 network 192.168.XXX.0 broadcast 192.168.XXX.255 gateway 192.168.XXX.ZZZ # Google's public DNS servers dns-nameservers 22.214.171.124, 126.96.36.199 # Secondary network interface - access to internal network only auto eth1 iface eth1 inet static address 192.168.AAA.YYY netmask 255.255.255.0 network 192.168.AAA.0 broadcast 192.168.AAA.255 gateway 192.168.AAA.BBB
Generally I have no problem running these two interfaces together, and the servers in the internal network are able to proxy through this Squid server to the outside world. However, sometimes things get a little messed up, and these servers lose the ability to connect through the proxy.
After much hair pulling, I discovered the problem. It happens occasionally (not always) when I restart my proxy server VM. It seems that the network interfaces don’t always get initiated in the way I expect. In a perfect world, eth0 is assigned, then eth1. However, it seems that sometimes eth1 is configured first, sets the default gateway in the routing table, and this prevents eth0 from initialising as the default gateway correctly.
If I try and bring up the interfaces on the proxy when things are going awry, this is what I get returned:
kristian@proxy:~$ sudo ifup eth0 RTNETLINK answers: File exists Failed to bring up eth0. kristian@proxy:~$ sudo ifup eth1 ifup: interface eth1 already configured
Checking my routing table, I see the following entries:
kristian@proxy:~$ netstat -rn Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface 0.0.0.0 192.168.AAA.BBB 0.0.0.0 UG 0 0 0 eth1 192.168.XXX.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 192.168.AAA.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
As you can see both eth0 and eth1 are there, but eth1 has snatched the default gateway. Since my 192.168.AAA (eth1) network is non internet facing, none of my other servers can access the net through the proxy anymore.
My solution is to comment out the entire eth1 interface in /etc/network/interfaces, and restart the machine (I’m aware that a restart is quite drastic – I’m always open to better solutions! If you have one, please let me know!). Once it comes back up, I can see that my routing table now has the information that I need to access the internet, but it’s missing my internal network as expected:
kristian@proxy:~$ netstat -rn Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface 0.0.0.0 192.168.XXX.ZZZ 0.0.0.0 UG 0 0 0 eth0 192.168.XXX.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
By then uncommenting eth1 again in the /etc/network/interfaces file, and bringing up the interface, you get the ‘RTNETLINK’ error on eth1, not eth0 this time. However a final check of the routing table shows that all is as it should be:
kristian@proxy:~$ sudo ifup eth1 RTNETLINK answers: File exists Failed to bring up eth1. kristian@proxy:~$ netstat -rn Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface 0.0.0.0 192.168.XXX.ZZZ 0.0.0.0 UG 0 0 0 eth0 192.168.XXX.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 192.168.AAA.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
Now eth0 controls the default gateway as it should, my internal network is configured, and my servers can proxy away to their hearts content!
Moral of the story: Don’t assume anything is initialised in a specific order based on its name.