Using Linux as a Router to configure Isolated VLANs w/ Multiple Subnets and Multiple Gateways, over a single (dumb) switch.

Category: Software and Systems Engineering
Author: Sean O'Donnell
Sat, Feb. 9th, 2013 @ 3:33:50 (MST)

Overview

There comes a time and a place in every system administrator's life, where he has to question the existence of a 3rd-party proprietary router device in the equation, and the costs that go with, vs. the simplicity, and autonomously-enabling nature of the native linux TCP/IP stack. That was part of what led me to linux in the 1st place... a stable OS that can run under minimal hardware conditions, and provide the flexibility of complete customization. Linux is just that, especially when it comes to the Internet Protocol (IP).

This article depicts a situation where you have a Dual-Gateway Network Matrix, where the application server connections to Gateway 1, which is connected to the primary NIC adapter (em1), and all systems connected to the application server via (dumb) ethernet switch on the secondary NIC adapter (em2), are routed to an alternate gateway that is plugged in to the switch, as well. Who needs Cisco?

dualgate_multinet.sh

I created a bash script to automate all of the steps (below). It should simplify the process. Simply flag the NIC and IP Address (of the NIC connected to your VLAN switch), and it will configure your linux router to support dual-gateways across multiple isolated subnets.

i.e. ./dualgate_multinet.sh -i eth1 -a 192.168.0.254

Download the latest revision from my github repository.

Kernel IP Forwarding

Here lies the basis for ALL essential Linux Router Configurations, and also applies to this configuration.

echo 1 > /proc/sys/net/ipv4/ip_forward
sed -i 's/net.ipv4.ip_forward = 0/net.ipv4.ip_forward = 1/' /etc/sysctl.conf 

Also, you will need to ensure that you have the nf_conntrack* modules compiled into your kernel, or else this will not work.

lsmod | grep "nf_conntrack"

DNS Masquerading

You'll want to install and configure dnsmasq to provide dns/dhcp masquerading on your VLAN NIC (only).

# install and configure dnsmasq
yum -y install dnsmasq
sed -i 's/#except-interface=/except-interface=em1/' /etc/dnsmasq.conf
sed -i 's/#no-dhcp-interface=/no-dhcp-interface=em1/' /etc/dnsmasq.conf
systemctl enable dnsmasq.service
systemctl start dnsmasq.service

Linux Application Server Network Configuration

  • IP Address (NIC1): 10.20.30.$undefined (wan/dhcp)
  • IP Address (NIC2): 192.168.0.254 (vlan)
  • Gateway: 10.20.30.254
    • DNS1: 10.20.30.10
    • DNS2: 10.20.30.11

VLAN "Client" Network Configuration

  • IP Address: 192.168.$subnet.$num
  • Gateway: 192.168.$subnet.254
    • DNS1: 192.168.0.254

Multiple Subnets

Configuring Multiple Subnets with more than (1) gateway is a bit more involved, but is very practical for many networking applications.

Create a custom routing table for each alternate Gateway. In this case, we only have (1) alternate gateway, so we only need (1) alternate routing table.

echo 200 gw2 >> /etc/iproute2/rt_tables

Add each subnet to the appropriate routing table

ip rule add from 192.168.1.0/24 table gw2
ip rule add from 192.168.2.0/24 table gw2
ip rule add from 192.168.3.0/24 table gw2
ip rule add from 192.168.4.0/24 table gw2
ip rule add from 192.168.5.0/24 table gw2
ip rule add from 192.168.6.0/24 table gw2
ip rule add from 192.168.7.0/24 table gw2
ip rule add from 192.168.8.0/24 table gw2

Define the 'default' alternate gateway for the custom routing table and VLAN subnets.

ip route add default via 192.168.0.254 dev em2 table gw2

Flush the routing cache to initiate the (new) alternate routing table

ip route flush cache

Verify that the routing tables are correct

ip -s route
default via 10.20.30.254 dev em1 
10.20.30.0/24 dev em1  proto kernel  scope link  src 10.20.30.254
192.168.0.0/24 dev em2  scope link 
192.168.1.0/24 dev em2  scope link 
192.168.2.0/24 dev em2  scope link 
192.168.3.0/24 dev em2  scope link 
192.168.4.0/24 dev em2  scope link 
192.168.5.0/24 dev em2  scope link 
192.168.6.0/24 dev em2  scope link 
192.168.7.0/24 dev em2  scope link 
192.168.8.0/24 dev em2  scope link

Append/Add the following rules to: /etc/sysconfig/iptables

*nat
:PREROUTING ACCEPT [209:32852]
:INPUT ACCEPT [200:32154]
:OUTPUT ACCEPT [134:9345]
:POSTROUTING ACCEPT [136:9513]
-A PREROUTING -d 192.168.1.20/32 -i em2 -p tcp -j DNAT --to-destination 192.168.0.254
-A PREROUTING -d 192.168.2.20/32 -i em2 -p tcp -j DNAT --to-destination 192.168.0.254
-A PREROUTING -d 192.168.3.20/32 -i em2 -p tcp -j DNAT --to-destination 192.168.0.254
-A PREROUTING -d 192.168.4.20/32 -i em2 -p tcp -j DNAT --to-destination 192.168.0.254
-A PREROUTING -d 192.168.5.20/32 -i em2 -p tcp -j DNAT --to-destination 192.168.0.254
-A PREROUTING -d 192.168.6.20/32 -i em2 -p tcp -j DNAT --to-destination 192.168.0.254
-A PREROUTING -d 192.168.7.20/32 -i em2 -p tcp -j DNAT --to-destination 192.168.0.254
-A PREROUTING -d 192.168.8.20/32 -i em2 -p tcp -j DNAT --to-destination 192.168.0.254
-A POSTROUTING -o em2 -j MASQUERADE
-A POSTROUTING -o em2 -j SNAT --to-source 192.168.0.254
COMMIT
*filter
:INPUT ACCEPT [2134:201734]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [1439:141782]
-A FORWARD -i em2 -j ACCEPT
-A FORWARD -s 192.168.1.0/24 -d 192.168.0.20/32 -j ACCEPT
-A FORWARD -s 192.168.1.0/24 -d 192.168.2.0/24 -j DROP
-A FORWARD -s 192.168.1.0/24 -d 192.168.3.0/24 -j DROP
-A FORWARD -s 192.168.1.0/24 -d 192.168.4.0/24 -j DROP
-A FORWARD -s 192.168.1.0/24 -d 192.168.5.0/24 -j DROP
-A FORWARD -s 192.168.1.0/24 -d 192.168.6.0/24 -j DROP
-A FORWARD -s 192.168.1.0/24 -d 192.168.7.0/24 -j DROP
-A FORWARD -s 192.168.1.0/24 -d 192.168.8.0/24 -j DROP
-A FORWARD -s 192.168.2.0/24 -d 192.168.0.20/32 -j ACCEPT
-A FORWARD -s 192.168.2.0/24 -d 192.168.1.0/24 -j DROP
-A FORWARD -s 192.168.2.0/24 -d 192.168.3.0/24 -j DROP
-A FORWARD -s 192.168.2.0/24 -d 192.168.4.0/24 -j DROP
-A FORWARD -s 192.168.2.0/24 -d 192.168.5.0/24 -j DROP
-A FORWARD -s 192.168.2.0/24 -d 192.168.6.0/24 -j DROP
-A FORWARD -s 192.168.2.0/24 -d 192.168.7.0/24 -j DROP
-A FORWARD -s 192.168.2.0/24 -d 192.168.8.0/24 -j DROP
-A FORWARD -s 192.168.3.0/24 -d 192.168.0.20/32 -j ACCEPT
-A FORWARD -s 192.168.3.0/24 -d 192.168.1.0/24 -j DROP
-A FORWARD -s 192.168.3.0/24 -d 192.168.2.0/24 -j DROP
-A FORWARD -s 192.168.3.0/24 -d 192.168.4.0/24 -j DROP
-A FORWARD -s 192.168.3.0/24 -d 192.168.5.0/24 -j DROP
-A FORWARD -s 192.168.3.0/24 -d 192.168.6.0/24 -j DROP
-A FORWARD -s 192.168.3.0/24 -d 192.168.7.0/24 -j DROP
-A FORWARD -s 192.168.3.0/24 -d 192.168.8.0/24 -j DROP
-A FORWARD -s 192.168.4.0/24 -d 192.168.0.20/32 -j ACCEPT
-A FORWARD -s 192.168.4.0/24 -d 192.168.1.0/24 -j DROP
-A FORWARD -s 192.168.4.0/24 -d 192.168.2.0/24 -j DROP
-A FORWARD -s 192.168.4.0/24 -d 192.168.3.0/24 -j DROP
-A FORWARD -s 192.168.4.0/24 -d 192.168.5.0/24 -j DROP
-A FORWARD -s 192.168.4.0/24 -d 192.168.6.0/24 -j DROP
-A FORWARD -s 192.168.4.0/24 -d 192.168.7.0/24 -j DROP
-A FORWARD -s 192.168.4.0/24 -d 192.168.8.0/24 -j DROP
-A FORWARD -s 192.168.5.0/24 -d 192.168.0.20/32 -j ACCEPT
-A FORWARD -s 192.168.5.0/24 -d 192.168.1.0/24 -j DROP
-A FORWARD -s 192.168.5.0/24 -d 192.168.2.0/24 -j DROP
-A FORWARD -s 192.168.5.0/24 -d 192.168.3.0/24 -j DROP
-A FORWARD -s 192.168.5.0/24 -d 192.168.4.0/24 -j DROP
-A FORWARD -s 192.168.5.0/24 -d 192.168.6.0/24 -j DROP
-A FORWARD -s 192.168.5.0/24 -d 192.168.7.0/24 -j DROP
-A FORWARD -s 192.168.5.0/24 -d 192.168.8.0/24 -j DROP
-A FORWARD -s 192.168.6.0/24 -d 192.168.0.20/32 -j ACCEPT
-A FORWARD -s 192.168.6.0/24 -d 192.168.1.0/24 -j DROP
-A FORWARD -s 192.168.6.0/24 -d 192.168.2.0/24 -j DROP
-A FORWARD -s 192.168.6.0/24 -d 192.168.3.0/24 -j DROP
-A FORWARD -s 192.168.6.0/24 -d 192.168.4.0/24 -j DROP
-A FORWARD -s 192.168.6.0/24 -d 192.168.5.0/24 -j DROP
-A FORWARD -s 192.168.6.0/24 -d 192.168.7.0/24 -j DROP
-A FORWARD -s 192.168.6.0/24 -d 192.168.8.0/24 -j DROP
-A FORWARD -s 192.168.7.0/24 -d 192.168.0.20/32 -j ACCEPT
-A FORWARD -s 192.168.7.0/24 -d 192.168.1.0/24 -j DROP
-A FORWARD -s 192.168.7.0/24 -d 192.168.2.0/24 -j DROP
-A FORWARD -s 192.168.7.0/24 -d 192.168.3.0/24 -j DROP
-A FORWARD -s 192.168.7.0/24 -d 192.168.4.0/24 -j DROP
-A FORWARD -s 192.168.7.0/24 -d 192.168.5.0/24 -j DROP
-A FORWARD -s 192.168.7.0/24 -d 192.168.6.0/24 -j DROP
-A FORWARD -s 192.168.7.0/24 -d 192.168.8.0/24 -j DROP
-A FORWARD -s 192.168.8.0/24 -d 192.168.0.20/32 -j ACCEPT
-A FORWARD -s 192.168.8.0/24 -d 192.168.1.0/24 -j DROP
-A FORWARD -s 192.168.8.0/24 -d 192.168.2.0/24 -j DROP
-A FORWARD -s 192.168.8.0/24 -d 192.168.3.0/24 -j DROP
-A FORWARD -s 192.168.8.0/24 -d 192.168.4.0/24 -j DROP
-A FORWARD -s 192.168.8.0/24 -d 192.168.5.0/24 -j DROP
-A FORWARD -s 192.168.8.0/24 -d 192.168.6.0/24 -j DROP
-A FORWARD -s 192.168.8.0/24 -d 192.168.7.0/24 -j DROP
COMMIT

Restart the iptables service.

systemctl restart iptables.service

Copyleft (<) 1998-2019 www.seanodonnell.com