Ubuntu 18.04, netplan, 2 gateways.

Discussion in 'ISPConfig 3 Priority Support' started by xrstokes, May 16, 2019.

  1. xrstokes

    xrstokes Member

    I have a perfect server mirror setup. Runs great, thanks so much to these forums. In front of the servers is a haproxy. In this case the haprox is at another data centre. The haprox is also a open-vpn server, the clients and prox all communicate with each other over this vpn network. I was recently burned really bad by a reputable data centre having a long outage on their SAN making me look really bad. I'm not happy to have a single point of failure any where, which is what the vpn represents. The data recovery strategy is to re boot-strap the sql cluster at a single location. Open up the firewall and serve all traffic from the best servers external ip. Failing over to the ip via dns changes.
    This where this falls over, is that the backup location also serves other data from it's external ip all the time.
    So I've got 2 interfaces, ens3 and tun0. I need some genius policy routing to reply on the interface that the request comes from. If I set tun0 to the default gateway of the prox, all the traffic coming from the prox reply's fine, but reply's from it's external ip obviously go out the wrong way.
    The opposite is also true if the gateway is not redirected through the tun0 interface, the external traffic is great, but the haprox stops using it.
    Here is what it looks like
    Code:
    [email protected]*****:~# netstat -anr
    Kernel IP routing table
    Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
    0.0.0.0         external_ip    0.0.0.0         UG        0 0          0 ens3
    10.8.0.0        10.8.0.1        255.255.255.0   UG        0 0          0 tun0
    10.8.0.1        0.0.0.0         255.255.255.255 UH        0 0          0 tun0
    external_ip     0.0.0.0         255.255.255.255 UH        0 0          0 ens3
    
    The answer maybe contained on this page
    https://blog.scottlowe.org/2013/05/29/a-quick-introduction-to-linux-policy-routing/
    What i need is a policy routing rule like the following.
    echo 200 vpn2 >> /etc/iproute2/rt_tables
    ip rule add to 10.8.0.10 lookup vpn2
    ip rule add to prox_external_ip lookup vpn2
    ip route add default via 10.8.0.1 dev tun0 table vpn2

    The goal is to see what has come through the prox by looking at the to field and setting the default gateway to out the VPN interface.

    Thanks in advance.
     
  2. xrstokes

    xrstokes Member

    If any one else runs into this. Check out this resource. https://osric.com/chris/accidental-developer/2019/03/linux-policy-based-routing/ Mark as solved. It's murphy's law, if I get really stuck all I need to do is ask on this forum. Then I usually discover what I need...….Sorry. Thanks anyway.
    Code:
    sudo iptables -A PREROUTING -t mangle -i eth1 -j MARK --set-mark 1
    sudo iptables -A PREROUTING -t mangle -i eth1 -j CONNMARK --save-mark
    sudo iptables -A OUTPUT -t mangle -j CONNMARK --restore-mark
    /etc/iproute2/rt_tables
    100    eth1_table
    sudo ip rule add priority 1000 fwmark 0x1 table eth1_table
    ip rule show
    sudo ip route add table eth1_table 0.0.0.0/0 via 192.168.100.1 dev eth1 src 192.168.100.9
    ip route show table eth1_table
    default via 192.168.100.1 dev eth1 src 192.168.100.9
    /etc/sysctl.d/10-eth1.conf
    net.ipv4.conf.eth1.src_valid_mark=1
    $ chmod 0644 /etc/sysctl.d/10-eth1.conf
    $ chown root:root /etc/sysctl.d/10-eth1.conf
    sudo sysctl -p /etc/sysctl.d/10-eth1.conf
    just incase it goes missing
     
    till likes this.

Share This Page