Xen Networking Problem

Discussion in 'Installation/Configuration' started by atjensen11, Jan 27, 2009.

  1. atjensen11

    atjensen11 New Member

    I have finally installed Xen on LVM on top of a RAID 1 array. That alone is a big accomplishment for me.

    So now I am adding NICs to my machine and configuring Xen to use them.

    First, I created a new file /etc/xen/scripts/my-network-script. It reads:
    Code:
    #!/bin/sh
    dir=$(dirname "$0")
    "$dir/network-bridge" "$@" vifnum=0 netdev=eth0 bridge=eth0
    "$dir/network-bridge" "$@" vifnum=1 netdev=eth1 bridge=eth1
    
    I replaced the default line in /etc/xen/xend-config.sxp to tell Xen to use my custom bridge script:
    Code:
    #(network-script network-bridge)
    (network-script my-network-script)
    
    After booting the machine, the physical NICs are brought up. The two NICs are in different Class C networks. The ouput of /etc/network/interfaces is:
    Code:
    # This file describes the network interfaces available on your system
    # and how to activate them. For more information, see interfaces(5).
    
    # The loopback network interface
    auto lo
    iface lo inet loopback
    
    # The primary network interface
    auto eth0
    iface eth0 inet static
            address 192.168.10.11
            netmask 255.255.255.0
            network 192.168.10.0
            broadcast 192.168.10.255
            gateway 192.168.10.1
    
    auto eth1
    iface eth1 inet static
            address 192.168.20.11
            netmask 255.255.255.0
            network 192.168.20.0
            broadcast 192.168.20.255
            gateway 192.168.20.1
    
    The output of ifconfig is:
    Code:
    eth0      Link encap:Ethernet  HWaddr 00:12:3f:3b:11:9c
              inet addr:192.168.10.11  Bcast:192.168.10.255  Mask:255.255.255.0
              inet6 addr: fe80::212:3fff:fe3b:119c/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:334 errors:0 dropped:0 overruns:0 frame:0
              TX packets:280 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:32361 (31.6 KB)  TX bytes:37356 (36.4 KB)
    
    eth1      Link encap:Ethernet  HWaddr 00:1b:21:1d:41:21
              inet addr:192.168.20.11  Bcast:192.168.20.255  Mask:255.255.255.0
              inet6 addr: fe80::21b:21ff:fe1d:4121/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:103 errors:0 dropped:0 overruns:0 frame:0
              TX packets:15 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:14560 (14.2 KB)  TX bytes:1166 (1.1 KB)
    
    lo        Link encap:Local Loopback
              inet addr:127.0.0.1  Mask:255.0.0.0
              inet6 addr: ::1/128 Scope:Host
              UP LOOPBACK RUNNING  MTU:16436  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
    
    peth0     Link encap:Ethernet  HWaddr 00:12:3f:3b:11:9c
              inet6 addr: fe80::212:3fff:fe3b:119c/64 Scope:Link
              UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
              RX packets:333 errors:0 dropped:0 overruns:0 frame:0
              TX packets:291 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:39661 (38.7 KB)  TX bytes:39882 (38.9 KB)
              Base address:0xdcc0 Memory:dfde0000-dfe00000
    
    peth1     Link encap:Ethernet  HWaddr 00:1b:21:1d:41:21
              inet6 addr: fe80::21b:21ff:fe1d:4121/64 Scope:Link
              UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
              RX packets:112 errors:0 dropped:0 overruns:0 frame:0
              TX packets:33 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:100
              RX bytes:17242 (16.8 KB)  TX bytes:2840 (2.7 KB)
              Base address:0xdc80 Memory:dfda0000-dfdc0000
    
    And brctl show:
    Code:
    bridge name     bridge id               STP enabled     interfaces
    eth0            8000.00123f3b119c       no              peth0
    eth1            8000.001b211d4121       no              peth1
    
    And finally, netstat -arn:
    Code:
    Kernel IP routing table
    Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
    192.168.20.0    0.0.0.0         255.255.255.0   U         0 0          0 eth1
    192.168.10.0    0.0.0.0         255.255.255.0   U         0 0          0 eth0
    0.0.0.0         192.168.20.1    0.0.0.0         UG        0 0          0 eth1
    0.0.0.0         192.168.10.1    0.0.0.0         UG        0 0          0 eth0
    
    The problem is that I cannot ping any outside machine from eth0. I can ping the gateway (192.168.10.1) and another machine on the 192.168.10.0 network. From another machine on the 192.168.10.0 network, I can ping the eth0 address 192.168.10.11.

    I can ping any machine from eth1.

    Both networks use the same DNS server on the 192.168.10.0 network. Therefore, I don't believe it is a DNS issue. I can issue a dig or nslookup
    and get a correct response, but I can't be sure which interface the request occurred over.

    If I enter ifdown eth1, then I can ping out on eth0. If I bring eth1 back up, I can no longer ping out to external machines on eth0.

    Any suggestions?
     
    Last edited: Jan 27, 2009
  2. falko

    falko Super Moderator

    I'm not sure if it helps, but you could try to add the following two lines at the end of /etc/network/interfaces:

    Code:
    up route add -net 192.168.10.0 netmask 255.255.255.0 gw 192.168.10.1 eth0
    up route add -net 192.168.20.0 netmask 255.255.255.0 gw 192.168.20.1 eth1
     
  3. atjensen11

    atjensen11 New Member

    Falko,

    I think you are on to something. Last night, I was researching the issue more and noticed that the only thing that really changed when I took eth1 up and down was that the default route changed.

    I am getting in over my head a little bit, but one forum post I read stated that a machine should only have one default route to the internet. But in my case, I have two NICs that each have a possible route to the internet.

    I guess I am wondering if what I am attempting to do is somehow violating the logic of basic networking principles. I try not to over think things, but is seems pefectly plausible to me that both NICs should be able to ping a public address.

    I will attempt to include a schematic of the current situation.

    Xen Dom0:
    eth0 is on the 192.168.10.0 network. The gateway device for this network is the external facing router. That is, the gateway router has a private address of 192.168.10.1 and a public WAN address.

    eth1 is on the 192.168.20.0 network. The gateway device for this network is a private side router. That is, the router has a private address on this network of 192.168.20.1. From the router's perspective, the WAN address is 192.168.10.2.

    So ultimately, everything does make its way to the 192.168.10.0 network, but traffic out of eth1 goes through an extra router. I do this to protect all my personal machines on the 192.168.20.0 network from incoming connections from the public. Those are all dealt with and routed to the 192.168.10.0 network.
     

Share This Page