Private vlans are not routing to each other by default..

Peter -

Joyent recently discovered a networking issue with different private vlans and the inability to route between them in the same datacenter.

Problem

Network routing components (L3 Edge and distribution switches) show no signs of misconfiguration and routing works as designed to the host gateways. We are not seeing traffic destined for host-to-host via JoyentSDC-10.112.0.0/21 JoyentSDC-10.112.72.0/21 private subnets working. It appears that the packets are not leaving the host interface when routing to the adjacent subnet.

We found that IP AntiSpoof is blocking the packets. In short, Box A tries to talk to private interface of Box B, however the L2 private subnet is different and therefore Box A packats traverse the public interface arriving at the public interface of Box B and are rejected by IP Antispoof because the destination address isn't an "allowed-ip".

Solution

For SmartOS systems regardless of which vlan they're on, assuming the private is net1. 

route -p add -interface 10.0.0.0 -gateway `ifconfig net1 | grep inet | awk '{print $2}'`

Here's a centos example:

/sbin/route add -net 10.0.0.0 netmask 255.0.0.0 dev eth1

and add the above to the end of /etc/rc.local so the route persists on reboot

Have more questions? Submit a request

3 Comments

  • 0
    Avatar
    Nick Pimshin

    Hmm... I'm not sure how it helps os to respond over the routed path.. The matter I'm dealing with is to have multiple interfaces with different IPs on the same subnet and make them accessible over the routed path. For instance, this is the routing table on the machine where 2 out of 4 interfaces are working as expected due to default routes being set in the table, and the other 2 are not routed behind current vlan:

    Routing Table: IPv4
      Destination           Gateway           Flags  Ref     Use     Interface
    -------------------- -------------------- ----- ----- ---------- ---------
    default              10.57.76.1           UG        1          0 net3
    default              10.57.76.1           UG        1          0 net2
    10.57.76.0           10.57.76.37          U         2          0 net3
    10.57.76.0           10.57.76.38          U         2          0 net2
    10.57.76.0           10.57.76.39          U         2          0 net1
    10.57.76.0           10.57.76.55          U         2          0 net0
    127.0.0.1            127.0.0.1            UH        3          8 lo0

    So I can ping 10.57.76.37  and 10.57.76.38 from outside, and 10.57.76.55  and 10.57.76.39 are unreachable.

    Any idea on how to make them all accessible over the routed paths?

     

    Thank you in advance,

    Nick

     

     

  • 0
    Avatar
  • 0
    Avatar
    ashey sullivan

    I think this is a common issue that is prevalent among the VLAN users . The problem usually arises out of misconfiguration that is designed to cater the queries and support of the host-to-host traffic. Apparently IP AntiSpoof has been blocking the packets.

Article is closed for comments.