I’ll cover a recent design decision we had to make on whether or not to inject a default route from your on-premises network into VMConAWS.
This may at first glance not sound like a big deal however depending on the customer’s topology and footprint i.e. if they have existing on-premises locations then it could be something you need to consider carefully.
For example egress costs for internet connectivity directly out of AWS are charged at a higher rate then egressing across a direct connect connection. Therefore if your customer already has an on-premises presence and infrastructure in-place it may well be more cost-effective to route and egress out of that instead.
The internet breakout within VMConAWS via the IGW is also unfiltered and not inspected, we only have the NSX L4 firewalls to protect us. If you did want to egress directly out of AWS you would need to stand up a transit VPC and deploy your own Layer 7 Next-Gen firewalls to inspect that traffic for you.
We also have an AWS limitation of only being able to receive 100 routes from your network in to AWS via BGP depending on the scale and topology of your network trying to summarise that network may be very difficult and may well push you over the 100 routes. If you do exceed the 100 route limit the BGP session will be terminated and connectivity lost over the direct connect so it is something you need to design against.
This particular customer has already invested in standing up an on-premises SDDC as well as the VMConAWS SDDC’s. This meant from a connectivity perspective they had already invested in L7 firewalls and security appliances for internet connectivity. Which meant in the immediate term we would route traffic across the direct connect and out of the on-premises internet breakout. This could always be revisited in the future as the VMC instance outgrows the on-premises deployments.
The above diagram covers how this would look from a traffic flow perspective, with internet traffic originating in VMC being routed over the direct connect and out of the on-premises egress point.
However; in making that design decision it raised a question around how this might affect VPN connectivity this is especially important if you want to use a Route Based IPSEC VPN as a back connectivity method. You have to understand that internet connectivity is terminated on the IGW (Internet Gateway) inside the shadow VPC and not directly on the Tier-0 (That looks to be a future release item). Therefore if we receive a default route from on-premises we would not be able to route out to the IGW from the Tier-0 for VPN connectivity as all traffic would be sent down the direct connect.
VMware has a solution for this little issue and that is to inject a /32 route into the Tier-0 for the destination VTI (Virtual Tunnel Interface). As an example to get to the destination VTI1 which is on-premises go via the Tier-0 —> IGW rather than over say the direct connect where the default route would push you.
vWud.net 