How to peer two VPCs and access it using a VPN

Although I have some experience, I am not a sysadmin. So this might be obvious for a sysadmin or someone with an AWS certification but not for a developer O:)

We had 2 VPCs that are connected through the Internet. We wanted to peer them both so the traffic goes internally within the AWS region. Reasons?

1. Efficiency: it is faster to use the internal network.
2. Security: the traffic does not leave the data center.
3. Price: AWS charges per data transfer.

We access one of the VPCs using a VPN server (specifically an OpenVPN). The other VPC is managed by a provider.

This post applies to any VPCs, but our setup was:

1. AWS VPC with our docker cluster.
2. Atlas VPC with a MongoDB cluster.
3. Traffic goes through the Internet.

As we are connecting an Atlas VPC to an AWS VPC, I am going to use those names (although they both were in AWS of course).

Steps to connect them:

1. Peer the VPCs
2. Route the traffic from one VPC to another
3. Route your clients through the VPN

Peering the VPCs:

1. You have to start the process from one of the VPC. In our case, we started it from Atlas. You have to provide the other vpc id, the region, and VPC CIDR. For example vpc-1aa12345, eu-central-1, 192.16.8.0.0/16
2. You have to make sure that you have DNS hostnames enabled in the target VPC. That makes the DNS of your VPC resolve internally the hostname of your cluster.
3. Now in your target VPC under AWS > VPC > Peering connections you will find the peering to Atlas.
4. Now you may secure your Atlas cluster allowing access only to some target groups in your AWS VPC.

With this configuration, every host you have deployed in your AWS VPC will have access to the Atlas VPC (and vice versa). The magic happens at DNS level: If you resolve one of the hostnames from Atlas locally, you will get a public IP. If you do it from within VPC, you will get an internal IP.

Accessing the second VPC from the VPN

Our OpenVPN server was configured to access the AWS VPC. Our clients connect to that VPN server and then they route all the traffic to that VPC through the VPN server. If you are wondering how the OpenVPN client works, you should check your network interfaces :) You will find that the traffic to the internal IPs is routed through that interface.

For example, in macOS you can find those routes:

❯ netstat -rn |grep utun16
192.168     11.7.0.20          UGSc           utun16

So… how do we make our VPN route traffic to the new peered VPC? Just add the rule to /etc/openvpn/server.conf. You must have a previous rule there O:)

# This is AWS VPC
push "route 192.168.0.0 255.255.0.0"

# This is Atlas VPC
push "route 172.31.0.0 255.255.0.0"`

Now the key. How do we make our clients resolve the hostnames from the peered VPC? Just use the internal DNS from the VPC. This is where the hostnames are registered.

# Add AWS VPC DNS server so we can use internal mappings
push "dhcp-option DNS 192.168.0.2"

This overwrites our local /etc/resolv.conf to use that DNS server. If you want to add more DNS servers just in case the client is not able to restore back your configuration, you can add a fallback rule (for example, use the Google DNSs or any other public server)

push "dhcp-option DNS 192.168.0.2"
push "dhcp-option DNS 8.8.8.8"

Disclaimer: Those are not our real IPs. You should also check what’s your internal DNS, don’t assume it’s the IP that ends in 2.