Migration of IT resources to the public cloud is a strategic decision for companies nowadays. In most of the cases not all the IT resources (servers, storage, applications, etc) are migrated, thats is, some remain on premises (in a company datacenter, private cloud or in a hosted facility).
In many cases, it might be required to connect the resources moved to the public cloud with the ones remaining on-premises and there many options depending on the cloud provider and existing technology.
- The Big Picture
The need of having applications / solutions that have interacting components on-premises (i.e. datacenter) and in public clouds (i.e. AWS), give businesses great flexibility and data deployment options. This hybrid approach brings benefits but also present technical, business and management challenges. Key aspects that need to be considered include security and a solid network “connection” because the applications’ components must be able to talk to each other over a reliable, fast and secure channel. In some cases, there are stric requirements in terms of latency, security, etc.
This “connection” is usually done using a secure path over the internet as shown in the figure 1. As seen, the data between the cloud “VPN gateway” and the on-promises router is encrypted but the link is not dedicated. It means, Internet is used, which sometimes falls short in terms of speed, latency, security, and manageability.
The good news is that public cloud providers (like AWS, Azure and GCP) offer the possibility of establishing a dedicated and resilient network connection between your on-premises and cloud resources using private WAN links as show in Figure 2. That is, the cloud provider “extends” its reach to customers by having a Point-of-Presence (POP), usually co-located in an ISP datacenter. AWS calls it Direct Connect (DX), Azure calls it ExpressRoute and Google’s GCP call it Dedicated Interconnect.
- Connection to resources on AWS - Direct Connect
AWS Direct Connect (DX) allows to establish a dedicated network link between your on-premise resources (at corporate office or data center) and the resources on the AWS cloud.
In this case you connect to one Direct Connect partner location, usually an Internet Service Provider (ISP) where AWS has resources, getting access to the AWS cloud within that specific AWS geographic region. The connection is done using an 802.1q trunk between your datacenter and the ISP, allowing to use VLANs to have logical separation between public and private cloud resources. There are two flavors of DX:
- Dedicated: Ordered directly from AWS, with speeds of 1 Gbps or 10 Gbps
- Hosted: Ordered from an AWS DX partner, with speeds from 50M to 10Gbps.
- Microsoft Azure ExpressRoute
Microsoft’s ExpressRoute works as an extension of a private network to any of Microsoft’s cloud services, including Azure and Office365.
Similarly to AWS, you connect to an ExpressRoute peering location using a layer 3 redundant connection (as opposed to layer 2). It also uses BGP to exchange routes between Azure and the customer network. Bandwidth supported options are similar to the ones with AWS. With ExpressRoute Premium you can extend the connectivity to an entire geopolitical boundary, such as North America and not only to a region.
- Google Dedicated Interconnect
Google’s GCP Cloud Dedicated Interconnect is available in many global locations and supports speeds from 10 Gbps up to 100 Gbps. By default, it allows access to the entire global cloud network . For provisioning, you connect your “customer” router to a Google Cloud Router at a common colocation facility, and configure BGP for exchanging routing information.
- Real Life Scenario with AWS’ Direct Connect
For the sake of illustrating the usage of Direct Connect, let’s use a real-life scenario we recently had with one of our customers:
Summary of Requirements:
- Customer has workloads in USA on a specific AWS region. These workloads run on virtual servers (EC2) and EKS (AWS’ managed Kubernetes service for containers).
- The customer has operations in Europe, where it has a small datacenter.
- Applications running on EC2 and EKS in USA needed to communicate to applications running on the remote servers in Europe (different AWS Region).
- Customer wanted to have a 200 Mbps, resilient, secure connectivity between AWS resources in USA and datacenter in Europe. The local ISP in Europe is an AWS partner that provides hosted DX service with this speed, as well as connectivity from its facilities to the customer datacenter using an MPLS network.
- Traffic needed to be encrypted using a specific VPN tunneling option deployed on Cisco Routers.
- The EKS machines (workers) used private addressing.
Direct Connect can be used with multiple use cases.
- Solution Overview
In order to comply with the strict connectivity requirements between locations was required to have a dedicated network connection from the AWS Point of presence (POP) and the customer datacenter in Europe. That is, a private circuit from the local ISP (AWS Direct Connect POP) and our customer site. A Hosted DX with 200 Mbps was ordered to comply with the requirements.
On the other hand, in order to comply with specific traffic encryption requirements, it was necessary to implement a VPN using a virtual Cisco Router (CSR) on AWS. On this virtual device terminated the VPN tunnels established between AWS and the remote datacenter. The details are covered later.
Finally, the private addressing used by EKS’ workers, required to implement NAT and other advanced networking functionality available on AWS. This machines had private addressing but also needed to communicate to the internet for specific functionality related to EKS, outside of the scope of this article.
- Direct Connect (DX) Details
For DX to work, it is established a Layer 2 link (using a 802.1q trunk) between your datacenter and an AWS endpoint (a.k.a. DX router) co-located at the ISP. This trunk can have multiple VLANs, allowing to use the same connection to access public resources (i.e. Amazon S3, Glacier) using public IP address space, and private resources (i.e. EC2 instances, EKS, etc) running within an Amazon Virtual Private Cloud (VPC) using private IP space, while maintaining network separation between the public and private environments. The following diagram shows these components:
These VLANs terminate on Virtual interfaces (VIF) on the DX Router co-located at the ISP. There is a VIF for each VLAN. Since our customer only required remote servers to access devices within the VPC (Private IP addressing), only the private VIF was needed.
BGP is the protocol used to exchange routes between DX and the customer datacenter. Specifically, it was configured an external BGP (eBGP) session (using AWS VIFs and the customer router’s IP), to exchange routes between the AWS Autonomous Systems (AS) and Customer network’s AS.
As mentioned before, there are two flavors of DX: dedicated and hosted that allow having several combinations including:
Dedicated:
- Ordered directly from AWS, with speeds of 1 Gbps or 10 Gbps, and support of multiple virtual interfaces
- Can bundle multiple 1G or 10G ports into a single managed connection called LAG (Link aggregation) that will load-balance traffic across the links, per flow. You can have up to 4 links in a LAG.
Hosted:
Provided by an AWS partner (ISP).
- The ISP orders a 1G or 10G connection to AWS and then it can offer it to multiple end-customers, assigning chunks from from 50M to 10Gbps.
- It supports a single virtual interface.
Due to the requirements, a hosted DX was used. Since this was ordered from an ISP, there was a specific procedure that should be followed. The procedure is covered at: https://docs.aws.amazon.com/directconnect/latest/UserGuide/WorkingWithConnections.html
Please notice that there are multiple options for physical connectivity to the DX service:
- Customer presence in the same DX location
- Circuit between customer data center and DX location
- Service provider network extending to DX location
For our particular case, the service provider extended its network using MPLS. in these cases, the ISP usually takes care of this configuration because it is a DX hosted service.
The procedure for ordering and configuring Direct Connect as well as configuring the router at the Customer datacenter was straightforward and is well covered in AWS documentation found at: https://docs.aws.amazon.com/directconnect/
- VPN for Data Encryption
AWS offers the possibility of configuring a site-to-site Virtual Private Network (VPN) using the IPSEC protocol. This VPN enables to securely connect the on-premises network or branch office site to your Amazon Virtual Private Cloud (Amazon VPC).
To terminate the VPN tunnels on AWS, you usually use an AWS Virtual Private Gateway, a logical, fully redundant distributed edge virtual device that sits at the edge of your VPC. As it is capable of terminating VPN connections from your on-prem or customer environments, the VPG is the VPN concentrator on the Amazon side of the Site-to-Site VPN connection.
You also have the option to terminate the VPN tunnel on a virtual appliance on AWS like the Cisco Cloud Services Router (CSR) 1000V. This is a router that runs on a virtual machine (EC2) and is configured on a similar way you configure a Cisco physical router. The performance of the CSR1000V depends on the size of the EC2 machine you use .
For our specific scenario, it was required to implement a redundant IPSEC VPN solution comprised of two tunnels (for more reliability) between the datacenter in Europe and the DX location. Specific requirements from the customer and the Service provider determined to use a Cisco Cloud Service Router (CSR) 1000V.
- Additional Cloud Networking Considerations
The combination of multiple functionalities like VPN support with CSR1000V, EKS with private addressing, etc implies having a VPC with Public and Private Subnets and AWS Site-to-Site VPN Access, and implement advanced networking features available on AWS like NAT, custom route tables, Access Lists and security. These topics will be covered in another article.
- Conclusion
Cloud providers have “extended” their networks to specific ISP’s datacenters from where is possible to have dedicated links that offer reliable, fast and secure communication to the cloud. Implementing a hybrid cloud approach might require to use these links in order to comply with customer requirements. This service may have different name depending on the cloud provider: AWS’ Direct Connect, Azure’s ExpressRoute and Google GCP’s Dedicated Interconnect. Implementing it, opens to multiple security, high availability and advanced networking deployment options and features.
Do you need help with Hybrid Cloud Networking?
Contact us today for a free consultation.
