Cloud networking at scale: hub and spoke and hybrid links
Lesson 3 of 5
3Cloud networking at scale: hub and spoke and hybrid links
The big pictureThe cloud makes creating a network a one-line job, which is exactly the danger: each team creates its network on the default 10.0.0.0/16, and a year later the company wants to join them to each other and to the data centre and finds they all overlap. A senior sets the plan before anyone starts, and builds a central network where everything passes, is seen and is inspected.
What the picture shows: Every team network (a spoke) connects only to the hub, and the hub connects to the inspection network and the data centre. Traffic between any two spokes crosses the hub, where it is seen and controlled.
Map of this part
The concepts in order, click any one to jump to it:
Two ways into the same idea
Picture an international hub airport. Instead of every small city flying to every other (thousands of routes), each flies to the hub and on from there. And the hub has one security check everyone passes. Cloud networks are built the same way: each team network is a spoke joined to a hub, and the hub joins the data centre over a steady dedicated line, or a VPN tunnel over the internet.
Peering between two networks is not transitive: link A to B and B to C and A still cannot reach C, and with n networks you need n(n-1)/2 peerings. AWS Transit Gateway and Azure Virtual WAN are routing hubs with route tables that keep production apart from development. Build an inspection network holding a firewall (such as AWS Network Firewall or a third-party one) for east-west and internet traffic to pass. Hybrid links: a Site-to-Site VPN (IPsec, up to about 1.25 Gbps per tunnel on AWS, quick to set up) or Direct Connect / ExpressRoute (a physical link with fixed capacities, steady latency, and weeks to arrange), often both together for backup, both exchanging routes over BGP. Hybrid DNS through inbound and outbound resolver endpoints. Private access (PrivateLink or Private Endpoint) reaches a service with no internet. And watch the cost: data moving between regions and to the internet, and NAT gateways, are billed by the gigabyte. AWS reserves 5 addresses in every subnet.
Before You Start
🌐Modern networks
You created a VPC with a public and a private subnet.
🌐BGP
Hybrid links exchange routes over BGP.
The Core Idea
1The address plan before anything
Give each region a slice of 10.0.0.0/8, each network a /16 or /20 by its size, reserve what the data centres and branches use, and never use the default 10.0.0.0/16 or 172.31.0.0/16 because every new account comes with them. The reason: two overlapping networks cannot be routed between without messy NAT, and re-addressing a live network means rebuilding everything in it.
Each region's /12 holds 16 /16 networks. The network a team made on the default clashes with the data centre at once.
import ipaddress
on_prem = [ipaddress.ip_network(n) for n in ("10.0.0.0/12", "192.168.0.0/16")] # data centres and branches
cloud = ipaddress.ip_network("10.64.0.0/10") # everything in the cloud
regions = dict(zip(["me-central", "eu-west", "us-east"], cloud.subnets(new_prefix=12)))
plan = {}
for region, block in regions.items():
nets = block.subnets(new_prefix=16)
for env in ("prod", "dev", "shared"):
plan[f"{region}/{env}"] = next(nets)
for name, net in plan.items():
clash = [str(o) for o in on_prem if net.overlaps(o)]
print(f"{name:<18} {str(net):<15}", "OVERLAP " + ", ".join(clash) if clash else "ok")
team_default = ipaddress.ip_network("10.0.0.0/16")
print("a team using the default 10.0.0.0/16 clashes with on-prem:", any(team_default.overlaps(o) for o in on_prem))
print("usable hosts in a /24 subnet on AWS:", ipaddress.ip_network("10.64.1.0/24").num_addresses - 5)2Why a hub?
Peering is simple for two or three networks. But it is not transitive, so every network needs a link to every other, and there is no central place to inspect. With a hub each network connects once, and the hub's route tables decide who reaches whom: production cannot see development, and anything bound for the internet passes the inspection network.
With 200 networks: about twenty thousand peerings, or 200 hub attachments.
for n in (3, 10, 50, 200):
print(f"{n:>3} networks: peering mesh {n * (n - 1) // 2:>6} links | hub {n:>3} attachments")3Linking the data centre: VPN or a dedicated link?
A VPN over the internet works within an hour and is cheap, but its latency moves with the internet and each tunnel's capacity is limited. A dedicated link (Direct Connect on AWS, ExpressRoute on Azure) is a physical cable through a telecom partner into the cloud: steady latency and 1 to 100 Gbps, but it takes weeks and costs more. A common design: two dedicated links in two different locations, a backup VPN, and BGP choosing.
| Site-to-Site VPN | Direct Connect · ExpressRoute | |
|---|---|---|
| Set-up | Hours | Weeks |
| Path | The public internet, encrypted | A private link |
| Latency | Variable | Steady |
| Capacity | About 1.25 Gbps per tunnel on AWS | 1 to 100 Gbps |
| Use | Getting started and backup | Large and critical production |
4Hybrid DNS, private access and cost
A server in the data centre needs to resolve private cloud names, and the other way round. The answer is inbound and outbound resolver endpoints and a forwarding rule per domain. Private access (PrivateLink) makes a service such as storage or a database appear at an internal address in your network, so its traffic never goes out to the internet. Cost is the hidden part: traffic between regions, to the internet and through NAT gateways is billed per gigabyte, so a poor design shuttling data back and forth can cost more than the servers.
# Terraform: the hub and two spokes, reviewed in Git like any code
resource "aws_ec2_transit_gateway" "hub" {
description = "core hub"
default_route_table_association = "disable" # every attachment picks its table on purpose
}
resource "aws_ec2_transit_gateway_vpc_attachment" "prod" {
transit_gateway_id = aws_ec2_transit_gateway.hub.id
vpc_id = aws_vpc.prod.id
subnet_ids = aws_subnet.prod_tgw[*].id
}Worked example
The task: a company will move 40 apps to the cloud across two regions, and owns a data centre on 10.0.0.0/12. Two teams already built their networks on 10.0.0.0/16. Design the network.
The answer: (1) The plan: the whole cloud is 10.64.0.0/10, a /12 per region, a /16 per environment. (2) The two teams' networks overlap the data centre: rebuild them now before they grow. (3) A Transit Gateway per region with a peering between the hubs, and separate route tables for production, development and shared. (4) An inspection network with a firewall for internet egress and traffic between environments. (5) Two Direct Connect links in two locations with a backup VPN, and BGP preferring the dedicated ones. (6) Resolver endpoints for hybrid DNS, and PrivateLink for storage. (7) All of it in Terraform with review in Git.
Myth vs Reality
| Myth | Reality |
|---|---|
| Cloud networks need no network engineer. | They need the same ideas: addressing, routing, BGP and firewalls, with different tools and costlier mistakes. |
| Peering passes routes on to every linked network. | Not transitive: every pair of networks needs its own link. |
| Traffic inside the cloud is free. | Traffic between regions, zones and through NAT is billed, and can become the biggest line on the bill. |
Where you meet it at work
🏢Moving to the cloud
The address plan and the hub are built before the first app.
🛡️Compliance
A central inspection network makes all egress logged and inspected.
💸Cutting the bill
Moving chatty services to PrivateLink and removing duplicate NAT gateways.
Test your understanding
You peered A with B and B with C. Can A reach C?
Each pair needs its own link, or you use a hub.
Why avoid the default 10.0.0.0/16?
Overlap blocks routing between networks later.
When do you pick Direct Connect or ExpressRoute over a VPN?
A dedicated link is steady and large, but slow to set up and unencrypted by default.
What is PrivateLink for?
The service appears inside your network so its traffic never crosses the internet.
How many addresses can you use in a /24 subnet on AWS?
AWS reserves 5 addresses in every subnet: 256 − 5.
Quick review cards
Check Your Understanding
Why fix overlap now rather than later?
Because re-addressing a network with dozens of services means rebuilding it, and the cost grows every day.
How does the hub stop development from reaching production?
Through separate route tables: the development table carries no production routes.
What is the risk of relying on one dedicated link?
A single point of failure; the fix is two links in two locations and a backup VPN with BGP.
Resources for this lesson
Grouped by type. Start with the official docs for precision, or an article or video for the explanation.
Done with this lesson?
Sign in to save your progress.