Routing, BGP, and the Internet's Geography

We have spent eleven chapters at the level of cables, packets, layers, and addresses. This chapter zooms out one notch further. The internet is not a single network. It is a network of networks, and the way they talk to each other determines almost everything you experience as a user.

Hub, switch, router, gateway — the device chain, properly.

We have used these words loosely. Let me line them up cleanly, one at each layer.

A hub operates at Layer 1. It is electrically dumb. A bit on any port goes out every other port. It does not know who anyone is.

A switch operates at Layer 2. It learns MAC addresses and forwards frames only to the right port. It does not know about IP addresses at all.

A router operates at Layer 3. It looks at IP addresses, consults a routing table, and forwards packets toward their destination. A switch can talk only to the machines plugged into it. A router stitches different networks together, including the public internet.

Above that, you encounter L4 devices — application gateways, load balancers, web application firewalls. They inspect TCP, UDP, HTTP, and sometimes the contents of an HTTP request itself. A UTM ("unified threat management") firewall, for example, can say "block any HTTP traffic to this destination, regardless of which port it uses." It is operating on the protocol, not just the addresses.

In a real corporate or data-center network, you will find all of these in a chain: ports on a switch, switches uplinked to a router, the router uplinked to a gateway, the gateway connected to the internet. Each layer's job is to ignore what the layer below already handled.

Can two servers share one public IP address?

A question worth chewing on. The standard answer is no — IP addresses are unique on the public internet by design, otherwise routing collapses into ambiguity.

The real answer is yes, with the right routing trick. The technique is called anycast.

In anycast, multiple servers in different physical locations advertise the same IP address into the global routing fabric. Each one says, "I can reach 192.0.2.1." The routers of the world build their tables based on these announcements via BGP — which we are about to meet — and each router independently decides which announcement is "nearest" based on routing cost. When a packet for 192.0.2.1 arrives at a router, the router sends it to whichever server it considers closest. Different routers will make different decisions. The same IP address resolves to a different physical machine depending on where you are sitting.

This is how 8.8.8.8 (Google's public DNS) and 1.1.1.1 (Cloudflare's) work. They are not a single server in California. They are hundreds of servers around the world, all advertising the same address. The reason your ping 8.8.8.8 returns in 5 ms from Bangalore and also in 5 ms from London is that you are hitting two completely different machines, neither of which is in California.

CDNs do the same thing at higher layers. So do large cloud providers for their global edges. Anycast turns one IP address into a global service.

BGP — the routing protocol that holds the internet together.

Border Gateway Protocol is the agreement by which independently-run networks tell each other what they can reach.

An autonomous system (AS) is a network under one administrative authority — Comcast, Reliance Jio, Cloudflare, your university. Each AS has a number (Cloudflare is AS13335, Google is AS15169). When two ASes peer with each other at a physical exchange point, they exchange BGP announcements: "I can reach these IP ranges. Here is the AS path to get there."

Each router on each AS aggregates these announcements into a routing table. When a packet arrives, the router looks up the destination in its table and forwards it via whichever neighboring AS offered the best path according to the AS's local policy.

A few important truths about BGP.

It does not optimize for latency by default. It optimizes for policy — peering agreements, transit costs, contractual preferences. A path that looks short on a map can be much longer in practice because the announcer's policy preferred sending you through three countries away.

It is trust-based. There is no central authority that signs announcements. Historically, this has caused real outages — a misconfigured ISP announces "I can reach the entire internet" and the internet routes itself into a hole for an hour. There are emerging mechanisms (RPKI, BGPsec) to fix this, but BGP's openness is also its strength: it lets anyone join.

It is automatic. As soon as a cable goes down or a peering relationship changes, BGP propagates the new state of the world to every router within seconds. The route through the failed link disappears; alternative paths take over. Most of the time you never notice. The internet is, at its core, a system of routers continuously gossiping about reachability.

The physical geography of the internet.

Now zoom even further out.

Every country has a small number of international gateways — the physical points where its domestic network connects to the world. The submarine cables we met in chapter 2 land at these gateways; the international transit traffic flows through them; the legal authority over that traffic is exercised at them.

India has roughly seven such gateways, depending on the source: Mumbai, Chennai, Kolkata, Kochi, and a few more. Every packet you send from your laptop in Bangalore to a server in Frankfurt physically exits the country through one of those points. The geography is the architecture.

This has consequences.

It is a latency floor. The shortest possible path from Bangalore to Frankfurt is constrained by how the cables actually run from Mumbai or Chennai. No software optimization beats that geometry.

It is a policy choke point. If a government wants to throttle, block, or monitor international traffic, it does not have to touch every laptop. It can act at seven points. This is true in India, in China, in every country with serious internet infrastructure. It is one of the most important and least discussed facts about the modern internet.

It is a resilience problem. In 2025 there were credible threats — geopolitical, military — to undersea cables in several regions. If enough cables are cut, the relevant traffic falls back to satellites, which are slower, lower-bandwidth, and not designed for the full traffic load. The internet does not collapse. It degrades.

IReC and the domestic backbone.

India's response, like many large countries', was to build an internal network. The Indian Research and Educational Network (and related domestic backbones) interconnects major metros so that traffic between, say, Bangalore and Delhi does not have to leave the country.

Before this, traffic between two Indian cities often physically traveled to Singapore and back, because that was where the regional internet exchange lived. Latency was bad and sovereignty was worse. Building an internal backbone is a strategic act — it gives the country control over its own routing in a way that submarine cables alone cannot.

When you design an application for a global audience, this geometry matters. You cannot serve Brazilian users from a Frankfurt data center with snappy latency, because the packet has to cross an ocean. You cannot serve users in countries with strict gateway controls from outside those countries without dealing with the gateway. Where your servers physically live is, at scale, the most consequential technical decision you will make.

Inside the room: rack vs. blade, and the cooling math.

If you have ever toured a data center, you have seen two main form factors of server hardware.

A rack server is a single, self-contained machine — a chassis with its own power, its own disks, its own ports — that you slide into one slot of a 42U rack. They are general-purpose, easy to swap, and the workhorse of most data centers.

A blade server is more compact. A blade chassis holds the shared components (power, cooling, networking, sometimes storage), and individual blades plug in to share them. Where you might fit three rack servers in a given space, you can fit sixteen blades. Density is the point.

Either way, the actual constraint is power, not floor space. Heat is energy loss. Every watt of electricity that goes into a CPU or GPU eventually exits as heat. A rack full of idle servers needs barely any cooling. A rack full of saturated AI accelerators needs a small power plant's worth.

This is the math behind the AI data-center boom. The newest GPU racks consume tens of kilowatts each. The cooling system has to remove all of that heat in real time, or the chips throttle. Many of the world's new data centers are being built next to cheap, reliable power and cheap, abundant cooling water. Geography again.

Back to the ROI lesson.

It is tempting, reading all this, to want to rip everything out and replace it with the newest, most capable version. New fiber. New IPv6 everywhere. New gateways. New cooling.

Production infrastructure does not work that way. Every running system has paid for itself in part with an assumed lifetime; every business has a budget for the next migration, not for an unscheduled one. A senior engineer reads the gap between "what the system could be" and "what it currently is" and weighs the cost of closing the gap against the return of closing it. If the return is real, they make the case. If the return is just aesthetic, they let it lie.

The same logic explains why IPv6 has been "the future" for twenty years, why NAT is a load-bearing hack we never quite retired, and why your office is still wired with copper. The economics of upgrading rarely match the economics of inventing.

Push On It

1. Run traceroute (or mtr) to a server in another country. Identify which hop is the international gateway from your country. How does that hop's latency compare to all the hops before and after it? That jump is the geography.
2. Look up your country's international internet gateways. Sketch them on a map. Now sketch which undersea cables land where. Where would a single, well-placed accident leave your country with degraded internet?
3. dig +short google.com from two different networks (your laptop on Wi-Fi, then on cellular). Did you get the same IP? Look up what anycast and GeoDNS are doing to your answer.
4. Find the public BGP looking-glass service for any major network. Look up the AS path from that network to your domain. How many ASes does the packet cross? Are any of them in countries you would not have guessed?