The Internet is not one network but hundreds of thousands of independently owned networks, arranged in a hierarchy that money built.
Words you will meet
- Network of networks — the Internet’s real structure: many separate networks joined together.
- Access ISP (Internet Service Provider) Internet Service Provider Connects hosts, and other ISPs, to the rest of the Internet. introduced in ch. 1 — the ISP that reaches you.
- Regional ISP — an ISP covering an area, sitting between access ISPs and the top.
- Tier-1 ISP — one of about a dozen at the very top. They pay nobody.
- Customer–provider — the paying relationship: the lower one pays the higher.
- Peering — a direct link between two ISPs at the same level, usually free for both.
- IXP — a building where many ISPs peer with each other at once.
- PoP — a place in a provider’s network where customer ISPs plug in.
- Multi-homing — connecting to two or more providers, so one failure does not cut you off.
Why this matters
The book says understanding the phrase network of networks is the key to understanding the Internet. It is also the key to two later chapters.
Chapter 5 explains BGP (Border Gateway Protocol) Border Gateway Protocol The protocol ISPs use to tell each other which destinations they can reach; its rules follow the customer-provider and peering relationships. introduced in ch. 5 , the protocol these networks use to tell each other which destinations they can reach. BGP’s rules make no sense until you know who pays whom. Section 2.6 explains content distribution networks, which exist to shorten the path drawn in the last step below.
Why a hierarchy at all?
Hosts reach the Internet through an access ISP. That ISP might be a residential cable or telephone company, a mobile operator, a university providing access to its students, or a company providing it to its employees.
But connecting hosts into access ISPs is only a small piece of the puzzle. The access ISPs themselves must be interconnected. Doing that creates a network of networks network of networks Simple The Internet is not one network but many separate networks joined together. Precise The structure of today's Internet: hundreds of thousands of access ISPs interconnected through regional ISPs and about a dozen tier-1 ISPs, together with PoPs, multi-homing, peering, IXPs and content-provider networks. introduced in ch. 1 — open in glossary .
One approach would be to connect every access ISP directly to every other. That mesh is far too costly: each of hundreds of thousands of access ISPs would need a separate link to every other one, all over the world.
What actually shaped the Internet
The structure that emerged instead was driven by economics and national policy, rather than by performance considerations. It is not the arrangement an engineer would draw if asked for the fastest possible network. It is the arrangement that made commercial sense, one step at a time.
Keep that in mind as you step through the five structures below. Almost every addition exists to save somebody money.
Five steps to today’s Internet
One imaginary global transit ISP, with at least one router near every access ISP. Each access ISP pays it for connectivity, so the access ISP is the customer and the transit ISP is the provider. Building such a network really would cost a fortune — which is exactly why it could charge.
Amber links are customer → provider: the lower one pays. Teal links are settlement-free peering: neither pays. Almost every step below is added to save somebody money.
Read all steps as text
- Structure 1 — One imaginary global transit ISP, with at least one router near every access ISP. Each access ISP pays it for connectivity, so the access ISP is the customer and the transit ISP is the provider. Building such a network really would cost a fortune — which is exactly why it could charge.
- Structure 2 — If one global transit ISP is profitable, competitors appear. Access ISPs prefer this: they can now choose on price and service. But the transit ISPs must interconnect with each other, or their customers could not reach one another. This is a two-tier hierarchy.
- Structure 3 — In reality no ISP has a presence in every city. So regional ISPs appear in between: access ISPs pay a regional ISP, and regional ISPs pay a tier-1 ISP. There are about a dozen tier-1 ISPs, and they pay nobody. In some countries the ladder is longer still — in China, city access ISPs connect to provincial ISPs, then national ISPs, then tier-1.
- Structure 4 — Four additions, and every one of them saves money or adds resilience. PoPs are the places where a customer plugs into a provider. Multi-homing means connecting to two providers, so one failure does not cut you off. Peering is a direct, settlement-free link between ISPs at the same level. An IXP is a building where many ISPs peer at once — there are over 600 of them.
- Structure 5 — Today’s Internet. A content provider such as Google runs its own private network joining its data centres, separate from the public Internet and carrying only its own traffic. It peers settlement-free with lower-tier ISPs, directly or at IXPs, to bypass the tiers it would otherwise pay. But many access ISPs can only be reached through a tier-1 network, so it still buys transit there too.
What each step added
Structure 1 connects all access ISPs through a single global transit ISP. Because the access ISP pays, the access ISP is the customer and the transit ISP is the provider. The pricing reflects how much traffic the access ISP exchanges — though not necessarily in direct proportion.
Structure 2 adds competition. Once one global transit ISP is profitable, others build their own. Access ISPs prefer this, because they can now choose between providers on price and service. Note the requirement this creates: the global transit ISPs must interconnect with each other, or their respective customers could not reach one another.
Structure 3 faces reality. No ISP has a presence in every city in the world. So regional ISPs appear in the middle: access ISPs connect to a regional ISP, and regional ISPs connect to a tier-1 ISP tier-1 isp Simple One of about a dozen very large ISPs at the top; they pay nobody for transit. Precise An ISP at the top of the Internet hierarchy, consisting of high-speed routers interconnected with high-speed fiber-optic links. Tier-1 ISPs pay no provider and peer settlement-free with one another. No group officially sanctions tier-1 status. introduced in ch. 1 — open in glossary . There are about a dozen tier-1 ISPs — Level 3 Communications, AT&T, Sprint and NTT among them. In some countries the ladder is longer: in China, city access ISPs connect to provincial ISPs, which connect to national ISPs, which connect to tier-1 ISPs.
Nobody hands out tier-1 status
No group officially sanctions membership of tier-1. The book quotes the saying: if you have to ask whether you are a member of a group, you probably are not.
The practical test is simpler than any certificate. A tier-1 ISP pays no provider for connectivity. It reaches everywhere through settlement-free peering with the other tier-1 ISPs.
Structure 4 adds four things at once.
-
A PoP (Point of Presence) Point of Presence A place in a provider ISP's network where customer ISPs can connect in. introduced in ch. 1 point of presence Simple A place in a provider's network where customer networks can connect in. Precise A group of one or more routers at the same location in a provider ISP's network, where customer ISPs can connect into the provider. PoPs exist at all levels of the hierarchy except the bottom, access-ISP level. introduced in ch. 1 — open in glossary is a group of one or more routers at one location in a provider’s network, where customer ISPs connect in. A customer leases a high-speed link from a third-party telecommunications company to reach a router at the PoP. PoPs exist at every level except the bottom, access-ISP level.
-
Multi-homing multi-homing Simple Connecting to two or more providers, so you stay online if one of them fails. Precise The practice of an ISP connecting to two or more provider ISPs. Any ISP except a tier-1 ISP may multi-home; it can then continue to send and receive packets even if one of its providers has a failure. introduced in ch. 1 — open in glossary means connecting to two or more provider ISPs. Any ISP except a tier-1 may do it. The reward is that the ISP keeps working even when one of its providers fails.
-
Peering peering Simple Two ISPs connect directly so their traffic does not have to travel through a third one. Precise A direct connection between a pair of nearby ISPs at the same level of the hierarchy, so that all traffic between them passes over that direct connection rather than through upstream intermediaries. Peering is typically settlement-free: neither ISP pays the other. introduced in ch. 1 — open in glossary is a direct connection between a pair of nearby ISPs at the same level of the hierarchy. Their mutual traffic then travels over that link instead of through upstream intermediaries. Peering is typically settlement-free: neither ISP pays the other.
-
An IXP (Internet Exchange Point) Internet Exchange Point A building where many ISPs peer with each other; there are over 600 today. introduced in ch. 1 internet exchange point Simple A building where many ISPs plug into each other in one place. Precise A meeting point, created by a third-party company, where multiple ISPs can peer together. An IXP is typically a stand-alone building with its own switches; there are over 600 in the Internet today. introduced in ch. 1 — open in glossary is a meeting point created by a third-party company, where many ISPs can peer at once. It is usually a stand-alone building with its own switches. There are over 600 in the Internet today.
Structure 5 is today’s Internet, and it adds content-provider networks.
The content provider’s shortcut
Google is the book’s example. It has 19 major data centres across North America, Europe, Asia, South America and Australia, each with tens or hundreds of thousands of servers. It also runs smaller data centres of a few hundred servers, often placed inside IXPs.
All of them are joined by Google’s own private network, which spans the globe but is separate from the public Internet and carries only traffic to and from Google’s servers.
Everyday picture
A large retailer that runs its own lorry fleet instead of paying a courier. It builds depots near the big towns and drives its own goods directly there. For the few remote villages it still cannot reach, it hands the parcels to the national courier and pays the fee.
That is exactly the shape of the last diagram: Google peers directly with lower-tier ISPs where it can, and buys transit from tier-1 ISPs for everything else.
Where the picture breaks: lorries on a road do not slow each other down much, but traffic on a shared Internet link genuinely competes. That competition is what §1.4 measures.
Its private network attempts to bypass the upper tiers by peering settlement-free with lower-tier ISPs, either directly or at IXPs. But many access ISPs can still only be reached by transiting a tier-1 network, so Google also connects to tier-1 ISPs and pays them for that traffic.
Building its own network gives a content provider two things: smaller payments to upper-tier ISPs, and greater control over how its services finally reach users. Section 2.6 returns to this in detail.
The four kinds of network
| Access ISPreaches you | Regional ISPcovers an area | Tier-1 ISPabout a dozen exist | Content providere.g. Google | |
|---|---|---|---|---|
| Who it connects | ||||
| Pays | ||||
| Paid by | ||||
| Reach | ||||
| Peers settlement-free? |
Cells marked ⓘ have a reason behind them — click to read it.
Read the “Pays” row down the table. That single column is most of why the Internet has the shape it does.
In plain words — the whole section in four lines
- Users and content providers are customers of lower-tier ISPs.
- Lower-tier ISPs are customers of higher-tier ISPs.
- Higher-tier ISPs interconnect with one another, paying nobody.
- Wherever two networks can avoid paying a third, they connect directly.
Check yourself
Check yourself — §1.3.3
0 of 6 answered1.Why does the Internet not simply connect every access ISP directly to every other?
2.In a customer–provider relationship between two ISPs, who pays?
3.predictStep from Network Structure 3 to Structure 4. What do the new teal links between two regional ISPs achieve?
Read the note under the diagram at each step, and watch which links appear.
4.Why does an ISP multi-home?
5.Google's private network carries only traffic to and from Google's own servers. Why build it?
6.What does the book say actually drove the Internet into this shape?
What to remember
- The Internet is a network of networks: about a dozen tier-1 ISPs and hundreds of thousands of lower-tier ones, each independently managed.
- Customer–provider: the lower ISP pays. Peering: two ISPs at the same level connect directly and neither pays. Tier-1 ISPs pay nobody, and that is what makes them tier-1.
- Content-provider networks run their own global private network and peer around the paid tiers wherever they can, buying transit only for what is left.