Five applications, one architecture repeated four times and rejected once, and an interface narrow enough that nine lines of Python are a working network program.
What this page is for
- A map of everything the chapter introduced, with every node linking back to the section that taught it.
- A recap of the argument, in the order the chapter made it.
- A comparison of the five applications on the axes that actually separate them.
- An honest statement of what chapter 2 did not answer, which is what chapter 3 is for.
Why this matters
This is the end of the first layer. Everything from here down — transport, network, link — exists to support what you have just read.
That is worth pausing on before continuing. The applications came first in this book on purpose. When chapter 3 explains how TCP (Transmission Control Protocol) Transmission Control Protocol The Internet transport protocol that delivers data reliably and in order, with flow control and congestion control. introduced in ch. 1 recovers a lost segment, you already know which applications were counting on it — and what they were counting on it for.
The map
Hover an idea to see what it connects to. Click one to go back to the section that taught it.
Hover a node to dim everything it does not touch. Click one to go back to the section that taught it. Notice how many arrows point at caching.
Read this map as a list
- application architecture (§2.1.1) — connects to client-server, peer-to-peer
- client-server (§2.1.1) — connects to application architecture, HTTP · the Web, SMTP · e-mail, DNS, DASH · video
- peer-to-peer (§2.1.1) — connects to application architecture, BitTorrent
- processes and sockets (§2.1.2) — connects to TCP or UDP, UDP sockets, TCP sockets
- TCP or UDP (§2.1.4) — connects to processes and sockets, application-layer protocol, UDP sockets, TCP sockets
- application-layer protocol (§2.1.5) — connects to TCP or UDP, HTTP · the Web, SMTP · e-mail, DNS
- HTTP · the Web (§2.2) — connects to client-server, application-layer protocol, HTTP/2, persistent connections, caching, DASH · video
- HTTP/2 (§2.2.6) — connects to HTTP · the Web
- SMTP · e-mail (§2.3) — connects to client-server, application-layer protocol, DNS
- DNS (§2.4) — connects to client-server, application-layer protocol, DNS caching, CDNs, SMTP · e-mail
- BitTorrent (§2.5) — connects to peer-to-peer, self-scalability
- DASH · video (§2.6) — connects to client-server, HTTP · the Web, CDNs
- CDNs (§2.6.3) — connects to caching, DNS, DASH · video
- persistent connections (§2.2.2) — connects to HTTP · the Web
- caching (§2.2.5) — connects to HTTP · the Web, CDNs
- DNS caching (§2.4.2) — connects to DNS
- self-scalability (§2.5) — connects to BitTorrent
- UDP sockets (§2.7.1) — connects to TCP or UDP, processes and sockets
- TCP sockets (§2.7.2) — connects to TCP or UDP, processes and sockets
The recap
The ground rules
Applications run only on end systems. Network-core devices work at the network layer and below, so there is no application layer inside a router for your code to live in. That is why anyone can deploy a new application without asking permission from anyone who owns a router.
The developer picks an application architecture application architecture Simple How an application spreads its work across the end systems. The developer chooses it. Precise The structure, designed by the application developer, that dictates how the application is organised over the various end systems. It is distinctly different from the network architecture (the fixed five-layer Internet architecture), which from the developer’s point of view is given and provides a specific set of services. introduced in ch. 2 — open in glossary .
A client-server client-server architecture Simple One always-on server answers many clients. The clients never talk to each other. Precise An application architecture with an always-on host, called the server, which services requests from many other hosts, called clients. Clients do not directly communicate with each other, and the server has a fixed, well-known IP address, so a client can always contact it. The Web, FTP, Telnet and e-mail use this architecture. introduced in ch. 2 — open in glossary design has an always-on server with a fixed, well-known address, and clients that never talk to each other.
A peer-to-peer peer-to-peer architecture Simple Users’ own machines send files straight to each other, with little or no help from servers. Precise An application architecture with minimal or no reliance on dedicated servers in data centres. The application exploits direct communication between pairs of intermittently connected hosts, called peers, which are controlled by users rather than owned by the service provider. BitTorrent is the best-known example. introduced in ch. 2 — open in glossary design has users’ own intermittently connected machines serving each other.
Programs do not communicate; processes do, through sockets socket Simple The door of a program, through which its messages leave and enter the network. Precise The software interface between the application layer and the transport layer within a host, through which a process sends messages into, and receives messages from, the network. It is the concrete thing a program holds; the socket interface of §1.1 is the service it offers. It is also called the Application Programming Interface between the application and the network. The developer controls everything on the application side of the socket and almost nothing on the transport side. introduced in ch. 2 — open in glossary . Reaching one needs two numbers: an IP (Internet Protocol) Internet Protocol The network-layer protocol that defines the datagram format and addressing every Internet device must use. introduced in ch. 1 address for the host and a port number for the socket. The developer’s one real choice is the transport protocol.
In plain words — the gap the whole chapter works around
A transport protocol could offer four things: reliable data transfer, throughput, timing and security. Today’s Internet offers one and a half.
TCP gives reliable data transfer. Security is available, but only because the application adds TLS (Transport Layer Security) Transport Layer Security Encryption, integrity and authentication added above TCP, in the application layer. introduced in ch. 2 above the socket. Throughput and timing guarantees do not exist, and every design decision in the rest of this chapter is made in that knowledge.
The five applications
| The WebHTTP · §2.2 | E-mailSMTP · §2.3 | DNS§2.4 | BitTorrent§2.5 | VideoDASH · §2.6 | |
|---|---|---|---|---|---|
| Architecture | |||||
| Transport | |||||
| Port | |||||
| Who keeps state | |||||
| How it gets content near you | |||||
| Its hardest problem |
Cells marked ⓘ have a reason behind them — click to read it.
Read down a column for one application, or across a row to see the same decision made five different ways. Click any ⓘ.
The Web. HTTP (HyperText Transfer Protocol) HyperText Transfer Protocol The application-layer protocol that requests and transfers Web documents. introduced in ch. 1 is stateless, plain ASCII (American Standard Code for Information Interchange) American Standard Code for Information Interchange The 7-bit text encoding to which SMTP restricts mail bodies. introduced in ch. 2 text, and runs over TCP on port 80. Its engineering content is the cost of a connection: two round-trip times per object with non-persistent connections, three round trips for a whole page with persistent connections and pipelining. Cookies rebuild state without changing the protocol. Caches cut both response time and traffic on the access link. HTTP/2 breaks messages into interleaved HTTP/2 frames, which removes head-of-line blocking and the browser’s incentive to open six connections at once.
E-mail. Three components — user agents, mail servers, SMTP (Simple Mail Transfer Protocol) Simple Mail Transfer Protocol The application-layer protocol that transfers e-mail messages. introduced in ch. 1 — and a protocol from 1982 that still restricts message bodies to 7-bit ASCII. Mail travels directly from the sender’s mail server to the recipient’s, with no intermediate server, and waits in a queue if the far side is down. Because SMTP pushes, a separate mail access protocol is needed to pull: HTTP for web mail, IMAP (Internet Mail Access Protocol) Internet Mail Access Protocol Lets a user agent read and manage a mailbox held on a mail server (RFC 3501). introduced in ch. 2 for clients.
DNS. The chapter’s best distributed database. A centralized design fails four ways. So DNS is a hierarchy of root, TLD (Top-Level Domain) Top-Level Domain The last label of a hostname — com, edu, uk — and the DNS servers responsible for it. introduced in ch. 2 and authoritative servers, fronted by a local DNS server that queries on your behalf and caches what it learns. A cold lookup costs eight messages; caching turns that into zero for the next person to ask.
BitTorrent. The only peer-to-peer application here, and the only one whose hardest problem is human. Rarest first equalises how many copies of each chunk exist; tit-for-tat makes contributing worth more than freeriding.
Video. About 80 % of Internet traffic in 2020. DASH (Dynamic Adaptive Streaming over HTTP) Dynamic Adaptive Streaming over HTTP Chunk-by-chunk quality selection driven by the throughput the client measures. introduced in ch. 2 stores several qualities and lets the client choose chunk by chunk; content distribution networks content distribution network Simple A fleet of servers spread around the world, each holding copies of the content. Precise A CDN manages servers in multiple geographically distributed locations and stores copies of videos and other Web content in those servers. It then tries to direct each user request to the CDN location that will give the best user experience. It may be a private CDN owned by the content provider itself, such as Google’s or Netflix’s, or a third-party CDN serving many providers, such as Akamai or Limelight. introduced in ch. 2 — open in glossary put the copies near the viewer. Netflix and YouTube both run huge private CDNs and disagree on almost every mechanism inside them.
And then you write it yourself
Section 2.7 puts it on the wire. Nine lines of Python are a working
UDP (User Datagram Protocol) User Datagram Protocol A simple transport protocol with no reliability, no flow control and no congestion control.
introduced in ch. 1 client. SOCK_DGRAM becomes SOCK_STREAM, sendto becomes send, and a
connect() appears — and the same exchange goes from two packets to eight,
because six of them are the connection.
One idea appears at three scales — notice it
Put a copy nearer the user.
- A local DNS server caches a hostname-to-address mapping, and the root servers are bypassed for all but a very small fraction of queries.
- A proxy cache holds objects for an institution, cutting the average response time from minutes to 1.21 seconds without buying a faster link.
- A CDN holds video in hundreds of locations worldwide, so the bytes never cross an ocean at all.
Three sections, three orders of magnitude, one idea. When you meet it again in §5.4 and §6.7, you will already know its shape.
The road ahead
What the book says next
At the very beginning of this book, in Section 1.1, we gave a rather vague, bare-bones definition of a protocol… The material in this chapter, and in particular our detailed study of the HTTP, SMTP, and DNS protocols, has now added considerable substance to this definition.
And then, honestly, what it did not do:
we know that TCP provides a reliable data service, but we haven’t said yet how it does so. In the next chapter, we’ll take a careful look at not only the what, but also the how and why of transport protocols.
In plain words
This chapter told you what the transport layer offers and let you build on it. It said nothing about how any of it works.
You know TCP delivers every byte in order. You have not been told how it does that over a network that drops packets, reorders them and has no memory. That is chapter 3, and it is the longest chapter in the book for a reason.
Check yourself
Check yourself
0 of 6 answered1.Four of the five applications in this chapter are client-server. Which is not, and what does that buy it?
2.Three of this chapter's protocols run over TCP and one over UDP. Which is which, and why?
3.The chapter shows the same idea — put a copy nearer the user — at three different scales. Where?
4.Section 1.1 defined a protocol as the format and order of messages, plus the actions taken. Which part of this chapter added the most substance to "actions taken"?
5.What does chapter 3 promise that this chapter deliberately did not deliver?
6.Which claim about this chapter's five applications is true?
The chapter in three lines
- Applications run only on end systems. Nothing in this chapter lives in the network core — not even DNS, which implements a core function at the edge.
- A transport protocol could offer four services; the Internet offers reliable data transfer, and security only if you add TLS yourself. Throughput and timing guarantees do not exist.
- The recurring idea is put a copy nearer the user, at three scales: DNS caching, web caching, CDNs.