A transport protocol could offer an application four things — reliable data transfer, throughput, timing and security — and today’s Internet offers one and a half of them.
Words you will meet
- Reliable data transfer — a promise that everything sent arrives, complete and in order.
- Loss-tolerant application — one that still works when a little data goes missing.
- Bandwidth-sensitive application — one that needs a fixed rate or is no use at all.
- Elastic application — one that works with whatever rate it can get.
- Connection-oriented service — the two sides greet each other before data flows.
- Handshaking — that opening exchange.
- Congestion control — slowing a sender down when the network is overloaded.
- TLS — Transport Layer Security: encryption added above TCP.
- Application-layer protocol — the rules for what two programs say to each other, and when.
Why this matters
This is the section where you make the one decision the socket leaves you: TCP (Transmission Control Protocol) Transmission Control Protocol The Internet transport protocol that delivers data reliably and in order, with flow control and congestion control. introduced in ch. 1 or UDP (User Datagram Protocol) User Datagram Protocol A simple transport protocol with no reliability, no flow control and no congestion control. introduced in ch. 1 . Everything else about the transport layer is out of your hands.
It is also where the book states a gap that shapes the rest of the Internet. Two of the four services an application might want are not available at all. Chapter 3 explains how the two protocols that do exist are built. Sections 2.6 and 3.7 show what applications had to invent because the other two services were missing.
What a transport protocol could offer
A socket is the interface between the application process and the transport-layer protocol. The application pushes messages through the socket. On the other side, the transport-layer protocol has the job of getting those messages to the socket of the receiving process.
Many networks, including the Internet, provide more than one transport-layer protocol. When you develop an application you must choose one. How?
Everyday picture
The situation is like choosing between the train and the plane for a journey between two cities. You must pick one, and each offers different services. The train picks you up and sets you down in the city centre. The plane is faster. Neither is better in general; it depends on what your journey needs.
Where the picture breaks: you can compare a train and a plane on price and comfort, and there is a timetable for both. The Internet publishes no timetable at all. As this section is about to show, no transport protocol will tell you when your data will arrive. None will even tell you how fast it moves.
So study the services the available protocols offer, then pick the one that best matches your application’s needs. The possible services fall into four dimensions.
1. Reliable data transfer
Packets can get lost in a computer network. A packet can overflow a buffer in a router, or be discarded by a host or router after some of its bits are corrupted. Section 1.3.1 showed how.
For many applications — e-mail, file transfer, remote host access, web document transfers, financial applications — data loss is devastating. So something must guarantee that the data sent by one end is delivered correctly and completely to the other. A protocol that provides that guarantee provides reliable data transfer reliable data transfer Simple A promise that everything sent arrives, complete, without errors, and in the right order. Precise A service in which the transport protocol guarantees that the data sent by one end of the application is delivered correctly and completely to the other end. The sending process can pass its data into the socket and know with complete confidence that it will arrive without errors at the receiving process. introduced in ch. 2 — open in glossary . The sending process can pass its data into the socket and know with complete confidence that it will arrive without errors.
When a transport protocol does not provide it, some data sent may never arrive. That can be acceptable for a loss-tolerant application loss-tolerant application Simple An application that still works when a little of its data goes missing. Precise An application — most notably a multimedia application such as conversational audio or video — that can tolerate some amount of data loss. Lost data might result in a small glitch in the audio or video rather than a crucial impairment. introduced in ch. 2 — open in glossary , most notably multimedia applications such as conversational audio and video. A lost packet there means a small glitch, not a crucial impairment.
2. Throughput
Section 1.4.4 introduced available throughput throughput Simple How many bits per second actually arrive at the receiver. Precise The rate, in bits/second, at which the receiving host receives data. Instantaneous throughput is that rate at an instant; the average throughput of a transfer of F bits taking T seconds is F/T. With no competing traffic it is min{R₁, …, R_N}. introduced in ch. 1 — open in glossary : the rate at which the sending process can deliver bits to the receiving process. Because other sessions share the path, and because those sessions come and go, the available throughput fluctuates over time.
That suggests a service: guaranteed available throughput at some specified
rate. An application could request r bits/second, and the transport protocol
would ensure the available throughput is always at least r.
Worked example — why a guarantee would matter
An Internet telephony application encodes voice at 32 kbps. It must send data into the network, and have data delivered to the receiving application, at that rate.
Suppose the transport protocol can only manage 16 kbps. What does the application do with half the throughput it needs?
Nothing useful. It must either encode at a lower rate — and then get enough throughput to sustain that rate — or give up. Receiving half of the needed throughput is of little or no use.
Applications with throughput requirements are called bandwidth-sensitive applications bandwidth-sensitive application Simple An application that needs a fixed number of bits per second or it is no use at all. Precise An application that has a specific throughput requirement. An Internet telephony application encoding voice at 32 kbps needs that rate delivered; receiving half of the needed throughput is of little or no use. Some multimedia applications avoid this by using adaptive coding. introduced in ch. 2 — open in glossary . Many multimedia applications are bandwidth sensitive, although some use adaptive coding: they encode voice or video at a rate that matches whatever throughput is currently available. Section 2.6.2 builds a whole streaming system on that idea.
The opposite kind is the elastic application elastic application Simple An application that works with whatever rate it can get. More is simply better. Precise An application that can make use of as much, or as little, throughput as happens to be available. Electronic mail, file transfer and Web transfers are all elastic applications. introduced in ch. 2 — open in glossary , which can make use of as much, or as little, throughput as happens to be available. Electronic mail, file transfer and web transfers are all elastic. More is better, but any amount works.
3. Timing
A transport protocol can also provide timing guarantees. An example guarantee might be: every bit the sender pushes into the socket arrives at the receiver’s socket no more than 100 milliseconds later.
That would appeal to interactive real-time applications — Internet telephony, virtual environments, teleconferencing, multiplayer games. All of them need tight timing constraints to be effective. Long delays in Internet telephony produce unnatural pauses in the conversation. In a multiplayer game, a long delay between taking an action and seeing the result makes the whole thing feel unreal.
For non-real-time applications, lower delay is always preferable to higher delay, but no tight constraint applies.
4. Security
Finally, a transport protocol can provide security services. In the sending host it could encrypt all data transmitted by the sending process; in the receiving host it could decrypt the data before delivering it. That would give confidentiality between the two processes even if the data is observed on the way. A transport protocol could also provide data integrity and end-point authentication end-point authentication Simple Proving that a message really came from the sender it claims to come from. Precise A mechanism that allows a receiver to determine with certainty whether a message originates from where it appears to originate. introduced in ch. 1 — open in glossary , which chapter 8 covers in detail.
The four dimensions, applied
| This row is the awkward one. It tolerates loss but demands both a rate and a deadline — exactly the two things no Internet transport protocol will promise. | |||
Cells marked ⓘ have an explanation — click to read it. Sortable columns have a ↕ in the heading.
Sort any column. Click a ⓘ for the reasoning. Notice that no single row needs everything, which is why one transport protocol cannot serve them all.
Read the table as a designer, not as a list. No single row needs everything. File transfer needs perfect delivery and no timing at all. Telephony needs the opposite. That is why the Internet ended up with two transport protocols rather than one, and why neither of them is the right answer to every question.
What the Internet actually provides
Now stop asking what a network could offer and look at what the Internet does. The Internet, and more generally TCP/IP (Transmission Control Protocol / Internet Protocol) Transmission Control Protocol / Internet Protocol The collective name for the Internet's principal protocols. It became the ARPAnet standard on 1 January 1983. introduced in ch. 1 networks, make two transport protocols available: UDP (User Datagram Protocol) User Datagram Protocol A simple transport protocol with no reliability, no flow control and no congestion control. introduced in ch. 1 and TCP (Transmission Control Protocol) Transmission Control Protocol The Internet transport protocol that delivers data reliably and in order, with flow control and congestion control. introduced in ch. 1 . When you create a new network application, one of your first decisions is which to use.
TCP
The TCP service model includes two things, and an application that invokes TCP receives both.
A connection-oriented service connection-oriented service Simple The two sides greet each other and agree on state before any real data flows. Precise A service in which the client and server exchange transport-layer control information with each other before the application-level messages begin to flow. After this handshaking phase, a connection is said to exist between the sockets of the two processes; when the application finishes sending it must tear the connection down. introduced in ch. 2 — open in glossary . TCP has the client and server exchange transport-layer control information with each other before the application-level messages begin to flow. This handshaking handshaking Simple A short opening exchange that lets both sides get ready before the real data arrives. Precise The procedure by which a client and server exchange transport-layer control information before application-level messages flow. It alerts the client and server, allowing them to prepare for the packets to come. introduced in ch. 2 — open in glossary procedure alerts the client and server, so they can prepare for the packets to come. After the handshaking phase, a TCP connection is said to exist between the sockets of the two processes.
The connection is full-duplex full-duplex Simple Both sides can send at the same time over the same connection. Precise A property of a TCP connection whereby the two processes can send messages to each other over the connection at the same time. introduced in ch. 2 — open in glossary : the two processes can send messages to each other over it at the same time. When the application finishes sending, it must tear the connection down.
A reliable data transfer reliable data transfer Simple A promise that everything sent arrives, complete, without errors, and in the right order. Precise A service in which the transport protocol guarantees that the data sent by one end of the application is delivered correctly and completely to the other end. The sending process can pass its data into the socket and know with complete confidence that it will arrive without errors at the receiving process. introduced in ch. 2 — open in glossary service. The communicating processes can rely on TCP to deliver all data sent, without error and in the proper order. Pass a stream of bytes into a socket at one side. You can count on TCP to deliver the same stream to the receiving socket, with no missing or duplicate bytes.
TCP also includes a congestion-control congestion control Simple A rule that slows a sender down when the network between the two ends is overloaded. Precise A TCP mechanism that throttles a sending process, client or server, when the network is congested between sender and receiver. It is a service for the general welfare of the Internet, not for the direct benefit of the two communicating processes. It also tries to limit each TCP connection to its fair share of the link’s capacity. UDP has no congestion control. introduced in ch. 2 — open in glossary mechanism. This one is different in kind from the other two.
Congestion control is not for you
The book is precise about this, and the precision matters:
a service for the general welfare of the Internet rather than for the direct benefit of the communicating processes
TCP congestion control throttles a sending process when the network is congested between sender and receiver. It also attempts to limit each TCP connection to its fair share of the link’s capacity.
So it is a service you cannot switch off, that exists to protect everybody else from you. Remember this when section 2.2.6 shows browsers opening six connections at once specifically to get around it.
Focus on security: neither protocol encrypts anything
Neither TCP nor UDP provides any encryption. The data the sending process passes into its socket is the same data that travels over the network. A password sent as cleartext travels as cleartext over every link between sender and receiver, and can be sniffed at any of them.
The book puts this in a sidebar. It is worth stepping through, because the placement of TLS decides whose job the encryption is.
Read all steps as text
- Without TLS: the password travels as cleartext — Neither TCP nor UDP provides any encryption. The data the sending process passes into its socket is the same data that travels over the network. A password sent as cleartext travels as cleartext over every link between sender and receiver, and can be sniffed at any of them.
- Add TLS — but notice where it goes — TLS is not a third Internet transport protocol on the same level as TCP and UDP. It is an enhancement of TCP, and the enhancement is implemented in the application layer. To use it, an application includes TLS code — existing, highly optimised libraries — in both its client and its server.
- The sending side: cleartext in, ciphertext out — The sending process passes cleartext data to the TLS socket. TLS in the sending host encrypts the data and passes the encrypted data to the TCP socket. From TCP downwards, nothing knows or cares that the bytes are encrypted.
- The receiving side: the mirror image — The encrypted data travels over the Internet to the TCP socket in the receiving process. That socket passes it to TLS, which decrypts it. Finally TLS passes the cleartext through its TLS socket to the receiving process. TLS also supplies data integrity and end-point authentication, which chapter 8 covers in full.
In plain words
TLS (Transport Layer Security) Transport Layer Security Encryption, integrity and authentication added above TCP, in the application layer. introduced in ch. 2 transport layer security Simple A layer of code above TCP that encrypts what an application sends. Precise TLS, an enhancement of TCP defined in RFC 5246, providing process-to-process encryption, data integrity and end-point authentication. It is not a third Internet transport protocol alongside TCP and UDP. The enhancements are implemented in the application layer, and an application that wants them must include TLS libraries on both the client and the server side. introduced in ch. 2 — open in glossary is badly named. Its name says “transport layer”, but it is not a third Internet transport protocol on the same level as TCP and UDP. It is an enhancement of TCP whose enhancements are implemented in the application layer.
The practical consequence for you: encryption does not arrive by choosing a transport protocol. You include TLS libraries in both the client and the server side of your application. TLS has its own socket API (Application Programming Interface) Application Programming Interface The published interface through which one program can use another program's services. introduced in ch. 1 , similar to the traditional TCP one, so the code looks familiar.
UDP
UDP is a no-frills, lightweight transport protocol providing minimal services.
It is connectionless connectionless service Simple Data is sent with no greeting first, and with no promise that it will arrive. Precise The service provided by UDP. There is no handshaking before the two processes start to communicate, and no guarantee that a message will ever reach the receiving process. Messages that do arrive may arrive out of order. introduced in ch. 2 — open in glossary : there is no handshaking before the two processes start to communicate. It provides an unreliable data transfer service — when a process sends a message into a UDP socket, UDP gives no guarantee the message will ever reach the receiving process. Messages that do arrive may arrive out of order.
UDP does not include a congestion-control mechanism, so the sending side can push data into the network layer at any rate it pleases. Note carefully what that does not mean. The actual end-to-end throughput may still be far less than that rate. The intervening links have limited capacity, and they may also be congested. Pushing harder does not make the bottleneck link wider.
| TCPTransmission Control Protocol | UDPUser Datagram Protocol | |
|---|---|---|
| Reliable data transfer?dimension 1 of 4 | ||
| Throughput guarantee?dimension 2 of 4 | ||
| Timing guarantee?dimension 3 of 4 | ||
| Security?dimension 4 of 4 | ||
| Connection first? | ||
| Order preserved? | ||
| Congestion control? | ||
| Who uses it |
Cells marked ⓘ have a reason behind them — click to read it.
The four service dimensions come first, then everything else. Click any ⓘ.
Services not provided
Go back to the four dimensions and check them off.
TCP provides reliable end-to-end data transfer. TCP can be enhanced at the application layer with TLS to provide security. But in the description of TCP and UDP above, any mention of throughput or timing guarantees was conspicuously missing. Those services are not provided by today’s Internet transport protocols.
Does the absence of guarantees mean time-sensitive applications such as Internet telephony cannot run on today’s Internet? Clearly not — the Internet has hosted them for many years. They often work fairly well because they have been designed to cope, as far as possible, with the lack of guarantee.
But do not overstate it
Clever design has limits. When delay is excessive, or the end-to-end throughput is limited, no amount of application cleverness rescues the call.
The honest summary is the book’s own: today’s Internet can often provide satisfactory service to time-sensitive applications, but it cannot provide any timing or throughput guarantees.
| Streaming video runs over HTTP over TCP — the same machinery as an ordinary web page. That choice is the subject of §2.6. | ||
Cells marked ⓘ have an explanation — click to read it. Sortable columns have a ↕ in the heading.
Sort the last column. Almost everything picked TCP, and for the same reason. Click a ⓘ to see it.
E-mail, remote terminal access, the Web and file transfer all use TCP. They chose it for the same reason: reliable data transfer, which guarantees that all data will eventually reach its destination.
Internet telephony is the exception. Such applications can often tolerate some loss but need a minimal rate to be effective, so their developers usually prefer UDP. Doing so avoids TCP’s congestion-control mechanism and its packet overheads. But there is a practical catch: many firewalls are configured to block most types of UDP traffic. Internet telephony applications are therefore often designed to use TCP as a backup if UDP communication fails.
What an application-layer protocol is
Processes communicate by sending messages into sockets. But how are those messages structured? What do their fields mean? When does a process send one?
Those questions define the application-layer protocol application-layer protocol Simple The rules for what two application programs say to each other, and when. Precise A protocol that defines how an application’s processes, running on different end systems, pass messages to each other. It defines four things: the types of messages exchanged, the syntax of each message type, the semantics of the fields, and the rules for when and how a process sends messages and responds to them. It is only one piece of a network application. introduced in ch. 2 — open in glossary . It defines how an application’s processes, running on different end systems, pass messages to each other. Specifically, it defines four things:
- The types of messages exchanged — for example, request messages and response messages.
- The syntax of the various message types — the fields in the message, and how the fields are separated from one another.
- The semantics of the fields — the meaning of the information in them.
- Rules for determining when and how a process sends messages and responds to messages.
Some application-layer protocols are specified in RFC (Request For Comments) Request For Comments The name of an IETF standards document. There are currently nearly 9000 of them. introduced in ch. 1 s and are therefore in the public domain. The Web’s protocol, HTTP (HyperText Transfer Protocol) HyperText Transfer Protocol The application-layer protocol that requests and transfers Web documents. introduced in ch. 1 , is one of them ( RFC (Request For Comments) Request For Comments The name of an IETF standards document. There are currently nearly 9000 of them. introduced in ch. 1 7230). That is what makes the Web work at all. If a browser developer follows the rules of the HTTP RFC, the browser can retrieve pages from any web server that also followed them.
Many other application-layer protocols are proprietary and deliberately not public. Skype uses proprietary application-layer protocols.
A protocol is not an application
It is important to keep these apart, and easy to blur them.
The Web is a client-server application that lets users obtain documents from web servers on demand. It consists of many components: a standard for document formats ( HTML (HyperText Markup Language) HyperText Markup Language The document format of the Web, created by Tim Berners-Lee at CERN between 1989 and 1991. introduced in ch. 1 ), browsers such as Chrome, web servers such as Apache, and an application-layer protocol. HTTP defines only the format and sequence of messages exchanged between browser and server. It is one piece — an important piece — of the Web application.
The same holds for Netflix, as section 2.6.4 shows. It has servers that store and transmit video, other servers that handle billing, and clients such as the app on your phone. It also has an application-level protocol, DASH (Dynamic Adaptive Streaming over HTTP) Dynamic Adaptive Streaming over HTTP Chunk-by-chunk quality selection driven by the throughput the client measures. introduced in ch. 2 . DASH defines the format and sequence of messages between a Netflix server and a Netflix client, and it is one piece of the application, not the whole of it.
The five applications this chapter studies
New applications appear every day. Rather than covering a large number of them, the book focuses on a small number that are both pervasive and important.
| Application | Section | Why it is here |
|---|---|---|
| The Web | 2.2 | Enormously popular, and its protocol, HTTP, is straightforward and easy to understand. It goes first for that reason. |
| Electronic mail | 2.3 | The Internet’s first killer application. More complex than the Web, because it uses not one but several application-layer protocols. |
| DNS | 2.4 | The domain name system: a directory service. Most users never interact with it directly; they invoke it through other applications. It shows how a piece of core network functionality — translating names to addresses — can be implemented at the application layer. |
| P2P file sharing | 2.5 | Peer-to-peer sharing — the one non-client-server application, and the place the self-scalability argument becomes an equation. |
| Video streaming | 2.6 | Distributing stored video over content distribution networks. About 80 % of Internet traffic in 2020. |
Then section 2.7 writes real code, over both TCP and UDP.
Check yourself
Check yourself
0 of 6 answered1.An Internet telephony application encodes voice at 32 kbps. It is given only 16 kbps. What kind of application is it, and why does that matter here?
2.Which of the four service dimensions does today's Internet actually guarantee to an application?
Count what TCP and UDP promise between them.
3.Time-sensitive applications such as Internet telephony run on today's Internet every day. How, given that no timing guarantee exists?
4.Where does TLS sit, and why does the answer matter to you as a developer?
5.Why do developers of Internet telephony applications often prefer UDP, and why do those applications frequently fall back to TCP?
6.The Web is a network application. HTTP is an application-layer protocol. What is the difference?
What to remember
- An application-layer protocol defines four things: message types, their syntax, the meaning of their fields, and the rules for when to send. It is one piece of an application, not the whole of it — HTTP is not the Web.
- TCP gives a connection-oriented service plus reliable data transfer, and imposes congestion control — which exists for the Internet’s benefit, not yours. UDP is connectionless and unreliable, may reorder, and has no congestion control. It is chosen for what it does not do.
- Throughput and timing guarantees do not exist on today’s Internet. Time-sensitive applications work because they are designed to cope — and that coping has limits.