Boxes that sit in the data path and do something other than forward. Whether they are an abomination or a necessity is the argument this whole chapter has been building to.
Words you will meet
- Middlebox middlebox Simple A box in the path that is not a router. Precise RFC 3234: any intermediary box performing functions apart from the normal, standard functions of an IP router on the data path between a source host and a destination host. Examples are NAT boxes, firewalls, intrusion detection systems, caches and load balancers. introduced in ch. 4 — open in glossary — anything in the path performing functions apart from a router’s normal ones.
- NFV (Network Function Virtualization) Network Function Virtualization Running middlebox functions as software on commodity hardware. introduced in ch. 4 — running middlebox functions as software on ordinary hardware network function virtualization Simple Run middlebox jobs as software on ordinary hardware. Precise NFV: implementing middlebox services with commodity networking, computing and storage hardware and specialised software on a common stack — the approach SDN took for the control plane a decade earlier. introduced in ch. 4 — open in glossary .
- Narrow waist — there are many protocols at every layer except the network layer, where there is one.
- Spanning layer — Clark’s name for that waist: it hides the differences below and presents one interface above.
- The end-to-end argument — a function that needs the endpoints’ knowledge cannot be completely implemented in the network.
Why this matters
Routers are the workhorses of the network layer, and this chapter has been about how they do their job. But you have kept meeting other boxes: web caches in section 2.2.5, TCP (Transmission Control Protocol) Transmission Control Protocol The Internet transport protocol that delivers data reliably and in order, with flow control and congestion control. introduced in ch. 1 connection splitters in section 3.7, NAT (Network Address Translation) Network Address Translation Letting a whole private network share one public IP address by rewriting addresses and ports. introduced in ch. 4 , firewalls and intrusion detection in section 4.3.3.
This section names them, counts the cost, and then asks the question section 4.3.3 deferred: is any of this legitimate?
It is the right place to ask, because section 4.4 has just shown that a modern router can perform firewalling and load balancing natively, with generalized match-plus-action. So the question is no longer only “should these boxes exist” but “should this function be in the network at all”.
What a middlebox is
[RFC 3234]‘s definition, which the book quotes:
any intermediary box performing functions apart from normal, standard functions of an IP (Internet Protocol) Internet Protocol The network-layer protocol that defines the datagram format and addressing every Internet device must use. introduced in ch. 1 router on the data path between a source host and destination host
Note how much that admits. Not “a box that breaks things” — just anything in the path doing something other than the standard job.
| NAT translation | Security services | Performance enhancement | |
|---|---|---|---|
| What it does | |||
| Examples | |||
| Which headers it reads | |||
| Where you met it |
Cells marked ⓘ have a reason behind them — click to read it.
The book groups them into three. Every one of them you have already met somewhere earlier in this book — click any cell.
Drag any device to rearrange the picture. Hover a link to see its rate, delay and length.
Every box on this path does something other than plain forwarding. In the "good old days" picture there would be nothing here but routers.
Read this diagram as text
- — wired link —
- — wired link —
- — wired link —
- — wired link —
- — wired link —
- — wired link —
- — wired link —
The last twenty years
The book’s framing: in the past 20 years, we’ve seen tremendous growth in such middleboxes. They appear in both wired and wireless cellular networks.
And with the proliferation comes a cost that is not architectural at all. Separate specialised hardware boxes, separate software stacks, and separate management and operation skills translate into significant operational and capital costs. Somebody has to buy, install, configure, patch and staff each kind of box.
That practical pressure produced the obvious response: use commodity hardware — networking, computing and storage — with specialised software built on a common software stack. This is network function virtualization ( NFV (Network Function Virtualization) Network Function Virtualization Running middlebox functions as software on commodity hardware. introduced in ch. 4 ).
The book notes what that approach is: exactly the approach taken in SDN (Software-Defined Networking) Software-Defined Networking Separating the control plane from the data plane by computing forwarding tables in a remote controller. introduced in ch. 4 a decade earlier. Section 4.4 separated the control plane from cheap forwarding hardware; NFV does the same for middlebox functions. An alternative that has also been explored is outsourcing middlebox functionality to the cloud.
The clean picture, and how it broke
The good old days
For many years the Internet architecture had a clear separation:
- The network layer consisted of routers, operating in the network core, forwarding datagrams using fields only in the IP datagram header.
- The transport and application layers were implemented in hosts, at the network edge.
Hosts exchanged transport-layer segments and application-layer messages among themselves. Nothing in the middle looked inside.
Today’s middleboxes clearly violate this separation, and the book gives three specific violations:
- a NAT box, sitting between a router and a host, rewrites network-layer IP addresses and transport-layer port numbers;
- an in-network firewall blocks suspect datagrams using application-layer, transport-layer and network-layer header fields;
- an e-mail security gateway is injected between sender and receiver, filtering messages on whitelisted and blacklisted addresses as well as on the content of the message itself.
Read those in order and the reach grows one layer at a time, until the last one is reading what people wrote to each other.
The argument, with both sides stated
The book presents this as a genuine disagreement, and it is worth keeping both halves.
Against. Some have considered middleboxes “a bit of an architectural abomination” [Garfinkel 2003]. The objection is section 4.3.3’s. Layer-3 devices should process packets only up to the network layer, and hosts should talk directly without interfering nodes modifying their addresses — much less their port numbers.
For. Others adopt the philosophy that such middleboxes “exist for important and permanent reasons”, filling a real need. On that view we will have more, not fewer of them in future [Walfish 2004].
Neither side is presented as obviously right. What has changed since the argument began is section 4.4. If a router can perform firewalling and load balancing natively with generalized forwarding, some of these functions no longer need a separate box at all. That does not settle whether the function belongs in the network — it only settles where it runs.
The architectural principles
The book steps back and asks what principles guided the largest and most complex engineered system ever built. [ RFC (Request For Comments) Request For Comments The name of an IETF standards document. There are currently nearly 9000 of them. introduced in ch. 1 1958], “Architectural Principles of the Internet”, suggests they are truly minimal:
Many members of the Internet community would argue that there is no architecture, but only a tradition, which was not written down for the first 25 years. However, in very general terms, the community believes that the goal is connectivity, the tool is the Internet Protocol, and the intelligence is end to end rather than hidden in the network.
Three things: a goal, a tool, and a placement. The rest of this section is the last two.
The IP hourglass
Figure 4.31 — the narrow-waisted Internet hourglass
Many protocols above, many below, and exactly one in the middle. Every one of the billions of Internet-connected devices must implement that one.
Why the waist being narrow is the whole trick
The Internet has many protocols at the physical, link, transport and application layers. It has one network-layer protocol.
That one protocol is the only universal requirement for Internet connectivity. Its relative simplicity is what let a rich variety of networks — Ethernet, WiFi (IEEE 802.11 wireless LAN) IEEE 802.11 wireless LAN Wireless LAN access; a user must be within a few tens of metres of the access point. WiFi is a trade name, not an abbreviation. introduced in ch. 1 , cellular, optical — all become part of the Internet. The book says this narrow waist has played a critical role in the Internet’s growth.
Clark calls it a spanning layer [Clark 1997], and his description is worth reading slowly:
Its role is to “hide the detailed differences among these various underlying technologies and present a uniform service interface to the applications above.”
“How does the IP spanning layer achieve its purpose? It defines a basic set of services, which were carefully designed so that they could be constructed from a wide range of underlying network technologies.”
Now recall section 4.1. The Internet’s network layer offers best-effort service and guarantees nothing. That looked like poverty at the time. Here is the other half of the trade: a service model that promises almost nothing is a service model that almost any underlying technology can provide. The waist is narrow because the promise is small.
And the waist may be widening
The book’s own wry observation, as the Internet architecture enters middle age — and at 40 to 50 years old it qualifies:
“one might observe that its narrow waist may indeed be widening a bit (as often happens in middle age!) via the rise of middleboxes.”
That is the connection between the two halves of this section. Every middlebox that inspects a transport or application header is a device in the middle that needs to understand more than IP. The waist widens by exactly that much, and every protocol that has to accommodate them becomes harder to change.
Section 3.8 recorded the consequence without naming it. QUIC (Quick UDP Internet Connections) Quick UDP Internet Connections A transport-like protocol built in the application layer over UDP, carrying HTTP/3. The IETF now treats QUIC as a name rather than an abbreviation. introduced in ch. 2 encrypts everything, partly so that middleboxes cannot see enough to interfere.
The end-to-end argument
RFC 1958’s third principle — intelligence is end to end rather than hidden in the network — is about where a function should be placed.
The argument, in its original words
The book notes a contrast worth keeping. The 20th-century telephone network had dumb, non-programmable endpoints and smart switches. The Internet has always had smart endpoints — programmable computers — which makes complex functionality at the edge possible in the first place.
But possibility is not justification. The principled argument came from an extremely influential paper [Saltzer 1984]:
The function in question can completely and correctly be implemented only with the knowledge and help of the application standing at the end points of the communication system. Therefore, providing that questioned function as a feature of the communication system itself is not possible. (Sometimes an incomplete version of the function provided by the communication system may be useful as a performance enhancement.)
Read the parenthesis. It is not a purist’s argument that the network must do nothing. It is a claim about what the network cannot finish, plus an allowance for partial help that speeds things up.
The example: reliable data transfer
Packets can be lost inside the network — and not only through buffer overflow. A router holding a queued packet could crash. A portion of the network holding a queued packet could become detached by a link failure.
So the endpoints, via TCP (Transmission Control Protocol) Transmission Control Protocol The Internet transport protocol that delivers data reliably and in order, with flow control and congestion control. introduced in ch. 1 , must perform error control.
Chapter 6 will show that some link-layer protocols do perform local error control. But that local control alone is incomplete, and not sufficient for end-to-end reliable data transfer. It protects one hop, and the packet can be lost at any of the others, or between them.
Therefore reliable data transfer must be implemented end to end.
That is section 3.4’s whole existence, justified. It built checksums, sequence numbers, acknowledgements and timers at the transport layer not because no lower layer could have helped, but because no lower layer could have finished.
And the parenthesis applies too: link-layer error control is exactly the “incomplete version useful as a performance enhancement” the paper allows for. It is worth having, and it is not enough.
Two papers RFC 1958 points at
RFC 1958 deliberately includes only two references, both called fundamental papers on Internet architecture:
- [Saltzer 1984] — the end-to-end paper itself.
- [Clark 1988] — the design philosophy of the DARPA (Defense Advanced Research Projects Agency) Defense Advanced Research Projects Agency The later name of ARPA; it sponsored Cerf and Kahn's work on interconnecting networks. introduced in ch. 1 Internet protocols.
The book calls both must reads for anyone interested in Internet architecture. Follow-ons are [Blumenthal 2001] and [Clark 2005], which reconsider the architecture in light of the far more complex environment today’s Internet operates in. That is, largely, the environment this section has described.
Check yourself
Check yourself
0 of 7 answered1.RFC 3234 defines a middlebox as any intermediary box performing functions apart from a router's normal ones. Which of these is NOT a middlebox?
2.predictThe Internet's network layer promises almost nothing. What does that have to do with the narrow waist?
3.What does the end-to-end argument actually claim?
4.predictSome link-layer protocols perform local error control. Why must TCP still do error control end to end?
5.What is network function virtualization, and what earlier idea does the book compare it to?
6.predictThe book says the narrow waist "may indeed be widening a bit". What widens it?
7.How does the book present the disagreement about middleboxes?
What to remember
- A middlebox is anything in the data path doing something other than a router’s normal job [RFC 3234]. There are three service types: NAT translation, security services, performance enhancement — and you met all three before this section.
- The narrow waist is one network-layer protocol beneath many of everything else, and it is what let Ethernet, WiFi, cellular and optical networks all join one Internet. The waist is narrow because the promise is small.
- The end-to-end argument is not “the network must do nothing”. It is that a function needing the endpoints’ knowledge cannot be completed in the network — though an incomplete version may be a useful performance enhancement.