Chapter 4 showed a router looking up its forwarding table; this chapter shows where that table came from, and there are exactly two answers.
Words you will meet
-
Control plane control plane Simple How the routers work out where everything should go. Precise The network-wide logic that controls how a datagram is routed among routers along an end-to-end path from source host to destination host. introduced in ch. 4 — open in glossary — the network-wide logic that decides what the forwarding machinery should do.
-
Routing algorithm routing algorithm Simple Works out good paths through the network. Precise An algorithm determining good — typically least-cost — paths from senders to receivers through the network of routers. introduced in ch. 5 — open in glossary — the computation that works out good paths and fills a forwarding table in.
-
Per-router control per-router control Simple Every router works out its own table. Precise A control-plane approach in which a routing algorithm runs in each and every router, and the routing components of different routers communicate with one another to compute each forwarding table. OSPF and BGP are built this way. introduced in ch. 5 — open in glossary — a routing algorithm runs in every router, and the routers talk to each other.
-
Logically centralized control logically centralized control Simple One remote brain fills in every table. Precise A control-plane approach in which a controller computes and distributes the forwarding tables used by every router. "Logically" centralized means the service is reached as if it were one point, even though it is normally run on several servers for fault tolerance and scale. introduced in ch. 5 — open in glossary — one remote controller computes every table and sends them out.
-
Control agent control agent Simple The small piece of a router that takes orders from the controller. Precise A component in each router that communicates with a logically centralized controller and does as it commands. Control agents have minimum functionality: they do not talk to each other and take no part in computing the forwarding table. introduced in ch. 5 — open in glossary — written CA: the small piece inside a router that takes orders from that controller.
Why this matters
Section 4.1 drew two figures and then walked away from one of them. Figure 4.2 had a routing algorithm inside the router. Figure 4.3 had a remote controller instead. Both boxes were empty rectangles labelled with a name.
This section reopens those two figures, and the rest of the chapter fills them in. Sections 5.2 to 5.4 are the first box. Section 5.5 is the second.
There is a second reason to read this short section carefully. The chapter is long, and it can feel like a list of unrelated protocols: two algorithms, then OSPF (Open Shortest Path First) Open Shortest Path First The Internet’s intra-AS link-state routing protocol. Each router builds a complete map of its autonomous system and runs Dijkstra locally. introduced in ch. 5 , then BGP (Border Gateway Protocol) Border Gateway Protocol The protocol ISPs use to tell each other which destinations they can reach; its rules follow the customer-provider and peering relationships. introduced in ch. 5 , then SDN (Software-Defined Networking) Software-Defined Networking Separating the control plane from the data plane by computing forwarding tables in a remote controller. introduced in ch. 4 , then ICMP (Internet Control Message Protocol) Internet Control Message Protocol Carries error and diagnostic messages between hosts and routers. A ping is an ICMP message. Covered in §5.6. introduced in ch. 2 and SNMP (Simple Network Management Protocol) Simple Network Management Protocol The Internet’s network-management protocol, typically run over UDP so that it still works when the network is under stress (§3.3, built in §5.7). introduced in ch. 3 . It is not a list. Almost all of it is one of these two pictures, drawn at a different scale.
What chapter 4 left unfinished
The book opens by recalling Figures 4.2 and 4.3, and it names the thing those figures had in common. In destination-based forwarding it was the forwarding table. In generalized forwarding it was the flow table match-plus-action table Simple The generalised version of a forwarding table. Precise The flow table in a packet switch, generalising destination-based forwarding: each entry pairs a match over header fields with a list of actions. It is typically computed, installed and updated by a remote controller. introduced in ch. 4 — open in glossary . Either way, one table was the element that linked the network layer’s data plane to its control plane.
Those tables specify the local forwarding behaviour of one router. Section 4.4 showed how much behaviour that can be. The action is not only sending a packet to an output port. It can also be dropping the packet, replicating it, or rewriting header fields at layer 2, layer 3 or layer 4.
What chapter 4 never said is how any of those tables get their contents. This chapter says. And section 4.1 already told us there are two possible approaches.
The first way: a routing algorithm in every router
This is everything chapter 4 built. Each router holds one table, and that table is its entire forwarding decision. Nothing here says where the table came from.
The five forwarding tables and the network below the dashed line are the data plane of chapter 4, unchanged. Watch only what appears above the line. The routers are named R1 to R5 here; the book leaves them unnamed.
Read all steps as text
- 1 — Five routers, five forwarding tables — This is everything chapter 4 built. Each router holds one table, and that table is its entire forwarding decision. Nothing here says where the table came from.
- 2 — A routing algorithm runs inside every router — One per router, above the dashed line. This is the control plane, and in this design it is spread across all five machines. No router is in charge.
- 3 — The routing components talk to one another — They exchange routing messages according to a routing protocol. Every line here is a message between two routers, and this exchange is what sections 5.2 to 5.4 are about.
- 4 — Each one computes its own table — From the messages it received, a router works out the values for its own forwarding table and installs them. It computes nobody else’s table, and nobody computes its.
- 5 — This is how the Internet has worked for decades — OSPF, in section 5.3, is this picture inside one organisation. BGP, in section 5.4, is this picture between organisations. Both are per-router control.
Each router has a routing component. That component communicates with the routing components in other routers, and from what they say it computes the values for its own forwarding table.
Notice how little any one router knows in this design. It has never seen the network. It has heard from its neighbours, and it has done a computation. Section 5.2 is the two ways that computation can be organised, and one of them fails in an interesting way.
This approach has been used in the Internet for decades. OSPF, in section 5.3, and BGP, in section 5.4, are both built this way.
The second way: one controller for all of them
Compare this with step 1 of Figure 5.1. It is not similar; it is identical. Whatever changes next, the data plane will not.
The base of this diagram is the same array of shapes as Figure 5.1 above — same five tables, same network, same positions. That is deliberate. Only what sits above the dashed line has changed.
Read all steps as text
- 1 — The same five routers, the same five tables — Compare this with step 1 of Figure 5.1. It is not similar; it is identical. Whatever changes next, the data plane will not.
- 2 — The routing computation has left the routers — One controller now computes the forwarding tables for all five. It holds a copy of each table, which is why the book draws five small tables inside it.
- 3 — Each router keeps a small control agent — The control agent, written CA, has minimum functionality. Its whole job is to talk to the controller and do what the controller commands.
- 4 — Controller and control agents talk, up and down only — Every line runs between the controller and one control agent. Look for a line joining two control agents. There is none, and that absence is the key distinction from Figure 5.1.
- 5 — The control agent installs the table it is given — It did not compute the table and takes no part in computing it. Chapter 4’s data plane then runs exactly as before, on a table someone else worked out.
- 6 — And "logically" centralized is not one machine — The service is reached as if it were a single point. Underneath it normally runs on several servers, for fault tolerance and for performance. One box in the picture, many boxes in the rack.
A logically centralized controller computes the forwarding tables to be used by each and every router, and distributes them.
The controller reaches a control agent in every router, over a well-defined protocol, and uses it to configure and manage that router’s table. The control agent has minimum functionality. It communicates with the controller and does as the controller commands.
The one sentence that separates the two designs
The book states the key distinction and does not decorate it:
Unlike the routing algorithms in Figure 5.1, the CAs do not directly interact with each other nor do they actively take part in computing the forwarding table.
Read that as two separate claims, because they fail independently.
They do not talk to each other. In Figure 5.1 the control-plane arrows run sideways, between routers. In Figure 5.2 every arrow runs up or down.
They take no part in the computation. A control agent is not a small routing algorithm. It carries a result it did not produce and could not have produced.
Neither design changes the data plane
The two diagrams above are drawn from the same list of shapes below the dashed line. That is not a shortcut in how this page was built. It is the point section 4.1 made and this section repeats.
A packet arriving at R3 is handled the same way in both pictures. One header field is read, one row of the forwarding table is matched, one output interface is chosen. The packet cannot tell which control plane filled the table in, and neither can the hardware doing the lookup.
| Per-router controlFigure 5.1 — the traditional design | Logically centralized controlFigure 5.2 — the software-defined design | |
|---|---|---|
| Where the algorithm runswhich machine does the computing | ||
| Who talks to whomthe shape of the control traffic | ||
| What each computer can seehow much of the network is visible to the thing deciding | ||
| What sits in the routerthe control-plane component inside each box | ||
| Where you meet itthe sections and the deployments |
Cells marked ⓘ have a reason behind them — click to read it.
The book states this contrast in two bullet points. Click any cell for what the difference actually costs you.
Everyday picture — timetables at a railway
Every station needs a timetable saying which platform each train leaves from. The timetable is the forwarding table. A passenger reads one line of it.
Per-router control is stationmasters ringing each other up. Each one learns what the neighbouring stations are doing, works out their own timetable, and posts it. Nobody has seen the whole railway, and the timetables still fit together — most of the time.
Logically centralized control is a head office that knows every line, writes all the timetables, and sends each station its own. The clerk who pins it to the wall does not know how it was decided and did not help decide it.
Where the picture stops. A stationmaster could ignore head office. A control agent cannot: it has no routing algorithm to fall back on. And a railway timetable is written once a season, while a control plane may have to rewrite every table because one link failed a second ago.
What “logically centralized” is careful not to say
The word logically is doing real work in that phrase, and the book defines it rather than leaving it to be guessed.
By logically centralized control the book means that the routing control service is accessed as if it were a single central service point. The service itself is likely to be implemented across several servers, for two reasons: fault tolerance, and performance scalability.
In plain words
One controller in the diagram does not mean one computer in the building.
If it did, the design would be obviously worse than the one it replaces. A single machine that computes every forwarding table is a single machine whose failure stops the entire network from adapting to anything.
“Logically centralized” claims something weaker and far more useful: one place to ask, not one machine to lose. How that is actually built is section 5.5’s problem, and section 5.5.3 shows a controller reacting to a link changing state.
Who actually runs a controller
The book lists real deployments, and the list is worth reading because it answers the obvious question about whether any of this left the laboratory.
- Google uses it to control the routers in B4, its internal global wide-area network, which connects its data centres.
- SWAN, from Microsoft Research, manages routing and forwarding between a wide-area network and a data-centre network.
- Comcast’s ActiveCore and Deutsche Telekom’s Access 4.0 are integrating it into major ISP (Internet Service Provider) Internet Service Provider Connects hosts, and other ISPs, to the rest of the Internet. introduced in ch. 1 networks.
- China Telecom and China Unicom use it both inside data centres and between them.
- 4G and 5G cellular networking depend on it, as chapter 8 shows.
The book also quotes AT&T, in 2019, on how settled the question is:
… SDN, isn’t a vision, a goal, or a promise. It’s a reality. By the end of next year, 75% of our network functions will be fully virtualized and software-controlled.
Two things are worth noticing about that list. Every entry is one organisation controlling its own network. None of them is the public Internet, where sections 5.3 and 5.4’s per-router protocols still decide where packets go.
Check yourself
Check yourself
0 of 6 answered1.What is the question this whole chapter answers?
2.In per-router control, which direction does the control traffic run?
3.predictStep through both figures on this page. What is different below the dashed line?
The two diagrams are drawn from the same list of shapes. Look at what that list contains.
4.The book calls one fact the key distinction between per-router and logically centralized control. Which is it?
5.What does the word "logically" add to "logically centralized controller"?
6.predictEvery deployment the book names — Google's B4, Microsoft's SWAN, Comcast, Deutsche Telekom, China Telecom — has something in common. What?
What to remember
- This chapter answers one question: where does a forwarding table come from? Chapter 4 used the table and never said.
- There are two answers, and they are Figures 5.1 and 5.2. Per-router control puts a routing algorithm in every router and lets the routers talk sideways — OSPF and BGP. Logically centralized control has one controller compute every table and send it down.
- The data plane is identical either way. The two diagrams on this page share the same shapes below the dashed line, and a packet cannot tell the difference.