§5.5.1–5.5.2The SDN Control Plane · The OpenFlow Protocol

Network layer Kurose & Ross pp. 411–417 · ~18 min read

  • sdn controller
  • network-control application
  • northbound interface
  • southbound interface
  • openflow
  • unbundling

Where you are

  • Application layer
  • Transport layer
  • Network layer you are here
  • Link layer
  • Physical layer

In a software-defined network ( SDN (Software-Defined Networking) ) the switches keep only their flow tables. A remote controller, with applications above it, computes what those tables should contain.

Words you will meet

  • SDN controller — the remote software that holds the network’s state and programs the switches. Also called a network operating system.

  • Network-control application — a program above the controller that decides what the network should do.

  • Northbound interface — the controller’s API (Application Programming Interface) upward, to those applications.

  • Southbound interface — the controller’s interface downward, to the devices.

  • OpenFlow — one protocol that fills the southbound interface, and the earliest.

  • Unbundling — splitting one product into parts that different companies can supply.

Why this matters

Section 5.1 named two ways to build a control plane and then spent four sections on only the first. Per-router control gave us OSPF (Open Shortest Path First) and BGP (Border Gateway Protocol) .

This is the second way. The routing algorithm moves out of the routers entirely and becomes an ordinary program on an ordinary server.

Section 4.4 already built the half of this that lives in the switch: generalised forwarding, match plus action, and the OpenFlow abstraction. What it never said was who fills in those tables. This section is the answer.

Four things that make an architecture SDN

SDN is a set of four characteristics, and the book lists them before drawing anything.

Flow-based forwarding. Forwarding by SDN-controlled switches can be based on any number of header field values in the transport, network or link layer. The OpenFlow 1.0 abstraction of section 4.4 allows eleven different header fields. That contrasts sharply with sections 5.2 to 5.4, where forwarding used the destination IP (Internet Protocol) address alone.

Separation of the data plane and the control plane. The data plane is the switches: relatively simple but fast devices that execute match-plus-action rules in their flow tables . The control plane is servers and software that determine and manage those tables.

Network control functions live outside the switches. The control plane is software, and it runs on servers that are distinct and remote from the network’s switches. That is what a traditional router does not do.

The network is programmable. It is programmed through network-control applications running in the control plane. They are the brains, and they use the controller’s APIs to specify and control the data plane.

The four characteristics of an SDN architecture
A traditional routerone box from one vendorAn SDN architecturethree separate things
What forwarding is based on
Where the control plane lives
Who runs the control software
How behaviour is changed

Cells marked ⓘ have a reason behind them — click to read it.

The book’s own four bullets, each set against the arrangement it replaces. Click any cell for the detail.

In plain words

The word the book uses for the result is unbundling.

Before, a router was one product. The hardware, the control-plane software and the protocol implementations arrived together, from one vendor, and could not be separated.

Now there are three kinds of thing: switches, controllers, and applications. Each may come from a different vendor or organisation, and each is a market of its own.

The book’s comparison is the move from mainframes to personal computers. Separating hardware, operating system and applications produced innovation in all three, and the hope for SDN is the same.

The architecture

Figure 5.14 — the three parts of an SDN architecture
The data plane: switches that only match and actstep 1 of 4
Control planeData planeSDN-controlled switchesfast, simple, and they only run match plus action

Relatively simple but fast devices that execute the match-plus-action rules in their flow tables. They compute nothing. Section 4.4 built exactly this.

Drawn top-down, like the book. Read it bottom-up and it is a stack: dumb fast hardware, one program that knows the whole network, and applications that decide what the network should do.

Read all steps as text
  1. The data plane: switches that only match and actRelatively simple but fast devices that execute the match-plus-action rules in their flow tables. They compute nothing. Section 4.4 built exactly this.
  2. The control plane: a controller, on separate machinesServers and software that determine and manage the switches’ flow tables. Unlike a traditional router, this software does not run inside the switch. It runs on machines that are distinct and remote from it.
  3. And above it, the network-control applicationsThese are the brains. A routing application might compute end-to-end paths by running Dijkstra over the controller’s state. An access-control application decides which packets a switch should block. A load balancer spreads requests over servers. All three are section 4.4.3’s examples, now given a home.
  4. Two interfaces, and they have namesThe controller talks upward to the applications across its northbound API, and downward to the devices across its southbound API. Almost everything in section 5.5 is about one of these two boundaries.

”Logically centralized” is not “one server”

Figure 5.14 draws the controller as a single central server, and that picture is convenient rather than true.

The controller is logically centralized. From outside — from a switch, or from an application — it looks like one monolithic service. Inside, it is implemented on several servers that give coordinated, scalable performance and high availability.

That is not a detail. It brings every hard problem distributed systems have: ordering events in time, consistency, consensus. Modern controllers such as ODL (OpenDaylight) and ONOS (Open Network Operating System) put considerable effort into being logically centralized and physically distributed at once.

Inside the controller

The control plane divides into two components: the controller and the network-control applications. The book examines the controller first, and organises its work into three layers.

It then does something unusual, and says so: it presents them bottom-up.

Figure 5.15 — inside the controller, built bottom-up
Bottom: talking to the devicesstep 1 of 4
RoutingAccess controlLoad balancerNorthbound APISouthbound APISDN controllerCommunication to and from controlled devicesOpenFlowSNMP

To control a remote switch you need a protocol. It must carry instructions down, and it must carry locally observed events back up — a link going up or down, a device joining, a heartbeat saying a device is alive. Those events are what keep the controller’s picture current. This is the southbound side, and OpenFlow lives here.

The book presents these three layers "in an uncharacteristically bottom-up fashion", and this diagram follows it. Each layer only exists because the one above needs something from it.

Read all steps as text
  1. Bottom: talking to the devicesTo control a remote switch you need a protocol. It must carry instructions down, and it must carry locally observed events back up — a link going up or down, a device joining, a heartbeat saying a device is alive. Those events are what keep the controller’s picture current. This is the southbound side, and OpenFlow lives here.
  2. Middle: remembering what the network looks likeEvery real decision needs up-to-date information about hosts, links, switches and other controlled devices. The layer holds link-state information, host and switch information, the counters read out of flow tables, and often a copy of the flow tables themselves. This is the network-wide state.
  3. Top: offering that state to the applicationsThe applications need to read and write network state and flow tables, and to register for notification when something changes. That is the northbound API. Two popular controllers offer it as a RESTful request-response interface; others offer a network graph or an intent abstraction.
  4. And it is only logically one machineEvery earlier picture drew the controller as a single central server. From outside — from a switch, or from an application — it does look like one monolithic service. Inside, it is a distributed set of servers, for fault tolerance, high availability and performance. That brings the usual distributed-systems problems: ordering events in time, consistency, consensus.

Everyday picture

A building manager for a large office block.

The bottom layer is the radio the manager uses to talk to the caretakers. It works both ways: instructions go down, and reports of a broken lift come back up. The middle layer is the manager’s own notes — which lifts work, which rooms are booked, what every caretaker last reported. The top layer is the office where tenants come to ask for things.

A tenant never speaks to a caretaker, and a caretaker never decides anything.

Where the picture stops. One manager is genuinely one person. An SDN controller only looks like one from outside; behind the desk there is a team keeping identical notes, and keeping them identical is the hard part.

The OpenFlow protocol

OpenFlow operates between an SDN controller and an SDN-controlled switch, or any other device implementing the OpenFlow API from section 4.4.

It runs over TCP (Transmission Control Protocol) , with a default port number of 6653. That detail is worth pausing on. The southbound interface is an ordinary transport connection, so the controller can be anywhere the network can reach — which is exactly what “distinct and remote” required.

The OpenFlow protocol, in both directions

Four message types travel down. Everything the controller wants a switch to do arrives as one of these.

message 4 of 4
SDN controllerremote, logically oneTimeSDN-controlled switchflow table lives hereTimeflow table updatedConfigurationModify-StateRead-StateSend-Packet

Click any arrow to see what that message says and why it is sent.

OpenFlow runs over TCP, default port 6653. Every arrow here is control traffic, never user data — except the payloads inside Send-Packet and Packet-in, which carry one real packet each.

Read this diagram as text
  1. SDN controller sends Configuration to SDN-controlled switch. Lets the controller query and set a switch’s configuration parameters.
  2. SDN controller sends Modify-State to SDN-controlled switch. Adds, deletes or modifies entries in the switch’s flow table, and sets switch port properties. This is the message that actually programs the data plane.
  3. SDN controller sends Read-State to SDN-controlled switch. Collects statistics and counter values from the switch’s flow table and its ports. Those counters are the third column of the match-plus-action table from section 4.4.
  4. SDN controller sends Send-Packet to SDN-controlled switch. Tells the switch to send one specific packet out of one specified port. The packet itself travels in the message payload.

Lifelines, left to right: SDN controller (server), SDN-controlled switch (link-layer switch).

The six OpenFlow messages the book names
MessageDirectionWhat it does

Cells marked ⓘ have an explanation — click to read it.

Direction is the thing to remember: the controller programs and reads, and the switch reports. Click a cell marked ⓘ.

In plain words

Six messages, and the split between them is the whole protocol.

The controller programs and reads. Configuration and Modify-State write; Read-State reads; Send-Packet injects one packet.

The switch reports. Port-status says something changed. Packet-in says “I have a packet and no rule for it — you decide.”

Packet-in is the interesting one, because it is how a switch that computes nothing can still handle something it has never seen. It asks. Section 4.4 called sending unmatched packets to the controller the default behaviour; this is the message that does it.

Principles in practice: Google’s B4

Google runs a dedicated wide-area network, B4, connecting its data centres and server clusters, with a Google-designed SDN control plane built on OpenFlow. Section 2.6.4 met the same network from the content side.

The results are the reason it is famous. B4 drives its WAN (Wide Area Network) links at near 70 % utilization over the long run, two to three times typical link utilization. It also splits application flows over several paths, by application priority and current demand.

The book gives three reasons B4 suits SDN particularly well.

  1. Google controls every device, from edge servers in IXP (Internet Exchange Point) s and ISP (Internet Service Provider) s to the routers in its core.
  2. Its most demanding applications are large data copies between sites, and those can wait when interactive traffic needs the link rate.
  3. Only a few dozen data centres are connected, so centralized control is feasible.
Google’s B4, mapped onto Figure 5.15
In B4Corresponds to

Cells marked ⓘ have an explanation — click to read it.

The sidebar on book page 417 is a real deployment described in the same vocabulary. Every piece corresponds to something in the diagram above.

Two more details are worth keeping. The agents reach their controllers over a separate out-of-band network, distinct from the one carrying data-centre traffic — so a data-plane problem cannot cut the control plane off. And Paxos is used to run hot replicas of the network control server components, which is the “logically centralized, physically distributed” idea again, in production.

Check yourself

Check yourself

0 of 7 answered
  1. 1.What does an SDN-controlled switch compute?

  2. 2.The book says the SDN controller is "logically centralized". What does that mean in practice?

  3. 3.Which of the controller's three layers does OpenFlow belong to?

  4. 4.predictA packet arrives at an SDN-controlled switch and matches no entry in its flow table. What happens?

  5. 5.Which OpenFlow message actually installs forwarding behaviour in a switch?

  6. 6.OpenFlow runs over TCP with a default port of 6653. Why does that matter architecturally?

  7. 7.Google's B4 keeps running BGP and IS-IS. Does that contradict its being an SDN network?

What to remember

  • Four characteristics define SDN: flow-based forwarding, data and control planes separated, control functions outside the switches, and a programmable network. The switches compute nothing — they run match plus action from a flow table somebody else wrote.
  • Northbound goes up to the applications; southbound goes down to the devices. Almost every question in section 5.5 is about one of these two boundaries.
  • Unbundling is the point. Switches, controllers and applications became three separate markets, as hardware, operating systems and applications did for computers.