§2.2.5–2.2.6Web Caching · HTTP/2

Web caching

A web cache web cache Simple A nearby machine holding copies of pages, so requests need not travel far. Precise Also called a proxy server: a network entity that satisfies HTTP requests on behalf of an origin Web server. It has its own disk storage and keeps copies of recently requested objects. A cache is both a server and a client at the same time — a server to the browser, a client to the origin server. Caches are typically bought and installed by an ISP. introduced in ch. 2 — open in glossary — also called a proxy server — is a network entity that satisfies HTTP (HyperText Transfer Protocol) HyperText Transfer Protocol The application-layer protocol that requests and transfers Web documents. introduced in ch. 1 requests on behalf of an origin web server origin server Simple The machine that really owns an object, as opposed to a cache holding a copy of it. Precise The Web server on which an object permanently resides. A Web cache contacts the origin server when it does not have a copy of a requested object, then stores a copy locally and forwards one to the browser. introduced in ch. 2 — open in glossary . It has its own disk storage and keeps copies of recently requested objects there.

A browser can be configured so that all of its HTTP requests go to the cache first.

Caches are typically purchased and installed by an ISP (Internet Service Provider) Internet Service Provider Connects hosts, and other ISPs, to the rest of the Internet. introduced in ch. 1 . A university might install one on its campus network and point every campus browser at it. A large residential ISP might install several and preconfigure the browsers it ships.

Why caches were deployed

Two reasons, and the second is the one people forget.

1. A cache can substantially reduce the response time for a request. That is especially true when the bottleneck link between client and origin server is much slower than the one between client and cache. If the cache sits on the same fast local network and holds the object, it delivers it almost at once.
2. A cache can substantially reduce traffic on an institution’s access link. By reducing traffic, the institution does not have to upgrade its link as quickly, which reduces cost.

Caches also reduce web traffic in the Internet as a whole, which improves performance for everybody.

The example that makes the case

The institutional network is a high-speed LAN (Local Area Network) Local Area Network A network covering one building, campus or home. introduced in ch. 1 . A router in it and a router in the Internet are joined by a 15 Mbps link. The origin servers are scattered around the globe. Suppose:

• the average object size is 1 Mbit;
• the average request rate from the institution’s browsers is 15 requests per second;
• HTTP request messages are so small they create no traffic worth counting;
• the time from the Internet-side router forwarding a request to receiving the response is 2 seconds on average. Call this the Internet delay.

The total response time is the LAN delay plus the access delay plus the Internet delay. Now use traffic intensity traffic intensity Simple How full a link is: the bits arriving each second divided by the bits it can send. Precise La/R, where a is the average packet arrival rate in packets/second, L the packet length in bits and R the link rate in bits/second. If La/R > 1 the queue grows without bound; as La/R approaches 1 the average queuing delay increases rapidly. introduced in ch. 1 — open in glossary from section 1.4.2:

Two ways to fix it

Option 1: buy a bigger link. Raise the access rate from 15 Mbps to 100 Mbps. Traffic intensity falls to 0.15 and the delay between the two routers becomes negligible. The total response time becomes roughly 2 seconds — the Internet delay, which no link upgrade can touch. It also means paying for a 100 Mbps link, which is a costly proposition.

Option 2: install a cache. Hit rates hit rate Simple The share of requests a cache can answer from its own copies. Precise The fraction of requests that are satisfied by a cache. Hit rates typically range from 0.2 to 0.7 in practice. introduced in ch. 2 — open in glossary typically range from 0.2 to 0.7 in practice. Suppose 0.4 here. Then 40 % of requests are satisfied almost immediately — say within 10 milliseconds — by the cache on the same fast LAN. The other 60 % still go to the origin servers, so the access link now carries 60 % of the traffic and its intensity falls from 1.0 to 0.6. Below about 0.8, the delay is small. So:

$0.4 \times (0.01\ \text{s}) + 0.6 \times (2.01\ \text{s}) = 1.21\ \text{s}$

only the misses cross the access link: 60 % of requests
arrival rate on the link = 0.60 x 15 = 9.00 requests/s
intensity = 9.00 x 1,000,000 bits / 15.0 Mbps
          = 0.60
below 0.8 the queuing delay is small, tens of milliseconds

a hit is served from the cache over the LAN: 10.0 ms
a miss also crosses the access link and the Internet: 10.0 ms + 0.000 ns + 2.00 s = 2.01 s
average = 0.40 x 10.0 ms + 0.60 x 2.01 s
        = 4.00 ms + 1.21 s
        = 1.21 s

Through CDN (Content Distribution Network) Content Distribution Network Geographically distributed servers holding copies of content close to users. introduced in ch. 2 s, web caches are increasingly important in the Internet. A CDN company installs many geographically distributed caches, which localises much of the traffic. There are shared CDNs such as Akamai and Limelight, and dedicated ones such as Google’s and Netflix’s. Section 2.6.3 covers them properly.

The conditional GET

Caching introduces a new problem: the copy in the cache may be stale. The object at the origin server may have been modified since the copy was taken.

HTTP’s answer is the conditional GET conditional get Simple A request that says: send this file only if it has changed since the date I give you. Precise An HTTP request message that uses the GET method and includes an If-Modified-Since header line. The value of that line is the Last-Modified date the server sent when the object was cached. If the object has not been modified, the server replies 304 Not Modified with an empty entity body, so the cache may forward its own copy. introduced in ch. 2 — open in glossary . A request is a conditional GET if it uses the GET method and includes an If-Modified-Since: header line.

The value of If-modified-since: is exactly the value of the Last-Modified: line the server sent a week earlier. That is why section 2.2.3 called that header critical for caching.

HTTP/2

HTTP/2 was standardised in 2015. It was the first new version of HTTP since HTTP/1.1, which dates from 1997. It has taken off: over 40 % of the top 10 million websites supported it in 2020, and most browsers do too.

Its primary goals are to reduce perceived delay by multiplexing requests and responses over a single TCP (Transmission Control Protocol) Transmission Control Protocol The Internet transport protocol that delivers data reliably and in order, with flow control and congestion control. introduced in ch. 1 connection. It also provides request prioritization and server push, and compresses HTTP header fields efficiently.

The problem: head-of-line blocking

Recall that HTTP/1.1 uses persistent connections, so a whole page can travel over one TCP connection. That has two benefits: the server keeps fewer sockets open, and each page gets a fair share of the link.

But browser developers quickly discovered a problem.

Consider a page with an HTML (HyperText Markup Language) HyperText Markup Language The document format of the Web, created by Tim Berners-Lee at CERN between 1989 and 1991. introduced in ch. 1 base file, a large video clip near the top, and many small objects below it. Suppose the path has a low-to-medium speed bottleneck link, such as a slow wireless link. Over a single connection the video clip takes a long time to cross that link, and the small objects are delayed behind it. That is head-of-line blocking head-of-line blocking Simple One large item at the front of a queue makes everything behind it wait. Precise The problem in which a large object being sent over a single connection delays the small objects queued behind it. The book’s example is a video clip at the top of a page blocking the small objects below it over a slow bottleneck link. HTTP/1.1 browsers work around this by opening up to six parallel TCP connections; HTTP/2 solves it by breaking messages into frames and interleaving them. introduced in ch. 2 — open in glossary .

HTTP/1.1 browsers work around it by opening multiple parallel TCP connections, so objects arrive in parallel and small ones render sooner.

So one primary goal of HTTP/2 is to reduce or remove parallel connections. That reduces sockets at servers and lets congestion control work as intended. But with only one connection, HTTP/2 needs a careful mechanism to avoid head-of-line blocking.

The solution: framing

The framing sub-layer framing sub-layer Simple The part of HTTP/2 that cuts messages into frames and puts them back together. Precise The sub-layer of HTTP/2 that breaks a message into frames and binary-encodes them. It interleaves those frames with the frames of other messages over the single persistent TCP connection, and reassembles the original messages at the receiver. Binary encoding is more efficient to parse, gives slightly smaller frames, and is less error-prone. introduced in ch. 2 — open in glossary breaks each message into small HTTP/2 frames http/2 frame Simple A small piece of an HTTP/2 message. Pieces of different messages take turns on one connection. Precise The unit into which the HTTP/2 framing sub-layer breaks each HTTP message. Frames belonging to different messages can then be interleaved on one TCP connection and reassembled at the other end. On this site it is always called an HTTP/2 frame, never simply a frame, because “frame” already names the link-layer packet of §1.5. introduced in ch. 2 — open in glossary and interleaves the request and response messages on the same TCP connection.

When the server wants to send a response, the framing sub-layer breaks it up. The header field of the response becomes one HTTP/2 frame, and the body becomes one or more further ones. Those are interleaved with the pieces of other responses and sent over the single persistent TCP connection. At the client they are reassembled into the original messages, then processed by the browser as usual. A client’s requests are broken up and interleaved the same way.

The framing sub-layer also binary encodes what it sends. Binary protocols are more efficient to parse, give slightly smaller pieces, and are less error-prone.

Prioritization and server push

Message prioritization message prioritization Simple Telling the server which answers matter most, so it sends those frames first. Precise An HTTP/2 feature in which a client assigns each concurrent request a weight between 1 and 256, a higher number meaning higher priority. The client may also state a message’s dependency by giving the ID of the message it depends on. The server uses these weights to send the highest-priority frames first. introduced in ch. 2 — open in glossary lets developers customise the relative priority of requests. When a client sends concurrent requests it can assign each message a weight between 1 and 256, where higher means higher priority, and the server sends the highest-priority pieces first. A client can also state that one message depends on another, by giving the ID of the message it depends on.

Server push server push Simple The server sends files it knows you will need, before you ask for them. Precise An HTTP/2 feature in which the server answers the original request and also sends objects the client has not asked for. It can do this because the HTML base page states which objects are needed to render the page. The server analyses that page and sends them, which removes the delay of waiting for those requests. introduced in ch. 2 — open in glossary is the ability of a server to send several responses for a single client request. The HTML base page states which objects are needed to render the page. So the server need not wait for the browser to ask. It reads that page, works out what is needed, and sends those objects before any request for them arrives. That removes the delay of waiting for the requests.

HTTP/3

QUIC (Quick UDP Internet Connections) Quick UDP Internet Connections A transport-like protocol built in the application layer over UDP, carrying HTTP/3. The IETF now treats QUIC as a name rather than an abbreviation. introduced in ch. 2 , covered in chapter 3, is a new transport-like protocol implemented in the application layer over bare UDP (User Datagram Protocol) User Datagram Protocol A simple transport protocol with no reliability, no flow control and no congestion control. introduced in ch. 1 . It has several features HTTP wants: message multiplexing, per-stream flow control, and connection setup with very little delay.

HTTP/3 is designed to run over QUIC. As of 2020 it was described in Internet drafts and not yet fully standardised. Many HTTP/2 features — message interleaving among them — are subsumed by QUIC, which allows a simpler, streamlined design for HTTP/3.

Check yourself