§labChapter 2 Labs, Assignments and Interview

Application layer Kurose & Ross pp. 175–180 · ~14 min read

  • conditional get
  • resource record
  • welcoming socket
  • simple mail transfer protocol

Where you are

  • Application layer you are here
  • Transport layer
  • Network layer
  • Link layer
  • Physical layer

Two Wireshark labs rebuilt as captures you can click through, four socket assignments written out in full, and four questions to the man who invented the Web.

What is on this page

  • Wireshark Lab: HTTP (HyperText Transfer Protocol) — a real GET and its reply, field by field.
  • Wireshark Lab: DNS (Domain Name System) — a real query and answer, with the DNS message format §2.4.3 drew now filled with actual bytes.
  • Four socket programming assignments — complete working Python, with the expected output explained.
  • An interview with Tim Berners-Lee.

Why this matters

The book’s labs assume you have Wireshark and a live network. This site has neither, so each lab is rebuilt as a canned capture — and a canned capture is worth nothing unless it is actually correct.

Both captures here were generated byte by byte with real checksums, and both were then verified by an independent implementation that parsed them back. The DNS one round-trips through an RFC (Request For Comments) 1035 parser, compression pointer and all. If you check the arithmetic, it will hold.

Wireshark Lab: HTTP

The book’s lab explores the basic GET and reply interaction, HTTP message formats, persistent and non-persistent connections, and authentication. Here is the first and most important part of that: one complete page fetch, from the TCP (Transmission Control Protocol) handshake to the response.

This is the chapter 1 capture, reused deliberately

Section 1.6 built this capture for its own lab, and it is exactly what this one needs. Rebuilding a near-identical HTTP exchange would have added bytes and taught nothing.

What has changed is what you now know to look for. In chapter 1 you could see the request line. Now you can read the header lines, and you know why each is there.

A web page fetch, captured
No.TimeSourceDestinationProtocolLengthInfo
10.000000192.168.1.24198.51.100.7TCP5449152 → 80 [SYN] Seq=0 Win=64240
20.180216198.51.100.7192.168.1.24TCP5480 → 49152 [SYN, ACK] Seq=0 Ack=1 Win=65535
30.180298192.168.1.24198.51.100.7TCP5449152 → 80 [ACK] Seq=1 Ack=1 Win=64240
40.180452192.168.1.24198.51.100.7HTTP173GET /index.html HTTP/1.1
50.360912198.51.100.7192.168.1.24TCP5480 → 49152 [ACK] Seq=1 Ack=120
60.361847198.51.100.7192.168.1.24HTTP175HTTP/1.1 200 OK (text/html)
70.361998192.168.1.24198.51.100.7TCP5449152 → 80 [ACK] Seq=120

Packet 1 — The greeting. No web page is requested yet — this packet exists only to ask whether the server is willing to talk. Compare it with the first “Hi” in the human-protocol ladder of §1.1.

Protocol tree — click a field

The actual bytes

0000  aa bb cc 00 00 01 aa bb  cc 00 00 11 08 00 45 00  ..............E.
0010  00 28 1c 46 40 00 40 06  32 8f c0 a8 01 18 c6 33  .(.F@.@.2......3
0020  64 07 c0 00 00 50 9e 1c  3a 55 00 00 00 00 50 02  d....P..:U....P.
0030  fa f0 30 34 00 00                                ..04..

Seven real frames: the TCP three-way handshake, the HTTP request, and the start of the response. Every byte in the hex pane is genuine, including both checksums — you can verify them by hand if you like.

Work through it — five things to find

  1. How many packets pass before any HTTP appears? Three. Packets 1 to 3 are the TCP three-way handshake, invisible to both programs, and they cost a full round-trip time. Section 2.7.2 measured that from the other side.
  2. Find the Host: header line in packet 4. Section 2.2.3 asked why it exists when a connection to the host already exists. The answer was web proxy caches.
  3. Find Content-Length: in packet 6, and check it. Compare it against the actual number of body bytes in the hex pane. Problem P16 explains why HTTP needs this field where SMTP does not.
  4. Compare Date: with Last-Modified:. One is when this response was sent, the other when the object last changed. Only the second is any use to a cache.
  5. Packet 5 carries no data at all. It is a pure acknowledgement. Count how many of the seven packets carry application data — the answer is two, and that ratio is the argument section 2.7.2 made about connections.

What this capture cannot show you

The book’s lab also covers large files split across many segments, embedded objects fetched over the same persistent connection, and HTTP authentication. This capture has none of those, because it is one small object on one connection.

Section 2.2.2’s ladders show the multi-object case, and section 2.2.5’s ladder shows the conditional GET reaching 304 Not Modified. Neither is a substitute for seeing it on the wire, and this site is honest about the gap rather than pretending a two-packet capture covers a six-part lab.

Wireshark Lab: DNS

The book’s own framing is worth keeping: from the client’s standpoint DNS is simple — a query goes to the local DNS server and a response comes back. Much goes on under the covers, invisible to the client, as the hierarchy resolves the name recursively or iteratively.

So this capture is deliberately just those two messages. Everything section 2.4.2 described happens between them, and none of it is here, because none of it reaches the client.

A DNS lookup, byte by byte
No.TimeSourceDestinationProtocolLengthInfo
10.000000192.168.1.2441.72.0.53DNS75Standard query 0x1a2b A www.example.edu
20.01060041.72.0.53192.168.1.24DNS91Standard query response 0x1a2b A www.example.edu A 198.51.100.7

Packet 1 — One datagram out. Nothing was established first, and this single packet contains the entire question. Everything §2.4.2 described — the root, the TLD server, the authoritative server — happens after this, out of sight, and only if the resolver does not already know the answer.

Protocol tree — click a field

The actual bytes

0000  aa bb cc 00 00 01 aa bb  cc 00 00 11 08 00 45 00  ..............E.
0010  00 3d 62 a1 00 00 40 11  2c d2 c0 a8 01 18 29 48  .=b...@.,.....)H
0020  00 35 cf 7a 00 35 00 29  eb 1d 1a 2b 01 00 00 01  .5.z.5.)...+....
0030  00 00 00 00 00 00 03 77  77 77 07 65 78 61 6d 70  .......www.examp
0040  6c 65 03 65 64 75 00 00  01 00 01                 le.edu.....

The two messages that turn a name into an address. Click any field — the DNS section is exactly the format §2.4.3 drew, now with real bytes under it.

Work through it — six things to find

  1. The transaction ID, in both packets. 0x1a2b in the query, and the same value copied into the reply. This is the only thing tying the two together — UDP (User Datagram Protocol) keeps no connection, so without it a client with several queries in flight could not tell the answers apart.
  2. The flags. The query has 0x0100: QR=0, and RD=1, recursion desired. That single bit is what asks the local server to walk the hierarchy on your behalf — the one recursive query in section 2.4.2’s chain.
  3. The reply’s flags: 0x8180. QR=1, RD echoed, RA=1. And AA=0 — this resolver is not authoritative for the name. It answered from its cache, which is exactly what section 2.4.2 said a cache may do.
  4. The name, in the hex pane. It is not plain text. Read the bytes: 03 w w w 07 e x a m p l e 03 e d u 00. Each label carries its own length, and a zero byte ends the name.
  5. The four number-of fields. Query: 1 question, 0 answers. Reply: 1 question, 1 answer. There is no length field in a DNS message anywhere — these counts are what tell the receiver how much follows.
  6. The answer’s name: c0 0c. Two bytes, not a name. This is a compression pointer: the top two bits mark it as a pointer, and the remaining 14 bits give offset 12, where the question’s name already sits. It is why the reply is shorter than you would expect, and it is what real DNS replies do.

In plain words

Two packets, 75 and 91 bytes, 10.6 milliseconds apart. That is the whole of DNS as a client experiences it.

Section 2.4.1 put this exchange in front of the entire HTTP conversation and measured what it adds. Now you can see the bytes it adds them with — and how few there are.

Socket Programming Assignments

The book summarises four assignments and puts the skeleton code on its companion website. Complete working versions are given here, with the expected output explained. Each builds directly on section 2.7.

Assignment 1 — Web Server

A web server in Python capable of processing one request. It creates a welcoming socket , accepts a browser, receives the request, parses it for the filename, fetches the file, builds a response with header lines, and sends it. If the file does not exist, it returns 404 Not Found.

from socket import *
import sys

serverSocket = socket(AF_INET, SOCK_STREAM)
serverPort = 6789
serverSocket.bind(('', serverPort))
serverSocket.listen(1)
print(f'Ready to serve on port {serverPort}')

while True:
    connectionSocket, addr = serverSocket.accept()
    try:
        message = connectionSocket.recv(1024).decode()
        filename = message.split()[1]          # "GET /index.html HTTP/1.1" -> /index.html
        with open(filename[1:], 'rb') as f:    # strip the leading slash
            body = f.read()
        header = (
            'HTTP/1.1 200 OK\r\n'
            'Content-Type: text/html\r\n'
            f'Content-Length: {len(body)}\r\n'
            'Connection: close\r\n'
            '\r\n'
        )
        connectionSocket.send(header.encode())
        connectionSocket.send(body)
    except IOError:
        connectionSocket.send(
            b'HTTP/1.1 404 Not Found\r\n'
            b'Content-Type: text/html\r\n'
            b'Connection: close\r\n'
            b'\r\n'
            b'<html><body><h1>404 Not Found</h1></body></html>'
        )
    except IndexError:
        pass                                   # not an HTTP request at all
    connectionSocket.close()

What you should see, and why

Put a file index.html beside the program, run it, then visit http://<your-host>:6789/index.html. The page renders. Ask for /nothere.html and the browser shows your 404 page.

Three details worth understanding rather than copying:

  • filename = message.split()[1] works because the request line is GET /index.html HTTP/1.1 and splitting on whitespace makes the URL (Uniform Resource Locator) field the second token. That is section 2.2.3’s request line, being parsed.
  • filename[1:] strips the leading slash, because the path name in the request is absolute and the file on disk is not.
  • The blank line before the body is not optional. '\r\n' ends the last header and the second '\r\n' is the blank line. Omit it and the browser waits for headers that never end.

If you run this on a host that already has a web server, use a port other than 80 — as this code does.

Where this is not a real web server. It handles one request at a time, so a second browser waits. It has no security whatsoever and will serve any file it can open, including ones above its own directory. And it guesses Content-Type. Do not expose it to a network you do not control.

Assignment 2 — UDP Pinger

Send 10 ping messages over UDP, print the round-trip time for each, and wait no more than one second before assuming a packet was lost. The server code is given by the book; the client is the assignment.

from socket import *
import time

serverName = 'localhost'
serverPort = 12000

clientSocket = socket(AF_INET, SOCK_DGRAM)
clientSocket.settimeout(1)          # the whole point of the assignment

for seq in range(1, 11):
    sendTime = time.time()
    message = f'Ping {seq} {sendTime}'
    try:
        clientSocket.sendto(message.encode(), (serverName, serverPort))
        reply, serverAddress = clientSocket.recvfrom(1024)
        rtt = time.time() - sendTime
        print(f'Ping {seq}  reply from {serverAddress[0]}  rtt = {rtt * 1000:.2f} ms')
    except timeout:
        print(f'Ping {seq}  Request timed out')

clientSocket.close()

What you should see, and why

Ten lines. Some show a round-trip time of a fraction of a millisecond on localhost. Others say Request timed out, because the book’s server deliberately discards about 30 % of the packets to simulate loss.

The line that matters is clientSocket.settimeout(1). Without it, recvfrom() blocks forever the first time a packet is lost, and the program hangs on ping 1. Section 2.7.1 noted exactly this: a UDP client waiting for a reply that will never come has nothing to tell it so.

That is why the assignment specifies a timeout, and it is the whole difference between a program that copes with an unreliable service and one that assumes a reliable one.

Where this differs from the real ping: the standard program uses ICMP (Internet Control Message Protocol) , which is a network-layer protocol requiring privileges, and chapter 5 covers it. This one is an ordinary UDP application, which is why it needs a cooperating server at the far end while real ping does not.

Assignment 3 — Mail Client

Establish a TCP connection to a mail server, conduct the SMTP dialogue by hand, send a message, and close the connection. This is section 2.3.1’s transcript, written as code.

from socket import *

mailserver = ('smtp.example.edu', 25)
msg = '\r\nI love computer networks!'
endmsg = '\r\n.\r\n'

clientSocket = socket(AF_INET, SOCK_STREAM)
clientSocket.connect(mailserver)

def expect(code):
    reply = clientSocket.recv(1024).decode()
    print(reply, end='')
    if reply[:3] != code:
        raise RuntimeError(f'expected {code}, got: {reply.strip()}')

expect('220')                                        # server greets first
clientSocket.send(b'HELO client.example.net\r\n')
expect('250')
clientSocket.send(b'MAIL FROM: <alice@example.net>\r\n')
expect('250')
clientSocket.send(b'RCPT TO: <bob@example.edu>\r\n')
expect('250')
clientSocket.send(b'DATA\r\n')
expect('354')                                        # "go ahead"
clientSocket.send(b'Subject: a test\r\n')
clientSocket.send(msg.encode())
clientSocket.send(endmsg.encode())                   # the lone period
expect('250')
clientSocket.send(b'QUIT\r\n')
expect('221')
clientSocket.close()

What you should see, and why

The printed output is the server’s half of section 2.3.1’s transcript, in order: 220, 250, 250, 250, 354, 250, 221.

Three details:

  • The server speaks first. The first expect('220') happens before the client sends anything. An HTTP client would never do this.

  • endmsg is \r\n.\r\n — the CRLF (Carriage Return, Line Feed) . CRLF of section 2.3.1. A line containing a single period is how SMTP marks the end of a message body, and problem P16 explains why HTTP cannot use the same trick.

  • MAIL FROM: is not From:. The command tells the server where to deliver bounces; the Subject: and any From: you add inside DATA are part of the message. Envelope and letterhead.

    Where this will fail today: almost no mail server will relay a message from an unauthenticated client, and many providers block outbound port 25 entirely to slow spam. Expect the 220 greeting and then a refusal at MAIL FROM: or RCPT TO:. That refusal is the lesson — the protocol in the book is still the protocol on the wire, wrapped in checks the 1982 RFC never imagined. To see it succeed you need a local mail server you control.

Assignment 4 — Web Proxy

Receive an HTTP request from a browser, generate a new request for the same object, send it to the origin server, and pass the response back. This is section 2.2.5’s proxy, in about thirty lines.

from socket import *

tcpSerSock = socket(AF_INET, SOCK_STREAM)
tcpSerSock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
tcpSerSock.bind(('', 8888))
tcpSerSock.listen(5)
print('Proxy ready on port 8888')

while True:
    tcpCliSock, addr = tcpSerSock.accept()
    message = tcpCliSock.recv(4096).decode()
    if not message:
        tcpCliSock.close()
        continue
    print(message.split('\n')[0])                 # the request line

    # "GET http://host/path HTTP/1.1" -> host and /path
    url = message.split()[1]
    host = url.split('//')[1].split('/')[0]
    path = '/' + '/'.join(url.split('//')[1].split('/')[1:])

    try:                                          # serve from the cache if we can
        with open('cache/' + host + path.replace('/', '_'), 'rb') as f:
            tcpCliSock.send(f.read())
            print('  cache hit')
    except IOError:
        originSock = socket(AF_INET, SOCK_STREAM) # the proxy is now a CLIENT
        originSock.connect((host, 80))
        originSock.send(
            f'GET {path} HTTP/1.0\r\nHost: {host}\r\nConnection: close\r\n\r\n'.encode()
        )
        body = b''
        while True:
            chunk = originSock.recv(4096)
            if not chunk:
                break
            body += chunk
        originSock.close()
        tcpCliSock.send(body)
        with open('cache/' + host + path.replace('/', '_'), 'wb') as f:
            f.write(body)                         # store a copy for next time
        print('  cache miss — fetched and stored')
    tcpCliSock.close()

What you should see, and why

Create a cache/ directory, run it, set your browser’s proxy to your host and port 8888, then visit a plain http:// site. The first request prints cache miss; reload and it prints cache hit and returns instantly.

This program is section 2.2.5’s central claim, made concrete:

a cache is both a server and a client at the same time

Look at the code. tcpSerSock is a welcoming socket, so towards the browser this program is a server. Then originSock.connect(...) — towards the origin server it is a client. Both roles, in the same loop body, on the same machine.

Notice also that it needs the hostname from the request. The browser sends GET http://host/path to a proxy rather than GET /path, and the Host: header carries it too. That is the reason section 2.2.3 gave for Host: existing at all.

Where this is not a real cache: it never checks whether its copy is stale, so it will serve a year-old page forever. A real cache issues the conditional GET of §2.2.5 and honours the reply. It also handles only HTTP, not HTTPS — a proxy cannot read an encrypted connection, which is why real proxies tunnel rather than cache it. And the filename scheme is naive enough to collide.

An interview with Tim Berners-Lee

Voices from the field

Sir Tim Berners-Lee invented the World Wide Web. In 1989, working as a fellow at CERN, he proposed an Internet-based distributed information management system including the original version of the HTTP (HyperText Transfer Protocol) protocol. In the same year he implemented it on a client and a server. He received the 2016 Turing Award for “inventing the World Wide Web, the first Web browser, and the fundamental protocols and algorithms allowing the Web to scale”. He co-founded the World Wide Web Foundation and is a professor at Oxford and at MIT.

You originally studied physics. How is networking similar to physics?

When you study physics, you imagine what rules of behavior on the very small scale could possibly give rise to the large-scale world as we see it. When you design a global system like the Web, you try to invent rules of behavior of Web pages and links and things that could in the large create a large-scale world as we would like it. One is analysis and the other synthesis, but they are very similar.

What influenced you to specialize in networking?

After my physics degree, the telecommunications research companies seemed to be the most interesting places. The microprocessor had just come out, and telecommunications was switching very fast from hardwired logic to microprocessor-based systems. It was very exciting.

What is the most challenging part of your job?

When two groups disagree strongly about something, but want in the end to achieve a common goal, finding exactly what they each mean and where the misunderstandings are can be very demanding. The chair of any working group knows that. However, this is what it takes to make progress toward consensus on a large scale.

What people have inspired you professionally?

My parents, who were involved in the early days of computing, gave me a fascination with the whole subject. Mike Sendall and Peggie Rimmer, for whom I worked at various times at CERN are among the people who taught me and encouraged me. I later learned to admire the people, including Vanevar Bush, Doug Englebart, and Ted Nelson, who had had similar dreams in their time but had not had the benefit of the existence for PCs and the Internet to be able to realize it.

Worth sitting with

The first answer is the best short description of protocol design anybody has given. Physics asks what small-scale rules would produce the world we observe. Protocol design asks what small-scale rules would produce the world we want.

Everything in this chapter is that exercise. Rarest first is a rule about one peer that produces an even spread across a torrent. Tit-for-tat is a rule about four neighbours that produces a system where contributing beats freeriding. DNS caching is a rule about one server that keeps thirteen root servers from being asked. In each case the rule is local and small, and the behaviour that matters is global and emergent.

The third answer is worth noticing too. The hardest part of the job is not technical.

Check yourself

Check yourself

0 of 6 answered

  1. 1.predictIn the DNS capture, compare the transaction ID in the two packets. What would break if the reply carried a different one?

  2. 2.The DNS reply has AA = 0. What does that tell you about the server that answered?

  3. 3.The two bytes `c0 0c` appear where the answer's name should be. What are they?

  4. 4.In the UDP pinger, what breaks if you remove `clientSocket.settimeout(1)`?

  5. 5.The web proxy creates a welcoming socket and also calls connect(). What does that make it?

  6. 6.Berners-Lee compares designing the Web to physics. What is the comparison he actually draws?

What to remember

  • The HTTP capture shows three handshake packets before any HTTP appears, and only two of seven carrying application data.
  • A DNS name on the wire is length-prefixed labels, not text: 03 www 07 example 03 edu 00, and the answer’s name is a compression pointer to offset 12. The transaction ID is the only thing tying the reply to the query, because UDP keeps no connection.
  • The web proxy is a server towards the browser and a client towards the origin, in the same loop body. That is §2.2.5’s claim, in code you can point at.