Networking: The Physics of Packets
Why 'fast' internet feels slow. A deep dive into TCP Congestion Control, Head-of-Line Blocking, and the BGP map of the world.
🎯 What You'll Learn
- Deconstruct the TCP 3-Way Handshake at the packet level
- Analyze Head-of-Line (HOL) Blocking in HTTP/1.1 vs HTTP/2
- Calculate Bandwidth-Delay Product (BDP) for tuning
- Trace a BGP Route hijack
- Differentiate MTU and MSS and why fragmentation kills performance
📚 Prerequisites
Before this lesson, you should understand:
🔌 Try It: TCP Handshake Builder
Click packets in the right order to establish a TCP connection:
🔌 TCP Handshake Builder
Click packets in the correct order to establish a TCP connection.
⚡ Try It: Port Number Speed Run
Quick! What port is SSH? Race the clock:
⚡ Port Number Speed Run
Learn common service ports! Pick the correct port for each service.
15 seconds per question. Use hints if you're stuck.
Introduction
“The Network” is not a cloud. It is a hostile environment where packets are dropped, reordered, and corrupted by cosmic rays. Your application succeeds only because TCP spends 30% of its energy apologizing for errors.
In this lesson, we stop treating the network like a reliable pipe and start treating it like a probabilistic delivery service.
The Physics: TCP Congestion Control
How does your computer know how fast to send data? If it sends too slow: Bandwidth is wasted. If it sends too fast: Router buffers overflow, packets drop, and throughput collapses.
The Solution: The Congestion Window (cwnd).
TCP starts slow (Slow Start). It doubles packets every RTT.
1, 2, 4, 8, 16… DROP!
When a packet drops, TCP panics. “The network is full!”
It cuts cwnd in half (Reno) or by 30% (Cubic) and linearly inches back up.
Physics: Your download speed is practically a Sawtooth Wave oscillating around the bottleneck capacity.
The Bottleneck: Head-of-Line (HOL) Blocking
Imagine a one-lane highway. If the front car crashes, everyone waits. This is HOL Blocking.
HTTP/1.1 (The Crash)
You open a TCP connection. You request style.css. It takes 2 seconds.
Result: You cannot request script.js until style.css arrives. The pipe is idle.
HTTP/2 (The Multi-Lane Highway)
You multiplex streams. Block style.css? Fine. script.js drives in the lane next to it.
But TCP is still one lane. If a TCP Packet is lost, the Operating System holds back all HTTP/2 streams until that packet is retransmitted.
HTTP/3 (QUIC)
The solution is to ditch TCP. UDP has no lanes. QUIC implements “lanes” in userspace. One lost packet only stalls one stream.
Deep Dive: BGP (Border Gateway Protocol)
TCP is how you talk. IP is where you go. BGP is the map. The internet is a mesh of 70,000 Autonomous Systems (AS). BGP is the protocol where they whisper to each other: “I can reach IP range 192.168.0.0/16”.
The Flaw: BGP is (mostly) trust-based. If I announce “I own YouTube’s IP Address”, and enough routers believe me, I can hijack global traffic. This happens (accidentally or maliciously) every year.
Code: Inspecting the Handshake
Don’t trust the browser. Trust tcpdump.
# Capture packets on port 80
sudo tcpdump -i eth0 port 80 -n -S
# Output Physics:
# 1. SYN (Seq=0) -> "Let's talk."
# 2. SYN-ACK (Seq=0, Ack=1) -> "Okay. I hear you."
# 3. ACK (Seq=1, Ack=1) -> "Connection Open."
# 4. PSH (Seq=1:500) -> "Here is 500 bytes of data."
Note the Ack number: It is not “I received packet 5”. It is “I am expecting byte number 501”. TCP counts bytes, not packets.
Practice Exercises
Exercise 1: The Bandwidth-Delay Product (Beginner)
Scenario: 1Gbps Link. 100ms RTT. Task: How much data must be “in flight” (on the wire) to fill the pipe? (Hint: . If your TCP Buffer is 64KB, your speed is capped at 5Mbps!).
Exercise 2: Fragmentation (Intermediate)
Scenario: MTU (Max Transmission Unit) is 1500 bytes. Typical Ethernet. You try to send a 2000 byte UDP packet. Task: What happens? (Fragmented into 1500 + 500 headers). Why is this bad for performance?
Exercise 3: Traceroute Analysis (Advanced)
Task: Run traceroute google.com.
Identify:
- Your Gateway.
- Your ISP’s backbone.
- The moment you enter Google’s network (AS15169).
Knowledge Check
- Why does TCP start huge downloads slowly?
- What is the difference between Flow Control and Congestion Control?
- Why is HTTP/3 built on UDP?
- What does “ACK 500” mean?
- Why does having a larger TCP Window Size improve speed on high-latency links?
Answers
- Slow Start. It probes network capacity exponentially to avoid immediate congestion.
- Flow: Receiver says “I’m full”. Congestion: Network router says (implicitly) “I’m full”.
- To avoid TCP HOL Blocking. It moves reliability logic to userspace where streams are independent.
- “I have received everything up to 499. Send me 500 next.”
- Filling the pipe. With high RTT, ACKs take a long time to return. You need to send more data “blindly” to keep the link busy. (Little’s Law again!).
Summary
- TCP: A state machine that guesses bandwidth.
- BGP: The glue holding the internet together (barely).
- Latency: The ultimate speed limit. You cannot beat the speed of light.
Questions about this lesson? Working on related infrastructure?
Let's discuss