HTTP Fundamentals

The last module ended with a working pipe. Two machines complete a three-way handshake, TCP guarantees the bytes arrive in order with nothing missing, and after that the network stops caring what travels through. HTTP is what a browser pours into that pipe. And it is text.

That is the part worth sitting with. When a client asks a server for a web page, it does not send a special binary blob. It sends a few lines of readable ASCII, the server sends a few lines back, and everything the browser paints on screen started as that plain conversation. HTTP, the HyperText Transfer Protocol, is the agreed format for those lines: what the client may say, what the server says back, and where each message ends. Anyone who can read the raw text can see everything the two sides said.

Where HTTP sits

HTTP does not move bytes on its own. It hands them to TCP, which handles the delivery. A browser opens a TCP connection to port 80 on the server, writes the request as text, and reads the response off the same connection. Port 80 is HTTP's home by convention, one of the well-known ports reserved for common services.

When the traffic is encrypted it runs over HTTPS on port 443 instead, wrapped in TLS so anyone reading the wire sees ciphertext. The message format inside is identical either way. TLS changes who can read the conversation, not what the conversation says. Everything in this lesson is that inner format.

Anatomy of a request

A request is three parts stacked top to bottom, in a fixed order the server counts on.

POST /login HTTP/1.1
Host: shop.example.com
User-Agent: Mozilla/5.0
Content-Type: application/x-www-form-urlencoded
Content-Length: 27

user=admin&password=hunter2

The first line is the request line: a method, a path, and the protocol version. POST is the method, /login is the path on the server being addressed, and HTTP/1.1 is the version both sides speak. Then come the headers, one per line, each a name and a value. They carry metadata about the request: which host, what software is asking, what kind of data follows. A single blank line marks the end of the headers. Everything after that blank line is the body, the payload the client is submitting. A request that only reads a page carries no body, so it stops at the blank line.

Two headers in that example do real work. Host names which site the request is for, because one server often answers for many domains on the same IP address and has to be told which one. Content-Length tells the server exactly how many bytes of body to read, which is how it knows where this request ends and the next one could begin on a reused connection. A wrong Content-Length leaves the server hanging, waiting for bytes that never arrive.

The methods

The method is the verb. It tells the server what the client wants done with the resource at that path.

• GET reads a resource and changes nothing. Any parameters ride in the URL after a ?, so GET /search?q=shoes carries its input in the address itself. That input lands in server logs and browser history, which is why passwords never belong in a GET.
• POST submits data in the body, below the blank line. Logins, uploads, and form posts use it. The body is not more secret than a URL, but it stays out of the address bar and the logs that record URLs.
• PUT writes a resource at the given path, creating or replacing it. DELETE removes it. Both change server state, and both are often left switched on by a misconfiguration when nobody meant to expose them.

Two more are worth knowing on sight. HEAD asks for the response headers with no body, a fast way to check whether a path exists and what it would return. OPTIONS asks which methods a path accepts, and the server answers in an Allow header. A server that lists PUT or DELETE there has told an attacker how to write to it.

The response

The server answers in the same shape, flipped. First a status line, then headers, then a blank line, then the body.

HTTP/1.1 200 OK
Server: Apache/2.4.41 (Ubuntu)
Content-Type: text/html; charset=UTF-8
Content-Length: 3021
Set-Cookie: session=9f2c8a1b; HttpOnly

<!DOCTYPE html>
...

The status line leads with the version, then a three-digit status code, then a short reason phrase. 200 OK means the request succeeded and the body is the thing that was asked for. The reason phrase (OK, Not Found) is for humans, and clients ignore it; the number is what they act on. Below the status line the headers describe the response: what kind of content the body is, how long it is, what software produced it. Then the blank line, then the body itself, which for a web page is HTML.

Status codes

The first digit of the code sorts every response into one of five classes, and knowing the class is usually enough to know what happened.

A 2xx means the request worked. A 3xx means the thing moved, and the response carries a Location header pointing where to go next; a browser follows it automatically, which is why a login often ends in a 302 to a dashboard. A 4xx blames the request: 401 means no valid credentials were sent, 403 means the server understood the request and refuses it, 404 means nothing lives at that path. A 5xx means the server broke while handling a request it accepted, and the error page it returns sometimes spills a stack trace or a framework version.

The gap between two 4xx codes is information. A 404 on one path and a 403 on another draws a map: the 403 confirms something real exists there and the server is guarding it, which is exactly where an attacker pushes next. A wall of identical 404s says nothing is there. One 403 in the middle says something is.

One connection, many messages

Opening a TCP connection costs a full three-way handshake, and a single page pulls dozens of resources: HTML, stylesheets, scripts, images. HTTP/1.1 keeps the connection open by default so those requests reuse one handshake instead of paying for a new one each time. The header that governs it is Connection. A Connection: keep-alive holds the socket open for the next request, and a Connection: close tells the other side to hang up after this exchange.

Reusing a connection raises the one question the protocol has to answer: where does a message stop and the next one start? Two mechanisms settle it. A Content-Length header gives the exact byte count, so the reader consumes that many bytes and knows the message is finished. When the length is not known up front, the server switches to chunked transfer encoding (Transfer-Encoding: chunked), sending the body in labeled pieces followed by a final zero-length chunk that marks the end. Either way, the boundary is explicit, because a connection carrying request after request cannot leave it to guesswork.

Headers that carry identity

Most headers are bookkeeping. A few decide who the server thinks you are.

HTTP is stateless: every request stands completely alone, and the server keeps no memory of the last one. That is a problem the moment a site needs to know a visitor already logged in. The fix is the cookie. On a successful login the server sends Set-Cookie: session=9f2c8a1b, the browser stores that value, and it attaches the value to every later request to the same site as Cookie: session=9f2c8a1b. That echoed value is the session id, and it is the only thing tying a stream of unrelated requests into one logged-in session. Whoever holds a valid session cookie is, as far as the server can tell, the user.

A handful of other headers matter on sight. User-Agent names the client software and is trivially faked. Content-Type declares what the body is, and a server that trusts it blindly can be fed the wrong type. Authorization carries credentials on APIs. Server and X-Powered-By name the software and often its version, which is free reconnaissance: the version string points straight at whatever known weaknesses that release shipped with, before an attacker has sent anything real.

Do it by hand

The whole exchange is reproducible from a shell. curl speaks HTTP directly, and its -v flag prints the full request and response, every header in both directions, so nothing hides behind a browser.

$curl -v http://example.com/

$curl -I http://example.com/

The > block is the request curl built and sent. Four lines, and every one is readable: a request line, three headers, and the blank line that ends them. The < block is the server's answer, opening with the status line the client acts on. This is the same conversation a browser has, minus the rendering.

A request and a response are the whole vocabulary of the web, and reading them cold is the skill everything else here rests on. What none of this shows is what happens between the server accepting a request and sending the response back: the application code, the database sitting behind it, and the seams where one component trusts another it should have checked. The next lesson, how web apps work, opens that box.