What happens when...

What happens when you enter google.com into your browser and hit Enter?

This question is often asked on interviews. It’s an extremely broad one, which can be answered in an infinite amount of detail, akin to the coastline paradox. I’m guessing that the purpose of this question is not only to probe the candidate on their networking knowledge but equally to test their ability to summarise a complex topic. So, in practice, when it comes to this question, I would try to keep it short but still showcase my knowledge about some aspects of multiple layers of the Internet protocol suite - mention a few key steps from each level but don’t go into too much detail.

In order to limit the length of this blog post, and because I don’t want to get lost in a rabbit hole, I’m not going into the details below the transport layer of the OSI stack. I will also ignore the stuff going on with your keyboard, interrupts, URL parsing, the browser rendering the webpage, and many, many other details. Here we go!

# The Domain Name System

First, your browser has to figure out the IP address of the host that belongs to the google.com domain. This is achieved using the Domain Name System and it’s a quite complex, hierarchical process. The DNS query goes through a series of steps including these main DNS server types:

  • the DNS recursor (recursive resolver)
  • the root nameserver
  • the TLD nameserver
  • the authoritative nameserver
# DNS caching and configuration

However, not all of these servers have to be contacted for each query all the time, due to DNS caching which is implemented at different layers of this process. Caching in the browser, in the OS, and caching at the recursive resolver help reducing the number of DNS queries.

dns-process

image from Wikimedia Commons

Caching speeds up the lookup process and saves bandwidth, but can be an attack vector (see DNS cache poisoning), and also introduces a propagation time for any IP address change, meaning that it will take some time before the latest IP data is present in the resolver (see DNS TTL).

With regards to what transport protocol DNS uses: it can use both UDP and TCP, with UDP being the default, and it only falls back to TCP in certain scenarios where the packet size is too large.

DNS configuration warrants a whole blog post, but I will still mention some points of configuration:

  • the hosts file (at /etc/hosts in Unix systems) can override the DNS resolution by mapping a hard-coded IP address to a given domain name.
  • which nameservers are contacted can be configured in config files like /etc/resolv.conf and /etc/nsswitch.conf (on Unix-like systems at least). The nsswith file defines the order of the hosts file vs the DNS servers (and is absent from macOS).
  • if you are on a VPN, your VPN server might push you the address of the DNS server to use.
# The DNS steps in action

Getting the IP address of google.com with dig looks like this:

click to see details 

$ dig google.com

G 9.11.14-3-Debian <<>> google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23920
;; flags: qr rd ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;google.com.			IN	A

;; ANSWER SECTION:
google.com.		0	IN	A	216.58.213.110

;; Query time: 0 msec
;; SERVER: 172.21.0.1#53(172.21.0.1)
;; WHEN: Tue Mar 23 17:56:19 GMT 2021
;; MSG SIZE  rcvd: 54

What we see here is that querying google.com returns an A record with the value 216.58.213.110. This is the host we want to connect to.

For reference and more information about DNS, see this excellent article by CloudFlare about DNS.

# TCP

Once we have the IP address, we have to establish a connection with the Google server. TCP is the usual choice for the connection, because TCP is a reliable protocol. HTTP, the application layer protocol above the transport layer, assumes a reliable protocol upon which it can operate. Interestingly however, newer versions of HTTP can work using UDP.

TCP is a connection-oriented protocol and so a connection must be set up between your local computer and the remote computer, both identified by an IP address and a port number. The remote port number depends on the protocol you are using - for HTTP, it’s usually 80, for HTTPS, it’s 443. The local port number is usually randomly assigned. The abstraction of of the combination of IP and port number is an endpoint called a socket, and a connection is thus created and maintained between two sockets. A TCP connection is thus point-to-point (not supporting broadcast) and is full duplex, meaning traffic can go in both directions at the same time.

The connection is set up by a three-way TCP handshake. There is an interesting theoretical problem when it comes to setting up a connection over an unreliable network. The problem is called the two armies or two generals problem and it states that it is impossible to know if the other party has received our message and their reply acknowledging our message is lost, or they haven’t even received our message in the first place. It’s a problem that’s proven to be unsolvable, however, in practice, using timeouts works around this. Here’s a really practical description about the solution on StackOverflow.

There are lots of things which TCP does for us, including:

  • flow control: it uses a sliding window approach to make sure the sender is not overloading the receiver
  • congestion control: it uses many different techniques to avoid overloading the network (slow-start, additive increase/multiplicative decrease, etc)

A few other notable things it does are error correction, guaranteed ordering of data and retransmission of lost packets. See more on Wikipedia about these topics.

# HTTP

As we mentioned, HTTP is the application layer protocol that the web browser uses to connect to the Google server. The browser issues a GET request with some request headers, and receives an HTTP response with a number of response headers, a status code and some content.

The HTTP request and response (not using TLS) are pretty straightforward. The client issues a GET request with the following headers (and no body):

 GET / HTTP/1.1
 Host: www.google.com
 User-Agent: curl/7.68.0
 Accept: */*

Note the Host header - it might seem odd, because at this point we already know the IP address and port number we want to connect to, so why specify the host here? To the best of my knowledge, this is to support virtual hosts - multiple backends running on the same physical machine, on the same IP.

The HTTP response might look something like the following:

 HTTP/1.1 200 OK
 Date: Sat, 27 Mar 2021 17:28:04 GMT
 Expires: -1
 Cache-Control: private, max-age=0
 Content-Type: text/html; charset=ISO-8859-1
 P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
 Server: gws
 X-XSS-Protection: 0
 X-Frame-Options: SAMEORIGIN
 Set-Cookie: NID=212=LJ[..] expires=Sun, 26-Sep-2021 17:28:04 GMT; path=/; domain=.google.com; HttpOnly
 Accept-Ranges: none
 Vary: Accept-Encoding
 Transfer-Encoding: chunked

<!doctype html><html [...]

I can’t go into details about all the headers here in the response, but they are dealing with caching (Expires, Cache-Control), they might set a cookie (Set-Cookie), they might relate to security (X-Frame-Options, X-XSS-Protection), and so on. There are also the option for custom headers for your server to set.

The most important parts of the response are the status code and the response body. Here, the status code is 200 meaning OK, successful request, and the body starts after all the headers, plus a newline, and it contains all the HTML document of the web page.

# HTTP versions

In the above example, the client and server agreed to use HTTP 1.1. However, there are more recent version of HTTP which can provide some benefits. Using Application-Layer Protocol Negotiation, the client and the server can agree to use HTTP/2 for instance, which has features like multiplexing using a single TCP connection, compression of headers, and a lot more.

For more information about HTTP/2, check out this excellent article by Ilya Grigorik.

# HTTPS / TLS

We’ve seen how to get data from Google via the TCP and HTTP network protocols. What we have ignored, and what this workflow lacks big time so far is security: HTTP is not a secure protocol in itself, by design. Anyone who has access to the transmitted packets could decode your communication between the client (you) and the server, and thus they could eavesdrop on your messages.

With HTTPS, however, even if an attacker can intercept all the raw data that is sent between you and the server, they will not be able to make any sense of it. It will look like a bunch of gibberish to them. They might be able to tell roughly how much data is communicated, but that’s about it.

Discussing the theoretical background and implementation of TLS is outside of the scope of this post, but I still want to summarise the basics:

  • there is a TLS handshake which crucial for the process. During this,
    • the client and server agree on which TLS version to use
    • the client and server agree on which cipher suite to use
    • the client verifies the identity of the server using the server’s public key and the SSL certificate, via the certificate authority’s digital signature
    • the client and server generate session keys so they can use symmetric encryption for subsequent messages
  • from then on, the parties use the session keys to encrypt messages
  • TLS also ensures that the data has not been tampered with (by a man-in-the-middle, for instance)

It’s important to mention the concept of SSL certificates: a certificate is a file on the server’s side that proves the identity of the server - it ensures that the server the client is talking to is actually the one which it says it is. Certificates are signed by certificate authorities, trusted companies who verify the server’s identity.

To see why TLS is important, see the following practical example.

We can use tcpdump to dump the traffic of our network to a file so we can have a look at the contents.

In one terminal (on Linux), run: sudo tcpdump -w tcpdump_log -A In another terminal, issue a request to an insecure website using plain HTTP: curl -k http://http.badssl.com/

If we inspect the output file, it will contain (among other data) the following things:

GET / HTTP/1.1
Host: http.badssl.com
User-Agent: curl/7.68.0
Accept: */*

HTTP/1.1 200 OK
Server: nginx/1.10.3 (Ubuntu)
Date: Mon, 31 May 2021 11:18:21 GMT
Content-Type: text/html
Content-Length: 483
Last-Modified: Tue, 23 Feb 2021 21:28:41 GMT
Connection: keep-alive
ETag: "60357389-1e3"
Cache-Control: no-store
Accept-Ranges: bytes

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="shortcut icon" href="/icons/favicon-red.ico"/>
  <link rel="apple-touch-icon" href="/icons/icon-red.png"/>
  <title>http.badssl.com</title>
  <link rel="stylesheet" href="/style.css">
  <style>body { background: red; }</style>
</head>
<body>
<div id="content">
  <h1 style="font-size: 8vw;">
    http.badssl.com
  </h1>
</div>

</body>
</html>

What we see here is that the content of our request and also that of the response are in plain sight, without encryption. If an attacked can eavesdrop on our traffic, they can see the actual content of it, including form contents such as passwords.

If we do the same with a secure website however, things will look different.

Again, let’s run tcpdump: sudo tcpdump -w tcpdump_log -A Send a request to a secure website: curl https://tls-v1-2.badssl.com:1012/

The output will be very different:

ÁS'Àp%µtFã þ¸Æú§4íÑbhÇ9SøÈé÷4@^ªdLÕS-Bh9S+kÀtvtHêfôäøD·ãr&cHb¶.Û/Kf`íÅIyʧ­¿Í¯çl7eyë^Âðîgi¯^º'Ýïá;o\ÿCѬO7oR%ÛSÑ}Mj<Eì	αTP³3F¬Î¤·I_'/à¼el¡ÁÝyÓÞ*¶®Å ë#4ßQl¼*ÓñO-5ÎêK]'1ÌèªÈ&ë­7ôCé3 00| ý£ënÊuÈCrKϼ0 *H÷ 0a10	UUS10U DigiCert Inc10Uwww.digicert.com1 0UDigiCert Global Root CA0 130308120000Z 230308120000Z0M10	UUS10U DigiCert Inc1'0%UDigiCert SHA2 Secure Server CA0"0 *H÷ 0 Ü®XMÁÄ05[n<õ,\½ãÛÿqCúd%Ôî¢MðfÐ sn6d¯7ýúA¯Ç¯þsMÏ3¢S+¹¦uH-V7{Ú12׬«ôª]K·GFÝ*Ã.yïj;µ¾ÂeNüÚüÿz®Ü\~U1è9¤×¾/ÓjÒ±ß_þWtS;5Ý®D³ÓÚà×ôk)«D§KXmKÃÚsHEuÝ71èÍT ;äÁ?9^.¸ó\~A EfG°¡eΠª)	Nóëè.«r§0úÇôýwä[(W³ùý·EX £Z0V0Uÿ0ÿ 0Uÿ04+(0&0$+0http://ocsp.digicert.com0{Ut0r07 5 31http://crl3.digicert.com/DigiCertGlobalRootCA.crl07 5 31http://crl4.digicert.com/DigiCertGlobalRootCA.crl0=U 60402U 0*0(+https://www.digicert.com/CPS0Ua1aÕ/(çF8´,áÆÙâ0U#0ÞP5VÑL»fð£âò=ÑU0 *H÷
#>ßKÒ1B¥¶~B\DÌiÑh´]Kà!lKâm̱à¦S	ͪ*eå9O¥n\¢$&æû¡íÇ.ÆMJ¿°BßxÚ³¨ùmÿ!S6`LvÎì8ÜÖQðÅÖåÔM'd«Ç>qûH¸3mÉî¢ö\L@í³ÂìÿqÁãGÿÔ¹ ´7BÚ Éênî®}¢Y¨o-ôòÉ_&Ï,~í7À©Õ9¹¿ê4¯ !hø­sâÉ2Ú8%UÓðhí.A4ï|¥P¿:ùÓÁæíX%ä¸w­-nõRÝ´t«I.;4(xÎêǽÓÉmÞ\2óM I AhȨOo`j¤¨á¼þ°õi¬ó?Ö?×tòBFNÏv,8:¤6ê.ÙÏÍ;l|@Ф9Ý ¬¡w'ZÂ!UÆ~))k ñ úQ@[ì	ì'+«¾Ñ.èóØ#¥ ÑõG¶U§@7Ý,îÎó«Kª}(¹H_Û&öø}ºº2©ù¨½9h¥ãRýÜÈ·¡jìQHg66]óºps~ÎúA'Ðb1³ØY~¶ÿ!_pκ_EwúÍ Û(XµI¹ù§¸à2-gD˾Wl =`£Þ8¸@þÉ,Àù"{Eì®ôö-l	Ã.8aã}q·â?VFÃJHJã-F¶¢ Õ¢LlZL¦Ñ[dÐs3÷ Ç´`S B B ]óp¡ ]ðÀ E 49:@ @2ö¬`~hYiþôaõugSdìκ ®#~P÷ÞµiÇ´`Æ À À ]óp¡ ]ðÀ E ²9;@ @2w¬`~hYiþôaõugSdõÏ8 ®#~R÷Þµi F BA©µFUôC: $00yKN¹p­Ï$+gôµ&0za ô ÆÏ6è Tþ#}éÑkØé{Á(Cª4[Û} (¨7ÆúE·§·È¸EYpº:D z¿ õúûêPUïÇ´`y% u u ]ðÀ ]óp¡ E g#@ 0XdhYi¬`~ôþgSdaõó å}v ÷Þ¶!®#~R (ÖjþeÃ|ô=ÈϬ5ºÝàyÙª® ;îÖ©ÊsS¸íÿ|'pUÇ´`­% B B ]óp¡ ]ðÀ E 49<@ @2ô¬`~hYiþôaõógSõκ ®#~Ø÷Þ¶!Ç´`Ñ( · · ]óp¡ ]ðÀ E ©9=@ @2~¬`~hYiþôaõógSõÏ/ ®#~Ù÷Þ¶!  p¨7ÆúE¸§&íü gÎÂUÀ(5=÷(fÏd+°}¬ZqôñÝluØcÇ:V ÷F¤ÎÕÓãü¢cb-iÚ¥^¤°ä¢^`ÒØù- ¦±Í7¶â*tl*KÊXÀt½_AèÇ´` e e ]ðÀ ]óp¡

It’s total gibberish for an attacker, except for some information about the SSL certificate. They won’t be able to decrypt this without our private key.

For more information, see this excellent blog post on SSL by CloudFlare. This is also an interesting article on Keyless SSL

# The TCP, HTTP and TLS steps in action

Click to see the outputs of curl -v https://google.com. You can notice the negotiation to upgrade to HTTP/2 via ALPN,the TLS handshake, information about the server’s certificatie, the use of headers to set cookies, and various other things.

click to see details 


$ curl -v https://www.google.com
*   Trying 172.217.20.132:443...
* TCP_NODELAY set
* Connected to www.google.com (172.217.20.132) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=www.google.com
*  start date: Feb 23 15:43:15 2021 GMT
*  expire date: May 18 15:43:14 2021 GMT
*  subjectAltName: host "www.google.com" matched cert's "www.google.com"
*  issuer: C=US; O=Google Trust Services; CN=GTS CA 1O1
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55eaf031d190)
> GET / HTTP/2
> Host: www.google.com
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200
< date: Tue, 23 Mar 2021 18:05:07 GMT
< expires: -1
< cache-control: private, max-age=0
< content-type: text/html; charset=ISO-8859-1
< p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
< server: gws
< x-xss-protection: 0
< x-frame-options: SAMEORIGIN
< set-cookie: NID=211=nbqKuoIefpF3Asl0Q69gRExHTpq2V6EociMDG73Ab3pfjhekkSv4XhhlS4xAd-9YBkG4D73l0WQlXS51S2gpHA4DuUBQFUOCVX0gF-MrV6u9UWZivk3OPisbK3MT3IRGU6AO0D6ylMp_9v37zVEEdk6QgBpKxtSr1-I1iNmNhxk; expires=Wed, 22-Sep-2021 18:05:07 GMT; path=/; domain=.google.com; HttpOnly
< set-cookie: CONSENT=PENDING+963; expires=Fri, 01-Jan-2038 00:00:00 GMT; path=/; domain=.google.com
< alt-svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
< accept-ranges: none
< vary: Accept-Encoding
<

[OUTOUT TRUNCATED]


# Conclusion

Once you have the returned HTML in the reponse HTTP body, your browser will display it with nice colours and formatting - but the inner workings of that are out of the scope of this post.

We have seen how DNS, TCP, HTTP and TLS work in harmony to provide us with a seamless browsing experience to present us with the search page of Google.

We have only brushed the surface of this topic and we could dig a lot deeper, but we have to end the blog post somewhere. Happy browsing and catchy ou later!

# Further resources

Written on May 31, 2021
If you notice anything wrong with this post (factual error, rude tone, bad grammar, typo, etc.), and you feel like giving feedback, please do so by contacting me at samubalogh@gmail.com. Thank you!
Back