The Principles of CDNs
Let's start at CDNs. What is a CDN? It's a Content (Distribution|Delivery) Network, depending on who you consult. The expansion of the initialism itself
isn't extraordinarily revelatory, though, because the natural next question is
"great, what's a Content (Distribution|Delivery) Network?"
To understand that, let's briefly go back to basic principles.
Basic Principles
Physics
I promise this won't hurt. The only element of physics we need to understand
relevant to our discussion is that nothing moves faster than c, the speed of
light in a vacuum, 299,792,458 m/sec (metres per second).
Remember, everything will be slower than this. Electrical pulse through a copper cable? Slower than light in a vacuum. Sound wave through water? Slower than light in a vacuum. Message written on paper, tied to a rock, and thrown out of a moving vehicle? Slower than light in a vacuum.
If instead of a vacuum, we're sending that light through strands of glass, as with fiber-optics, then we get about two thirds of that speed.
Now, two thirds of 299,792,458 m/sec, or 199,861,638 m/sec is still a good
pace, but it's not immediate.
Rear Admiral Grace Hopper used
30cm pieces of wire
to demonstrate how far an electrical signal travels in a nanosecond. Let's
instead focus on distance here, through fiber-optic cable.
At a distance of 1 mile, it takes ~0.0000080523 seconds
for light to travel from A to B.
Pretty fast. How about the ~2400 miles from San Jose CA, in Silicon Valley, to
Ashburn VA, where many datacenters are located (looking at you, us-east-1)?
Now we're looking at ~0.0193254975 seconds.
Let's go international. What about Sydney NSW, Australia to Ashburn VA, USA? That's
about 9700 miles, at a time of ~0.0781072191 seconds.
We're starting to talk serious times here. 0.0781072191 s
may not seem like a lot, but imagine we need some kind of acknowledgement signal
in response: now we're looking at a round-trip time of
0.1562144382 s, and that's
best-case–we're likely not lucky enough to have a fiber-optic cable running
directly across the globe, in a straight line, from our origin to our
destination. About
7 of these round-trips
would take in the vicinity of 1 s, an easily perceptible amount of time.
We can't beat light, and while light is fast, even the distances involved on our own lil' planet Earth are large.
Want to get more than 2/3rds of the speed of light? That's where line-of-sight systems like microwave transmission come into play. The speed of electromagnetic waves through mostly-empty space like air is faster than through fiber-optic.
Digital Representation
Yes, that far back. Accepting that we're dealing with digital computing means taking these two statements to be ground truths:
- Someone really smart figured out how to express Boolean algebra in electronic circuits.
- Boolean algebra means we're
dealing with only two values,
trueandfalse.
The nouns true and false are fairly arbitrary. The important thing is that
there are two, and only two of them. Some folks like yes and no. They might
be more familiar as 1 and 0.
We're only interested in digital computing here, since those are the computers we're working with. Analog computers are all well and good. Quantum computers are perhaps The Future and I do not understand them. Neither are pertinent to this discussion.
Having only two states is useful if we want to represent, say, whether a door is "open" or "closed":
| DOOR OPEN | T |
|---|---|
| DOOR CLOSED | F |
What if we want to represent "open", "closed", or "ajar"? Remember, we get
true and false, not true, false, and
possibly-maybe.
Well, that's fine. true and false are only two states, but that doesn't
mean we can't have more state.
| DOOR OPEN | T | T |
|---|---|---|
| DOOR CLOSED | F | F |
| DOOR AJAR | F | T |
Notice there is one more possible state here, TF. It is currently unused, but
we could use it for DOOR IS A STARGATE or something.
A single T or F value is what we know as a bit in computing. A single
bit can represent two states. Two bits can represent four states, three bits
eight states, and so on.
We picked some state combinations out of a hat and decided that TT represents
DOOR OPEN, while FF is DOOR CLOSED, for pretty much no good reason. We
could have just as easily decided that the inverse is true. What if we want to
do more than capture the state of a door, though? What if we actually want to
compute, and do things like get the result of 9 + 2?. Making one big list
of all the numbers and assigning a random bit pattern seems like it won't get us
very far.
We do in fact have tables that assign meaning to certain numbers. For example,
the ASCII table assigns the number 65,
or 01000001, the letter A.
Way before the advent of modern day computing, various cultures came to understand that two states can be used to construct a binary numeral system. As an example, let's take eight columns. We'll assign each column a value that is a power of two, highest to lowest:
| 128 (27) | 64 (26) | 32 (25) | 16 (24) | 8 (23) | 4 (22) | 2 (21) | 1 (20) |
|---|
Now let's assign eight bits to each of these columns. If the bit in a given
column is 0 (false), then that column's value is not added to the final
number. If the bit in a given column is 1, (true), then that column's value
is added to the final number. Here's number 9:
| 128 (27) | 64 (26) | 32 (25) | 16 (24) | 8 (23) | 4 (22) | 2 (21) | 1 (20) |
|---|---|---|---|---|---|---|---|
| 0 | 0 | 0 | 0 | 1 | 0 | 0 | 1 |
Great! Now we can use these bit patterns to count, in a logical and consistent manner.
Notice that the above means we can only count up to the sum of
128 + 64 + 32 + 16 + 8 + 4 + 2 + 1, or 255. The solution to that? We use
more bits to represent further powers of two. We call a set of eight
(rather arbitrarily) bits a byte.
We didn't address negative numbers, or decimal numbers, etc. It's not important for our purposes–what's important is the idea that we can use collections of bits to represent other values.
This is not meant to be a full dive into computation, so we won't go into the operations that can be performed on these bits. As mentioned earlier, electronic circuits can be built that can perform simple operations on bits. These can be combined to build more sophisticated circuits to carry out addition, subtraction, and so forth.
We store bits, and we operate on bits. This brings us to the transmission of bits.
Sending Messages
A single computer is pretty useful all on its own. For better and for worse, at various points in history people have decided that building multiple computers and then connecting them together would be a good idea1,2. Great work everybody, now there's a small computer that lives in my pocket, and every now and then it dings and shows me a little message about something terrible happening in the world. Are we sure about this?
Anyway. Remember, our digital computers are dealing with two states of
information, let's call them 1 and 0 from here on out. We want to transmit
the word hi, encoded in
ASCII. First, let's perform that
encoding:
| h | i |
|---|---|
| 104 | 105 |
Now lets convert each of those integers into a binary representation:
| 128 (27) | 64 (26) | 32 (25) | 16 (24) | 8 (23) | 4 (22) | 2 (21) | 1 (20) | |
|---|---|---|---|---|---|---|---|---|
| 104 | 0 | 1 | 1 | 0 | 1 | 0 | 0 | 0 |
| 105 | 0 | 1 | 1 | 0 | 1 | 0 | 0 | 1 |
Fantastic. Now we actually have to send these bits somehow. There are lots
of different ways of doing this, but
they're all trying to achieve the same goal: turning 1s and 0s into a
pattern to be sent down a wire, or a piece of glass, or through thin air itself.
Let's start turning a light on and off. If the light is on, that means 1, and
if the light is off, that's a 0.
Now let's turn that light on and off to send the bits for our message,
hi.
| 128 (27) | 64 (26) | 32 (25) | 16 (24) | 8 (23) | 4 (22) | 2 (21) | 1 (20) | |
|---|---|---|---|---|---|---|---|---|
| 104 | 0 | 1 | 1 | 0 | 1 | 0 | 0 | 0 |
| 105 | 0 | 1 | 1 | 0 | 1 | 0 | 0 | 1 |
We're currently changing the light once per second. We can make that a little faster if we'd like and send four bits every second.
| 128 (27) | 64 (26) | 32 (25) | 16 (24) | 8 (23) | 4 (22) | 2 (21) | 1 (20) | |
|---|---|---|---|---|---|---|---|---|
| 104 | 0 | 1 | 1 | 0 | 1 | 0 | 0 | 0 |
| 105 | 0 | 1 | 1 | 0 | 1 | 0 | 0 | 1 |
Notice that if we have multiple consecutive 1s or 0s that it might be hard
to discern if we mean a single, say, 1, versus two 1s in a row? That's
where the aforementioned line codes
come into play. There are ways to encode the 1s and 0s that account for this
problem, but we've chosen an extremely simple encoding mechanism (turning the
light on and off) for the sake of brevity.
Instead of flashing the one light on and off multiple times per second, maybe we can use multiple lights, one for each character.
| 128 (27) | 64 (26) | 32 (25) | 16 (24) | 8 (23) | 4 (22) | 2 (21) | 1 (20) |
|---|---|---|---|---|---|---|---|
| 0 | 1 | 1 | 0 | 1 | 0 | 0 | 0 |
| 128 (27) | 64 (26) | 32 (25) | 16 (24) | 8 (23) | 4 (22) | 2 (21) | 1 (20) |
|---|---|---|---|---|---|---|---|
| 0 | 1 | 1 | 0 | 1 | 0 | 0 | 1 |
We can transmit more bits per second. The maximum amount of bits we can send down a given medium per second is what we refer to as bandwidth. We can figure out how to send more signals down one piece of fiber-optic cable, or extend multiple wires between systems, but to go back to physics: we can't make those signals themselves arrive any faster than light itself. We can't beat light.
To make matters worse, none of this is ever as simple as just sending our raw
unencapsulated messages (e.g. hi) between computers. We have a whole suite of
protocols to enable
reliable messaging between systems, and many of these require multiple steps
to ensure data is being transmitted correctly. Take, for example, the
TCP three way handshake:
Let's say it takes 50ms to send a message between two machines. A three-way
handshake then takes a total of 150ms just to establish a connection. That's
a non-trivial amount of time, and that's just for one connection to one other
machine.
This time, the interval between a message being sent and the arrival at its destination, is called latency. This is the reason we're messing with CDNs at all. So if we can't beat light, then how does a CDN help us reduce latency?
CDN Mechanics
Cheating Light, Not Beating Light
We determined in Basic Principles that we can't beat the
speed of light, and that at global distances, latency adds up. Again, imagine
we're using a communications protocol that requires a 3-way handshake. We have
a client in Sydney NSW, Australia, and a server in Ashburn VA, USA, a distance
of about 9700 miles.
This gives us a total of ~234.3216573
ms to establish a connection, best-case. This is regarded very dimly by
users. For example, 234.3216573 ms
between clicking a link and a web page beginning to load is a poor user
experience. There's quite a body of research
on understanding these perceptions. So if we're not beating light, what can we
do to improve latency?
Well, how about we just put another computer closer to the client? One that can return the content the client requested that normally would have come from the server that's further away? What if we could turn this:
Into this:
Well, that's exactly what a CDN primarily does. It's a network of servers that distribute content so it's better placed to serve a given client with lower latency. Bandwidth is a consideration here as well, but we're most often concerned with latency. We often figure out how to squeeze more bandwidth out of a given medium, e.g. ADSL over copper phone lines as an improvement over dial-up, but we can't improve latency past the speed of light. So we just move clients and servers closer to each other.
As an example, here are the points-of-presence of our current CDN provider, Fastly:
As you can see, they have servers in multiple locations on every continent except for Antarctica. The penguins will have to tolerate poor latency while trying to find pirated copies of Happy Feet. Through the magic of anycast routing, a request is sent to the most-appropriate server in the network that can handle it. Clients experience lower latency, likely higher bandwidth, and we haven't had to break any laws of physics to achieve it.
Content Distribution
We now know why we use CDNs, but how is the content actually distributed? For each of these models, we're concerned with three entities:
- The
origin, the canonical source of the content. - The
client, the party requesting the content. - The
cdn, the network returning the content to the client.
Push CDNs
In certain cases we can push content to the CDN and let it handle moving it
around the globe appropriately. Imagine we have an entirely static website, like
this guide. Let's say we're deploying it at cdn-and-waf-guide.gironda.org.
With a push-based CDN model we could build the artifacts, all of the HTML and
CSS and images and JavaScript, and send this bundle away to get distributed
appropriately. It becomes the new cdn-and-waf-guide.gironda.org, and if we
change it, we just deploy again.
Use cases for this model work best when content is rarely changing and has simple caching semantics, or when we're able to predict demand to a certain extent. Netflix Open Connect is one specific use case where their private CDN pushes a subset of the Netflix catalog to nodes based on criteria like popularity or predicted demand for new releases. If Netflix are releasing a new season of a popular show on a specific date for our binge-viewing pleasure, then they can ensure all of the content is in place beforehand to best meet that demand.
Pull CDNs, or Caching Reverse Proxies
Other cases involve more dynamic and unpredictable content. Imagine, for instance, we operate a shopping website that has a catalog of items users can purchase. We might want to distribute the content on catalog pages so that users have the best avocado procurement experience, whether they be on the east coast, in the midwest, or the bay area.
Prices change pretty frequently, though, and so in a push-based model we might find ourselves pro-actively sending a lot of new content to the CDN. Furthermore, we're not really sure what the correct distribution of content looks like–do those catalog pages for stores based on the east coast really need to end up on a CDN node in the west coast ahead of time, even if we might have someone in SF ordering their NYC pal a gift delivery of guac ingredients?
This is where pull-based CDNs come into play. Instead of pushing content to the CDN as it's updated, we set up our origin to service requests from the CDN on an ad-hoc basis. Clients request content from the CDN as usual, but if the CDN doesn't have a copy, it reaches out to the origin to request one.
When the origin responds to the CDN with the content, it indicates a TTL, or a
time-to-live. This indicates to the CDN how long it can hold on to a copy of
the content and return it to clients before considering it expired. For example,
we might set a TTL of 300 s on our product page responses, knowing that prices
generally don't update every five minutes.
This is a model that is generally more powerful and flexible, but can have more moving parts to wrangle. Some use cases of pull-based CDNs can involve a bit of an inversion of the push-based model where the origin proactively informs the CDN not of new content, but when possibly-cached content should be manually expired. To go back to our shopping cart example, we may wire up our data ingestion process so it tells the CDN to expire a given product page if the price of the product has changed since the last update.
The CDN acting as a proxy layer between the client and the origin server gives us a place in our architecture where more smarts can be applied. We can perform tasks like image optimization here, or decide to serve stale content to clients in the event the origin is experiencing downtime.
Edge Computing
Finally, we have the new-ish kid on the block, edge computing. CDNs have traditionally been involved with pushing content, or data closer to clients, but these days we have more and more use cases where pushing computation closer to the client can make sense. Suppose, again, we're operating a shopping website. While we may allow people to browse and build carts without logging in, at some point we do require some manner of authentication.
With a regular pull-based CDN in front of the origin, we might encode and sign
these authentication details into a header (e.g. a Bearer token in an
Authorization header) and validate this at the origin itself:
You might notice a few potential downsides here, though. For example,
- We have to send traffic all the way to the origin to determine if the request should be allowed or denied. This costs both time and money in terms of CDN and origin bandwidth costs.
- Requiring the decision to be made at the origin means we can't cheat light anymore–the request might have to go all the way from one side of the world to the other and back.
- If we do want to return a cached response, how do we validate that the client should actually be allowed to see the cached content?
- Visibility into details of the client making the request is poor at the CDN level. Instead of being able to log the id of a user making a given request, the best we may be able to do is log their IP address.
We could solve this if we could make the CDN a little smarter–if we we could
have the CDN verify the cryptographic signature of the Bearer token, pull out
the user metadata, and make smarter decisions about when it's appropriate to
return cached content or contact the origin.
Edge computing allows us to pull this off. The edge here is the term for the CDN servers closest to the client, or at the edge of our architecture. Instead of largely storing data at the edge in the form of cached content, we execute code there as well. Now our previous example can look something like this:
Phew. That's a lot, but we've been able to cheat light again. More computation happens closer to the client, and we've gained some extra benefits like visibility into authentication state right from the edge of our architecture. It can be easier to think of edge computing as a hybrid of the pull-based CDN model and serverless computing. We can write code in any number of common programming languages and hand that to the CDN to execute as closely to the user as possible. The Content Distribution Network has now become somewhat of a Compute Distribution Network.
Fastly aren't the only folks doing this. For other examples take a look at Cloudflare Workers and Fly.io.
Light Aside: Other Use Cases
Distributing the content lets us cheat light, but are there other reasons we use CDNs? Absolutely. Here are a couple.
Resiliency and Graceful Degradation
Let's say that instead of a shopping website, we're operating a successful recipe blog. The star here is the content itself–there's extra functionality like being able to save recipes to personal collections and create shopping lists, but the recipes are what keep people showing up. Okay, maybe the comments also reel 'em in.
The product is successful, but some legacy architectural decisions have left it a little less stable than we'd like. Occasionally a client makes a request and the backend glitches out, serving up an error page:
One thing we can do in this case with many CDNs is instruct them to return stale content if an origin responds with an error, but the CDN has an old copy in the cache, resulting in this scenario:
This of course means we need to have had a cached entry in the CDN in the first place, but for our most popular recipes this is quite likely. This allows us to smooth over turbulence in reliability and try to return useful content to clients as frequently as possible.
DoS Mitigation
The internet is full of malicious dweebs. Some in it for money, some for thrills, but dweebs nonetheless. Commonly, these individuals or groups will send an enormous amount of traffic to a site to try and bring it to its knees. The idea is to just consume so many resources that either the site can't service valid requests, or something bugs out (like a disk filling up) and it all comes toppling down.
Having a CDN in front of our origin can protect us here even if the content being requested isn't well suited to caching. If the requests are coming from an easily identifiable source (e.g. one IP address), then we can block that address at the CDN and prevent the traffic from ever reaching our origin. In the case of more sophisticated attackers, many CDN offerings are able to automatically detect and block malicious traffic, even across a broad set of IP addresses.
Now that we know what problems CDNs aim to solve and why they exist, we can ask and answer another question in our next section–what kind of CDN is the one we use, Fastly?