Using Content Distribution Networks.

For some months now I have been participating in a Big Project for Dunkin Donuts in the United States, basically the construction of the site www.DunkinDonuts.com and other affiliated sites like www.myicedcoffee.com, these sites use different technologies like Flash, JavaScript, WebServices, Digital Certificates, and another list of things that are interesting to put to work together and that are tested in all the popular operating systems and browsers.

These sites expect to have massive amounts of visitors all the time with very high peaks in certain advertising campaigns and special days, so you have to face the problem of high availability seriously, and if you add to that the fact that many flash movies are used (running against web services) some of them very heavy, then you have a somewhat complicated combination.

In this site different solutions have been implemented like a DB cluster, a cluster of presentation servers, each server with 4 Gigas of RAM and redundant data access systems, etc, even dedicated network cards for the communications between the machines, the internet connection and even the one that lets me connect for administration tasks.

However, one of the most interesting technologies we have used has to do with the Content Distribution Network (CDN for Content Distribution Network) that has been implemented, here are a few lines about it. (It is not the only one, many application architecture, infrastructure and business decisions must be made and implemented correctly for a site to really support heavy traffic, starting with having an idea of how much traffic is estimated).

What is a CDN?

A content distribution network is a bunch of infrastructure generally distributed all over the world (at least in key points for the internet) that keep local copies of certain types of files to deliver them to the users that request them using geographic advantages (they are closer). (You didn't really believe youtube had the biggest hard drives in the world, did you?). This way when a user requests (through the browser) a video, song, image, or even one of the flash movies, it is sent to him from the server that is nearest. More references and information in English here

Since all that infrastructure is leased, important benefits are obtained in hardware costs and in the use of the bandwidth of your hosting provider, and for the users the experience is very good because they feel your site is much faster than it really is.

How is it used?

One of the really incredible things about this is the way it works from the developer's perspective (that is, mine) (the other incredible thing is how it selects who is actually near, but that is another topic), for me the location of the file is transparent, that is, I don't care.

For example:

Suppose I have some very big images that I want uploaded to this CDN network, in my file structure they are in a folder images and I reference them as src=”images/imagengrande1.jpg” well, when you join the content distribution network they are going to create you a pointer for your domain of the type assets.mydomain.com, so now instead of pointing the image scr like before you put scr=http://assets.midominio.com/images/imagengrade.jpg, the magic happens when, when a user makes the request for the first time, the CDN looks in its own list of files, since it doesn't find it it goes to your server and sends it to the user, but at the same time it takes a copy and they store it on their own servers, that is, the more people visit the site from different parts of the world or the country, the faster it works since it takes copies of the content as it distributes it.

That is, you upload nothing to the network, it feeds itself as your users ask for the files, and there are no crosssite scripting problems because the files are on your domain, just on other machines.

This generates two problems, both easy to solve and, well, for the benefit I can live with them.

1. Obviously if there is a change in the file that is loaded in the network, somehow it must be notified so the file is updated, otherwise the CDN network will keep sending the outdated file. However, all these networks have an administration console where you can see usage reports, but you can also request that a file be updated, it is an automated process generally based on queues but depending on the traffic of the site it can take from minutes to hours, so try to avoid changes.

2. As in the development process many times things have to be changed (it should not be, but that is how it is), it is better to have some way to “turn on and turn off” the service, so a good option is to put in the configuration file the route that is going to be used or in the load of the control or in a JS function or in CSS where the change can be made in a single place and affect all the files that are being changed.

To have one?

The providers of this service are Akamai, LimeLight Network (which I am using) (by the way, Akamai sued LimeLight for 45 million dollars, according to them Limelight violated the license granted by MIT to use the technology that allows the reconstruction of the URLs of the files used in the CDN), so you just have to go and subscribe, pay the monthly service and that's it, there are various services including having HD television over internet hosted in the network.

I hope this helps you to save some hours and to build sites of very high availability.

Juan Carlos Peláez

Software Architect

Keywords: Architecture, CDN, Content delivery Network, Content Distribution Network,