Peeking behind the scenes of the world’s largest sites

Nuts and boltsDo you want to know more about how big websites like Twitter, Facebook, Hotmail and others handle the technical challenges of dealing with massive amounts of visitors?

Well, you’re in luck, because many of those sites and services have engineering and/or developer blogs that share plenty of information about the challenges they have to deal with and the tools they use. This is an insider’s view that you usually can’t get anywhere else, giving us a unique view of what’s going on behind the scenes of some of the world’s largest web services.

Yes, you’ll have to be a tech geek to find them interesting, so consider yourself warned. 😉 But we here at Pingdom are geeks, and we are really into web tech so we love these kinds of blogs. They are blogs that web developers and sysadmins can learn from and be inspired by.

Let’s get right to it.

Twitter’s engineering blog

Twitter’s engineering blog gives insight into what makes Twitter tick and what technical solutions they are using to solve their scalability challenges.

URL: http://engineering.twitter.com/

A few recent posts:

Facebook’s engineering notes

Want to learn more about the technical ins and outs of the world’s largest social network? Look no further than Facebook’s engineering page.

URL: http://www.facebook.com/Engineering#!/Engineering?v=app_2347471856

A few recent posts:

Digg’s technology blog

Whether you use Digg or not this is interesting. Self-described as “a peek under the Digg hood for API developers, partners and technology geeks.”

URL: http://about.digg.com/blog/technology

A few recent posts:

Inside Windows Live

A blog by the engineering team behind Hotmail, Messenger and Windows Live. It has a large variety of articles with behind-the-scenes information and data analysis.

URL: http://windowsteamblog.com/blogs/windowslive/default.aspx

A few recent posts:

Flickr’s developer blog

A blog by Flickr’s development team that dives into the nuts and bolts of the photo-sharing site and working with the Flickr API.

URL: http://code.flickr.com/blog/

A few recent posts:

Meebo’s developer blog

Interesting developer-oriented articles about the inner workings of Meebo’s instant messaging service.

URL: http://devblog.meebo.com/

A few recent posts:

Amazon Web Services blog(s)

You have Amazon’s official AWS developer blog which has plenty of announcements and great advice for using Amazon EC2, S3, and so on, but the real nugget of gold is the insightful All Things Distributed, a blog written by Amazon’s CTO, Werner Vogels. So, with apologies to the official AWS blog, that’s the one we’ll list here below.

URL: http://www.allthingsdistributed.com/

A few recent posts:

There’s more, much more…

What we find interesting with these blogs is that they give unique insights into the nitty-gritty technical details of running sites that are much, much larger and more demanding than your average site.

There are a bunch of other sites you might also like to check out depending on your interests. The behind-the-scenes articles may be a bit sparser on these, but they still come up with the occasional gem. Here are some to get you started.

Then on top of these you can add all those great blogs that don’t actually belong to a company/service but still offer great articles about how the big sites are run. A great example is the High Scalability which is a blog dedicated to how sites deal with scalability and performance issues.

We’re sure we’ve missed several excellent blogs, so if you know of other services whose engineers or developers blog about their technology, let us know in the comments. We can build a great list here!

1 comment

Leave a Reply

Comments are moderated and not published in real time. All comments that are not related to the post will be removed.