Pingdom Home

US + international: +1-212-796-6890

SE + international: +46-21-480-0920

Business hours 3 am-11:30 am EST (Mon-Fri).

Do you know if your website is up right now? We do! LEARN MORE

When automatic software updates go horribly wrong

DisasterMcAfee had a nasty surprise in store for their customers a couple of weeks ago. An automatic update to its antivirus software suddenly pointed out a system-critical file in Windows XP as malicious. The result was that the file was removed, and Windows XP stopped working.

This crippled entire companies, which often have large sets of computers running XP. To make matters worse: every single computer had to be manually restored. Considering many companies had thousands of Windows XP machines, you can imagine the time it took and the outrage it caused.

Accidents such as these are uncommon, but they still happen way too often for comfort. And if you think the latest incident with McAfee was a one-off? Think again.

Here are a few examples of when automatic software updates have wreaked havoc.

McAfee mistake flags hundreds of files as viruses

McAfeeMarch 2006: An error in an update of virus definitions led McAfee’s anti-virus software to flag hundreds of legitimate executable files as a malicious virus.

Affected software included Microsoft Excel, Adobe Flash, Google’s Toolbar installer, Sun’s Java Runtime Environment. In total, at least 330 files were erroneously flagged as malicious.

Norton Antivirus breaks Windows XP

SymantecMay 2007: McAfee is not alone in giving Windows XP owners a headache. A Norton Antivirus virus definition update in May 2007 flagged two essential Windows XP system files as viruses by mistake, and once these files had been removed, computers couldn’t start properly.

Skype brought down by Windows Update

SkypeAugust 2007: Skype was down and unavailable for almost two days after a security update to Windows caused a massive amount of Windows computers to restart in a very short amount of time. Considering Skype has hundreds of millions of users, you can imagine the impact this had. You may remember it referred to as The Great Skype Outage (we listed it as one of the major outages on the Internet in 2007).

Admittedly, this was ultimately caused by a bug on Skype’s backend software (in other words: Skype’s fault, not Microsoft’s), but it was triggered by a Windows update.

Sitemeter update kills thousands of sites on IE

SitemeterAugust 2008: Widgets and web apps are by their very nature auto-updated, so if an update is made that breaks a widget in some way, it can have wide-ranging effects. Just ask Sitemeter, makers of a free web analytics tool. In August 2008, an update to the Sitemeter Javascript file suddenly made any website that included it break in Internet Explorer.

Since a ton of popular blogs used Sitemeter (at least at the time), sites like TechCrunch, Gawker, Lifehacker and many others stopped working on Internet Explorer until the bug had been fixed (after almost a day of problems).

Malware makes Windows update crash computers

WindowsFebruary 2010: Windows computers infected with the Alureon rootkit crashed during Windows a Windows security update in February 2010. The rootkit had made changes to the Windows kernel, and when the update in question was applied this created an unstable system, crashing Windows.

BitDefender breaks 64-bit Windows

BitdefenderMarch 2010: Once again an antivirus program wreaking havok… This time it was BitDefender’s turn, affecting all 64-bit versions of Windows. An update of virus signatures made the program think that Windows system files and even part of BitDefender itself were infected. The files were quarantined by BitDefender, leaving the system unable to boot.

The antivirus dilemma

Some software issues will have more serious implications than others. This is probably why antivirus software features heavily when you look around for automatic updates gone wrong. Antivirus software usually has deep-level system access and can even disable critical parts of the operating system itself if things go wrong. It’s less likely that your local word processor would be able to do that…

It all becomes extra sad when you think about the purpose of antivirus programs. They’re supposed to protect your computer, not break it.

Tip of the iceberg

These were just a few incidents, all of them easy to locate with a few web searches. They are just the tip of the iceberg, though. For every major incident there are a number of minor ones to consider. For example, in January this year, Norton Antivirus all of a sudden decided Spotify was a trojan, making the streaming music application impossible to run or install.

With so many applications and hardware platforms out there, testing is a nightmare for software developers. There are simply too many combinations to test all of them. Some problems will inevitably go undetected.

And the truth of the matter is that sometimes it’s not even the software update’s fault, it can simply be the trigger of some nasty side effect in some other software, like in the case of the Skype outage.

Most of us think automatic software updates are convenient, but when errors are introduced and are pushed out to hundreds of thousands, or even millions, of computers, things can go horribly wrong, as have been proven over and over again.



No Comments

I hope this makes people reconsider using Windows XP. That would make me happy if thousands of users ended up upgrading, rather than reinstalling XP, just because of the convenience.

Let’s not turn this into an antivirus debate club :)

The truth is they will never be able to test all the possible configurations.