Pingdom Home

US + international: +1-212-796-6890

SE + international: +46-21-480-0920

Business hours 3 am-11:30 am EST (Mon-Fri).

Do you know if your website is up right now? We do! LEARN MORE

Hardware malfunction: monitoring data unaffected

pingdom logoShortly after 9am CET today, Friday December 7, 2012, some of our systems started to experience downtime. This included our pingdom.com homepage and the customer control panel at my.pingdom.com. All systems have now been restored and are operating at normal capacity and functionality.

Monitoring of our customers’ websites was not affected, but alerts, unfortunately, were delayed. We are as passionate about uptime as our customers are, and this sort of thing should not happen. Since this is a rare occurrence for us, we want to take this opportunity to explain what happened, at least insofar as we know right now.

We received internal alerts as soon as things started to go wrong earlier today, and proceeded to investigate. It became apparent that this incident was caused by faulty hardware and here are some of the actions we’ve already taken so far:

  1. We have secured that monitoring, alerts, public reports and all our other systems are now operating normally.
  2. We have secured that monitoring data from all our probes and for all our customers has been collected as normal throughout the day. This data is secure, but there is a considerable amount of it. We’re currently processing it so that it can be rolled out to customers’ accounts. There may still be delays in sending out alerts and in data appearing in the my.pingdom.com control panel.
  3. We kept a dialogue going with our hardware and software providers during the entire incident, which helped us to quickly address the underlying issues, and move key hardware components to new systems.

This is obviously not a complete postmortem. That will have to wait until we’ve had an opportunity to perform a detailed investigation. When that is complete, we may return with an updated post.

We want to assure all our customers that although mistakes happen, they are not acceptable because we take considerable pride in delivering a service that you can depend upon.

If you were affected by this, we offer our sincere apologies.

Please contact us at support@pingdom.com if you have any questions or comments.



3 comments
rhbkwe
rhbkwe

Hi,

 

On this post you said and i quote:

 

"This is obviously not a complete postmortem. That will have to wait until we’ve had an opportunity to perform a detailed investigation. When that is complete, we may return with an updated post."

 

Until today no updated post was released and today we again experience a huge Pingdom downtime. 

 

As a client i request the Pingdom team can post on this blog a detailed explanation of what happen and what measures are you taking for this to happen with less frequency in the future.

 

Just one final note to say that when Pingdom is down, beside we cant manage our check, the thing that impacts me the most is my public network status page also be down.

 

Hope Pingdom team can improve their service uptime and reliability in a near future.

 

Regards

rhbkwe
rhbkwe

Hi,   On this post you said and i quote:   "This is obviously not a complete postmortem. That will have to wait until we’ve had an opportunity to perform a detailed investigation. When that is complete, we may return with an updated post."   Until today no updated post was released and today we again experience a huge Pingdom downtime.    As a client i request the Pingdom team can post on this blog a detailed explanation of what happen and what measures are you taking for this to happen with less frequency in the future.   Just one final note to say that when Pingdom is down, beside we cant manage our check, the thing that impacts me the most is my public network status page also be down.   Hope Pingdom team can improve their service uptime and reliability in a near future.   Regards

Mineral Water Plant
Mineral Water Plant

It's fantastic to be part of a group of blog writers. Your Blog Post Objectives are appropriate encouragement about the objective of weblogs and what I desire to as well. Keep composing a blog on blogs!