Author Archive

Blogger was unavailable for most users for over an hour. Service has now been restored.

RESOLVED: The issue affecting Blogger users in Europe has been resolved. There was a configuration problem in a new datacenter. We will provide a postmortem on Monday. Thanks to all who assisted us analyzing the problem.

Postmortem on recent issues affecting users in Europe

What happened
Beginning at approximately 2:30am (PDT) Saturday, October 31, a DNS configuration issue in a new datacenter in Europe caused requests to blogger.com and *.blogspot.com to be sent to an invalid IP address. Affected users throughout Europe were unable to connect to either Blogger or to blogs hosted by Blogger at blogspot.com for approximately 10 hours.

What we're doing about it
1. Blogger's systems engineers will proactively monitor and verify DNS settings during major system changes.
2. We have improved our automated monitoring so that on-call engineers will be paged earlier for DNS-related failures.
3. Checks in our DNS management system have been made to prevent the change that caused this to occur.

Eddie Kessler, Blogger Engineering Manager

By Eddie Kessler, Blogger Engineering Manager

After a recent service interruption, we started talking about how we could improve our communications about these (hopefully infrequent) issues. Going forward, in the case of significant service interruptions, we plan to publish a post mortem on the Blogger Status blog within 3 business days to provide details about what went wrong and what we're doing to help prevent similar problems in the future.

During any outage, we try to keep the Blogger Status blog updated once we know about an issue. Of course sometimes — as was the case on Saturday — affected users who cannot reach Blogger cannot reach the Status blog either. In such cases, we will try to post updates on the Blogger account on Twitter to keep users apprised of what we know and when we expect a resolution.

We don't like it when our users experience problems like what we saw on Saturday, but we hope the combination of transparency around these issues and our commitment to learn from our mistakes will help assure you that we're doing everything we can to keep Blogger a robust and reliable service for you. As always, thank you for using Blogger.