SRE Weekly Issue #291

A message from our sponsor, Rootly:

Manage incidents directly from Slack with Rootly 🚒. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging the right team, postmortem timeline, setting up reminders, and more. Book a demo:


Facebook’s outage caused significantly increased load on DNS resolvers, among other effects. Cloudflare also published this followup article with more findings.

Celso Martinho and Sabina Zejnilovic — Cloudflare

Shell (the oil company) reduced accidents by 84% by teaching roughnecks to cry. Listen to this podcast (or check it out in article form to find out how. Can we apply this to SRE?

Alix Spiegel and Hanna Rosin — NPR’s Invisibilia

Don’t have time to read Google’s entire report? Here are the highlights.

Quentin Rousseau — Rootly

I really like how open Facebook engineering has been about what went wrong on Monday. This article is an update on their initial post.

Santosh Janardhan — Facebook

Want to learn about BGP? Ride along as Julia Evans learns. I especially like how she whipped out strace to figure out how traceroute was determining ASNs.

Julia Evans

The Verica Open Incident Database is an exciting new project that seeks to create a catalog of public incident postings. Click through to check out the VOID and read the inaugural paper with initial findings. I’m really excited to see what this project brings!

Courtney Nash — Verica

Printing versus setting a date — they’re only separated by a typo. Perhaps something similar happened with Facebook’s outage.


Adopting a microservice architecture can strain your SRE. This article highlights an oft-missed section of the SRE book about scaling SRE.

Tyler Treat


Updated: October 10, 2021 — 8:47 pm
SRE WEEKLY © 2015 Frontier Theme