SRE Weekly Issue #300

300 issues. 6 years. Wow! I couldn’t have done it without all of you wonderful people, writing articles and reading issues. Thanks, you make curating this newsletter fun!

A message from our sponsor, Rootly:

Manage incidents directly from Slack with Rootly πŸš’. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging the right team, postmortem timeline, setting up reminders, and more. Book a demo:
https://rootly.com/demo/?utm_source=sreweekly

Articles

This is the best thing to hit incident analysis since the Etsy Debriefing Facilitation Guide and the PagerDuty retrospective guide!  This one’s even better because it’s not just about the retrospective, but the whole incident analysis process.

BONUS CONTENT: A preview/introduction by Lorin Hochstein.

  jeli

SysAdvent is back!

When teams only consult briefly on reliability or operational concerns, often the final output doesn’t adequately reflect customer or engineering expectations of reliability of the product or operability of the internals.

  Martin Smith (edited by Jennifer Davis) β€” SysAdvent

What can Dungeons and Dragons teach us about SRE?

  Jennifer Davis β€” SysAdvent

It’s so true. Don’t forget to read the alt text.

  Randall Munroe

This talk (with transcript) includes three stories about how incident analysis can be super effective.

  Nora Jones β€” InfoQ

I know this is SRE Weekly and not Security Weekly, but this vulnerability is so big that I’m sure many of us triggered your incident response process, and some of us may have even had to take services down temporarily.

  John Graham-Cumming β€” Cloudflare

What a colorful metaphor. This article discusses an effective technique for breaking up a monolith, one piece at a time.

  Alex Yates β€” Octopus Deploy

This article proposes a method of eliminating the need for a central team of architects, and it strikes me as very similar to the practice of SRE itself.

  Andrew Harmel-Law

More from the VOID, this piece is about the importance of analyzing “near miss” events.

  Courtney Nash β€” Verica

If you load-test in production, don’t include your load-test traffic in your SLO calculation.

  Liz Fong-Jones β€” Honeycomb

Outages

  • AWS us-east-1 region (and half the web)
    • Between the AWS outage and log4j, it’s been a busy week. Amazon has already posted a write-up about the incident, which includes the notable tidbit that their circuit-breaker/back-off code failed.

Updated: December 12, 2021 — 9:09 pm
A production of Tinker Tinker Tinker, LLC Frontier Theme