A message from our sponsor, incident.io:
“Lifting and shifting” noise to new tools just buys a different UI for the same burnout. incident.io’s migration framework prioritizes service cataloging and inventory to fix ownership, preventing team friction during the transition to a scalable on-call system.
A previously unpublished article by the late Dr. Richard Cook!
Organizational Second Hit Syndrome is an incident-related phenomenon analogous to neurological second-impact-syndrome (SIS). It occurs when a major incident creates a vulnerable period during which a second incident generates strong, widespread, and sometimes destructive organizational reactions.
John Allspaw and Dr. Richard I. Cook — Adaptive Capacity Labs
Over 20k mounts to run 100 containers! And NUMA issues too. This one really drives home the fact that SREs need to be cognizant of all layers of the stack.
Harshad Sane and Andrew Halaney — Netflix
Cost explosion is a reliability problem. I love the idea of surfacing sudden cost increase as an alert that something is probably going wrong.
David Iyanu Jonathan — DZone
Autoscaling is reactive, not resilient. Without caps, metrics, or overrides, it can worsen failures. True elasticity requires policy, testing, and bottleneck awareness.
Raise your hand if your system has ever autoscaled itself to death. ✋
David Iyanu Jonathan — DZone
Heinrich Hartmann argues AI’s most valuable role in SRE isn’t autonomous remediation. It’s making sure on-call engineers have the context to fix incidents fast.
Peter Farago — RunLLM
As usual, I enjoy reading Lorin’s analysis of GitHub’s writeup on their incidents just as much as the writeup itself, if not more. Saturation, a security mechanism causing an outage, and more.
Lorin Hochstein
Airbnb made a big move, migrating to a new observability stack. They explain how they structured the project to deliver a big win as early as possible, building buy-in.
Callum Jones — Airbnb
Each one of these is like a pile of War Stories all gathered up into a tidy package of we can learn from.
Karan Nagarajagowda — Uptime Labs