Newman Maps Disaster Recovery for AI-Era Software — SkimNews
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Sam Newman published 'Building Resilient Distributed Systems' with O'Reilly Media, spending three years instead of the planned two to rework chapters addressing AI's impact on resilient system design
- The book defines Recovery Point Objective (RPO) as the maximum time since data was last backed up and Recovery Time Objective (RTO) as acceptable downtime, noting cost and complexity rise sharply as both shrink toward zero
- Newman outlines four DR patterns ordered by complexity—backup-and-restore, pilot light failover, warm failover, and multisite active-active—that can be layered incrementally as needs grow
- Newman argues RPO and RTO don't map cleanly onto SLOs because disasters don't occur often enough to generate meaningful service level indicators
- Ongoing GPU and CPU shortages mean infrastructure availability can't be assumed, requiring engineers to understand vendor constraints and prioritize critical services for failover planning
- For AI-powered software, Newman recommends multi-model and multi-vendor strategies, transparent cost tracking, and frequent vendor analysis to handle model unpredictability and provider resilience gaps
Why it matters: For teams seeing more production incidents driven by AI-generated code, Newman's framework gives concrete DR targets and four escalating recovery architectures rather than abstract advice. The book's specific treatment of AI-layer resilience—multi-vendor strategies and observability for shifting models—addresses gaps that traditional SRE playbooks don't yet cover.
Ask SkimNews
