Arc Notes Weekly #101: Generator
This week, how AWS hits 1 petabyte per second on old HDDs, why a DNS bug killed Slack for 14 hours, and how to build a key-value database from scratch.
This week, discover how AWS serves 1 petabyte per second using commodity hard drives through massive parallelism and erasure coding, why a single DNS race condition brought down Slack, Signal, and a Premier League broadcast for 14 hours, and how to build your own key-value database from scratch with LSM trees and append-only files.
Sponsor Spotlight: QCon San Francisco — Enterprise AI Architecture Architecting AI systems? Evaluating LLM infra or planning migrations? QCon San Francisco (Nov 17–21) dives deep into real-world AI architecture, platforms, and patterns—before they go mainstream. Learn from early adopters. Make smarter decisions. Learn more at qconsf.com →
Enjoy this week's round-up!
— Mahdi Yusuf (@myusuf3) or LinkedIn
👋🏾 You are reading Architecture Notes - Your Sunday newsletter, which curates best system design and architecture news from around the web. We would appreciate you sharing it with like-minded people.
Articles
The Programmer’s Identity Crisis: Craft vs. AI
Explore the evolving identity of programmers as AI and ‘vibe-coding’ challenge traditional coding craft. Dive into the tension between maintaining deep engagement with code and the rise of AI-driven development, where programmers risk becoming mere operators.Avoiding Negativity Echo Chambers in Your Career
Marc Brooker shares a crucial piece of career advice: steer clear of negativity echo chambers that can drain your energy and hinder your progress. Instead, focus on positive communities and spend your time with people who inspire you to grow and succeed!Mastering Probabilistic Software: Key Tips Junior Developers
Dive into the world of probabilistic software with Jason Liu’s insights, where he shares essential tips for junior developers transitioning into machine learning. Learn how to focus on outcomes, metrics, and experiments to make informed decisions and improve your ability to handle uncertainty in software development.How to Build a Key-Value Database from Scratch
Ever wondered how databases work under the hood? Dive into this guide to learn how to build a key-value database from scratch, exploring concepts like append-only files, indices, and LSM trees. Discover the trade-offs and challenges of creating a database that efficiently stores and retrieves data!Enterprise AI Architecture at QCon San Francisco (Nov 17-21)
Whether you’re designing AI platforms, scaling LLM serving, or building GenAI infrastructure, QCon San Francisco 2025 connects you with senior architects, engineers, and technical leaders who’ve done it at scale.
Sessions explore balancing deterministic systems with probabilistic agents, autoscaling multi-model serving, vector stores and RAG pipelines at scale, and evolving data access patterns for AI workloads.
Hear what worked, what failed, and what to avoid from 60+ seasoned practitioners—so you can adopt emerging trends with confidence.
Inside the AWS Outage: How DynamoDB’s DNS Failure Caused Chaos
A 14-hour AWS outage took down major services like Signal, Slack, and even a Premier League game, all due to a DNS failure in DynamoDB. Discover how a race condition in DNS management led to this massive disruption and what AWS is doing to prevent it from happening again.When Code Gets Cheaper Every Day
AKent Beck explores the intriguing paradox of programming deflation, where the cost of coding plummets thanks to AI, yet the demand for skilled programmers may rise. Discover how this shift could transform every small business into a software company and why understanding and judgment are the new scarce resources in a world of abundant cheap code.How Obsidian Minimizes Supply Chain Attack Risks
Discover how Obsidian keeps your data safe by minimizing third-party code dependencies, implementing features from scratch, and using strict version control. Their deliberate approach to updates ensures a secure environment for your thoughts!Building Effective Agents
Discover how Anthropic's experience with LLM agents reveals that simplicity and composable patterns often outperform complex frameworks. Learn practical tips for developers to build effective AI agents without unnecessary complexity.How AWS S3 Achieves 1 Petabyte Per Second on Slow HDDs
Discover the engineering marvel behind AWS S3’s ability to serve over 1 petabyte per second using commodity hard drives. Learn how Amazon leverages massive parallelism, erasure coding, and clever load-balancing techniques to overcome the physical limitations of HDDs and deliver high throughput and availability at scale.Projects
canine
Discover Canine, a user-friendly deployment platform that combines the power of Kubernetes with the simplicity of Heroku, offering seamless GitHub integration, real-time monitoring, and more for small teams. Check it out to see how it can streamline your deployment workflows!harper
Tired of slow and privacy-invading grammar checkers? Meet Harper, the open-source, Rust-powered tool that checks your grammar offline in milliseconds, all while keeping your data private. Say goodbye to expensive and overbearing alternatives!
Mastering HTTP Caching: The Key to Faster, Cheaper Websites
Discover how effective HTTP caching can transform your website’s performance, resilience, and cost-efficiency. This comprehensive guide dives into the intricacies of caching headers, common pitfalls, and practical strategies to ensure your site is fast, reliable, and ready to handle traffic spikes.Understanding Good Taste in Software Engineering
Good taste in software engineering isn’t about technical skill but about aligning engineering values with project needs. Discover how taste influences design decisions and why flexibility and context matter more than rigid ‘best practices.’