Arc Notes Weekly #87: Mustard
This week, Netflix’s self-healing bug fix, Canva’s privacy-preserving search improvements, and the speed boost of binary vector embeddings—this week’s highlights are packed with innovation!
This week, learn how Netflix tackled a persistent concurrency bug with an innovative self-healing system. Canva’s engineers show how to improve search without peeking at queries or results. Plus, explore the power of binary vector embeddings that make data retrieval faster and more efficient.
Sponsor Spotlight: Start selling to enterprises with a few lines of code. WorkOS is a modern identity platform for B2B SaaS, offering flexible, easy-to-use APIs to integrate SSO, SCIM, and FGA in minutes instead of months. It's used by hundreds of the fastest growing companies in the world like Cursor, Vercel, and Perplexity.
Enjoy this week's round-up!
— Mahdi Yusuf (@myusuf3) or LinkedIn
👋🏾 You are reading Architecture Notes - Your Sunday newsletter, which curates best system design and architecture news from around the web. We would appreciate you sharing it with like-minded people.
Articles
Building a Self-Healing System to Tackle a Netflix Concurrency Bug
Netflix engineers tackled a concurrency bug that was crippling CPUs. With no immediate fix until Monday and rollback impractical, the team innovated a self-healing solution: disabling autoscaling and terminating problematic instances at regular intervals, allowing fresh ones to replace them by implementing a self-healing system that terminated failing instances every 15 minutes, replacing them with fresh ones. This quick fix maintained stability over the weekend until a proper solution could be deployed.Binary vector embeddings are so cool
Binary vector embeddings make high-dimensional data searches ultra-efficient, reducing storage by 32x and speeding up retrieval by 25x. This innovative approach uses single-bit representations to maintain high accuracy while slashing resource demands—worth a deep dive.Improving Search Without Analyzing Queries or Results
Canva’s engineering team enhanced private design search by generating synthetic datasets using large language models, enabling effective evaluation without accessing user data. This approach respects user privacy while improving search functionality. Unlock Enterprise Revenue Without the Hassle
Single Sign-On, Directory Sync (SCIM), Audit Logs, Fine-Grained Authorization — these are essential features when selling to enterprises. The problem is that building and maintaining them requires significant resources and can result in over $8M in lost revenue.
WorkOS simplifies this process with easy-to-use APIs that help you integrate complex enterprise features into your app in minutes. It's the fastest way to go upmarket while allowing your engineers to focus on building the core product.
Shrinking a Postgres Table
Facing a bloated Postgres table consuming 57 GB due to unprocessed records, the author employed a table-swapping technique to efficiently reclaim disk space. By creating a new table with only necessary data and swapping it with the old one, they reduced disk usage from 87% to 53% without significant downtime. Reflect Deeply on the Boundaries of the Unknown
Qwen’s latest model, QwQ-32B-Preview, showcases advanced reasoning by engaging in self-dialogue and introspection, achieving notable scores on benchmarks like GPQA and AIME. This approach enhances its problem-solving capabilities across scientific and mathematical domains.How Much Memory Is Needed in 2024 to Handle 1 Million Concurrent Tasks?
A recent benchmark evaluated the memory efficiency of various programming languages—Rust, C#, Go, Java, NodeJS, and Python—when running up to one million concurrent tasks. The findings revealed that C# (using NativeAOT) and Rust exhibited superior memory efficiency, while Go’s goroutines consumed more memory than anticipated.Projects
HyperDX
HyperDX is an open-source observability platform that unifies session replays, logs, metrics, traces, and errors, enabling engineers to diagnose production issues efficiently. Powered by Clickhouse and OpenTelemetry, it offers a developer-friendly alternative to tools like Datadog and New Relic.Conditional Writes with S3
Amazon S3 now supports conditional writes that evaluate if an object is unmodified before updating it, helping coordinate simultaneous writes and prevent unintentional overwrites. By providing the object’s ETag in the If-Match header during PutObject or CompleteMultipartUpload API requests, S3 ensures the object’s state matches expectations before committing the write.
Advent of Code 2024
Advent of Code 2024 is an annual coding challenge that kicks off on December 1st. With daily puzzles designed for all skill levels, it’s a festive way for programmers to sharpen their skills, compete, and have fun solving creative problems through the holiday season. Dive in solo or join a leaderboard with friends!