Arc Notes Weekly #79: Changing Order

This week, explore how Discord reduced WebSocket traffic by 40% using smart compression techniques, learn the challenges of concurrency and discover how tidying up code incrementally has big results.

Sep 29, 2024

This week, explore how Discord reduced WebSocket traffic by 40% using smart compression techniques, learn the challenges of concurrency in software development, and discover how tidying up code incrementally can lead to major architectural improvements.

Enjoy this week's round-up!

— Mahdi Yusuf (@myusuf3) or LinkedIn

👋🏾 You are reading Architecture Notes - Your Sunday newsletter, which curates best system design and architecture news from around the web. We would appreciate you sharing it with like-minded people.

Articles

Introducing Netflix’s Key-Value Data Abstraction Layer

Netflix developed a Key-Value Data Abstraction Layer (DAL) to streamline data access across its distributed databases, like Cassandra, improving consistency and efficiency. This system adapts to both simple and complex data models, reducing the complexity for developers and supporting Netflix’s diverse use cases with minimal effort.

How Discord Reduced WebSocket Traffic by 40%

Discord reduced WebSocket traffic by 40% by implementing zstandard compression with streaming support, replacing zlib, which significantly improved compression efficiency. They also introduced Passive Sessions V2 to reduce redundant data transmission, cutting bandwidth usage for inactive users. These optimizations improved performance across all platforms, leading to major bandwidth savings without affecting the user experience.

What makes concurrency so hard?

The article explores why concurrency in software development is difficult, highlighting how humans can manage concurrent reasoning but struggle with state space explosion. As systems become more complex, the number of possible states grows exponentially, making bugs harder to detect. The author suggests that the key to managing concurrency is reducing the size of the state space, using techniques like mutexes, memory isolation, and atomic instructions to simplify behavior and reduce potential errors.

Helping You Flex Your Product Muscles.

Don’t Be Left Out of the Engineering Revolution! Discover what top engineers are learning from PostHog’s Product for Engineers newsletter. Whether it’s curated advice or the very lessons that shaped PostHog, this is the go-to source for anyone looking to make an impact. Subscribe for free and be part of the future, today!

Reasons I Still Love The Fish Shell

This blog post outlines why the author continues to love the Fish shell for command-line work. They highlight Fish’s user-friendly features, such as syntax highlighting, auto-suggestions, and straightforward scripting. The post also emphasizes how Fish enhances productivity by simplifying routine tasks and making the shell experience more pleasant and efficient, even after years of use.

PostgreSQL 17 Released

PostgreSQL 17 introduces significant performance improvements, such as better memory management for vacuum processes and faster I/O operations. Key features include enhancements to JSON handling, logical replication for high availability, and better bulk data export performance. The release also adds tools for incremental backups and new developer-focused commands like JSON_TABLE. These updates make PostgreSQL 17 more efficient and scalable for a wide range of workloads.

Magic of Tidying Up Code

In this post, the author reflects on the practice of “tidying” code—small, safe cleanups that can lead to significant improvements over time. They share rules like avoiding difficult tasks when tired, landing changes immediately, and reverting when encountering issues. The process encourages incremental progress without requiring a grand vision of the final design. The post emphasizes how tidying can lead to major architectural improvements through disciplined, low-risk steps.

Should we decompose our monolith?

This article explores strategies for deciding whether to decompose a monolith into microservices. It outlines key considerations such as business and engineering constraints, trade-offs in managing complexity, and the overhead of both architectures. The piece emphasizes that while microservices can offer benefits, they often introduce operational burdens, making a careful evaluation essential before proceeding with decomposition.

Projects

JSON4U

JSON4U is a versatile tool for visualizing and processing JSON data. It offers features like graph and table view modes, structured and text comparisons, and CSV import/export. The tool also supports nested parsing and integrates with jq for advanced queries. Its user-friendly interface makes it a powerful resource for developers working with JSON data.

How I Computer 2024

Jon Seager shares his 2024 computer setup, covering hardware like his AMD Ryzen 9 desktop, Lenovo Z13 laptop, and iPhone 15 Pro. He discusses his use of tools like NixOS, Obsidian for note-taking, and Tailscale for secure connections. The article dives into his productivity apps, development environment, and home server setup, offering insights into how he streamlines work and personal computing with custom workflows.