Arc Notes Weekly #80: Pop
This week, explore how switching from Redis to SQLite for better performance and scalability. Discover the concept of “skin-shedding” code, where removing old and rewriting helps systems evolve.
This week, dive into how a company restructured their web firewall, switching from Redis to SQLite for faster performance and easier scaling. Discover the art of “skin-shedding” code, where deleting large swaths of old code leads to healthier, more maintainable systems. Plus, learn to cope with the relentless pace of technology by embracing continuous learning and focusing on practical trends rather than chasing every shiny new tool.
Enjoy this week’s insightful reads!
— Mahdi Yusuf (@myusuf3) or LinkedIn
👋🏾 You are reading Architecture Notes - Your Sunday newsletter, which curates best system design and architecture news from around the web. We would appreciate you sharing it with like-minded people.
Articles
Redis > SQLite
In “Rearchitecting: Redis to SQLite,” Wafris explains their decision to migrate from Redis to SQLite for their web application firewall. They detail performance challenges with Redis, especially network latency, and benchmarked SQLite as a faster, simpler alternative for their specific use case. The article highlights the improved deployment experience and scalability benefits of SQLite, while acknowledging the trade-offs, particularly with write operations.
Terminal Colours
In “Terminal Colours,” Julia Evans explores how terminal color schemes work, including how different terminals and environments handle color codes. She breaks down the 16-color palette and how to use ANSI escape codes to add color to terminal outputs, making it easier to create visually engaging command-line experiences. Evans also touches on customizing colors and dealing with discrepancies between different terminals.Git Absorb
Git Absorb is an automatic tool that streamlines fixing up Git commits. It helps developers absorb changes directly into previous commits without manually searching for commit SHAs or doing interactive rebases. It automates git commit --fixup by analyzing the working directory and efficiently applying the necessary changes. This tool is useful for improving the version control workflow, particularly when responding to code review feedback or cleaning up commit history.Shedding Code
In “Skin-Shedding Code”, Thorsten reflects on the practice of “shredding” code, which means deleting and completely rewriting existing code to build better solutions. Unlike simple refactoring, shredding involves large-scale deletions and recreations. He discusses how this approach is part of the culture at Zed and contributes to a healthier codebase, drawing parallels to controlled wildfires that eliminate unnecessary components while preserving and improving the core.Automated Architecture Documentation for Streamlined System Design
Multiplayer auto-documents your system, from the high-level logical architecture down to the individual components, APIs, dependencies, and environments. Perfect for teams looking to streamline system design and documentation management without the manual overhead.
How to avoid technology FOMO
In “How to Cope with Technology FOMO,” Avdi Grimm offers strategies for managing the fear of missing out on the rapid changes in tech. He emphasizes focusing on learning how to learn, identifying key technologies, and staying grounded in what’s practical rather than chasing every new trend. Grimm also encourages humility, pointing out that most real-world work involves maintaining older systems, and predicting future trends is rarely accurate.
Projects
Screenpipe
Screenpipe is a 24/7 AI-powered screen and mic recording tool designed to help developers build apps with full context from local data. It is an open-source alternative to Rewind.ai, supporting multi-monitor capture and audio input/output. Users can run Screenpipe through CLI or desktop apps, with options to customize via a plugin system. Screenpipe emphasizes privacy, giving users full control over their data, and offers Rust and WASM libraries for integration.Crawl4AI
Crawl4AI is an open-source web crawler and scraper designed to work seamlessly with large language models (LLMs). It offers features like extracting metadata, media, and structured data, supports asynchronous web crawling, and works with JavaScript-based dynamic content. With built-in LLM-friendly output formats like JSON and markdown, it’s ideal for AI-related applications. It also supports proxy configurations, session management, and advanced chunking strategies, ensuring privacy and high performance.
The Builder’s Guide to Better Mousetraps
In “The Builder’s Guide to Better Mousetraps,” Marc Brooker outlines key questions developers should ask when deciding to build a new solution versus adapting an existing one. He stresses the importance of evaluating costs, long-term ownership, technical risk, and scale differences. Brooker encourages thoughtful decision-making to ensure that building a new solution offers enough advantages to justify the effort over modifying what’s already available.