Techlusion’s Post

Wonderful diagram going over the basic considerations of system architecture.

System Design - High Availability Vs High Throughput Vs High Scalability

Ever wonder what it means when your manager asks you to design for "high availability," "high scalability," or "high throughput"? Let me break it down in simple terms. 𝗛𝗶𝗴𝗵 𝗔𝘃𝗮𝗶𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆 - 𝗞𝗲𝗲𝗽𝗶𝗻𝗴 𝘁𝗵𝗲 𝗦𝗲𝗿𝘃𝗶𝗰𝗲 𝗨𝗽 𝗮𝗻𝗱 𝗥𝘂𝗻𝗻𝗶𝗻𝗴 This refers to maximizing uptime of a service, usually targeting 99.9% availability or higher. Each additional nine usually means an exponential increase in complexity. To achieve this, we build in redundancy at multiple levels, with failover systems ready to step in if the main system crashes. 𝗛𝗶𝗴𝗵 𝗧𝗵𝗿𝗼𝘂𝗴𝗵𝗽𝘂𝘁 - 𝗛𝗮𝗻𝗱𝗹𝗶𝗻𝗴 𝗛𝗲𝗮𝘃𝘆 𝗟𝗼𝗮𝗱 Throughput refers to the number of requests a system can handle per second, measured in transactions per second (TPS) or queries per second (QPS). Common techniques include adding caches, tweaking thread usage, optimizing bottlenecks, and enabling asynchronous processing to handle more simultaneous requests. 𝗛𝗶𝗴𝗵 𝗦𝗰𝗮𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆 - 𝗚𝗿𝗼𝘄𝗶𝗻𝗴 𝗖𝗮𝗽𝗮𝗰𝗶𝘁𝘆 Scalability means a system can expand its workload capacity as needed. To scale out horizontally, it’s common to break services into independent modules or microservices. Leveraging load balancers and service registries enables seamlessly routing requests to new resources. Over to you: What stories do you have tackling these in system design? – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/496keA7

Ever wonder what it means when your manager asks you to design for "high availability," "high scalability," or "high throughput"? Let me break it down in simple terms. 𝗛𝗶𝗴𝗵 𝗔𝘃𝗮𝗶𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆 - 𝗞𝗲𝗲𝗽𝗶𝗻𝗴 𝘁𝗵𝗲 𝗦𝗲𝗿𝘃𝗶𝗰𝗲 𝗨𝗽 𝗮𝗻𝗱 𝗥𝘂𝗻𝗻𝗶𝗻𝗴 This refers to maximizing uptime of a service, usually targeting 99.9% availability or higher. Each additional nine usually means an exponential increase in complexity. To achieve this, we build in redundancy at multiple levels, with failover systems ready to step in if the main system crashes. 𝗛𝗶𝗴𝗵 𝗧𝗵𝗿𝗼𝘂𝗴𝗵𝗽𝘂𝘁 - 𝗛𝗮𝗻𝗱𝗹𝗶𝗻𝗴 𝗛𝗲𝗮𝘃𝘆 𝗟𝗼𝗮𝗱 Throughput refers to the number of requests a system can handle per second, measured in transactions per second (TPS) or queries per second (QPS). Common techniques include adding caches, tweaking thread usage, optimizing bottlenecks, and enabling asynchronous processing to handle more simultaneous requests. 𝗛𝗶𝗴𝗵 𝗦𝗰𝗮𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆 - 𝗚𝗿𝗼𝘄𝗶𝗻𝗴 𝗖𝗮𝗽𝗮𝗰𝗶𝘁𝘆 Scalability means a system can expand its workload capacity as needed. To scale out horizontally, it’s common to break services into independent modules or microservices. Leveraging load balancers and service registries enables seamlessly routing requests to new resources. Over to you: What stories do you have tackling these in system design? – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/496keA7

Ever wonder what it means when your manager asks you to design for "high availability," "high scalability," or "high throughput"? Let me break it down in simple terms. 𝗛𝗶𝗴𝗵 𝗔𝘃𝗮𝗶𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆 - 𝗞𝗲𝗲𝗽𝗶𝗻𝗴 𝘁𝗵𝗲 𝗦𝗲𝗿𝘃𝗶𝗰𝗲 𝗨𝗽 𝗮𝗻𝗱 𝗥𝘂𝗻𝗻𝗶𝗻𝗴 This refers to maximizing uptime of a service, usually targeting 99.9% availability or higher. Each additional nine usually means an exponential increase in complexity. To achieve this, we build in redundancy at multiple levels, with failover systems ready to step in if the main system crashes. 𝗛𝗶𝗴𝗵 𝗧𝗵𝗿𝗼𝘂𝗴𝗵𝗽𝘂𝘁 - 𝗛𝗮𝗻𝗱𝗹𝗶𝗻𝗴 𝗛𝗲𝗮𝘃𝘆 𝗟𝗼𝗮𝗱 Throughput refers to the number of requests a system can handle per second, measured in transactions per second (TPS) or queries per second (QPS). Common techniques include adding caches, tweaking thread usage, optimizing bottlenecks, and enabling asynchronous processing to handle more simultaneous requests. 𝗛𝗶𝗴𝗵 𝗦𝗰𝗮𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆 - 𝗚𝗿𝗼𝘄𝗶𝗻𝗴 𝗖𝗮𝗽𝗮𝗰𝗶𝘁𝘆 Scalability means a system can expand its workload capacity as needed. To scale out horizontally, it’s common to break services into independent modules or microservices. Leveraging load balancers and service registries enables seamlessly routing requests to new resources. Over to you: What stories do you have tackling these in system design? #Applicationdesign #systemdesign

We are often asked to design for high availability, high scalability, and high throughput. What do they mean exactly? The diagram below is a system design cheat sheet with common solutions. 1. High Availability This means we need to ensure a high agreed level of uptime. We often describe the design target as “3 nines” or “4 nines”. “4 nines”, 99.99% uptime, means the service can only be down 8.64 seconds per day. To achieve high availability, we need to design redundancy in the system. There are several ways to do this: - Hot-hot: two instances receive the same input and send the output to the downstream service. In case one side is down, the other side can immediately take over. Since both sides send output to the downstream, the downstream system needs to dedupe. - Hot-warm: two instances receive the same input and only the hot side sends the output to the downstream service. In case the hot side is down, the warm side takes over and starts to send output to the downstream service. - Single-leader cluster: one leader instance receives data from the upstream system and replicates to other replicas. - Leaderless cluster: there is no leader in this type of cluster. Any write will get replicated to other instances. As long as the number of write instances plus the number of read instances are larger than the total number of instances, we should get valid data. 2. High Throughput This means the service needs to handle a high number of requests given a period of time. Commonly used metrics are QPS (query per second) or TPS (transaction per second). To achieve high throughput, we often add caches to the architecture so that the request can return without hitting slower I/O devices like databases or disks. We can also increase the number of threads for computation-intensive tasks. However, adding too many threads can deteriorate the performance. We then need to identify the bottlenecks in the system and increase its throughput. Using asynchronous processing can often effectively isolate heavy-lifting components. 3. High Scalability This means a system can quickly and easily extend to accommodate more volume (horizontal scalability) or more functionalities (vertical scalability). Normally we watch the response time to decide if we need to scale the system. Over to you: Do you have other things to share in your design toolbox?

🚀 Exploring the world of system design! 🌐 Ever wondered about "High Availability," "High Scalability," and "High Throughput"? Let's break it down: 🔗 **High Availability - Keeping the Service Up and Running** Maximizing uptime, aiming for 99.9% availability or higher. Redundancy is key, with failover systems ready to step in if the main system crashes. It's all about resilience and continuous service delivery. 💨 **High Throughput - Handling the Load with Ease** Throughput measures how many requests a system can handle per second (TPS or QPS). Techniques like caching, thread optimization, bottleneck tweaking, and asynchronous processing boost the system's capacity to handle more simultaneous requests. 🚀 **High Scalability - Growing with Demand** Scalability ensures a system can expand its workload capacity as needed. Breaking services into independent modules or microservices, combined with load balancers and service registries, allows seamless routing of requests to new resources. It's all about growing gracefully with demand. Ready to dive into the world of resilient, high-performing systems? Let's connect and explore the intricacies of scalable, available, and high-throughput designs! 🌐💡 #SystemDesign #HighAvailability #Scalability #Throughput #TechTalks More Contents: Woongsik Dr. Su, MBA

Ever wonder what it means when your manager asks you to design for "high availability," "high scalability," or "high throughput"? Let me break it down in simple terms. 𝗛𝗶𝗴𝗵 𝗔𝘃𝗮𝗶𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆 - 𝗞𝗲𝗲𝗽𝗶𝗻𝗴 𝘁𝗵𝗲 𝗦𝗲𝗿𝘃𝗶𝗰𝗲 𝗨𝗽 𝗮𝗻𝗱 𝗥𝘂𝗻𝗻𝗶𝗻𝗴 This refers to maximizing uptime of a service, usually targeting 99.9% availability or higher. Each additional nine usually means an exponential increase in complexity. To achieve this, we build in redundancy at multiple levels, with failover systems ready to step in if the main system crashes. 𝗛𝗶𝗴𝗵 𝗧𝗵𝗿𝗼𝘂𝗴𝗵𝗽𝘂𝘁 - 𝗛𝗮𝗻𝗱𝗹𝗶𝗻𝗴 𝗛𝗲𝗮𝘃𝘆 𝗟𝗼𝗮𝗱 Throughput refers to the number of requests a system can handle per second, measured in transactions per second (TPS) or queries per second (QPS). Common techniques include adding caches, tweaking thread usage, optimizing bottlenecks, and enabling asynchronous processing to handle more simultaneous requests. 𝗛𝗶𝗴𝗵 𝗦𝗰𝗮𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆 - 𝗚𝗿𝗼𝘄𝗶𝗻𝗴 𝗖𝗮𝗽𝗮𝗰𝗶𝘁𝘆 Scalability means a system can expand its workload capacity as needed. To scale out horizontally, it’s common to break services into independent modules or microservices. Leveraging load balancers and service registries enables seamlessly routing requests to new resources. credits : Sahm

Unlocking the secrets of System Design Jargons is crucial for building robust and efficient systems. Here are some key terms you need to know: 1️⃣ SLA (Service Level Agreement): An agreement between a service provider and a customer that outlines the level of service expected. It defines parameters like uptime, response time, and resolution time, ensuring accountability and quality of service. 2️⃣ TTL (Time To Live): A value indicating the lifespan of data in a cache or a DNS record. Once the TTL expires, the data is considered stale and must be refreshed. Managing TTL optimally is essential for maintaining data freshness and reducing unnecessary cache overhead. 3️⃣ Scalability: The ability of a system to handle increasing workload or traffic without sacrificing performance. It's about designing systems that can seamlessly grow to meet user demand while maintaining efficiency. 4️⃣ Availability: The proportion of time that a system is operational and accessible. High availability ensures that users can rely on the system to be accessible whenever they need it, minimizing downtime and disruptions. 5️⃣ Load Balancing: Distributing incoming network traffic across multiple servers to ensure optimal resource utilization and prevent any single server from being overwhelmed. Load balancers help in improving performance and reliability by evenly distributing the workload. 6️⃣ Fault Tolerance: Designing systems that can continue to operate even in the presence of failures. It involves redundancy, graceful degradation, and fault detection mechanisms to ensure uninterrupted service despite component failures. 7️⃣ Caching: Storing frequently accessed data temporarily in a cache to reduce latency and improve performance. Caching helps in serving data faster by fetching it from a nearby cache instead of going to the original source every time. Understanding these jargons is key to designing systems that meet performance, reliability, and scalability requirements. #SoftwareArchitecture #Engineering #LinkedInLearning

Let’s imagine a world where we build applications not only with the aim of these working well but also handling a lot of users smoothly, even when things get busy. That’s what Reactive Architecture is all about. It’s just like creating a smart system that knows how to manage lots of information at once, without slowing down or crashing. This way, everyone using the app has a great experience, no matter how many people online or how much data is being used. That is what bring me here, pay attention for a few minutes, the time to empty your cup of coffee we will be done. Small Story Imagine it’s November 25, 2020, and you’re going about your day when suddenly, you can’t access your favorite tools and platforms online. Adobe won’t load, your Roku streaming is glitching, Twilio-based apps aren’t sending messages, Duolingo isn’t teaching languages, and even planning an in-person meetup using Meetup feels like a distant dream. To top it off, New York City’s Metropolitan Transit Authority is also facing tech hiccups. Why? Because AWS, a giant upon which the digital world leans, experienced an outage. This wasn’t just any outage. It was the kind that reminded everyone of a similar hiccup back in 2017, which also sent waves of disruption across the internet. Both instances shone a spotlight on a truth we often forget: our vast, intricate digital ecosystem is more interconnected than we realize. A snag in one corner can send ripples across the entire pond, affecting services and sectors far and wide. Reflecting on the 2020 AWS outage is like peering into a digital mirror, revealing our collective reliance on cloud services and the vulnerabilities that come with this dependence. Despite tech advances and beefed-up infrastructures, the incident underscored a lingering question: How do we safeguard our digital lives against such outages? It pushed companies to think harder about having backup plans and highlighted the broader conversation about the centralization of internet services and the risks this centralization poses to our digital society. In essence, the AWS outage of 2020 was more than just a technical glitch; it was a moment of realization about the fragile web that supports our digital world and a call to action for ensuring its strength and resilience. What is Reactive Architecture? At its core, Reactive Architecture is designed to create non-blocking, asynchronous applications that are capable of handling a vast number of concurrent data streams and operations. This approach not only enhances performance but also improves the user experience by ensuring responsive and resilient interactions. Reactive Architecture isn’t just about building applications; it’s about crafting experiences that feel alive. Imagine apps that breathe with the pulse of user interaction, systems that not only respond to demands but anticipate them, creating a seamless flow of information. This is technology that