Questions tagged [distributed-system]
This tag can be used by questions on distributed system concept, design, and implementations.
206
questions
0
votes
1
answer
37
views
How to rebalance data across nodes?
I am implementing a message queue where messages are distributed across nodes in a cluster. The goal is to design a system to be able to auto-scale without needing to keep a global map of each message ...
- 115
1
vote
2
answers
82
views
Subscribing to a range of topics per instance of a service [closed]
If I have a system where my distributed service is sending live scores of 1000 s of football games from some hypothetical event to millions of clients. The service subscribes to the games from a ...
- 244
0
votes
1
answer
33
views
how about using kubernetes statefulset to mapping the snowflake datacener id and worker id
I am developing a distribution id project, now using the twitter snowflake id as the fundation of the distribution id. In kubernetes cluster, to fetch the uniq and non-conflict datacenter id and ...
- 103
2
votes
1
answer
230
views
Problem with data consistency in Microservice architecture(2pc transaction) when external dependency envolved
I have microservice architecture. I implement a distributed transaction flow (2-phase commit), kafka, and 1 coordinator service. So, my problem is in each step of preparation I am calling an outside ...
- 157
-2
votes
1
answer
112
views
What is the difference between Eventual Consistency, Strong Eventual Consistency, and Causal Consistency?
I read articles about CRDTs and frequently come across the terms 'Strong Eventual Consistency,' 'Eventual Consistency,' and 'Causal Consistency.' What is the relationship between these consistency ...
- 11
-1
votes
1
answer
71
views
Designing a Distributed System for Indigenous Data Sovereignty Across Nations [closed]
I'm looking for some quick "back-of-the-napkin" thoughts from systems engineers on the following scenario:
A collection of Indigenous groups (in the US, Canada, and Australia) want to design ...
6
votes
4
answers
2k
views
Protecting against malicious duplicate IDs in a distributed environment
Let's say we have multiple (somewhat autonomous) (micro-)services, and when entities are created, the ID (UUIDs or whatever) can be set externally. How can we ensure that an ID remains unique across ...
- 71
0
votes
0
answers
50
views
Distributed caching system that scales and enables real-time UI updates?
I have a NextJS app for which I want to cache data from our backend APIs to reduce the load when we have high volumes of concurrent users scrolling through (or search) years-worth of historical data.
...
- 9
0
votes
1
answer
128
views
Architectural decision regarding at-most-once semantics cron jobs in distributed systems
please note that while I refer to a specific web application framework in the following, the problem also arises with most other web application frameworks I know so please don't be afraid to reply ...
- 109
0
votes
2
answers
180
views
Fault tolerance in aggregated distributed state
I have a scheduling system that is horizontally scaled, and stores shared state in a redis key.
The purpose of the system is to implement something similar to classic rate limiting, but a bit ...
- 21
1
vote
2
answers
140
views
Generating unique number in distributed environment
I have little app that i plan to run behind load balancer, with up to 5-8 instances.
Inside the app has service, that needs to generate unique number that has up to 10 digits ( so 1-10 digits) each ...
- 209
1
vote
2
answers
100
views
What is the starting point of a geo-distributed server farm?
Scenario:
A web application that wants to implement geographically high availability (HA), has multiple web servers in different cities around the globe. Each server is connected to a database in the ...
- 121
1
vote
1
answer
141
views
App that can be deployed both as monolith and microservices?
We have a monolith C++ application and I consider doing some major refactorings. Since we're not completely ready to go full "microservices only" (maybe in about two years time), I want to ...
- 419
0
votes
2
answers
629
views
How to effectively scale one microservice to multiple instances
Let’s say I have a booking microservice (single instance) with its own DB.
If I want to horizontally scale the service with multiple instances(exact same copy of the business logic code) running on ...
- 137
-2
votes
1
answer
76
views
Rate limiting design
I am designing rate limiting in my application in which the rules can be dynamic. I have a basic design in which every request is validated against the rate limiting quota, which is stored in redis. ...
- 199