2
$\begingroup$

I'm researching a subject of balancing the load between two black-box systems (with some twists). I thought that I could record latest response time log from each of those systems and process such a log to dynamically make decisions on how to balance the load.

E.g. if we start with system A handling 100% of the current load with 100ms 90p latency from some recent time window and the load grows so the latency grows to 150ms we should make a decision to duplicate some of those requests to system B so it warms up and eventually spread the load.

Unfortunately I don't really know where to start in looking for suitable algorithms that I could modify to suit this problem. I'd rather avoid ML, but I'm sure there are some other solutions to this problem.

If you could point me to some algorithms or even keywords to further research it'd be great. Thanks!

Update: I've looked at Markov Decision Process but it seems that it's not suited to problems where effects of an action depend not only on the current state, but also on history. Unless there would be a way to model some recent history window as a single state...

Improve this question
$\endgroup$
3
  • $\begingroup$ Can you explain why you need the whole history? Is a state that describes the load of each box and something as the average latency in the last x periods not sufficient? $\endgroup$
    – Michiel uit het Broek
    Commented Jun 17, 2020 at 8:30
  • $\begingroup$ Maybe the whole history is not needed but some change in the last x seconds would be useful since the traffic might be currently on the upwards or downwards trajectory in which case you want to make a different decision regarding load balancing. $\endgroup$
    – Bartek Andrzejczak
    Commented Jun 18, 2020 at 9:06
  • $\begingroup$ If the decision depends on a few history states, then MDPs may still work quite well in my experience. Just for my understanding (I'm not familiar with the problem you described), can we more or less freely reallocate load among the units at any moment in time? And is the total load that the units process together given by a (known) stochastic process or also a decision? Which accuracy do you need, e.g., would describing the current load of a box in steps of 1-5% suffice? Lastly, would it suffice to distinguish a few trajectory states (e.g., decreasing, stable, increasing, rapidly increasing)? $\endgroup$
    – Michiel uit het Broek
    Commented Jun 18, 2020 at 9:31

0

Reset to default