Critical view of a particular application design

Question

For a system, I have certain requirements

1. should be soft realtime.
2. should be able to handle lots of operations in parallel
3. should have ability to add, remove, alter features
4. should be able to increase/decrease memory, computation based on load

So I have implemented an app design based on microkernel architecture with a mediator bus pattern. Here is the design

Design Philosphy:

• Worker is any piece of code, running on a dedicated thread.
• Each worker has exactly one message box
• System worker has only one copy, starts/stops with the system only
• User worker can be created/ destroyed or multiple copies can run.
• Workers cannot share anything
• Workers can only communicate through immutable messages

Kernel

1. Can starts or stops a processes.

2. Allocates message boxes

3. can run multiple copies of user workers

Router

1. Can only move messages from one message box to another.

2. Undelivered messages are delivered to dead letter collector.

Dead Letter Collector

1. Messages which have no receiver are received and dumped

2. Messages undelivered because of no space in target message box are kept and retried after a certain interval 3 times at least.

Features can be built as workers and can be loaded and unloaded without modifying the source code at run time. (similar to plug-ins). If there is load on a certain feature then kernel can load multiple copies of that feature. In case of feature not available, the system does not crash but the messages are considered undelivered. new features can be added, existing ones can be removed or altered.

A feature can be an API, a database functionality, a computation etc. and a feature can be dependent of other features too.

---

Based on the requirements and the proposed design what are your views and what can we improve upon?

Answer 1

The Dead Letter Collector seems to be doing two things: handling undeliverable messages, and handling messages that need to be retried (delayed delivery). This could be split into two collectors, particularly as the name "Dead Letter" implies (to me) that anything that goes there will never leave.

Messages that can't be delivered for any reason can be sent to a Retry Collector. This will handle all of the retry logic. If too much time passes then the message can be passed along to the Dead Letter Collector.

Messages that arrive in the Dead Letter Collector won't ever leave, so it does not need an Outbox. They can be retained for diagnostic purposes before being purged, or they can be purged immediately.

The workers also need to be able to properly handle messages that don't get fully processed. Things like crashes, service restarts, or system updates may cause a worker to be stopped before the message has been fully processed. Sometimes this can be done by leaving messages in the inbox until all the output has been generated and saved, and any new messages sent to the outbox. If you have multiple workers reading from one inbox you'll need to keep track of which messages are being handled, and have some way to determine when a message is not being handled anymore and needs to be reprocessed (while keeping open to the possibility that handling the message is crashing the worker).