Possible Solutions for avoiding applications sharing database

Question

Background:

I work on a small group (5-10) of lightly used .Net web applications.

The applications typically have dedicated databases, but there are instances of an application accessing another application's database directly. There is also a common database that most of the applications use a source of truth for some data

I have been trying to decide on a methodology that we can begin to adopt to prevent the following types of questions:

• How will a proposed change to the database affect our family of applications?
• Which applications use this table?
• How did our database connections become a hot spaghetti mess?

Possible Solutions:

Here are the possible solutions I have thought of:

• Accept the current position and allow applications to access each other's databases as needed
• Follow a 1 database <--> 1 application model and have the applications communicate via API when they need to get information from each other
• Create dedicated code libraries / packages for database access, which are consumed by our applications
• Move toward a mono-repo solution where multiple applications share a single repo. Build modules within that repo to be responsible for database access
• Investigate using a message bus of some kind.
  • I do not have much experience in this area and it seems like overkill to me.

Question:

My goal with this question is to have a full list of reasonable approaches I could take, so I can consider the pros and cons and pick an approach.

Can anyone think of other possible solutions that I am not considering?

Answer 1

I suspect you did not simply divide and create different applications for any reason or new functionality, and there is actually a logic behind this grouping of sorts. So each application belongs to a context, with (kind of) defined boundaries and responsibilities. What you are dealing with has a name and it is called: micro services architecture (whether that was the intention or not, is another issue).

For micro services it is usually recommended that each should have its own database. Remember: Each application belongs to a context, and knows about a set of specific data (or entities), and it should be the only one writing or editing anything in its repository. If another application needs data that belongs to this application, it should be exposed through another mechanism, be it replication or, what is most commonly done: APIs. Of course, this will require that you build another set of apps for each context, but such is the nature of this architecture.

As for your other solutions:

• Accepting the current state will depend on the size and importance of the applications: If making changes is not that hard, then you should move on to something else that provides more value. But if it has become a nuisance, this will only get worse with time. Make changes now.

• Creating generic libraries for access just adds a layer of abstraction, but the external application would still need to know about the data, what and how to write, and changes would be harder to track. The API will provide a single point of access, for any other app that might require it, while still keeping the responsibilities intact.

• As said earlier, the best approach is for each application to be independent of each other and have their own databases. Access data through API. Keep in mind that in some situations, the apps could be so small they could share a single database, but in no situation is good for one app to access another apps database directly.

• The mono-repo approach sounds more like a monolithic approach. I am not particularly against building a monolith (actually, most solutions should start like that), but keep in mind why you have so many apps in the first place: Was it because maintaining a single huge model was complicated? The logical division of responsibility will persist even with all your apps in the same place. The problem is not how they access the database, but why.

• Messages are a solution for different problems, specially when you need asynchrony in your processes. Unless your replication strategy involves this (it could be), you would still need new APIs to provide access to the data.

Hope this helps!

Answer 2

It all comes to how logical it would be from the point of view of the business.

Imagine that you are a car dealer, and you have two APIs: one deals with the actual selling of the cars, and the other one—with the repairs. Both would at some point have a notion of a car model: you need to know that you sold a Ford or a GMC, and you need to know if you're repairing a Toyota or a Tesla.

Sharing the database between both APIs could quickly become a nightmare. What if the API that deals with cars to be sold needs a change in the data structure, but you don't have the resources to adapt to this change in the second API?

Making one API rely on the other doesn't make too much sense. Why would the repair API call the selling API, or the other way around? What makes one or the other the “owner” of the data that lists all the supported car manufacturers and models?

A common solution for the past ten years is to isolate the shared data into its own API. And yes, you may end up with a lot of APIs that talk to each other, and no, it's usually not a problem—microservices is exactly about that: interconnecting lots of APIs, but in a way that you don't end up with a maintenance nightmare where in order to change one API, you need to change dozen others.

In my example, you'll end up with an API that, at the beginning, may do just that: communicate the list of car manufacturers and car models. With time, it may grow first to accept not only the read-only mode, but also the updating of the data, and later, it may also accept some other types of data which may make sense to put together.

Make sure your architecture remains flexible at all times. Essentially, you will (no matter how experienced you are) have to adapt your services over time—you will decide that one service should actually be split into multiple ones, or that a bunch of services should be merged. Essentially, it's the same thing as with classes in OOP: some classes grow and you end up splitting them to follow the single responsibility principle, while other classes end up being merged into one.

Answer 3

Based on the solutions you propose, it seems to me that you're not really leveraging the microservice model effectively, instead being content with what is commonly referred to as a distributed monolith.

Distributed monoliths aren't great. They give you all the drawbacks from monoliths (tight coupling) and all the drawbacks from microservices (increased development overhead) with none of the benefits.

Can anyone think of other possible solutions that I am not considering?

Contrary to what you're expecting, I'm going to instead start excluding solutions that you currently consider to be viable.

Accept the current position and allow applications to access each other's databases as needed

I mean, if not solving problems is an acceptable outcome to you, sure. But then the question is mostly moot to begin with.

Follow a 1 database <--> 1 application model and have the applications communicate via API when they need to get information from each other

This is the microservice way.

Create dedicated code libraries / packages for database access, which are consumed by our applications

This doesn't solve the core issue. It gives you reusability of your logic, but when that logic gets updated it either forces you to update all services at the same time (thus maintaining a monolithic runtime) or your services are going to be running different versions of this package concurrently, which is going to massively complicate how you intend to serve this database in a version-friendly manner.

Move toward a mono-repo solution where multiple applications share a single repo. Build modules within that repo to be responsible for database access

A monorepo is concerned with where you store your code. It has no bearing on the runtime of the services.

Essentially, we get to the same argument as the previous point. While it enables code reusability, it ties your service runtimes together to the point of defeating the benefits from having microservices in the first place.

Investigate using a message bus of some kind.

I'm not even sure what you mean here, but given you point out a lack of knowledge I'm going to assume that this suggestion was a stab in the dark to begin with.

My goal with this question is to have a full list of reasonable approaches I could take, so I can consider the pros and cons and pick an approach.

The core issue here is one of communication between services. The question you should be focusing on is why you have multiple service in the first place. There's two reasonable outcomes to that consideration:

• You didn't need to have separate services

The answer here is to move back to a monolith and deal with it as it is. As much as the world is moving towards microservices, monoliths still have their use cases. At the end of the day, monoliths outperform microservices in per-request performance (by which I mean the resolution time of a single request) and they are organizationally easier to manage.

• You need separate microservices

The answer here is to actually leverage the benefits from microservices, which would inherently make it obvious that a lot of the options you considered to be viable would not be compatible with a microservice architecture (without losing the benefits of having it in the first place).

The microservice way is to define individual services for individual contexts. The underlying components that a service may or may not use, such as a persistence store, cache, or any external connections it relies on, are a private implementation detail that is hidden from sight from all the other microservices. Effectively, the microservice needs to be king of its own domain to ensure that it can change its implementation as it sees fit, without needing to first consider that other services might be making direct use of its private resource (which they simply shouldn't).

I know this answer somewhat sidesteps your concrete question, but you need to consider the bigger picture here as there are some inconsistencies on that level and answering those will inherently help answer questions you may have about the smaller picture.