Is an enterprise-wide data model truly feasible?

Question

I've consulted for a number of Fortune 500 and 100 companies all of whom have made some kind of effort at creating a "Master" data model even to the point that they'd attempt to have all applications work with a shared database (Imagine the change management issues in that world).

I've seen companies throw millions of dollars at these efforts and still come up short of what they envisioned (although there were in many cases some benefits realized from the effort).

My question here is has anyone seen a truly enterprise-wide master data model implemented and consistently used for anything larger than 100 or so tables?

My instinct leans more toward a message driven/SOA model where individual components were responsible for a subset of data and any component that needs access to that data would go through the appropriate channel. But even then, the "ideal" and reality tend to have a significant difference.

Answer 1

One hundred database tables is not a whole lot, once you consider ordinary data tables, lookup tables, many-to-many linking tables, OLAP, and so forth.

Is an enterprise-wide data model truly feasible?

It better be. How else are you going to maintain and extend it?

Rather than think of an enterprise data model as a "big ball of mud," it's more useful to think of it in terms of subsystems or departments. Every data processing system larger than a single application has multiple applications and multiple data stores that talk to each other through well-defined interfaces. You can readily capture such high-level architecture with a block diagram or data flow diagram, and have database schemas for each data store at the ready when you need to perform systems analysis.

Answer 2

My instinct leans more toward a message driven/SOA model where individual components were responsible for a subset of data and any component that needs access to that data would go through the appropriate channel.

We actually have something just like this where I work. Our company offers financial services, so the overwhelming majority of data we want to show customers boils down to "for security X, what is the value of field Y?" The security might be Google stock or US Treasury bonds, and the field might be today's price or the 52-week high or the short interest ratio or what have you.

Most of the time requesting a certain field through the company-wide standard interface will result in a call to some service that talks to some database. Each team of backend programmers owns a subset of these fields, along with the services and databases they use to implement them. Any one of those teams can completely change their implementation without affecting the other fields, and the frontend guys like me don't even notice the change. In fact, part of my new hire training was reimplementing a small group of relatively simple fields. I've never counted, but I'm fairly sure we have several hundreds if not thousands of database tables for all the data we support.

But we do have some data that simply doesn't fit this model, and it does get stored and accessed differently. It's probably impossible for a single model of anything to apply to 100% of a company's needs, but you can definitely get close to that ideal.

Answer 3

My question here is has anyone seen a truly enterprise-wide master data model implemented and consistently used for anything larger than 100 or so tables?

Kind of.

I've seen one implemented that defined the core list of products and configurations for the (~5k employee) company. It was used for sales, maintenance, engineering. The actual implementation was 10+ years old when I encountered it, and it was built from something even older (going back to non-electronic storage, from what I understood). And it consistently served as the source of truth for all product and client data across the company.

Is an enterprise-wide data model truly feasible?

"But".

The reason the thing was 100+ tables is because it grew with the company. It was an unmaintainable, fragile, unweildy mess. It had everything from results of physical tolerance tests to pictures of the products stored right in the DB. They ended up needing export processes anyways to seed smaller databases for specific needs.

So I mean... I wouldn't consider that feasible over the long term.