Rethinking Solo Monster Design
Your issue with strong monsters becoming boring and predicable over the course of a long encounter is a common issue with 5th Edition D&D (and to some degree all editions). There are several reasons for it, but most boil down to 1 big monster vs 4-5 adventurers being either A. an easy victory for the party due to action economy or B. a long drawn out slog because the DM buffed HP and AC to compensate for A.
Your instincts to add external elements such as lair actions or minions is a good one. A changing battlefield is a great way to keep long encounters interesting. But I would like to direct you to what I think is the single best resource on this topic:
This masterclass of a video focusing on addressing this exact issue by changing the way we think about running single powerful monsters. A quote from the video:
The heroes pound on the monster, because there are so many heroes and only one monster, the heroes do a ton of damage and then one of two things happens. The monster dies without ever getting to do anything (the definition of anticlimax) or the monster acts and if it was tough enough to survive the entire party pounding on it then it's really nasty and when it attacks it will probably kill a character. That is not fun.
So that's the problem, a challenging solo monster or boss can't be about "toughness" because the way the game is designed, any monster tough enough to survive the heroes pounding on it, is nasty enough to kill them.
The video focuses on building better solo monsters to address this. There are a few main themes:
- High level monster / PCs are complicated to run.
- Remove the complexity by focusing on combat stats.
- Ignore skills and non-combat abilities for monsters.
- Simplify spells but just picking one or two to cast and not tracking spell slots.
- Focus on Actions.
- Actions the solo monster or boss will take on their turn
- Bonus Actions they can use to influence the battle
- Give your monsters interesting Reactions to specific triggers.
- Adding a custom version of Legendary Actions called Villain Actions.
- Keep them on theme for the monster
- A unique villian action per round of combat
- Have your monsters actions make the fight more dynamic
- Design your Villian, Legendary and Lair actions in a way that makes the fight more interest
- Promote movement, changing the environment and forcing players to think on their feet.
Overall this video is a brilliant resource for building better solo and boss monsters. I highly recommend watching the whole thing, probably more than once.
Multiple HP pools
An alternative homebrew rule that's more in line with your original question, is to use multiple hit point pools for your solo and boss monsters. This can easily be used with the action oriented monsters I mentioned above.
Instead of buffing your monster to incredibly high numbers for AC and HP in order to keep them alive. Give them 2-3 HP pools and work through each sequentially. Each pool relates to a phase of the fight and once depleted there should be a noticeable narrative and mechanical effect on the fight.
Many DMs, online and otherwise have their own take on the HP pool method, so I encourage you to do your own research on this topic. If you want to keep it simple, just provide a simple buff or change of tactics after each stage. If you want a bigger change you can completely replace their statblock with a different one. Effectively you are running separate encounters back-to-back so you can make the different phases of the monsters statblock as different or similar as you wish.