How much do you trust your players?
You can make fundamentally broken characters using WotC rules with no 3rd party content. Doing so with 3rd party content is also possible.
If you trust your players to have both the self control and system mastery to tell that the rules they use to make their character have problems, then there really isn't an issue with using 3rd party rules for characters.
You should make it clear that any and all rules a player uses to make a character are up for discussion and modification later.
If someone makes a nuclear wizard based off of the "magic missile is one damage roll", or creates an infinite simulacrum chain, or a bunch of other frankly crazy builds, you'd have to have that conversation in a pure-WotC game.
An issue that can happen is that people who like playing around with rules are more likely to be the ones wanting to use a wider selection of rules to play around with. And if they lack self control or have a different perspective than you do, they could pick rules that cause problems at your table - be they annoying to adjudicate, overshadowing other players at the table, or out of the scope of what you want the PCs to be doing.
To prepare for such a thing:
Have practice with homebrewing rules yourself. Not just academic knowledge, but see what happens when your rules are used.
Get everyone on board with rulings not rules of 5e. Make it clear that you will be making on-the-fly rulings that could override the rules they used to make their character.
Talk to each player about how they view their character. Not just the mechanics, but what do they see their character as being good at, weak at, etc. Because matching player expectations to character capabilities (and calibrating those expectations!) is more important than the exact rules used to express character capabilities.
Do this with players you can trust. Let them know that you are relying on that trust; this can't work as an adversarial game where they try to "sneak" a rule into the game past the DM.
Communicate to the players that the purpose of the open homebrew rules should be to take a character idea and express it mechanically. You can be inspired by mechanics to make a character idea.
If you aren't sufficiently confident with letting your players do all of this, you can instead approve things on a case by case basis. Make it clear your approval is conditional! And that players (if they plan on some combo down the line) should describe that as well to avoid possible disappointment.
Most tables I've been at allow people to propose homebrew content to be used by their character. At a number of tables I've had DMs actually ask players to try out some specific homebrew content (not even theirs) as it fit the setting/story they are doing.
D&D it not a finely tuned machine. D&D for most of its existence has had groups playing with widely divergent sets of rules and having fun; often the different rules where because they misunderstood the rules in insanely different ways! And despite that, the game can work.
The social contract is far more important than the actual specific mechanics.