Though this question specifically discusses high-rep users, I would not feel comfortable having my vote be binding. Instead, I would propose a tiered close vote system since close votes can be cast beginning with 3,000 reputation. This would also remove the tag or downvote requirement.
Reputation Close vote weight
3,000 1.0
10,000 1.5
20,000 2.5
These are completely made up and for illustrative purposes
The minimum vote "count" could still remain at 5, but a combination of these users could satisfy a close:
1 @ 3,000 (1.0)
1 @ 10,000 (1.5)
1 @ 20,000 (2.5)
-----
5.0
Or
2 @ 20,000 (5.0)
-----
5.0
Or
5 @ 3,000 (5.0)
-----
5.0