Thanks everyone for the answers. It's possible there are different shittot about free will, and I got the gist from all of the answers offered that people are not holding in the same shita as me, or I was able to still view my shita in questions that told me my shita wasn't correct (or at least, I had unanswered kashes), and I have researched an answer now that fits with my exact question, so I will answer my own question.
Firstly, the Rambam is very strong with free will, and there's no need to make an argument against this in light of the question. As Deuteronomy shared here: שמונה פרקים: פרק שמיני and wbf's answer, the Rambam is very clear that we shouldn't assume that there's no free will even when it comes to pre-destined marriages. See also Hilchot Teshuva 5:1-4
So really, this calls into question how we resolve the idea of "pre-destined" and free will. The Rambam says in Hilchot Teshuva 5:5:
God certainly knows all that will transpire...but this is known without doubt: that the deeds of man are in the hands of man, and God does not induce or coerce a person to do anything.
Raavad's gloss on this:
Rather, it is like the knowledge of a fortune-teller, who knows, by some other means, what a particular person's parth will be... The Creator knows the strength of the influence of the constellations at any given moment, and if the person will muster the strength to overcome this influence or not. This knowledge does not coerce the person's choice...
Also see Avot 3:15
Everything is foreseen, and free choice is granted
And Tosfot Yom Tov's commentary on the above:
There is no contradiction in the first place. God's knowledge of the future is the result of His observing the deed that the person is doing... One cannot argue that because God knows the future actions of man He therefore compels them, because for God there is no "before" and "after"...So just like our knowledge of the present has no compelling effect, so, too, God's knowledge is always in the present and this is non-compelling.
So, what we conclude from all this is that you can have "meant to be", meaning, Hashem decided since before Bereshit who you will marry (and it will really be your actual other half of your soul), or if you will even have a zivug. The way He works this out, is by "looking" into "the future" to see what your choices are, and takes all of that into account in His Eternal Plan. This is what it means for us to be partner's in creation, and why this is all so infinitely serious. Hashem has, so to speak, put His Plan into our hands. When we keep mitzvot, then that's great, but when we sin, we are, so to speak, messing things up.
If someone chooses to not get married (or to have mamzerim, or to marry someone forbidden to them), this is, simultaneously his own choice and responsibility, and meant-to-be. Hashem's whole Plan took this into account since before creation, but was, so to speak, based around this bad choice (and therefore - before Rav Dessler's 'levels of free will' and Berdichever 'Hashem don's lechaf zechus' ideas is entirely our fault, not His i.e. we cannot say He pre-destined a sin). Hashem only knows what lengths He goes to, to "clean up" after us. Surely, it is considered difficult like קריעת ים סוף (see Sota 2a, thanks conceptualinertia for your answer).
Of course, there are still seeming paradoxes here, and things we can't possibly grasp. As the Rambam says:
Know that the answer to this question, "longer than the land is its measure, and broader than the sea"... "My thoughts are not as your thoughts, nor are your ways as My ways"... (Yishayahu 55:8) therefore we lack the capacity to know the nature of God's knowledge of all creations and all actions.
This answer is a very unified answer. It holds by zivug, it holds by hard free will, it respects the point that it's impossible to truly grasp Hashem and His ways, it fits well in all my Torah knowledge and I am happy with this answer. Feel free to comment your thoughts and kashes!