you would have to maintain your own portage tree which has ebuilds that would work with the libraries that ship with fedora.
you might also be able to make the ebuilds download the source rpm's and then compile the packages and install them, but you proably woldnt want to start applying the gentoo patches to the source since that is what makes gentoo, gentoo, in the first place.
this has been done before with other kernels, and in those cases, they still built everything up around portage, so in the same way, you would have to write ebuilds for everything that is currently installed.