Packages from main get five years of support/security-fixes; packages from the community repository (universe) meaning from flavor teams or upstream Debian come only with three years of guaranteed support/security-fixes; which ended at 2023-April.
You can use ubuntu-security-status
(or ubuntu-support-status
for older releases) to view what packages are still getting security fixes and guaranteed support and which are not (ESM/Ubuntu Pro can also extend security for LTS releases by an additional five years too), but there is no guarantee of support for universe packages, as any MOTU or dev with upload rights can still fix packages until 2025-April anyway (i.e. 5 years is technically still there; which is why I use the guarantee wording)
FYI: As far as I'm concerned I consider a Xubuntu 20.04 LTS system as a Ubuntu 20.04 LTS system where the Xfce desktop is being used.. and I actually have a 20.04 system here still (Xubuntu + Lubuntu installed), but both Xubuntu, Lubuntu and other flavors have ended support for it (so please don't ask for it). Your base Ubuntu system still gets upgrades, you've just lost guarantee for the flavor packages for security fixes, etc.
You can happily remove all packages from universe if you wish; as the Ubuntu Desktop install includes NONE of them anyway, however that tends to be rather limiting as many users need more than what is provided on the installation media (packages on Ubuntu Desktop install media are those from main only!) but want additional community packages.
You can install one Ubuntu system, and via package changes, convert it to another Ubuntu system or flavor desktop in any combination, though some settings may be different even after package changes to a install of final product.
i.e.
Yes, you can convert your Xubuntu 20.04 LTS system into an Ubuntu 20.04 LTS Desktop system (with five years of support), but personally I think that's a lot of effort that may not gain you much; as to really improve security can you live without universe packages? Adding Ubuntu Pro support may be a more security conscious option anyway.
ubuntu-security-status
(orubuntu-support-status
for older release) to view what packages are still getting security fixes and guaranteed support and which are not (ESM/Pro can also extend security too), but its more there is no guarantee of support for universe, as any MOTU or dev with upload rights can still fix packages until 2025-April anyway.