Changelogs seem to indicate VirtualBox 7.x is using the Apple Hypervisor frameworks, which does support this . Functionality is considered a developer preview and is unsupported at present, so I wouldn't suggest using it. Early versions had networking issues, but this appears to have been remediated (again according to changelogs). I've seen reports of more recent macOS versions not working in VirtualBox VMs too.
UTM, as mentioned in @tetsujin's comment above, is a user friendly GUI wrapper for QEMU. The comments on UTM's site seem to indicate it uses both the Apple frameworks and QEMU. Details are lacking though. It doesn't appear to have any APIs though according to various posts I've seen, which would make things a bit more difficult with respect to vagrant driver development should you choose to take that course. There is a command line interface though. The vagrant virtual box driver does make some use of CLI calls, so there are examples for that in the driver code should you need them.
As you indicated, you can use QEMU directly. Vagrant happens to have an official driver via libvirt, which interfaces with QEMU (this is also a developer preview, but given that it's from Hashicorp you may get more milage out of it). As you likely know there are libvirt and QEMU packages available for macOS via HomeBrew. This may, however, be less work than than community driver for a QEMU you mentioned.
There is also xhyve, which is a macOS port of BSD bhyve hypervisor that uses the Apple frameworks. There is an old vagrant plugin for xhyve, which utilizes the Apple frameworks as well. It's much more lightweight hypervisor than QEMU and should be more performant. But hasn't had a release since 2016 per Rubygems.org. You might have to do some work on this to bring it up to snuff for the latest vagrant releases. There is a vagrant plugin that leverages bhyve towards this end which you might be able to crib from if the default xyhve driver doesn't work well enough.
The TLDR here is that there are projects that might do what you want, but they're mostly in developer preview mode. As with anything YMMV with each of the above options. If you have the stomach to work on vagrant drivers, you might be able to get a stable version of what you need.
Hope this helps.
Disclosure: none of the above reflects the position or support of my employer.