This cannot be achieved unless both computers are in the same subnet.
If they are, then you'd first configure server A as the "default gateway" for B – and similarly the real gateway needs the opposite route towards B via A:
- on server B:
to 0.0.0.0/0 via <A's IP>
- on real gateway:
to <B's IP>/32 via <A's IP>
But if they're on different networks at all (as you said "hosted at a different ISP"), the only way you could use A as the gateway would be to set up a virtual layer-2 network – which is, yes, a VPN.
(This excludes IP "source routing", which has been removed with fire from most modern network stacks due to security issues.)
If you're concerned about privacy (traffic "leaks"), configure server B such that the only default route it has is via the VPN – if it only talks with server A, then it only needs a specific "host route" (a /32 or a /128) towards A. For example:
to ::/0 via <A's internal IP>
to <A's external IP>/128 via <real default gateway>