If both the modem and the Raspberry Pi are on the same logical network (e.g. no NAT or routing is involved) then your Raspberry Pi cannot act as a gateway, because it is inside the only network available, and the actual gateway is your modem. This is the case if your modem has an address (say 192.168.1.1/24
) and your RPi/computers an address in the same network (say 192.168.1.100
- 101
and so on). The computers can directly see the modem, so they don't need to use the Raspberry Pi as a gateway, but only as the DNS/DHCP server.
But you could have set up more IP addresses on the Raspberry Pi; maybe on two different networks (Say 10.0.0.0/8
for the internal machines and 192.168.1.0/24
for the modem) and the modem may be in the separate network (192.168.1.1
, a typical address for a modem). In this case you would set the Raspberry Pi as the gateway for your computers in the 10.0.0.0/8
network, beyond it being the DNS/DHCP server too.
To answer your actual question, you should set the modem as the gateway in the first case, and the Raspberry Pi in the second: you will avoid unnecessary packet forwarding if you are in the first case.
To elaborate on your second question, the DNS server and the gateway don't need to be on the same machine, so you should not confuse them: it is also true that many modems act as both.