Most routers (of the type you describe) can be configured to respond to Ethernet. Often, this is configurable using HTTP(S). The first trick, though, is to get the IP address assigned.
Plug the computer into a LAN port (not the WAN port) of the router.
Check the IP address of the computer. If the router has automatic addressing (e.g., DHCP/IPv4) disabled, you may need to manually assign the address. (Approximate way to do this, quoting from my memory: Start, Control Panel, Network Connections/Adapters, choose your adapter, Properties, TCP/IP configuration.)
You should assign an IP address that is similar, but not identical, to the router. For instance, if the router uses an address of 192.168.0.1, then use 192.168.0.5
One tricky part to this is to know the IP address that the router is using. Your devices with Wi-Fi adapters might be helpful, although the information could be misleading if the Wi-Fi communications are using a different subnet. (So the information could be useful, but might not be.) You may want to go online (somehow) and look up the default IP address, username, and password for the particular router that you're using. (Sometimes useful information is also provided on a sticker, perhaps on the bottom of the router.)
I think this answer provides you with a good general direction to head. You might need more help, but if you do, we probably will want some more specific information (like the router's model, and whatever network settings you do know) to efficiently help provide more specific answers.
ipconfig /all
and add the output to your question. Do this on your XP computer AND your laptop which is successfully connected to your Xfinity modem.