Before I purchased the wireless router, the internet cable from the ISP directly connected to my PC and I was able to see and access other users computers when I entered "Network" from "My Computer".
This is a bad thing for a lot of reasons:
First, very likely any of these users can do the same to you unless you've locked file sharing down on your own PC.
It sounds like the ISP's equipment is allowing broadcast traffic to propagate from users to within its network. This consumes bandwidth that could be used to transit Internet traffic instead.
Furthermore, if they are not filtering DHCP ports, a misconfigured DHCP server could wreak havoc among all visible users on that subnet.
ARP spoofing attacks are possible which could allow a machine to impersonate another server or site you are trying to get to.
Also note that unless you've secured permission from these people to access their systems, you are probably still committing a crime (IANAL though) even if it's easily accessible. I would recommend you not do this.
So keep your router, do not enable an external web interface to it, and enable all its firewall functions, to protect yourself.
Assuming you have permission to access another system on this ISP that you can see, and that you know for a fact it's not against your ISPs ToS, what I would do is get a 10/100Mbit hub, connect your ISP ethernet cable to it, and connect your router's WAN port to it. You can then connect another system to another port in the hub when you want to access other computers on the ISP, but systems behind your router will be protected from this mess. Don't do something silly like connect this back to a free LAN port on your router. Keep it physically separate.