0

Google has failed and the closest result I found pointed me here so, I will ask my question here.

I'm trying to turn a home computer into a multiple proxy machine. I saw someone manage to do this and run multiple crawlers through a home connection on different ports.

So I googled how to set up a home proxy but mainly what came up was CCProxy, FoxyProxy and WAMP with PHProxy. None of those are really what I want, this isn't browser related however I will be sending requests through http/https.

I found this link IBM Knowledge Center but I don't think it's partaning to what I am looking for. What I really want to do is find a way to make all the bots run from my IP but on different ports. I'm on a wireless home router and running windows 10.

I've considered installing Ubuntu onto another machine and squid as this tutorial Ubuntu/Squid Tutorial but I don't want to dive so deep without knowing it will give me the results I'm looking for.

Basically everything that I'm turning up is pointing me to solutions that will turn my computer into an active proxy for other computers on my network which is not what I'm trying to do. I want to have one computer with 25 different ports open to connect the bots through. Is this possible on a windows or do I have to go into the Ubuntu method explained using squid?

I only need HTTP/HTTPS I don't need FTP/TCP/SOCK5 and I saw mentioned SSH tunneling? Not sure if that would serve my purpose with the bots.

2
  • You should check your ISP terms of service as doing this may breach your contract. You should also be aware that if your bot crawler misbehaves it will result in your IP address being blocked. Consider renting a server with a professional hosting service.
    – Burgi
    Commented Aug 16, 2016 at 7:53
  • I am completely aware of the terms of service provided by my ISP. I do appreciate the insight because you are right and it is definitely important to check this first. I know this could result in an IP blacklist or legal issues if not done correctly.. I am not doing anything that will do either however. Thank you though, it is appreciated.
    – Axeon
    Commented Aug 16, 2016 at 16:12

1 Answer 1

0

You don't need a proxy or any special setups to have 25 different ports in use by a program. You will have to make sure that the ports you are wanting to use are allowed in your firewall on host computer, if any, through. And if you have a router, possibly allow it there as well. Also, you can assign more than 1 ip address on your host commputer, if you have a router as well.

However, you do need a proxy if you are trying to have your bots use a different WAN IP then what your ISP gives you. But the proxy needs to be on a different connection other than what you connect through which means you would need to specify a VPN or Tunnel for the connections for that to work.

The program that you are using either needs to support the use of multiple ports, or you could simply try launching multiple instances of the program and then assign them all different port to use, neither of these you can do, then you will have to probably look into setting up more phyiscal machines or Virtual machines. In either case they would all still use your same ip unless again you have vpn or tunneling set up. As for your business line, this would mean you have 2 WAN Connections, so its just a matter of telling your bot or a program to use the second connection instead and that would give your program or bot a different ip its using and don't have to vpn anywhere.

If you had a server somewhere else, like a friends house, and wanted to use your servers IP instead, you would set up a vpn or tunnel connection from your host computer to that server then programs can use that connection instead like a browser and every site you go to, would think the ip of your server, is your ip address.

If for some reason you need to use the same port for all the bots, its just a matter of using a virtual ip address, as only 1 port can be utilized by 1 program at a time. Each ip address can support 65535 ports, and a hundred or so of those ports are reserved for your computers use but its internal programs. So a program can utilize more than 1 port on an IP address given its not trying to use a duplicate port already in use either by itself or another program. So you can easilly use 25 separate bots on 25 separate ports.

7
  • EDIT: Sorry I should've also specified these will be 25 different bots connecting to 25 different ports not one bot using all 25 ports. If that matters. Thank you for your answer. So what you are saying is basically if I forward those ports on my PC/Router I can just assign 127.0.0.1:XX and 127.0.0.1:XZ etc as the bots connection IP and PORT and it will allow them to connect through those ports and crawl? Also say I have a business line which runs separately and a dedicated machine on that line.. Could I run that as the VPN like you suggested?
    – Axeon
    Commented Aug 14, 2016 at 4:21
  • @Axeon you are correct on the router part, edited my response to hopefully answer your additional questions
    – Frostalf
    Commented Aug 14, 2016 at 4:37
  • Dude I think you answered everything I need to know. I'm going to test it out right now using the forwarding and then telling each bot to use that port and see how that goes firstly before I try the more complicated approach of assigning them access through the other Internets connect. Again thank you very much.
    – Axeon
    Commented Aug 14, 2016 at 4:48
  • @Axeon Do note 127.0.0.1 is a loopback address meaning its not for use to connect to the internet, so hopefully you only used it as an example in your comment above.
    – Frostalf
    Commented Aug 14, 2016 at 4:52
  • I also appreciate that. Yes I did use it specifically as an example I guess 0.0.0.0 would've worked as well. Again thank you.
    – Axeon
    Commented Aug 14, 2016 at 4:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .