8

I use a ThreadPoolExecutor to quickly check a list of proxies to see which ones are dead or alive.

with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
    futures = []
    for proxy in proxies:
        future = executor.submit(is_proxy_alive, proxy)
        futures.append(future)
    
    for future in futures:
        print(future.result()) # prints true or false depending on if proxy is alive.
                               # how do I get the specific proxy I passed in the arguments 
                               # so that I can make a dictionary here?

My goal is to get the argument (the proxy) I passed to the executor when iterating through the results to know which exact proxies are dead or alive, so I could make a dictionary that might look like this:

{"IP1": False, "IP2": True, "IP3": True}

One method I can think of is returning the proxy I sent on top of returning true/false, but is there a better way to do it externally so the function doesn't have to return more than just a bool?

1
  • 2
    for index, future in enumerate(futures) and proxies[index]? Or, submit a function which returns whether the proxy is alive and the proxy. Up to you. Commented Aug 11, 2020 at 15:59

2 Answers 2

18

While submitting the task, you could create a mapping from future to its proxy.

with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
    future_proxy_mapping = {} 
    futures = []
    for proxy in proxies:
        future = executor.submit(is_proxy_alive, proxy)
        future_proxy_mapping[future] = proxy
        futures.append(future)
    
    for future in futures:
        proxy = future_proxy_mapping[future]
        print(proxy)
        print(future.result())
1
  • 3
    Thanks, if only the futures object were able to store this mapping off in the first place so we could just call it
    – Jennings
    Commented Apr 29, 2022 at 20:12
0

You have appended to futures in the same order that you read from proxies, so they are in the same order, so you can just zip them up.

with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
    futures = []
    for proxy in proxies:
        future = executor.submit(is_proxy_alive, proxy)
        futures.append(future)
    
    for future, proxy in zip(futures, proxies):
        # do whatever

Not the answer you're looking for? Browse other questions tagged or ask your own question.