Skip to main content

All Questions

0 votes
2 answers
1k views

Web scraping with python + selenium + multiprocessing in a docker container becomes slower over time

I have built a web crawling solution with python, selenium and multiprocessing which is deployed in a docker container in an EC2 instance (m4.2xlarge type). Whenever I run it with a large input, it ...
Saurabh Saxena's user avatar