The problem:
Client was paying multiple employees to manually find public information on specific websites. This resulted in great costs and delayed inaccurate data.
What we did:
We created a custom scraping application with the use of Selenium. Many scrapers need to run simultaneously. For this reason, several instances of the scraping app are run in parallel. Each scraper runs in its own virtual container (Docker). All virtual containers run on cloud and scrape data 24/7 ensuring timely and accurate data. Containers are lightweight and therefore easy on resources, which decreased cost for the client.
