WebXCrawler

• WebXCrawler is a fast static crawler to crawl a website and get all the links. • It has options for maximum depth of crawling and the number of threads to optimize performance. • It can also be used in Docker Container. • It also have the Poetry (better than PIP) installation part.