This web crawler would have to allow the user to specify the URLs to be crawled, the depth of recursive crawling, any filters that should be applied (e.g., use of a particular referrer, or limitation ...

Playwright, a Python library, allows developers to essentially control the browser with a few lines of Python code to automate cross-browser testing for single-page web apps and Progressive Web Apps.