[SOLVED] Iso Web Crawling Bot

[johnny_chronic]

I'm looking for a webcrawling bot that will download mass/bulk data from both google and youtube (and perhaps other sites).

I'd even like to use macros with it so I do not have to work on my computer and can still download high quality content.

Does anyone know where to find or how to build one of these?

Thank you!

 

[Ralston18]

FYI:

https://www.blog.datahut.co/post/how-to-build-a-web-crawler-from-scratch

https://www.geeksforgeeks.org/how-to-build-web-scraping-bot-in-python/

You can easily find other similar links.

Then write down the specific requirements needed and go from there.

Start simple and get each requirement working one at a time. Then add complexity, macros, etc. as necessary.

Do what you can then, if there are specific problems, post your work and questions. Include what you believe to be the solution.

Cite sources and references as well.

 

[ex_bubblehead]

Keep in mind that most of the big sites have anti bot measures in place to prevent bulk scavenging.