GitHub repo leaderboard by stars, growth rate and activity.

• brendonboshell

  #1brendonboshell/supercrawler

  A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.

  JavaScript382 starslast pushed 4 years ago

#	Repo	Language	Stars	30-day trend	Last updated
1	brendonboshell brendonboshell/supercrawler A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.	JavaScript	382		last pushed 4 years ago

All · 11,658