GitHub repo leaderboard by stars, growth rate and activity.

• firecrawl

  #1firecrawl/firecrawl

  The context API to search, scrape, and interact with the web at scale. 🔥

  TypeScript178,457 starslast pushed Yesterday
• D4Vinci

  #2D4Vinci/Scrapling

  🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

  Python79,739 starslast pushed 6 days ago
• scrapy

  #3scrapy/scrapy

  Scrapy, a fast high-level web crawling & scraping framework for Python.

  Python64,268 starslast pushed 2 days ago
• NaiboWang

  #4NaiboWang/EasySpider

  A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。

  JavaScript44,521 starslast pushed Yesterday
• iawia002

  #5iawia002/lux

  👾 Fast and simple video download library and CLI tool written in Go

  Go31,670 starslast pushed 6 months ago
• ScrapeGraphAI

  #6ScrapeGraphAI/Scrapegraph-ai

  Python scraper based on AI

  Python30,783 starslast pushed 3 days ago
• apify

  #7apify/crawlee

  Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

  TypeScript25,713 starslast pushed Yesterday
• gocolly

  #8gocolly/colly

  Elegant Scraper and Crawler Framework for Golang

  Go25,504 starslast pushed 1 week ago
• jhao104

  #9jhao104/proxy_pool

  Python ProxyPool for web spider

  Python23,686 starslast pushed 3 months ago
• Evil0ctal

  #10Evil0ctal/Douyin_TikTok_Download_API

  🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、TikTok、Bilibili数据爬取工具，支持API调用，在线批量解析及下载。

  Python20,035 starslast pushed 11 months ago

#	Repo	Language	Stars	30-day trend	Last updated
1	firecrawl firecrawl/firecrawl The context API to search, scrape, and interact with the web at scale. 🔥	TypeScript	178,457		last pushed Yesterday
2	D4Vinci D4Vinci/Scrapling 🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!	Python	79,739		last pushed 6 days ago
3	scrapy scrapy/scrapy Scrapy, a fast high-level web crawling & scraping framework for Python.	Python	64,268		last pushed 2 days ago
4	NaiboWang NaiboWang/EasySpider A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。	JavaScript	44,521		last pushed Yesterday
5	iawia002 iawia002/lux 👾 Fast and simple video download library and CLI tool written in Go	Go	31,670		last pushed 6 months ago
6	ScrapeGraphAI ScrapeGraphAI/Scrapegraph-ai Python scraper based on AI	Python	30,783		last pushed 3 days ago
7	apify apify/crawlee Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.	TypeScript	25,713		last pushed Yesterday
8	gocolly gocolly/colly Elegant Scraper and Crawler Framework for Golang	Go	25,504		last pushed 1 week ago
9	jhao104 jhao104/proxy_pool Python ProxyPool for web spider	Python	23,686		last pushed 3 months ago
10	Evil0ctal Evil0ctal/Douyin_TikTok_Download_API 🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、TikTok、Bilibili数据爬取工具，支持API调用，在线批量解析及下载。	Python	20,035		last pushed 11 months ago

All · 11,474