GitHub repo leaderboard by stars, growth rate and activity.
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台,支持任何语言和框架
蓝天采集器是一款开源免费的爬虫系统,仅需点选编辑规则即可采集数据,可运行在本地、虚拟主机或云服务器中,几乎能采集所有类型的网页,无缝对接各类CMS建站程序,免登录实时发布数据,全自动无需人工干预!是网页大数据采集软件中完全跨平台的云端爬虫系统
SpiderSuite (web security crawler) releases, wiki and roadmap
Uscrapper Vanta: Dive deeper into the web with this powerful open-source tool. Extract valuable insights with ease and efficiency, from both surface and deep web sources. Empower your data mining and analysis with Vanta's advanced capabilities. Fast, reliable, and user-friendly, Uscrapper Vanta is the ultimate choice for researchers and analysts.
Open-source Enterprise Grade Search Engine Software
| # | Repo | Language | Stars | 30-day trend | Last updated |
|---|---|---|---|---|---|
| 1 | Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台,支持任何语言和框架 | Go | 12,270 | last pushed 7 months ago | |
| 2 | 新一代爬虫平台,以图形化方式定义爬虫流程,不写代码即可完成爬虫。 | Java | 11,351 | last pushed 3 years ago | |
| 3 | 蓝天采集器是一款开源免费的爬虫系统,仅需点选编辑规则即可采集数据,可运行在本地、虚拟主机或云服务器中,几乎能采集所有类型的网页,无缝对接各类CMS建站程序,免登录实时发布数据,全自动无需人工干预!是网页大数据采集软件中完全跨平台的云端爬虫系统 | PHP | 2,090 | last pushed 1 week ago | |
| 4 | SpiderSuite (web security crawler) releases, wiki and roadmap | — | 976 | last pushed 4 weeks ago | |
| 5 | HTTP API for Scrapy spiders | Python | 884 | last pushed 2 months ago | |
| 6 | Uscrapper Vanta: Dive deeper into the web with this powerful open-source tool. Extract valuable insights with ease and efficiency, from both surface and deep web sources. Empower your data mining and analysis with Vanta's advanced capabilities. Fast, reliable, and user-friendly, Uscrapper Vanta is the ultimate choice for researchers and analysts. | Python | 796 | last pushed 2 years ago | |
| 7 | Open-source Enterprise Grade Search Engine Software | Java | 517 | last pushed 4 years ago | |
| 8 | An R web crawler and scraper | R | 362 | last pushed 4 years ago |
All · 11,474