GitHub repo leaderboard by stars, growth rate and activity.

• PuerkitoBio

  #1PuerkitoBio/gocrawl

  Polite, slim and concurrent web crawler.

  Go2,051 starslast pushed 5 years ago
• eliasdabbas

  #2eliasdabbas/advertools

  advertools - online marketing productivity and analysis tools

  Python1,452 starslast pushed 2 months ago
• PuerkitoBio

  #3PuerkitoBio/fetchbot

  A simple and flexible web crawler that follows the robots.txt policies and crawl delays.

  Go791 starslast pushed 5 years ago
• melbahja

  #4melbahja/seo

  🐘 Simple PHP library to help developers 🍻 do better on-page SEO optimization 🤖

  PHP361 starslast pushed 4 months ago
• beb7

  #5beb7/gflare-tk

  Open-Source Python Based SEO Web Crawler

  Python198 starslast pushed 3 years ago
• aza-ali

  #6aza-ali/webdownloader

  Terminal CLI to download complete web pages (HTML + assets) for offline viewing. Markdown export, robots.txt aware.

  JavaScript178 starslast pushed 4 months ago
• seantomburke

  #7seantomburke/sitemapper

  Parse through any sitemap in Node.js

  TypeScript137 starslast pushed 1 week ago
• alextim

  #8alextim/astro-lib

  Makes it easy to add robots.txt, sitemap and web app manifest during build to your Astro app.

  TypeScript134 starslast pushed 3 years ago
• rubenmarcus

  #9rubenmarcus/aeo.js

  Answer Engine Optimization for the modern web. Make your site discoverable by ChatGPT, Claude, Perplexity & AI search engines. Generates llms.txt, robots.txt, sitemap, JSON-LD & more.

  TypeScript134 starslast pushed Yesterday
• OranAi-Ltd

  #10OranAi-Ltd/orangeo-ai-visibility-skill

  Open-source GEO / AEO skill for AI visibility readiness: check robots.txt, llms.txt, schema, citation signals, competitor gaps, and buyer prompts

  Python133 starslast pushed 3 months ago

#	Repo	Language	Stars	30-day trend	Last updated
1	PuerkitoBio PuerkitoBio/gocrawl Polite, slim and concurrent web crawler.	Go	2,051		last pushed 5 years ago
2	eliasdabbas eliasdabbas/advertools advertools - online marketing productivity and analysis tools	Python	1,452		last pushed 2 months ago
3	PuerkitoBio PuerkitoBio/fetchbot A simple and flexible web crawler that follows the robots.txt policies and crawl delays.	Go	791		last pushed 5 years ago
4	melbahja melbahja/seo 🐘 Simple PHP library to help developers 🍻 do better on-page SEO optimization 🤖	PHP	361		last pushed 4 months ago
5	beb7 beb7/gflare-tk Open-Source Python Based SEO Web Crawler	Python	198		last pushed 3 years ago
6	aza-ali aza-ali/webdownloader Terminal CLI to download complete web pages (HTML + assets) for offline viewing. Markdown export, robots.txt aware.	JavaScript	178		last pushed 4 months ago
7	seantomburke seantomburke/sitemapper Parse through any sitemap in Node.js	TypeScript	137		last pushed 1 week ago
8	alextim alextim/astro-lib Makes it easy to add robots.txt, sitemap and web app manifest during build to your Astro app.	TypeScript	134		last pushed 3 years ago
9	rubenmarcus rubenmarcus/aeo.js Answer Engine Optimization for the modern web. Make your site discoverable by ChatGPT, Claude, Perplexity & AI search engines. Generates llms.txt, robots.txt, sitemap, JSON-LD & more.	TypeScript	134		last pushed Yesterday
10	OranAi-Ltd OranAi-Ltd/orangeo-ai-visibility-skill Open-source GEO / AEO skill for AI visibility readiness: check robots.txt, llms.txt, schema, citation signals, competitor gaps, and buyer prompts	Python	133		last pushed 3 months ago

All · 11,474