GitHub repo leaderboard by stars, growth rate and activity.
🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
Run a high-fidelity browser-based web archiving crawler in a single Docker container
News crawling with StormCrawler - stores content as WARC
Bitextor generates translation memories from multilingual websites
| # | Repo | Language | Stars | 30-day trend | Last updated |
|---|---|---|---|---|---|
| 1 | 🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more... | Python | 28,267 | last pushed 4 days ago | |
| 2 | The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns | Python | 1,611 | last pushed 1 year ago | |
| 3 | Run a high-fidelity browser-based web archiving crawler in a single Docker container | TypeScript | 1,132 | last pushed 1 hour ago | |
| 4 | News crawling with StormCrawler - stores content as WARC | Java | 375 | last pushed 2 days ago | |
| 5 | Bitextor generates translation memories from multilingual websites | Python | 300 | last pushed 2 years ago |
All · 11,658