scrapinghub/aduana
Frontera backend to guide a crawl using PageRank, HITS or other ranking algorithms based on the link structure of the web graph, even when making big crawls (one billion pages).
Stars: 55Language: C
Give AlbumentationsX a star on GitHub — it powers this leaderboard
Star on GitHubFrontera backend to guide a crawl using PageRank, HITS or other ranking algorithms based on the link structure of the web graph, even when making big crawls (one billion pages).