Design the crawl service behind a search engine: a fleet of fetchers leases URLs from a frontier, fetches each page, and reports the result back for parsing and bookkeeping.
Newly discovered links feed the frontier in turn, raw page bodies are archived for the indexing pipeline, and a seen-set records every URL the crawler has ever visited.