cmoncrawl.processor.pipeline.downloader#

Classes

AsyncDownloader([base_url, ...])

DownloaderDummy(files[, url, date])

Dummy downloader for testing It doesn't download anything but return files passed in the constructor and extracts metadata from the file

IDownloader()