cmoncrawl.common.types.PipeMetadata#

class cmoncrawl.common.types.PipeMetadata(domain_record: ~cmoncrawl.common.types.DomainRecord, article_data: ~typing.Dict[~typing.Any, ~typing.Any] = <factory>, warc_header: ~typing.Dict[str, ~typing.Any] = <factory>, http_header: ~typing.Dict[str, ~typing.Any] = <factory>, encoding: str = 'latin-1', name: ~typing.Optional[str] = None)#

Metadata for a pipe.

__init__(domain_record: ~cmoncrawl.common.types.DomainRecord, article_data: ~typing.Dict[~typing.Any, ~typing.Any] = <factory>, warc_header: ~typing.Dict[str, ~typing.Any] = <factory>, http_header: ~typing.Dict[str, ~typing.Any] = <factory>, encoding: str = 'latin-1', name: ~typing.Optional[str] = None) None#

Methods

__init__(domain_record[, article_data, ...])

Attributes

encoding

name

domain_record

article_data

warc_header

http_header