User Agent injector
This plugin automatically adds User-Agent
header if it’s not present.
It uses fake-useragent in order to generate fake real world user agents.
Configuration of the plugin is defined in UserAgentInjecterPluginConfig
.
How to configure plugin for the SneakpeekServer
(will be used globally for all requests):
from sneakpeek.plugins.user_agent_injecter_plugin import UserAgentInjecterPlugin, UserAgentInjecterPluginConfig
server = SneakpeekServer.create(
...
plugins=[
UserAgentInjecterPlugin(
UserAgentInjecterPluginConfig(
use_external_data = True,
browsers = ["chrome", "firefox"],
)
)
],
)
How to override plugin settings for a given scraper:
from sneakpeek.plugins.user_agent_injecter_plugin import UserAgentInjecterPluginConfig
scraper = Scraper(
...
config=ScraperConfig(
...
plugins={
"user_agent_injecter": UserAgentInjecterPluginConfig(
use_external_data = False,
browsers = ["chrome", "firefox"],
)
}
),
)