aws_ddk_core.stages.GlueTransformStage¶
- class aws_ddk_core.stages.GlueTransformStage(*args: Any, **kwargs)¶
Class that represents a Glue Transform DDK DataStage.
- __init__(scope: constructs.Construct, id: str, environment_id: str, job_name: Optional[str] = None, executable: Optional[aws_cdk.aws_glue_alpha.JobExecutable] = None, job_role: Optional[aws_cdk.aws_iam.IRole] = None, crawler_name: Optional[str] = None, database_name: Optional[str] = None, crawler_role: Optional[aws_cdk.aws_iam.IRole] = None, targets: Optional[aws_cdk.aws_glue.CfnCrawler.TargetsProperty] = None, job_args: Optional[Dict[str, Any]] = None, state_machine_input: Optional[Dict[str, Any]] = None, additional_role_policy_statements: Optional[List[aws_cdk.aws_iam.PolicyStatement]] = None, state_machine_failed_executions_alarm_threshold: Optional[int] = 1, state_machine_failed_executions_alarm_evaluation_periods: Optional[int] = 1) None ¶
DDK Glue Transform stage.
Stage that contains a step function that runs Glue job, and a Glue crawler afterwards. If the Glue job or crawler names are not supplied, then they are created.
- Parameters
scope (Construct) – Scope within which this construct is defined
id (str) – Identifier of the stage
environment_id (str) – Identifier of the environment
job_name (Optional[str]) – The name of a preexisting Glue job to run. If None, a Glue job is created
executable (Optional[JobExecutable]) – The job executable properties
job_role (Optional[IRole]) – The job execution role
crawler_name (Optional[str]) – The name of a preexisting Glue crawler to run. If None, a Glue crawler is created
database_name (Optional[str]) – The name of the database in which the crawler’s output is stored
crawler_role (Optional[IRole]) – The crawler execution role
targets (Optional[TargetsProperty]) – A collection of targets to crawl
job_args (Optional[Dict[str, Any]]) – The input arguments to the Glue job
state_machine_input (Optional[Dict[str, Any]]) – The input dict to the state machine
additional_role_policy_statements (Optional[List[PolicyStatement]]) – Additional IAM policy statements to add to the state machine role
state_machine_failed_executions_alarm_threshold (Optional[int]) – The number of failed state machine executions before triggering CW alarm. Defaults to 1
state_machine_failed_executions_alarm_evaluation_periods (Optional[int]) – The number of periods over which data is compared to the specified threshold. Defaults to 1
Methods
__init__
(scope, id, environment_id[, ...])DDK Glue Transform stage.
add_alarm
(alarm_id, alarm_metric[, ...])Add a CloudWatch alarm for the Data Stage
Get output event pattern of the stage.
Get input targets of the stage.
is_construct
(x)Checks if
x
is a construct.to_string
()Returns a string representation of this construct.
Attributes
cloudwatch_alarms
List[Alarm] List of CloudWatch Alarms linked to the stage
Optional[CfnCrawler] The Glue crawler
Optional[IJob] The Glue job
node
The tree node.
StateMachine The state machine
- property crawler: Optional[aws_cdk.aws_glue.CfnCrawler]¶
Optional[CfnCrawler] The Glue crawler
- Type
Return
- get_event_pattern() Optional[aws_cdk.aws_events.EventPattern] ¶
Get output event pattern of the stage.
Event pattern describes the structure of output event(s) produced by this stage. Event Rules use event patterns to select events and route them to targets.
- Returns
event_pattern – Event pattern
- Return type
Optional[EventPattern]
- get_targets() Optional[List[aws_cdk.aws_events.IRuleTarget]] ¶
Get input targets of the stage.
Targets are used by Event Rules to describe what should be invoked when a rule matches an event.
- Returns
targets – List of targets
- Return type
Optional[List[IRuleTarget]]
- property job: Optional[aws_cdk.aws_glue_alpha.IJob]¶
Optional[IJob] The Glue job
- Type
Return
- property state_machine: aws_cdk.aws_stepfunctions.StateMachine¶
StateMachine The state machine
- Type
Return