aws_ddk_core.stages.GlueTransformStage

class aws_ddk_core.stages.GlueTransformStage(*args: Any, **kwargs)

Class that represents a Glue Transform DDK DataStage.

__init__(scope: constructs.Construct, id: str, environment_id: str, job_name: Optional[str] = None, executable: Optional[aws_cdk.aws_glue_alpha.JobExecutable] = None, job_role: Optional[aws_cdk.aws_iam.IRole] = None, crawler_name: Optional[str] = None, database_name: Optional[str] = None, crawler_role: Optional[aws_cdk.aws_iam.IRole] = None, targets: Optional[aws_cdk.aws_glue.CfnCrawler.TargetsProperty] = None, job_args: Optional[Dict[str, Any]] = None, state_machine_input: Optional[Dict[str, Any]] = None, additional_role_policy_statements: Optional[List[aws_cdk.aws_iam.PolicyStatement]] = None, state_machine_failed_executions_alarm_threshold: Optional[int] = 1, state_machine_failed_executions_alarm_evaluation_periods: Optional[int] = 1) None

DDK Glue Transform stage.

Stage that contains a step function that runs Glue job, and a Glue crawler afterwards. If the Glue job or crawler names are not supplied, then they are created.

Parameters
  • scope (Construct) – Scope within which this construct is defined

  • id (str) – Identifier of the stage

  • environment_id (str) – Identifier of the environment

  • job_name (Optional[str]) – The name of a preexisting Glue job to run. If None, a Glue job is created

  • executable (Optional[JobExecutable]) – The job executable properties

  • job_role (Optional[IRole]) – The job execution role

  • crawler_name (Optional[str]) – The name of a preexisting Glue crawler to run. If None, a Glue crawler is created

  • database_name (Optional[str]) – The name of the database in which the crawler’s output is stored

  • crawler_role (Optional[IRole]) – The crawler execution role

  • targets (Optional[TargetsProperty]) – A collection of targets to crawl

  • job_args (Optional[Dict[str, Any]]) – The input arguments to the Glue job

  • state_machine_input (Optional[Dict[str, Any]]) – The input dict to the state machine

  • additional_role_policy_statements (Optional[List[PolicyStatement]]) – Additional IAM policy statements to add to the state machine role

  • state_machine_failed_executions_alarm_threshold (Optional[int]) – The number of failed state machine executions before triggering CW alarm. Defaults to 1

  • state_machine_failed_executions_alarm_evaluation_periods (Optional[int]) – The number of periods over which data is compared to the specified threshold. Defaults to 1

Methods

__init__(scope, id, environment_id[, ...])

DDK Glue Transform stage.

add_alarm(alarm_id, alarm_metric[, ...])

Add a CloudWatch alarm for the Data Stage

get_event_pattern()

Get output event pattern of the stage.

get_targets()

Get input targets of the stage.

is_construct(x)

Checks if x is a construct.

to_string()

Returns a string representation of this construct.

Attributes

cloudwatch_alarms

List[Alarm] List of CloudWatch Alarms linked to the stage

crawler

Optional[CfnCrawler] The Glue crawler

job

Optional[IJob] The Glue job

node

The tree node.

state_machine

StateMachine The state machine

property crawler: Optional[aws_cdk.aws_glue.CfnCrawler]

Optional[CfnCrawler] The Glue crawler

Type

Return

get_event_pattern() Optional[aws_cdk.aws_events.EventPattern]

Get output event pattern of the stage.

Event pattern describes the structure of output event(s) produced by this stage. Event Rules use event patterns to select events and route them to targets.

Returns

event_pattern – Event pattern

Return type

Optional[EventPattern]

get_targets() Optional[List[aws_cdk.aws_events.IRuleTarget]]

Get input targets of the stage.

Targets are used by Event Rules to describe what should be invoked when a rule matches an event.

Returns

targets – List of targets

Return type

Optional[List[IRuleTarget]]

property job: Optional[aws_cdk.aws_glue_alpha.IJob]

Optional[IJob] The Glue job

Type

Return

property state_machine: aws_cdk.aws_stepfunctions.StateMachine

StateMachine The state machine

Type

Return