aws_ddk_core.stages.GlueTransformStage

class aws_ddk_core.stages.GlueTransformStage(*args: Any, **kwargs)

Class that represents a Glue Transform DDK DataStage.

__init__(scope: constructs.Construct, id: str, environment_id: str, job_name: Optional[str] = None, executable: Optional[aws_cdk.aws_glue_alpha.JobExecutable] = None, job_role: Optional[aws_cdk.aws_iam.IRole] = None, crawler_name: Optional[str] = None, database_name: Optional[str] = None, crawler_role: Optional[aws_cdk.aws_iam.IRole] = None, targets: Optional[aws_cdk.aws_glue.CfnCrawler.TargetsProperty] = None, job_args: Optional[Dict[str, Any]] = None, state_machine_input: Optional[Dict[str, Any]] = None, additional_role_policy_statements: Optional[List[aws_cdk.aws_iam.PolicyStatement]] = None, state_machine_failed_executions_alarm_threshold: Optional[int] = 1, state_machine_failed_executions_alarm_evaluation_periods: Optional[int] = 1) None

DDK Glue Transform stage.

Stage that contains a step function that runs Glue job, and a Glue crawler afterwards. If the Glue job or crawler names are not supplied, then they are created.

Parameters
  • scope (Construct) – Scope within which this construct is defined

  • id (str) – Identifier of the stage

  • environment_id (str) – Identifier of the environment

  • job_name (Optional[str]) – The name of a preexisting Glue job to run. If None, a Glue job is created

  • executable (Optional[JobExecutable]) – The job executable properties

  • job_role (Optional[IRole]) – The job execution role

  • crawler_name (Optional[str]) – The name of a preexisting Glue crawler to run. If None, a Glue crawler is created

  • database_name (Optional[str]) – The name of the database in which the crawler’s output is stored

  • crawler_role (Optional[IRole]) – The crawler execution role

  • targets (Optional[TargetsProperty]) – A collection of targets to crawl

  • job_args (Optional[Dict[str, Any]]) – The input arguments to the Glue job

  • state_machine_input (Optional[Dict[str, Any]]) – The input dict to the state machine

  • additional_role_policy_statements (Optional[List[PolicyStatement]]) – Additional IAM policy statements to add to the state machine role

  • state_machine_failed_executions_alarm_threshold (Optional[int]) – The number of failed state machine executions before triggering CW alarm. Defaults to 1

  • state_machine_failed_executions_alarm_evaluation_periods (Optional[int]) – The number of periods over which data is compared to the specified threshold. Defaults to 1

Methods

__init__(scope, id, environment_id[, ...])

DDK Glue Transform stage.

add_alarm(alarm_id, alarm_metric[, ...])

Add a CloudWatch alarm for the Data Stage

build_state_machine(id, environment_id, ...)

Build state machine.

get_event_pattern()

Get output event pattern of the stage.

get_targets()

Get input targets of the stage.

is_construct(x)

Checks if x is a construct.

to_string()

Returns a string representation of this construct.

Attributes

cloudwatch_alarms

List[Alarm] List of CloudWatch Alarms linked to the stage

crawler

Optional[CfnCrawler] The Glue crawler

job

Optional[IJob] The Glue job

node

The tree node.

state_machine

StateMachine The state machine

property crawler: Optional[aws_cdk.aws_glue.CfnCrawler]

Optional[CfnCrawler] The Glue crawler

Type

Return

property job: Optional[aws_cdk.aws_glue_alpha.IJob]

Optional[IJob] The Glue job

Type

Return