Labs in the Data Engineering workshop are to be completed in sequence. This lab is designed to automate the Data Lake hydration with AWS Database Migration Service (AWS DMS), so we can fast forward to the following Glue lab.
If you prefer to get hands-on with AWS DMS service, please choose Option 1: DMS Main Lab.
Student PreLab Setup is not required. Otherwise, duplicate S3 buckets and IAM roles will appear in your lab environment. If that happens, please use the resources with a name prefix auto-dmslab-.
Instructor Prelab Setup - is completed and the source RDS database is fully populated.
RDS Database Server Name - If you are in an AWS hosted event, you can get this value from Event Engine Team Dashboard. Otherwise, check the Outputs tab on your CloudFormation Console, note down the RDS Server value named
dms-vpc-role - Check if the Identity and Access Managment (IAM) roles exist in your workshop AWS account. Go to the IAM console, copy & paste the names in the search box respectively.
Note whether these role are present or not. In this example screenshot the cloudwatch role is absent.
Make sure you select the appropriate AWS region
Select correct choice for the parameters DMSCWRoleCreated and DMSVPCRoleCreated. For the ServerName parameter enter the database endpoint that you obtained from the pre-requisite. Under Capabilities, Check the box “I acknowledge that …”, then click on “Create Stack” to create the stack.
It completes the following tasks on your behalf:
Select the Parameters as explained below:
Under Capabilities, check the box to acknowledge the policy and then click on Create Stack.
The stack launch may take 5-6 minutes. Wait until your stack status advances to “CREATE_COMPLETE”.
At this point, the source data has been fully loaded from RDS database to your S3 bucket via DMS. Go to AWS DMS console, you should see two Database migration tasks are 100% completed. If not, please wait until they are finished, then proceed to the Glue lab