Labs in the Data Engineering workshop are to be completed in sequence. This lab is designed to automate the Data Lake hydration with AWS Database Migration Service (AWS DMS), so we can fast forward to the following Glue lab.
PreLab Setup 2 is not required. Otherwise, duplicate S3 buckets and IAM roles will appear in your lab environment. If that happens, please use the resources with a name prefix “auto-dmslab-”.
Prelab Setup 1 - is completed and the source RDS database is fully populated.
RDS Database Server Name - If you are in an AWS hosted event, please ask your instructor. Otherwise, check the Outputs tab on your CloudFormation Console, note down the RDS Server value named
dms-vpc-role - Check if the Identity and Access Managment (IAM) roles exist in your workshop AWS account. Go to the IAM console, copy & paste the names in the search box respectively.
Note whether these role are present or not. In this example screenshot the cloudwatch role is absent.
It completes the following tasks on your behalf:
In Review page, review all the details, scroll down and check the box to acknowledge the policy and then click on Create Stack.
The stack launch may take 5-6 minutes. Wait until your stack status advances to “CREATE_COMPLETE”.
At this point, the source data has been fully loaded from RDS database to your S3 bucket via DMS. Go to AWS DMS console, you should see two Database migration tasks are 100% completed. If not, please wait until they are finished, then proceed to the Glue lab