ATP Conversion POC – Data ingestion from Oracle HRS/SFS to AWS S3
EDGC Decision Date: July 16, 2023
Decision: ATP Conversion POC – Data ingestion from Oracle HRS/SFS to AWS S3
Decision
This ATP conversion request is for actual data from our current HRS and SFS system (including PII data like full SSN, bank accounts, etc.) to be moved from the source system to Amazon S3, then to Workday for the conversion. The data will be converted into Workday and will follow the security policies of the ATP Workday team. Once the data is in Workday, the data will be secured using role-based security that is being designed by the ATP Functional teams. The data loaded to stage tables in AWS S3 is not going to persist past the go-live, all security is then passed to the Workday teams.
Background
The success of ATP Conversion depends, in part, on quickly and safely ingesting data from Oracle Exadata sources (including HRS and SFS), transforming that data to meet business requirements for Workday and ultimately loading the data to Workday via excel files.
The ATP Conversion Team has identified roughly 220 source tables needed to create the Workday files and present to our implementation partner to load to Workday. Some of these data sources contain highly restricted data such as social security numbers and bank accounts. This data will ultimately reside in the Workday cloud environment.
The team has been using a combination of Exadata, SQL, IICS and Python to build out the tenants for the project to date. The team requested permission to use AWS and AWS tools to ingest the data from SFS and HRS to an S3 raw layer.
The Conversion Team will utilize the following serverless services within AWS – S3, Glue, Athena, KMS (for encryption), IAM (for permissions) and Cloudwatch Logs (for logging). For ingestion, the team will use on-prem servers running hadoop/spark which won’t need any firewalls to be opened.
The goal for this POC is to evaluate the efficiency of using AWS. As we approach the go-live date, it is critical to continue to shrink the timelines for pulling and transforming the data to limit disruption to daily operations.
The team has anticipated this transition for 6 months and have been developing current code in a manner that will require little rework to test the POC. The team has been working with the DoIT Cloud Team to stand up the environment for ingestion. The team has met with Cyber Security, completed the assessment, and taking all necessary mitigation measures as it relates to this data.
This POC request is extremely limited to a small team of ATP Conversion developers and a key subset of HRS & SFS data. The team has tested the process of pulling public data to an S3 bucket in the AWS environment and the final step is to connect to Exadata to pull true data. The team has an expert AWS Engineer on the team who has been working to set up the environment and will manage the ingestion POC.
Who Developed the Recommendation?
- Michelle Weber, ATP Conversion Strategy Lead
- Brian Zehren, Associate Director, ERP App Support
- Shiva Prakash, Conversion Cloud Engineer
- Mike Vavrus, DoIT Cloud Engineer
Who Was Consulted in the Development of the Recommendation?
The following stakeholders were consulted about the proposed POC:
- Pam Snyder, Cloud Security Risk Analyst
- Steve Tanner, Cloud & Security Eng Associate Director
- Steffanie Johnson, Info Security Analyst IV
- Kristy Rogers, ATP IT Security Lead and HRS Security Lead
- Al Kantor, Cloud Data Architect