Senior Data Engineer
May 2023 – PresentTier One Analytics · Budapest, Hungary
- Led the data delivery team in the migration of more than 1500 legacy SAS DI jobs to Databricks, ensuring seamless transition by verifying consistency between SAS and Databricks results.
- Automated the conversion and testing of over 700 SAS DI jobs, saving over 1000 hours of development time with 0 manual intervention.
- Built a data delivery pipeline that decreased time to convert each job by more than 25%.
- Created a unit test framework that automatically generates Databricks jobs and notebooks for transform-level testing, significantly improving validation efficiency.
- Designed an end-to-end testing framework to validate entire jobs, ensuring that migrated jobs function as expected across the full workflow.
- Implemented regression testing to compare job performance across different versions of migration tools, ensuring consistent results and detecting improvements or regressions.
- Analyzed and visualized SAS job metadata to estimate the time and cost for converting over 1,500 jobs, optimizing project planning and resource allocation.