ETL Tester exp: 6 to 10 years

Lead: 10 years to 15 years

 

Lead ETL TESTER:

 

We are looking for an experienced Lead Quality Analyst Consultant to join our data engineering project team. As a Senior Quality Analyst Consultant, you will be responsible for ensuring the quality and reliability of our data engineering processes and systems. Your role will involve designing and executing test plans, developing test plan automations, identifying and documenting defects, and collaborating with cross-functional teams to deliver high-quality data engineering solutions.

 

Responsibilities:

- Develop and implement comprehensive test plans and test cases for data engineering projects, including data ingestion, transformation, integration processes, and PBI dashboards/reports.

- Create automated scripts in SQL/Python to validate data pipeline/transformations and generate reports after each data pipeline run.

- Validate data for consistency, accuracy, and completeness from source systems to reports.

- Execute test cases and analyze results to identify defects, inconsistencies, and areas for improvement.

- Collaborate with data engineers, data scientists, and other stakeholders to understand project requirements and ensure quality standards are met.

- Participate in design and code reviews to provide input on quality and testability aspects of data engineering solutions.

- Work closely with the development team to address and resolve defects in a timely manner.

- Continuously improve testing processes and methodologies to enhance efficiency and effectiveness.

- Provide guidance and mentorship to junior quality analysts, promoting a culture of quality within the team.

- Conduct functional, regression, performance, and load testing of DWH solutions.

- Review Power BI reports and dashboards to ensure they meet specifications and data accuracy.

- Document defects, issues, and improvements found during testing and collaborate with the development team for resolution.

- Automate data validation and testing processes using appropriate tools and frameworks.

- Support user acceptance testing (UAT) and provide timely feedback on testing results.

- Stay up-to-date with industry trends and advancements in data engineering and quality assurance practices.

 

Requirements:

- Bachelor's degree in computer science, engineering, or a related field.

- Experience testing ADF Pipelines, Databricks Notebooks, and PBI reports/dashboards.

- Proven experience as a Quality Analyst or similar role in data engineering or data analytics, preferably on Azure/Databricks environment.

- Strong understanding of data engineering concepts, including data ingestion, transformation, and integration.

- Proficiency in SQL and experience with data querying and analysis.

- Proficiency in SQL and Python scripting for developing reusable scripts for test case execution.

- Strong understanding of SQL queries, joins, and data manipulation techniques for data validation.

- Experience writing Python scripts to automate test case execution and perform data validation.

- Familiarity with testing frameworks and libraries in Python, such as pytest or unittest.

- Knowledge of data validation techniques, including data profiling, comparisons, and quality checks.

- Experience with testing tools and frameworks, such as PyTest.

- Familiarity with version control systems (e.g., Git) and continuous integration/continuous deployment (CI/CD) pipelines.

- Excellent analytical and problem-solving skills.

- Strong attention to detail and commitment to delivering high-quality results.

- Excellent communication and collaboration skills.

- Proven ability to work in a fast-paced, dynamic environment and manage multiple priorities.