Key Responsibilities:

• Architect and Design: Develop scalable data architectures using Databricks, Unity Catalog, Azure Data Lakehouse, Medallion Architecture, and Azure Data Factory.
• Leadership: Lead and mentor a team of data engineers, full stack developers, and QA teams, fostering a collaborative environment.
• Delivery Management: Oversee the entire lifecycle of data engineering projects, ensuring successful data extraction, transformation, and end-user consumption.
• Collaboration: Work with cross-functional teams, including data scientists, analysts, and business stakeholders, to meet data requirements.
• Problem-Solving: Apply strong analytical and problem-solving skills.
• Communication: Exhibit excellent verbal and written communication skills.
• Adaptability: Adapt to changing business needs in a fast-paced environment.
• Innovation: Stay updated with industry trends and integrate new technologies into data engineering practices.

Technical Expertise:

• Extensive experience with Databricks, Unity Catalog, Azure Data Lakehouse, Medallion Architecture, and Azure Data Factory.
• Proficiency in Python, PySpark, SQL, and Scala.
• Strong understanding of data modeling, ETL/ELT processes, and data warehousing.
• Experience with Microsoft Azure services for optimal performance and scalability.
• Design and implement use cases for Unity Catalog.
• Ensure data quality, integrity, and security.
• Familiarity with big data technologies like Hadoop, Spark, and Kafka.
• Experience with DevOps practices and CI/CD tools.
• Knowledge of full stack web development technologies (React, .Net, etc.).

Qualifications:

• Bachelor’s or master’s degree in computer science, Information Technology, or a related field.
• 10+ years of experience in data engineering, focusing on the specified technologies and architectures.