Data Engineer – Big Data Platforms

Job Summary:

We are seeking a skilled and experienced Data Engineer to join our team. The ideal candidate will have expertise in Big Data platforms and a strong background in designing, developing, and maintaining data pipelines. You will be responsible for ensuring data quality, optimizing data workflows, and working with distributed data systems. This role requires a deep understanding of modern data engineering workflows, ETL processes, and large-scale data processing.

Key Responsibilities:

Data Engineering:

• Design, develop, and maintain scalable and efficient data pipelines to support business needs.
• Implement ETL/ELT processes to extract, transform, and load data from various sources into Big Data platforms.
• Optimize data workflows for performance, scalability, and reliability.

Big Data Platforms:

• Work with Big Data technologies such as Hadoop, Spark, Hive, or HDFS to process and analyze large datasets.
• Develop distributed data processing pipelines using Spark (PySpark/Scala) and MapReduce.
• Optimize Big Data workflows for performance and scalability.

Data Integration:

• Integrate data from multiple sources, including APIs, flat files, and third-party systems.
• Collaborate with data analysts, data scientists, and business teams to understand data requirements.

Data Quality and Governance:

• Ensure data accuracy, consistency, and integrity across systems.
• Implement data validation and error-handling mechanisms.

Collaboration and Documentation:

• Work closely with cross-functional teams to understand business requirements and translate them into technical solutions.
• Document data pipelines, data designs, and processes for future reference.

Required Skills and Qualifications:

• Technical Expertise:
  • Expertise in Big Data platforms:
    • Experience with Hadoop, Spark (PySpark/Scala), Hive, or HDFS for distributed data processing.
  • Proficiency in SQL and database design principles.
  • Experience with ETL/ELT tools (e.g., Informatica, Talend, or Python-based ETL frameworks).
• Programming Skills:
  • Proficiency in scripting languages such as Python, Scala, or Shell scripting for data processing and automation.
• Data Governance:
  • Knowledge of data quality, data governance, and data security best practices.
• Soft Skills:
  • Strong problem-solving and analytical skills.
  • Excellent communication and collaboration skills.

Ability to work in a fast-paced, dynamic environment.
 

Qualifications:

• 7-9 years of relevant experience
• Experience in systems analysis and programming of software applications
• Experience in managing and implementing successful projects
• Working knowledge of consulting/project management techniques/methods
• Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements

Education:

• Bachelor’s degree/University degree or equivalent experience

This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.

------------------------------------------------------

Job Family Group:

Technology

------------------------------------------------------

Job Family:

Applications Development

------------------------------------------------------

Time Type:

Full time

------------------------------------------------------

Most Relevant Skills

Please see the requirements listed above.

------------------------------------------------------

Other Relevant Skills

For complementary skills, please see above and/or contact the recruiter.

------------------------------------------------------

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

 

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.

View Citi’s EEO Policy Statement and the Know Your Rights poster.