Project Role : Performance Engineer
Project Role Description : Diagnose issues that an in-house performance testing team has been unable to. There are five aspects to Performance Engineering: software development lifecycle and architecture, performance testing and validation, capacity planning, application performance management and problem detection and resolution.
Must have skills : Data Engineering
Good to have skills : Python (Programming Language), GitHub
Minimum 3 year(s) of experience is required
Educational Qualification : 15 years full time education

Summary:
As a Performance Engineer, a typical day involves investigating and resolving complex performance issues that have not been identified by the internal performance testing team. The role encompasses a broad spectrum of activities including analyzing software development processes and architecture, validating performance through rigorous testing, planning for capacity needs, managing application performance, and swiftly detecting and addressing problems to ensure optimal system functionality. This position requires a proactive approach to identifying bottlenecks and collaborating with various teams to enhance overall system efficiency and reliability.

KEY RESPONSIBILITIES

Set up and configure Databricks workspaces, including cluster management, access controls, and integration with cloud identity and access management services
Design and implement Medallion architecture (Bronze, Silver, and Gold layers) on cloud object storage using Delta Lake format, ensuring data quality and traceability at each layer
Build and maintain ETL and ELT data pipelines using PySpark and Spark SQL within Databricks, covering ingestion, transformation, deduplication, normalisation, and standardisation
Ingest data from multiple source systems including relational databases (Oracle, MySQL, SQL Server), flat files, and SaaS platforms such as Salesforce, into the Lakehouse
Execute data migration from Oracle and other legacy systems, storing transformed outputs in the cloud storage Gold Layer using Databricks batch processing
Orchestrate and automate data workflows using pipeline orchestration tools such as Apache Airflow, AWS Glue, or Azure Data Factory, ensuring timely, dependency-aware, and reliable data delivery
Create and manage Linked Services, Datasets, and connection configurations for source and sink systems
Monitor pipeline execution end-to-end, troubleshoot failures, implement corrective actions, and perform root cause analysis for recurring issues
Optimise Spark jobs for performance, scalability, and cost efficiency — including partitioning strategy, caching, and resource configuration
Ensure all data is stored in Delta format with appropriate schema enforcement, ACID compliance, and incremental load patterns
Manage sensitive credentials and connection strings securely using a secrets management service (e.g., AWS Secrets Manager, Azure Key Vault, or HashiCorp Vault)
Maintain data quality standards by implementing validation and reconciliation logic within pipeline workflows
Collaborate with data analysts, data scientists, and product teams across global delivery environments to understand data needs and deliver trusted datasets
Maintain thorough documentation of pipeline designs, data flows, architecture decisions, and operational runbooks


Required Skills & Experience
3–4 years of professional experience in data engineering, with a significant and demonstrable portion of that time working on Databricks in a production environment
Strong hands-on proficiency in PySpark and Python, with the ability to write clean, efficient, and maintainable transformation code
Proven experience designing and implementing Medallion architecture (Bronze / Silver / Gold) on cloud object storage, with Delta Lake as the storage layer
Solid understanding of Delta Lake capabilities: ACID transactions, schema evolution, time travel, and incremental ingestion patterns
Hands-on experience migrating and extracting data from Oracle and other relational databases (MySQL, SQL Server) into a cloud-based Lakehouse
Experience with batch data ingestion from SaaS platforms such as Salesforce
Working knowledge of at least one pipeline orchestration tool — Apache Airflow, AWS Glue, Azure Data Factory, or similar — for scheduling and dependency management
Familiarity with cloud-hosted relational databases and MySQL for structured data storage and maintenance
Sound understanding of ETL and ELT design patterns and the trade-offs between them
Proficiency in Git and GitHub for version control, branching, and collaborative code review
Experience working within Agile or Waterfall delivery frameworks with structured sprint or milestone-based planning
Strong analytical and troubleshooting skills ability to diagnose and resolve pipeline failures methodically
Effective communication and collaboration skills with the ability to work across cross-functional and geographically distributed teams

GOOD TO HAVE

Experience with Databricks Unity Catalog for data governance, lineage tracking, and fine-grained access control
Exposure to cloud secrets management integration (e.g., AWS Secrets Manager, Azure Key Vault, or HashiCorp Vault) within Databricks secret scopes
Background in legacy big data tooling — Hadoop, Hive, or Sqoop — particularly in the context of migration or modernisation projects
Familiarity with advanced Spark optimisation techniques including broadcast joins, adaptive query execution, and partition tuning
Basic understanding of data modelling concepts and dimensional or star schema design
Exposure to DevOps practices for data pipelines, including CI/CD, automated testing frameworks, or infrastructure-as-code
Databricks Certified Associate Developer for Apache Spark, or a relevant cloud data certification (AWS Certified Data Analytics, Microsoft Certified: Azure Data Engineer Associate, or equivalent)

EDUCATION

Bachelor s degree (B.Tech / B.E. or equivalent) in Computer Science, Information Technology, Data Science, or a related technical discipline is strongly preferred
A Master s degree in Computer Science, Data Engineering, or a related field is an advantage but not mandatory
Candidates with degrees in other engineering disciplines who can demonstrate strong, production-grade data engineering experience will also be considered
Databricks Certified Associate Developer for Apache Spark, or a relevant cloud data certification (AWS Certified Data Analytics, Microsoft Certified: Azure Data Engineer Associate, or equivalent) are a distinct advantage


Additional Information:
- The candidate should have minimum 3 years of experience in Data Engineering.
- This position is based at our Mumbai office.
- A 15 years full time education is required.

15 years full time education

About Accenture

Accenture is a leading global professional services company that helps the world’s leading businesses, governments and other organizations build their digital core, optimize their operations, accelerate revenue growth and enhance citizen services—creating tangible value at speed and scale. We are a talent- and innovation-led company with approximately 791,000 people serving clients in more than 120 countries. Technology is at the core of change today, and we are one of the world’s leaders in helping drive that change, with strong ecosystem relationships. We combine our strength in technology and leadership in cloud, data and AI with unmatched industry experience, functional expertise and global delivery capability. Our broad range of services, solutions and assets across Strategy & Consulting, Technology, Operations, Industry X and Song, together with our culture of shared success and commitment to creating 360° value, enable us to help our clients reinvent and build trusted, lasting relationships. We measure our success by the 360° value we create for our clients, each other, our shareholders, partners and communities.

Visit us at www.accenture.com 

Equal Employment Opportunity Statement

We believe that no one should be discriminated against because of their differences. All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, military veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by applicable law. Our rich diversity makes us more innovative, more competitive, and more creative, which helps us better serve our clients and our communities.