About the Role

As a Software Engineer 2 on the Data Platform team, you are a key contributor to the architectural backbone of our AI-native platform. You don’t just build one-off pipelines; you help engineer the reliable frameworks and robust data engines that power every product feature and AI workflow across the company.

Working closely with Senior Engineers, you will develop the systems that ingest, process, and serve data from hundreds of internal services and massive event streams. You will ensure our Databricks Lakehouse remains a high-performance, automated, and trusted foundation for our global ecosystem.

What You Will Do

Build & Enhance Data Platform Systems

• Design and develop core components of the data platform, including high-volume event ingestion pipelines and real-time processing workflows using Apache Spark.
• Develop reusable data access layers and "framework-based" ETL/ELT processes that transform structured business data and event streams into trusted Lakehouse assets.
• Contribute to the development of internal APIs and backend services that enable other engineering teams to interact with the data platform.

Engineer Reliable Data Flows

• Build robust, fault-tolerant pipelines with a focus on data correctness, consistency, and availability.
• Implement schema enforcement and validation gates to maintain strict data governance across systems, preventing "data swamp" scenarios.
• Optimize code and query performance to ensure workloads are both fast and cost-efficient.

Drive Observability & Quality

• Build and own the observability framework (logging, metrics, and alerting) to monitor pipeline health, data freshness SLAs, and system uptime.
• Proactively identify and resolve bottlenecks in production; participate in root-cause analysis and incident response.
• Apply rigorous automated testing (unit, integration, and regression) to all data code to ensure long-term maintainability.

Collaborative Platform Engineering

• Participate in design discussions and code reviews, bringing a software engineering lens to data problems.
• Follow governance and PII handling standards in day-to-day development, ensuring safe logging and access-aware implementations.

What We Are Looking For

• Experience: 3+ years of experience in software engineering, with a track record of building production-grade backend or data systems.
• Databricks/Spark Mastery: Strong hands-on experience with Apache Spark and the Databricks/Delta Lake ecosystem is essential.
• Coding Fundamentals: Expert-level Python and SQL skills, with a deep understanding of software design principles, debugging, and distributed computing concepts.
• Automation Mindset: Experience with CI/CD practices and a working knowledge of Terraform (or similar IaC) to help automate the platform’s lifecycle.
• Reliability Focus: A background in building monitored, observable systems where data quality and validation are treated as first-class citizens.

🌟 What Makes Us Different:

AI-Infused SDLC: We operate in an AI-accelerated environment. We encourage our engineers to leverage AI-first mindset throughout the SDLC from planning to AI-assisted development tools (e.g., GitHub Copilot, Cursor) to increase velocity, automate unit test generation, and focus on high-level architectural challenges rather than boilerplate code.