The role

We are seeking a Data Scientist with deep expertise in ML Ops, reliability, and performance monitoring to ensure our machine learning models deliver maximum business impact and consistently high accuracy in an ultra-low latency search ads ecosystem. In this role, you will closely collaborate with machine learning engineers, product managers, and data engineers to build and maintain robust, scalable ML infrastructure, while ensuring models remain stable, precise, and aligned with dynamic business objectives such as yield optimization, CTR prediction accuracy, and improved advertiser ROI.

You will have a unique opportunity to shape how we continuously improve and maintain our ML models in production. Your work will drive consistent performance, simplify troubleshooting, and streamline iterative experimentation, ultimately contributing to an efficient and reliable ML pipeline that underpins our entire ad serving ecosystem.

Responsibilities

• Develop and maintain dashboards, alerts, and reports to continuously monitor model accuracy (e.g., CTR, CVR), yield lift, and key KPIs across multiple placements and segments.
• Proactively identify performance regressions, model drifts, or data quality issues that may affect stability, accuracy, or latency.
• Collaborate with ML engineers and data scientists to iterate on model architectures, retraining schedules, and feature sets to ensure ongoing alignment with business goals.
• Establish and maintain a model monitoring framework, including triggers for anomaly detection, concept drift, data schema changes, and other production-level signals.
• Develop tools and processes to diagnose issues quickly and ensure rapid response to performance degradation.
• Work with ML engineers and DevOps personnel to implement appropriate failover strategies, fallback models, or safe experimentation frameworks that minimize revenue impact during model updates or unexpected downtime.
• Design and analyze A/B tests, synthetic experiments, and other evaluation methods to validate and refine model assumptions.
• Advocate for the adoption of new ML Ops tools, technologies, and industry best practices that improve team efficiency and ensure model quality.
• Collaborate closely with product managers, ML engineers, business stakeholders, and data scientists to understand evolving business requirements and proactively address production challenges.
• Translate complex model performance insights into actionable recommendations for technical and non-technical stakeholders.
• Serve as an internal expert on ML system maintenance best practices, guiding junior team members and promoting a data-driven culture of continuous improvement.

Basic Qualifications

• A PhD or an MS with 5+ years of experience in a quantitative field (e.g., Computer Science, Statistics, Operations Research, or related) or equivalent industry experience in large-scale ML projects.
• 5+ years of experience working with large-scale ML systems in production, including hands-on experience with ML Ops frameworks and monitoring solutions.
• Proficiency in Python and SQL, and familiarity with ML frameworks (e.g., TensorFlow, PyTorch, scikit-learn) as well as distributed processing tools (e.g., Spark, Hadoop).
• Strong understanding of model evaluation metrics, A/B testing methodologies, and methods to diagnose and mitigate performance drift.
• Excellent communication skills and the ability to work effectively in cross-functional teams, bridging gaps between ML, data engineering, product, and operations.

Preferred Qualifications

• Experience working in the ads domain, optimizing yield, CTR/CVR predictions, and bidding strategies.
• Familiarity with reinforcement learning or contextual bandit techniques, particularly for ad selection and exploration/exploitation scenarios.
• Demonstrated ability to handle limited engagement data scenarios, incorporate first-party data signals, and optimize ML models during periods of volatility (e.g., seasonal promotions, changing advertiser budgets).
• Knowledge of industry-leading ML Ops platforms and tools (e.g., MLflow, Kubeflow, Tecton) and how to integrate them into existing pipelines.

 

*Compensation Range: $130,000 - $170,000