As a Lead Data Scientist, you will be part of our Applied AI and Machine Learning team. You will own the work with our client-facing teams on testing and optimizing our DaVinci Personalization product to realize the best outcomes for our clients and for Movable Ink. This includes running a/b tests and conducting deep dives into data to identify opportunities for improving the product and how we deploy it. You will also work on driving improvements to our working model and process, driving training and technical investments for how to make more of our analytics self-serve. This is an opportunity to become Movable Ink’s resident expert for how we run experiments and how best to operate our modeling solutions to maximize their real-world impact.

Responsibilities:

• Run a/b tests for our DaVinci personalization product and own the reporting for their business outcomes - you will serve as our resident expert for a/b testing, work with advanced techniques for variance reduction, and influence metrics and test success criteria
• Determine and conduct deep dives as necessary to understand the behavior of our recommender models and to identify opportunities for improving our system – this includes both correlational and causal analysis
• Help customer-facing teams and our clients resolve issues that come up during tests and answer questions about how to optimize setups - you will own a ticket queue and the process for resolving issues, ensuring resolution happens within an SLA
• Help determine BI tooling needs to make testing, reporting, and analytics lower lift, and partner with engineers and client-facing teams to make improvements
• Help design documentation and other materials for educating adjacent teams on our recommender system behavior and self-serve insights generation

Qualifications:

• 5+ years of industry experience owning reporting and analytics required to operate a machine-learning system
• Solid foundations in statistics and online controlled experiments (a/b testing), including advanced measurement techniques such as Causal Inference or model-based variance reduction (e.g. ANCOVA, CUPED)
• High comfort level with SQL, reporting tools such as Looker/Google Sheets, and some programming skill (e.g. in Python) to be able to reason about logic in code
• Intuitive understanding of machine learning models and systems
• You enjoy helping others find pragmatic solutions to the daily challenges they face and have the ability to abstract from these challenges and identify process, training, and technical improvements we can make
• Ability to collaborate and drive projects that involve multiple teams
• A desire to always be learning and contributing to a collaborative environment