We are seeking a manager to grow and manage a team of engineers driving the implementation of GPU accelerated Apache Spark applications. Data scientists spend a considerable amount of time exploring data and iterating over machine learning (ML) experiments. Every hour of compute required to sort through datasets, extract features and fit ML algorithms impedes an efficient business workflow. NVIDIA believes that data science workflows can benefit tremendously from being accelerated, to enable data scientists to explore many more and larger datasets to drive towards their business goals, faster and more efficiently.

At NVIDIA, we are passionate about working on hard problems that have an impact. You will need to have previous experience working with Spark applications, implementing big data applications for a variety of customers, programming skills, and familiarity with open source data processing frameworks. You should have experience leading engineers, handling customer concerns and working with interdisciplinary teams. You will work with an engineering team accelerating Spark with GPUs using RAPIDS, CUDA and other libraries. Our goal is to achieve the best performance at the lowest cost and power utilization. The RAPIDS Spark library is integrated with cloud service providers and open source Apache Spark distributions. This is a strategic investment for NVIDIA! Are you up for the challenge?

What you’ll be doing

• Integrate GPU based Spark processing with cloud deployment services and IAM services. Ensure service observability, reliability and SLA.

• Guide development of software and tools to streamline the migration of Spark workloads from CPUs to GPUs.

• Conduct experiments for column based data processing to enable cooperation between CPUs and GPUs.

• Build and lead a distributed team to support business growth.

What we need to see:

• Solid understanding of the Apache Spark platform

• Experience with data ingestion, batch and stream processing, file formats, storage systems, data processing, workflow management, visualization and consumption.

• Background with cloud microservice technologies (orchestration, package management, deployment management)

• Experience supporting enterprise customers

• Experience recruiting, managing and developing a team

• Knowledge of SQL, Python and Scala

• Good at communicating, presenting and explaining technical topics

• 12+ overall years of software experience and 5+ years of management experience

• BS/MS/PhD in computer science or a related field (or equivalent experience)

Ways to stand out from the crowd:

• Apache Spark Committership or PMC membership

• End to end data processing background, including with data preparation, machine learning and deep learning applications.

• Experience with TensorFlow, PyTorch, SparkML, XGBoost

The base salary range is 248,000 USD - 385,250 USD. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions.NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.