The candidate will work in high visibility projects as a Data Scientist, bringing the Data Science and NLP expertise to projects. The candidate will work in RD Data Science team and collaborate with Product's managers, domain experts, Knowledge representation experts, to build high value outcome from Elsevier content. The candidate will have an opportunity to impact virtually all Elsevier applications related to Research and Operations.

Large Language Models (LLMs) have revolutionized the field of natural language processing, providing unprecedented capabilities in text generation, comprehension, and various other applications. Despite their impressive performance, these models often require carefully crafted input prompts to produce the best possible responses. Users, particularly those without expertise in prompt engineering, may find it challenging to formulate prompts that fully leverage the capabilities of these advanced models. This gap underscores the necessity for an automated system that can refine and optimize user prompts, making high-quality interaction with LLMs more accessible.

Existing methods for prompt optimization largely rely on manual adjustments and iterative testing, which are not only time-consuming but also require a certain level of expertise. To address this, we propose a novel approach: a task-specific prompt re-writer designed to automatically adapt and enhance user prompts for LLMs. This system involves a small, specialized model dedicated to re-writing prompts, coupled with a larger LLM that executes the task based on these optimized prompts.

The core idea is to use the prompt re-writer to generate better-suited prompts that maximize the performance of the LLM on a specific task. The re-writer will be trained and optimized using two primary feedback mechanisms. First, explicit feedback from the LLM will guide how prompts can be refined. Second, a reward system will assess how well the LLM’s output aligns with the desired task performance, providing a quantitative measure of success. Such feedback can be used to optimize the prompt re-writer using a reinforcement learning mechanism. This dual feedback loop ensures that the prompt re-writer continuously improves and adapts to produce prompts that significantly enhance the LLM's output quality in the task.

By implementing and fine-tuning this prompt re-writing model, we aim to democratize the use of LLMs, making their advanced capabilities more accessible to a wider audience. This project aims at improving the overall efficiency and effectiveness of interactions with LLMs across various applications and tasks. Through this automated and intelligent prompt optimization system, we can unlock the full potential of LLMs, enabling them to deliver consistently high-quality results with minimal user intervention.

-----------------------------------------------------------------------

Elsevier is an equal opportunity employer: qualified applicants are considered for and treated during employment without regard to race, color, creed, religion, sex, national origin, citizenship status, disability status, protected veteran status, age, marital status, sexual orientation, gender identity, genetic information, or any other characteristic protected by law. We are committed to providing a fair and accessible hiring process. If you have a disability or other need that requires accommodation or adjustment, please let us know by completing our Applicant Request Support Form: https://forms.office.com/r/eVgFxjLmAK , or please contact 1-855-833-5120.

Please read our Candidate Privacy Policy.