Forward Deployed Engineer, Assistant Vice President Greater New York City, NY Area

The Company is seeking a Forward Deployed Engineer, Assistant Vice President to join their team. This role is pivotal in the construction of automated evaluation harnesses, curation of regression testbeds, and optimization of prompts and output schemas to ensure the reliability, accuracy, and safety of production AI agents. The successful candidate will be responsible for building rigorous evaluation suites, driving automated CI/CD test execution, and maintaining domain-specific datasets in collaboration with domain experts. Additionally, the role involves optimizing prompt templates and system instructions to enhance deterministic agent behavior, as well as authoring and maintaining automated test execution and deployment pipelines. Applicants for this position should have 5-8 years' of relevant software engineering or systems analysis experience, with a strong focus on Python and Generative AI application development. Proficiency in Python programming, experience with AI evaluation frameworks, and a solid understanding of benchmarking methodologies are essential. The role requires the ability to manage and implement automated test pipelines in CI/CD environments, as well as the capacity to work effectively under pressure and meet tight deadlines. A Bachelor's degree in computer science, data science, or a related quantitative discipline is required, and the candidate should be adept at problem-solving, have good communication skills, and a detail-oriented approach to software quality and data integrity.

Hiring Company

Citi