Pinterest is hiring a remote Sr. Staff Software Engineer AI Platform!
Pinterest is hiring a Senior Staff Software Engineer to lead technical strategy for its AI Platform team, with a focus on turning the company's AI/ML evaluation tooling into a single, dependable system used across the organization. The platform underpins the work of hundreds of ML and AI engineers who build recommendations, advertising, visual search, growth and notifications, and trust-and-safety systems. In this role you will set direction rather than simply execute it: spotting opportunities to modernize the stack, shaping and elevating the roadmap, and making architectural calls that hold up as GenAI and ML workloads scale. Success means better launch decisions, higher-quality AI-powered products, and stronger confidence in the systems Pinterest ships. The position is remote anywhere in the country, with occasional in-person collaboration gatherings each quarter.
Key responsibilities
- Partner with technical leadership to identify modernization opportunities and expand the roadmap for a company-wide evaluation system
- Define cross-team technical strategy, align partner organizations, and drive adoption of shared evaluation capabilities
- Make durable architectural decisions that improve reliability and the overall user experience of the evaluation platform
- Establish evaluation standards that balance automated metrics, expert review, human feedback, safety considerations, and business results
- Lead complex, multi-team workstreams, resolve technical tradeoffs and risks, and deliver measurable impact
Key requirements
- 3+ years of hands-on work designing and operating evaluation systems for GenAI and complex ML, covering benchmark design, golden datasets, automated metrics, human evaluation, model-based grading, and failure analysis
- Strong Python programming ability with solid distributed-systems fundamentals, and comfort using AI tools for coding and design work
- Deep familiarity with GenAI and recommendation evaluation libraries and frameworks such as AutoEvals, OpenEvals, and Ragas
- 5+ years building scalable platforms spanning distributed experimentation, data and model versioning, orchestration, observability, release management, and production monitoring
- Sound judgment on GenAI risks including robustness, bias, privacy, security, and the limits of automated evaluation signals
- Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience
- Seniority: Senior Staff
- Work mode: Remote (with in-person collaboration 1-2 times per quarter)
- Salary: $208,592 - $429,454 USD
- Key skills: Python, distributed systems, GenAI evaluation, ML evaluation frameworks, benchmark design, human evaluation, observability, release management, model versioning, orchestration, risk assessment, technical strategy
Want more jobs?
Join our mailing list to receive updates about new data science and Sr. Staff Software Engineer AI Platform jobs.
