Elastic is hiring a remote Principal AI Engineer Context!
Elastic is hiring a Principal AI Engineer to join the Context Engine team, the group responsible for the knowledge layer that AI agents use to interact with enterprise data stored in Elasticsearch. The team turns information from varied sources into a structured AI Index, exposes it to agents through public APIs, MCP tools, and framework integrations (including Elastic's Agent Builder, Claude Code, and LangChain), then feeds agent traces back in so the engine improves with real usage. This principal-level, remote role sits at the intersection of engineering, data science, and product: the hire owns the production improvement loop end to end, deciding how agents, automations, and skills behave, how they are observed and evaluated, and how changes ship safely. The codebase is TypeScript and developed in the open alongside the rest of the Elastic Stack.
Key responsibilities
- Own the production improvement loop for Context Engine: diagnose failure modes in extraction automations, retrieval tools, and memory using offline evaluations, telemetry, and customer conversations, then fix and prove the fix.
- Define safe iteration practices for agents and skills, including versioning and rollout of prompts and automations, regression coverage, staged and shadow evaluation, and guardrails.
- Design telemetry capturing agent traces, tool calls, and knowledge retrieval, landing it in Elasticsearch to power evaluation, dashboards, and feedback loops.
- Partner with data scientists on evaluation strategy: golden datasets and evaluators gating on quality, latency, and cost.
- Raise the team's technical bar through design and PR review, mentoring in eval-driven development, and writing proposals that shape the roadmap.
Key requirements
- 10+ years of software engineering experience, with recent work shipping and operating AI-driven products on real production traffic.
- Demonstrated eval-driven improvement: diagnosing agent or LLM behaviour from traces and feedback, designing evaluations, shipping fixes, and measuring outcomes.
- Hands-on experience building agents with state and memory, and safely iterating on prompts, skills, and tool behaviour in production.
- Familiarity with MCP, including exposing public MCP servers and tools.
- Experience designing telemetry for AI systems and running product experiments end to end.
- Backend strength in Python or TypeScript for APIs, stateful workflows, pipelines, and production services.
- Comfort collaborating with data scientists, engineers, and product managers in a distributed, async-first, low-ego setting.
- Seniority: Principal
- Work mode: Remote
- Salary: $154,000 – $243,600 CAD
- Key skills: TypeScript, Python, AI agents, LLM evaluation, Telemetry and observability, MCP, Public APIs, Agent frameworks (LangGraph, CrewAI, Claude Agent SDK), Agentic retrieval, Elasticsearch
Want more jobs?
Join our mailing list to receive updates about new data science and Principal AI Engineer Context jobs.
