tripactions · New York, NY
About the Role We are building a fleet of specialized AI services that improve high-value human workflows across the company. As a Senior Software Engineer, Core AI, you will design, build, evaluate, and operate production-grade AI services and agents. You will own work end to end—from data and context design through tool use, application logic, evaluation, and production monitoring. What You’ll Do: Build specialized AI services: design and implement task-specific services and agents, including data serialization, retrieval, memory where appropriate, tool integrations, and bounded multi-step reasoning. Define and evaluate quality: work with partners to make expected behavior explicit, write and maintain evals against representative examples, and use results to measure performance against expectations. Improve quality systematically: use evaluation findings, failure analysis, and production feedback to improve context and data, service logic, tools, and model behavior. Build for production: integrate tracing, observability, versioning, monitoring, reliability practices, and success metrics into the services you own. Run controlled experiments: compare models, data, and implementation approaches using repeatable benchmarks so improvements are attributable and reproducible. Partner on architecture: work with AI Architecture, AI Ops, and Platform teams to make services secure, reliable, cost-aware, and deployable at scale. Ship iteratively: move quickly from user problem to prototype to production, learning from real workflow outcomes. What We’re Looking For: 5+ years of strong software engineering experience delivering production services or products, ideally at senior or staff level. Hands-on experience building AI/LLM-powered applications or agents with real users or operational use. Experience with evaluation methods for AI behavior: defining expected outcomes, building test sets or evals, tracking regressions, and acting on results. A data-first mindset: able to sh…
Sign in to see the original posting
Browse more developer jobs on RoleBurst