micro1
Skills: Python
Job Title: Member of Technical Staff, Realm Lab (frontier AI research)
Job Type: Full time
Location: Remote
The Role
We're hiring a Member of Technical Staff (MTS) to join Realm Lab and drive research initiatives focused on evaluating and improving frontier AI systems.
This is a hands-on technical role at the intersection of machine learning, data research, and evaluation engineering. You'll work closely with researchers and domain experts to develop benchmarks, reinforcement learning environments, and high-quality datasets that advance model capabilities and performance.
What You'll Do
1. Own research and evaluation initiatives end-to-end, from task design and dataset development to benchmark validation and performance analysis. 2. Design and validate capability evaluations, RL environments, and benchmark suites for computer use, agentic workflows, and emerging model capabilities. 3. Develop high-quality datasets, task specifications, evaluation rubrics, and methodologies to improve post-training data quality and model assessment. 4. Analyze model behavior through failure analysis, error taxonomies, and capability gap identification to inform research directions. 5. Collaborate with researchers and domain experts to design experiments, calibrate evaluation quality, and improve model performance. 6. Build reproducible evaluation pipelines and feedback loops to support iterative model development. 7. Contribute to benchmarks, research papers, technical reports, and open research artifacts.
What We're Looking For
1. Strong technical background in computer science, machine learning, AI, or a related field. 2. Hands-on experience designing ML evaluations, benchmark suites, research datasets, or experimental frameworks. 3. Proficiency in Python and experience building data pipelines, evaluation tools, or ML research infrastructure. 4. Understanding of LLMs, foundation models, post-training techniques, and evaluati …truncated… , expert-generated datasets, or post-training data. 5.…
Sign in to see the original posting
Browse more developer jobs on RoleBurst