Member of Technical Staff - Enterprise Model Evaluation

About the Role

The Model Evaluations team aims to design and implement xAI’s evaluations that shapes how we understand, measure and improve our model’s capabilities. You will work at the intersection of research and product to develop and implement model evaluations that give us high signal into merging model capabilities and robust evaluation infrastructure that enables fast iterations of our models.

Your work will be essential to xAI’s mission of understanding the universe. You will collaborate closely with the training and product teams to ensure our models meet the highest standards before deployment. This is a technical leadership role where you will be expected to drive both the vision and implementation of our model evaluations.

Responsibilities

• Design and implement next-generation evaluation suites beyond traditional benchmarks, creating frameworks that capture real-world utility and performance of Grok in production environments.
• Coordinate model evaluation efforts and collaborations to ensure comprehensive coverage and fast iterations.
• Integrate Grok into production systems, gain deep insights into real-world environments, and ensure alignment with user needs and business objectives.
• Partner with research teams to translate cutting-edge techniques and Grok models into production-ready implementations, optimizing for performance and impact.

Exceptional candidates may have:

• Proven expertise in designing and implementing sophisticated evaluation frameworks for machine learning models, especially LLMs.
• Experience with statistical analysis, experimental design, and benchmarking AI systems in real-world settings.

Location

• The role is based in Palo Alto. Our team usually works from the office 5 days a week but allow work-from-home days when required. Candidates are expected to be located near Palo Alto or open to relocation.

Interview Process

After submitting your application, the team reviews your CV and statement of exceptional work. If your application passes this stage, you will be invited to a 15 minute interview (“phone interview”) during which a member of our team will ask some basic questions. If you clear the initial phone interview, you will enter the main process, which consists of four technical interviews:

1. Coding assessment (2): in a language of your choice to solve a logical, algorithmic problem meant to be solved in 40 minutes setting.
2. System design: designing a scalable, reliable system for one of the real world problem demonstrating first-principles thinking by breaking down requirements, justifying architectural choices, and addressing challenges.
3. Meet the Team: Present your past exceptional work and your vision with xAI to a small audience.

Our goal is to finish the main process within one week. Final interviews will be conducted in person.

Annual Salary Range

$180,000 - $440,000 USD

Benefits

Base salary is just one part of our total rewards package at xAI, which also includes equity, comprehensive medical, vision, and dental coverage, access to a 401(k) retirement plan, short & long-term disability insurance, life insurance, and various other discounts and perks.