Software Engineer, Simulator Evaluation

Waymo’s simulator is one of the most complex virtual environments ever built. It blends deterministic logic, physical dynamics, and state-of-the-art Generative AI to create a training ground for the Waymo Driver. The Simulator Evaluation team faces the ultimate data challenge: How do you mathematically prove that a virtual world is "real"?

We are looking for a Software Engineer to build the metrics and pipelines that grade this hybrid environment. You will work at the intersection of systems engineering and AI, writing the code that processes petabytes of simulation logs to answer a simple but vital question: "Did the simulator do the right thing?"

In this role, you will report to Senior Staff Software Engineering Manager and serve as a core contributor to the simulator evaluation stack.

You will:

• Implement Evaluation Metrics and mine Datasets: You will translate mathematical concepts and product requirements into robust, efficient code (C++/Python). You will implement metrics that measure everything from agent rule compliance (e.g., "Did it stop at the red light?") to the statistical quality of AI-generated traffic. You will use a variety of methods to mine from Waymo's enormous corpse of driving logs to select the most relevant examples to evaluate the simulator on.
• Build Scalable Pipelines: You will own key components of our data processing infrastructure. You will write and maintain the pipelines that ingest simulation logs, compute metrics at scale, and surface results to the broader engineering team.
• Analyze & Debug: You will act as the first line of defense for simulation quality. You will dig into metric results to distinguish between genuine regressions (e.g., a code bug causing cars to stall) and evaluation noise.
• Collaborate & Iterate: You will work closely with Tech Leads to refine our evaluation capabilities, quickly prototyping new ideas and hardening them for production use.

You have:

• Engineering Craftsmanship:
• 2+ years of industry experience in software development.
• Solid proficiency in Python or C++, with experience writing clean, testable, and maintainable production code.
• Experience with version control, code review, and debugging complex systems.
• Data Fluency:
• You are comfortable working with data. You know how to query a dataset (SQL/Pandas), visualize a result, and sanity-check your own metrics.
• You have a curiosity for "why" the numbers look the way they do.
• Adaptability:
• You are excited to work in a hybrid environment where you might evaluate a physics engine one day and a generative model the next.

We prefer:

• Experience with data processing frameworks (e.g., MapReduce, Flume, Spark), data analysis libraries (Pandas, NumPy), or machine learning.
• Background in Autonomous driving, Robotics, Simulation, or AdTech.
• Familiarity with basic statistics or machine learning concepts.
• Experience validating software performance or quality (e.g., building test infrastructure).