Research Engineer, Live Experiment Analysis

At Google, research-focused Software Engineers are embedded throughout the company, allowing them to setup large-scale tests and deploy promising ideas quickly and broadly. Ideas may come from internal projects as well as from collaborations with research programs at partner universities and technical institutes all over the world.

From creating experiments and prototyping implementations to designing new architectures, engineers work on real-world problems including artificial intelligence, data mining, natural language processing, hardware and software performance analysis, improving compilers for mobile platforms, as well as core search and much more. But you stay connected to your research roots as an active contributor to the wider research community by partnering with universities and publishing papers.

Artificial intelligence will be one of humanity’s most transformative inventions. At Google DeepMind, we are a pioneering AI lab with exceptional interdisciplinary teams focused on advancing AI development to solve complex global challenges and accelerate high-quality product innovation for billions of users. We use our technologies for widespread public benefit and scientific discovery, ensuring safety and ethics are always our highest priority.

Responsibilities

• Evaluate and model launch analysis.
• Work cross-functionally on data collection, management, quality measurement and improvements.
• Improve end-to-end evaluations by incorporating user and product feedback, and providing continuous production evaluation/verification for model releases.
• Provide user feedback analysis and insights to improve our frontier models and the user experiences.
• Solve complex, high-impact research and engineering challenges where the solution is non-obvious to executive leaders.

Minimum qualifications:

• Bachelor's degree in Computer Science, Electrical Engineering, Mathematics, Statistics, a related technical field, or equivalent practical experience.
• Experience with machine learning.
• Experience in software engineering.
• Experience in Python programming.
  
• Experience in data analysis and related tools and processes.

Preferred qualifications:

• Master’s degree or PhD in Engineering, Computer Science, a technical related field, or equivalent practical experience.
• Extensive experiences on evaluations and user feedback analysis.
• Experience using internal/standard data analysis platforms and interactive notebooks (e.g., Colab).
• Strong quantitative and engineering skills and proven knowledge and experience working on large scale data analysis, autorater development, and human-in-the-loop quality improvements.
  
• Savvy users of LLM products (e.g., Gemini, etc.) who is familiar with standard LLM evaluation and testing methodologies.