Position Summary...
What you'll do...
Role Summary
As a
Principal Engineer on the Walmart Advertising technology team, you will define and lead the test engineering strategy for AI-powered advertising systems — spanning ML model validation, intelligent audience pipelines, real-time bidding infrastructure, and programmatic data platforms. You'll architect test frameworks, embed quality into AI/ML development lifecycles, and operate as a senior technical leader across engineering, data science, and product.
About the Team
The mission of Walmart Ads is to connect brand owners and shoppers with relevant ads. We bring brand awareness and rich product information to shoppers and help brand owners grow their business. The Adtech QE team is a horizontal team, with cutting edge technologies, to understand shopper intent and surrounding context, serve relevant ads and optimize campaign performance end-to-end. We are a small, high-performance team that believes quality is a first-class engineering discipline. If you want to work on AI systems that operate at Walmart scale, this is the team .
What You'll Do
- Architect and own enterprise-scale test automation frameworks for APIs, data pipelines, UI, and services layers — built for maintainability, scale, and CI/CD integration
- Drive shift-left quality — embedding automated quality checks at every stage of the development lifecycle, from schema design through production monitoring
- Define and enforce quality gates in CI/CD pipelines at an organizational level; own release criteria and go/no-go decisions for major platform changes
- Lead data pipeline testing at scale — ETL/ELT validation, schema governance, event-level tracking accuracy, audience segment integrity, and aggregation correctness (BigQuery, Spark, Kafka)
- Own test strategy for AI/ML systems — including model validation, data drift detection, feature pipeline integrity, inference correctness, and algorithmic fairness testing for advertising models (lookalike, propensity, audience prediction)
- Define model quality gates in ML training and deployment pipelines — validating model outputs against golden datasets, statistical thresholds, and business KPIs before promotion to production
- Build automated evaluation frameworks for GenAI and LLM-integrated components — including prompt regression testing, output consistency checks, and hallucination detection pipelines
- Partner with data scientists to implement shadow testing, A/B experiment integrity validation, and statistical significance guardrails for algorithmic experiments
- Implement performance and chaos engineering for latency-sensitive systems — bidding engines, ad decisioning, and audience lookup at millions of QPS
- Represent quality in architecture and design reviews — surfacing testability gaps, observability blind spots, and data integrity risks before they are built in
- Author quality frameworks, testing playbooks, and SDET standards adopted across WMX and Growth Tech
- Define SLOs and production quality posture — integrating quality signals into observability dashboards and driving proactive incident prevention
- Mentor and grow Staff and Senior SDETs; raise the engineering bar through design reviews, code reviews, and hiring
What you'll bring: Required
- 8+ years in quality/test engineering; 3+ years at Staff or Principal SDET level in distributed, large-scale systems
- Proven expertise testing AI/ML systems — model validation, feature store integrity, inference pipeline testing, or data quality for ML training datasets
- Expert-level experience designing test automation frameworks for APIs, data, and services layers (Pytest, Selenium, Appium, WebDriver, or equivalent)
- Deep expertise in data pipeline quality — ETL/ELT validation, schema governance, data completeness, transformation correctness at scale (BigQuery, Hive, Spark)
- Strong experience testing event-driven and async systems — Kafka pipelines, REST APIs, webhook integrations
- SQL as a first-class testing tool — writing complex queries against large analytical datasets to validate data correctness
- Experience enforcing quality gates in CI/CD at organizational scale
Preferred - Experience with LLM/GenAI quality — prompt regression testing, output evaluation frameworks, hallucination detection, or RAG pipeline validation
- Expertise in programmatic advertising quality — RTB auction integrity, DSP/SSP integrations, ad serving correctness, measurement/attribution validation
- Hands-on with GCP data platform: BigQuery, Dataflow, Pub/Sub, Vertex AI
- Experience in privacy and compliance testing — GDPR, CCPA, consent signal propagation, data deletion verification, CMP validation
- Proficiency with performance and load testing for high-throughput, low-latency systems
- Familiarity with MLOps tooling — experiment tracking (MLflow, Vertex AI), model registries, feature stores, and deployment pipelines
- Experience with data clean room and privacy-preserving measurement validation
- Background in building and growing SDET teams — hiring, mentoring, technical interview design
At Walmart, we offer competitive pay as well as performance-based bonus awards and other great benefits for a happier mind, body, and wallet. Health benefits include medical, vision and dental coverage. Financial benefits include 401(k), stock purchase and company-paid life insurance. Paid time off benefits include PTO (including sick leave), parental leave, family care leave, bereavement, jury duty, and voting. Other benefits include short-term and long-term disability, company discounts, Military Leave Pay, adoption and surrogacy expense reimbursement, and more.
You will also receive PTO and/or PPTO that can be used for vacation, sick leave, holidays, or other purposes. The amount you receive depends on your job classification and length of employment. It will meet or exceed the requirements of paid sick leave laws, where applicable.
For information about PTO, see
https://one.walmart.com/notices.
Live Better U is a Walmart-paid education benefit program for full-time and part-time associates in Walmart and Sam's Club facilities. Programs range from high school completion to bachelor's degrees, including English Language Learning and short-form certificates. Tuition, books, and fees are completely paid for by Walmart.
Eligibility requirements apply to some benefits and may depend on your job classification and length of employment. Benefits are subject to change and may be subject to a specific plan or program terms.
For information about benefits and eligibility, see
One.Walmart.
The annual salary range for this position is $143,000.00 - $286,000.00
Additional compensation includes annual or quarterly performance bonuses.
Additional compensation for certain positions may also include :
- Stock
ㅤ
ㅤ
ㅤ
ㅤ
Minimum Qualifications...
Outlined below are the required minimum qualifications for this position. If none are listed, there are no minimum qualifications.
Option 1: Bachelor's degree in computer science, computer engineering, computer information systems, software engineering, or related area and 5 years’ experience in software engineering or related area.
Option 2: 7 years’ experience in software engineering or related area.
Preferred Qualifications...
Outlined below are the optional preferred qualifications for this position. If none are listed, there are no preferred qualifications.
Master’s degree in computer science, computer engineering, computer information systems, software engineering, or related area and 3 years' experience in software engineering or related area., We value candidates with a background in creating inclusive digital experiences, demonstrating knowledge in implementing Web Content Accessibility Guidelines (WCAG) 2.2 AA standards, assistive technologies, and integrating digital accessibility seamlessly. The ideal candidate would have knowledge of accessibility best practices and join us as we continue to create accessible products and services following Walmart’s accessibility standards and guidelines for supporting an inclusive culture.
Primary Location...
1345 Crossman Ave, Sunnyvale, CA 94089-1114, United States of America
Walmart and its subsidiaries are committed to maintaining a drug-free workplace and has a no tolerance policy regarding the use of illegal drugs and alcohol on the job. This policy applies to all employees and aims to create a safe and productive work environment.