Amazon's Big Data Technologies (BDT) organization builds the data platform that connects millions of businesses of all sizes to hundreds of millions of customers across Amazon marketplaces worldwide. The PartiQL/HubSchema team sits at the heart of this platform: we build the query language, schema modeling, and data validation technologies that make exabytes of data discoverable, trustworthy, and queryable across Amazon.
We own PartiQL, Amazon's open-source, SQL-compatible query language for semi-structured and nested data, and HubSchema, Amazon's unified schema modeling and operation solution. HubSchema implements a hub-and-spoke architecture with a canonical schema model that serves as the universal intermediary for schema operations - conversion, validation, and compatibility analysis - across diverse compute and storage systems including
AWS Glue, Andes (Amazon's data catalog), Apache
Iceberg, Apache
Avro,
Parquet,
Redshift, DynamoDB, and more. When a BDT service needs to convert, validate, or reason about schemas, HubSchema provides a single consistent answer - detecting type compatibility issues and data precision loss before an actual data transformation even occurs.
Our systems define how tens of thousands of datasets are modeled, validated, evolved, and queried. HubSchema is integrated across the BDT ecosystem - Cradle (the data loading engine), Maestro (the orchestration platform), DataCraft v3, External Tables, and Andes Views all rely on HubSchema converters to ensure consistent schema interpretation and unified conversion logic. We are actively driving adoption across remaining BDT services and eliminating legacy schema definition approaches in favor of a single unified standard (Andes Schema Spec v1.1+).
We are looking for a passionate and innovative engineer with a solid technical background to join the team. You will design and build core language and schema infrastructure used by virtually every data producer and consumer at Amazon:
- Extending HubSchema's canonical model and spoke converters to support new storage formats and compute engines
- Evolving the PartiQL specification, its
Kotlin/JVM and
Rust implementations, and runtime performance for latency-sensitive use cases
- Building schema validation, conversion, and compatibility-checking services that guard data quality at Amazon scale
- Delivering HubSchema service APIs that power schema operations across the BDT platform
- Contributing to PartiQL as an open-source project
The successful candidate will have a background in building distributed systems or data infrastructure, strong computer science fundamentals, good communication skills, and the motivation to achieve results in a fast-paced environment. Experience with query engines, compilers, type systems, data serialization formats, or schema management is a strong plus - but curiosity and rigor matter more than prior exposure to any specific technology.
Key job responsibilities
- Design, implement, and operate core components of PartiQL and HubSchema used across Amazon's data platform
- Build and extend HubSchema spoke converters (
Iceberg,
Avro,
Parquet,
Glue,
Redshift, Ion) and the canonical schema model
- Develop schema validation and compatibility APIs that detect type mismatches, precision loss, and breaking changes before they reach production
- Enhance PartiQL runtime performance (lazy evaluation, async execution, zero-copy Ion integration) for latency-sensitive consumers
- Drive HubSchema integration across BDT services, replacing legacy one-off conversion logic with unified library converters
- Contribute to the open-source PartiQL specification and reference implementations
- Collaborate with partner teams across BDT (Catalog, Compute, Cradle, Maestro, DataCraft) to deliver end-to-end customer experiences
- Raise the bar on operational excellence, testing, and engineering quality for systems in the critical path of Amazon's data ecosystem
About the team
The Sahale team owns PartiQL and HubSchema - the query language and schema technologies at the foundation of Amazon's data platform. We are a team of engineers who care deeply about language design, type systems, and data correctness at scale. Our work is unusual in the best way: we operate open-source projects with an external community, publish a formal language specification, and ship libraries and services that virtually every data producer and consumer at Amazon depends on. If you want your code to be in the critical path of exabytes of data, this is the team.
Why BDT?
The Business Data Technologies (BDT) organization exists to serve Amazon's growing analytics needs. BDT's mission is to accelerate Amazon's data-driven business, enable the next generation of analytics and machine learning technologies at scale, and raise the bar on global customer trust by cataloging, protecting, enriching, and brokering all SDO data through its lifecycle. Our teams develop and evolve services for storage and access to the authoritative repository of all data published by source teams across Amazon - enhanced with aggregations and transformations for use by consuming teams using modern compute services. BDT enterprise data products are available through DataCentral, the one-stop hub for data analytics tools at Amazon, spanning the Andes data catalog, ingestion, processing, egress, compliance, and infrastructure. The problems here are genuinely hard: schema evolution across tens of thousands of datasets, query processing over semi-structured data, and unifying data experiences across formats and engines - at a scale few organizations ever reach.
- 3+ years of non-internship professional software development experience
- 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
- 1+ years of software development engineer or related occupational experience
- 1+ years of designing and developing large-scale, multi-tiered, multi-threaded, embedded or distributed software applications, tools, systems, and services using:
C#,
C++,
Java, or
Perl experience
- 1+ years of Object Oriented Design experience
- Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field
- Experience programming with at least one software programming language
- 3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
- Bachelor's degree in computer science or equivalent
Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit
https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.
The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.
USA, WA, Seattle - 143,700.00 - 194,400.00 USD annually