airbnb · United States
Skills: Docker
Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way. The Community You Will Join Airbnb’s Data Infrastructure organization builds the platforms that power data across the company. Within it, the Data Movement & Transformation (DMT) pillar builds the platform that moves data between Airbnb’s storage engines and transforms it along the way. Today, data moves through many bespoke, one-off pipelines that are built and maintained differently by each team. We’re replacing them with one unified, declarative platform. You define data once, and it is replicated everywhere it needs to go. This work underpins Airbnb’s Data as a Service initiative. You’ll join the team that owns three connected layers of that platform: Control plane: the layer that turns declarative data definitions into running pipelines, including provisioning, runtime selection, and lifecycle management. Schema: how schemas are defined, tracked, and evolved safely across every system and engine. Realtime Data Infrastructure (RTDI): Kafka and the streaming transport layer that real-time data flows through at scale. As a Staff Software Engineer, you’ll set technical direction across these layers and stay hands-on in the code. Examples of the Difference You’ll Make Design and evolve the control plane that automatically provisions Kafka topics, registers schemas, and deploys pipelines from a declarative definition. Build schema management that makes evolution safe and backward compatible across streaming, storage, and warehouse systems. Improve the reliability, scalability, and cost efficiency of RTDI and Kafka at Airbnb’s scale, including capacity, multi-tenancy, and failure handling. Replace fragmented pipe…
Sign in to see the original posting
Browse more developer jobs on RoleBurst