AWS re:Invent 2016: Streaming ETL for RDS and DynamoDB (DAT315)

Kelvion New Plate Heat Exchanger


During this session Greg Brandt and Liyin Tang, Data Infrastructure engineers from Airbnb, will discuss the design and architecture of Airbnb’s streaming ETL infrastructure, which exports data from RDS for MySQL and DynamoDB into Airbnb’s data warehouse, using a system called SpinalTap. We will also discuss how we leverage Spark Streaming to compute derived data from tracking topics and/or database tables, and HBase to provide immediate data access and generate cleanly time-partitioned Hive tables.


Duration: 41:32
Publisher: Amazon Web Services
You can watch this video also at the source.