How Twitter Is Migrating 300 PB of Hadoop Data to GCP (Cloud Next '19)


Twitter has been migrating their complex Hadoop workload to Google Cloud. In this session, we deep dive into how Twitter’s components use Cloud Storage Connector and describe our initial usage, features we implemented, and how Google helped us build those features in open source. We describe how Cloud Storage fits into our ecosystems and the experience and features which have helped us. We’ll also talk about unique challenges we discovered in data management at scale.

Migrating Hadoop to GCP → http://bit.ly/2KbS1Ap

Watch more:
Next ’19 Storage Sessions here → https://bit.ly/Next19Storage
Next ‘19 All Sessions playlist → https://bit.ly/Next19AllSessions

Subscribe to the GCP Channel → https://bit.ly/GCloudPlatform

Moderator: Sameer Farooqui
Panelists: Vrushali Channapattan, Joep Rottinghuis, Lohit VijayaRenu

Session ID: STO201
product:Cloud Storage,Storage Transfer Service;


Duration: 49:58
Publisher: Google Cloud
You can watch this video also at the source.