How Twitter Migrated its On-Prem Analytics to Google Cloud (Cloud Next ’18)

Bit Ninja


Twitter is migrating its existing on-premises revenue analytics infrastructure into Google Cloud. Using technologies such as Bigtable, BigQuery, and Dataflow, we have been able to provide a more robust analytic solution for significantly less cost.

This talk will focus on our work building an ETL pipeline to migrate an existing Lambda architecture into Google Cloud and building a query service to manage analytics queries to BigTable.

We will cover:
– Quickly and efficiently exporting data from our on-prem data stores into BigQuery
– Replacing analytic MapReduce jobs with BigQuery
– Loading results from BigQuery into Bigtable for high throughput online queries
– Querying data from Bigtable

Event schedule → http://g.co/next18

Watch more Data Analytics sessions here → http://bit.ly/2KXMtcJ
Next ‘18 All Sessions playlist → http://bit.ly/Allsessions

Subscribe to the Google Cloud channel! → http://bit.ly/NextSub


Duration: 32:6
Publisher: Google Cloud
You can watch this video also at the source.